Importing Time Series Into MATLAB: Difference between revisions

From CCN Wiki
Jump to navigation Jump to search
No edit summary
 
(10 intermediate revisions by 2 users not shown)
Line 1: Line 1:
The next step is to load your data matrices. Assuming the time series are plaintext files containing only structured numerical data (e.g., as produced by the gettimecourses.sh script), you can use the <code>load</code> function to import these data.
The next step is to load your data matrices. Assuming the time series are plaintext files containing only structured numerical data (e.g., as produced by the gettimecourses.sh script), you can use the <code>load</code> function to import these data.
= Functionally-assisted loading =
= Functionally-assisted loading =
== June 2019 MNI305 Update ==
loadFSTS() has been updated in June 2019 to also automatically load time series extracted in mni305 voxel space. Update your local copies accordingly.
== Running the loadFSTS Script ==
The steps described in this subsection have been simplified with the inevitable development of a MATLAB function, <code>loadFSTS.m</code> that can be found in the ubfs Scripts/Matlab folder. If you have copied it to a folder in your MATLAB path, it can be invoked thus:
The steps described in this subsection have been simplified with the inevitable development of a MATLAB function, <code>loadFSTS.m</code> that can be found in the ubfs Scripts/Matlab folder. If you have copied it to a folder in your MATLAB path, it can be invoked thus:
  M=loadFSTS(); %A dialog box will prompt you to select the files you want to open
  M=loadFSTS(); %A dialog box will prompt you to select the files you want to open
One convenient feature is the ability to omit specified columns when loading the data (e.g., the '' 'unknown' '' region or the '' 'corpuscallosum' ''). Two optional paired arguments include ''ldrop'' and ''rdrop'', which are matched with arrays corresponding to the columns (regions) to be dropped.  
One convenient feature is the ability to omit specified columns when loading the data (e.g., the '' 'unknown' '' region or the '' 'corpuscallosum' ''). Optional argument sets include ''ldrop'', ''rdrop'' and ''mnidrop'', which are matched with arrays corresponding to the columns (regions) to be dropped.  
''NOTE: If using ROI's, you likely only want to remove column 1 (unknown).''
 
  %Omit columns 1 and 5 from both hemispheres
'''NOTE: If using ROI's, you likely only want to remove column 1 (unknown).''' First check the *.sum.txt files to confirm which columns you want to discard, because the UNKNOWN region might already be excluded from the .wav.txt files.
 
  %Omit columns 1 and 5 from both hemispheres, column 1 from subcortical regions
  ldropregions=[1 5];
  ldropregions=[1 5];
  rdropregions=[1 5];
  rdropregions=[1 5];
  M=loadFSTS('ldropregions', ldropregions, 'rdropregions', rdropregions);
  mnidropdregions=[1];
Of course, you can and should read the help documentation to find out the full capabilities of the function, which I assure you is very clever. But the above functionality will probably get you through 99.99% of the time. Rather than load each time series for each hemisphere into a separate array, loadFSTS will return a single cell array, where each cell contains the merged lh &amp; rh data for each session. Thus, if you open up the lh/rh time series values for 6 fMRI runs, '''''M''''' will be a 6&times;1 cell array, where '''''M'''''{i} is a ''t timepoints'' &times; ''r regions'' matrix (''r regions'' is the sum of the regions retained for lh and rh data sets)
[M, hemis]=loadFSTS('ldropregions', ldropregions, 'rdropregions', rdropregions, 'mnidropregions', mnidropregions);
Of course, you can and should read the help documentation to find out the full capabilities of the function, which I assure you is very clever. But the above functionality will probably get you through 99.99% of the time. Rather than load each time series for each hemisphere into a separate array, loadFSTS will return a single cell array, where each cell contains the merged lh &amp; rh &amp; subcortical data for each session. Thus, if you open up the lh/rh/subcortical time series values for 6 fMRI runs, '''''M''''' will be a 6&times;1 cell array, where '''''M'''''{i} is a ''t timepoints'' &times; ''r regions'' matrix (''r regions'' is the sum of the regions retained for lh, rh and mni305 data sets). '''''hemis''''' will be a 1 &times; r vector, with a -1 assigned to all columns  of '''M''' from the left hemisphere, a 0 assigned for the columns corresponding to the subcortical regions, and a 1 assigned to the right hemisphere regions (note that this is a change from previous versions of the script that assigned 1=left, 0=right, -1=subcortical).


If each matrix '''''M'''''{i} is the same size, you can convert the cell array into a 3D matrix, which can facilitate a number of tasks. There are a number of ways to do this. For example:
If each matrix '''''M'''''{i} is the same size, you can convert the cell array into a 3D matrix, which can facilitate a number of tasks. There are a number of ways to do this. For example:
  new_M=cat(3,M{1}, M{2}, ... , M{i}); %concatenates each 2D matrix stored in M along a 3rd dimension
  new_M=cat(3,M{1}, M{2}, ... , M{i}); %concatenates each 2D matrix stored in M along a 3rd dimension
  %new_M is now a 3D matrix of size ''t'' &times; ''r'' regions &times; ''i'' input series
  %new_M is now a 3D matrix of size ''t'' &times; ''r'' regions &times; ''i'' input series
= Batch FSTS =
list=load('participants.txt');
for p=1:length(list)
    %dynamically figure out subject ID
    subid=sprintf('FS_%d',list(p));
    dirname=[pwd() filesep subid filesep 'bold'];
    lfiles=dir([dirname filesep '*lh*.wav.txt']);
    lfilenames={lfiles.name};
    rfiles=dir([dirname filesep '*rh*.wav.txt']);
    rfilenames={rfiles.name};
    for n=1:length(lfilenames)
        lfilenames{n}=fullfile(dirname, lfilenames{n});
    end
    for n=1:length(rfilenames)
        rfilenames{n}=fullfile(dirname, rfilenames{n});
    end
   
[M, hemis]=loadFSTS('lnames', lfilenames, 'rnames', rfilenames);


=Bulk Functionally-Assisted Loading=
=Bulk Functionally-Assisted Loading=
Line 20: Line 46:
  %% General Config %%
  %% General Config %%
  rootdir=pwd();
  rootdir=pwd();
  annot='myaparc_125';
  annot='lausanne';
  pattern='fmcpr.down.sm4';
  pattern='fmcpr.up.sm4';
  ldrop=[1 5];
  ldrop=[1];
  rdrop=[1 5];
  rdrop=[1];
mnidrop=[1:7,12:14,17,20:27, 36:43]; %after reviewing the sum.txt file, I don't want these columns
   
   
  %% 1. Get the subjects to process %%
  %% 1. Get the subjects to process %%
Line 55: Line 82:
     lnames=cellfun(@(x) [subj '_lh_' annot '_' x '.' pattern '.wav.txt'], Runs, 'UniformOutput', false);
     lnames=cellfun(@(x) [subj '_lh_' annot '_' x '.' pattern '.wav.txt'], Runs, 'UniformOutput', false);
     rnames=cellfun(@(x) [subj '_rh_' annot '_' x '.' pattern '.wav.txt'], Runs, 'UniformOutput', false);
     rnames=cellfun(@(x) [subj '_rh_' annot '_' x '.' pattern '.wav.txt'], Runs, 'UniformOutput', false);
     [M, hemis]=loadFSTS('lnames', lnames, 'rnames', rnames, 'ldropregions', ldrop, 'rdropregions', rdrop);
    mninames=cellfun(@(x) [subj '_subcort_' annot '_' x '.' pattern '.wav.txt'], Runs, 'UniformOutput', false);
     save(fullfile(rootdir, [subj '.' annot '.timeseries.mat']), 'M', 'hemis', 'ldrop', 'rdrop');
     [M, hemis]=loadFSTS('lnames', lnames, 'rnames', rnames, 'mninames', mninames, ...
      'ldrop', ldrop, 'rdrop', rdrop, 'mnidrop', mnidrop);
     save(fullfile(rootdir, [subj '.' annot '.timeseries.mat']), 'M', 'hemis', 'ldrop', 'rdrop', 'mnidrop');
     cd(rootdir);
     cd(rootdir);
  end
  end

Latest revision as of 13:10, 25 May 2020

The next step is to load your data matrices. Assuming the time series are plaintext files containing only structured numerical data (e.g., as produced by the gettimecourses.sh script), you can use the load function to import these data.

Functionally-assisted loading

June 2019 MNI305 Update

loadFSTS() has been updated in June 2019 to also automatically load time series extracted in mni305 voxel space. Update your local copies accordingly.

Running the loadFSTS Script

The steps described in this subsection have been simplified with the inevitable development of a MATLAB function, loadFSTS.m that can be found in the ubfs Scripts/Matlab folder. If you have copied it to a folder in your MATLAB path, it can be invoked thus:

M=loadFSTS(); %A dialog box will prompt you to select the files you want to open

One convenient feature is the ability to omit specified columns when loading the data (e.g., the 'unknown' region or the 'corpuscallosum' ). Optional argument sets include ldrop, rdrop and mnidrop, which are matched with arrays corresponding to the columns (regions) to be dropped.

NOTE: If using ROI's, you likely only want to remove column 1 (unknown). First check the *.sum.txt files to confirm which columns you want to discard, because the UNKNOWN region might already be excluded from the .wav.txt files.

%Omit columns 1 and 5 from both hemispheres, column 1 from subcortical regions
ldropregions=[1 5];
rdropregions=[1 5];
mnidropdregions=[1];
[M, hemis]=loadFSTS('ldropregions', ldropregions, 'rdropregions', rdropregions, 'mnidropregions', mnidropregions);

Of course, you can and should read the help documentation to find out the full capabilities of the function, which I assure you is very clever. But the above functionality will probably get you through 99.99% of the time. Rather than load each time series for each hemisphere into a separate array, loadFSTS will return a single cell array, where each cell contains the merged lh & rh & subcortical data for each session. Thus, if you open up the lh/rh/subcortical time series values for 6 fMRI runs, M will be a 6×1 cell array, where M{i} is a t timepoints × r regions matrix (r regions is the sum of the regions retained for lh, rh and mni305 data sets). hemis will be a 1 × r vector, with a -1 assigned to all columns of M from the left hemisphere, a 0 assigned for the columns corresponding to the subcortical regions, and a 1 assigned to the right hemisphere regions (note that this is a change from previous versions of the script that assigned 1=left, 0=right, -1=subcortical).

If each matrix M{i} is the same size, you can convert the cell array into a 3D matrix, which can facilitate a number of tasks. There are a number of ways to do this. For example:

new_M=cat(3,M{1}, M{2}, ... , M{i}); %concatenates each 2D matrix stored in M along a 3rd dimension
%new_M is now a 3D matrix of size t × r regions × i input series

Batch FSTS

list=load('participants.txt');
for p=1:length(list)
   %dynamically figure out subject ID
   subid=sprintf('FS_%d',list(p));
   dirname=[pwd() filesep subid filesep 'bold'];
   lfiles=dir([dirname filesep '*lh*.wav.txt']);
   lfilenames={lfiles.name};
   rfiles=dir([dirname filesep '*rh*.wav.txt']);
   rfilenames={rfiles.name};
   for n=1:length(lfilenames)
       lfilenames{n}=fullfile(dirname, lfilenames{n});
   end
   for n=1:length(rfilenames)
       rfilenames{n}=fullfile(dirname, rfilenames{n});
   end
   
[M, hemis]=loadFSTS('lnames', lfilenames, 'rnames', rfilenames);

Bulk Functionally-Assisted Loading

The structure of the Freesurfer directories and some of the existing helper files can be leveraged in a Matlab script to automatically call loadFSTS on all the .wav.txt time series data in a SUBJECTS_DIR. The following code parses the contents of the subjects file (used by preproc-sess, et al.) and a runs file that can be placed in each subject's bold directory. The code will iterate through each subject to navigate to that subject's functional data (where the .wav.txt files should be located). There, it will parse the runs file to string together the names of all the .wav.txt files that should be found there. These dynamically-generated file names can be passed as parameters to the loadFSTS() function. The .mat files that are produced will be saved in SUBJECTS_DIR.

%% General Config %%
rootdir=pwd();
annot='lausanne';
pattern='fmcpr.up.sm4';
ldrop=[1];
rdrop=[1];
mnidrop=[1:7,12:14,17,20:27, 36:43]; %after reviewing the sum.txt file, I don't want these columns

%% 1. Get the subjects to process %%
fid=fopen('subjects');
Subjects={};
while 1
   s = fgetl(fid);
   if ~ischar(s)
       break
   end
   Subjects{length(Subjects)+1}=s;
end
fclose(fid);
%% 2. For each subject, get the runs to process %%
for s=1:length(Subjects)
   subj=Subjects{s};
   dirname=[subj filesep 'bold'];
   cd(dirname);
   
   fid=fopen('runs');
   Runs={};
   while 1
       r = fgetl(fid);
       if ( ~ischar(r) || length(r)<1 )
           break
       end
       Runs{length(Runs)+1}=r;
   end
   fclose(fid);
   % Cross the subject with each of the runs
   lnames=cellfun(@(x) [subj '_lh_' annot '_' x '.' pattern '.wav.txt'], Runs, 'UniformOutput', false);
   rnames=cellfun(@(x) [subj '_rh_' annot '_' x '.' pattern '.wav.txt'], Runs, 'UniformOutput', false);
   mninames=cellfun(@(x) [subj '_subcort_' annot '_' x '.' pattern '.wav.txt'], Runs, 'UniformOutput', false);

   [M, hemis]=loadFSTS('lnames', lnames, 'rnames', rnames, 'mninames', mninames, ...
     'ldrop', ldrop, 'rdrop', rdrop, 'mnidrop', mnidrop);
   save(fullfile(rootdir, [subj '.' annot '.timeseries.mat']), 'M', 'hemis', 'ldrop', 'rdrop', 'mnidrop');
   cd(rootdir);
end