Configure preproc-sess: Difference between revisions

From CCN Wiki
Jump to navigation Jump to search
 
(35 intermediate revisions by 7 users not shown)
Line 1: Line 1:
You can preprocess multiple participants in a batch or individual participants, depending on how you invoke the preproc-sess command.
You can preprocess multiple participants in a batch or individual participants, depending on how you invoke the preproc-sess command. This took ~40 min to run one participant on Brocasarea.
= Before you start =
Check down at the Problems section at the bottom of this page. In particular, there is an additional step you will have to do to work with data collected from the DENT Institute ('''Header Problems 1''').
==Ensure your BOLD timing is correct!==
There is a chance that the data collected at UB does not have the correct header information. In particular, the TR information may be inaccurately set to 1.0 (we have never run an experiment with a 1-second TR). Both the LDT and the Semantic Categories experiment used a 2.047 TR, and so you should consult the [[Freesurfer_BOLD_files]] page for information on how to correct these files -- if the timing is incorrect, then the slice timing calculations will be incorrect. Discover this too late (like I just did), and you'll have to run preprocessing all over again!


== Create a subjectname file ==
== Create a subjectname file ==
Line 11: Line 15:
  echo `basename "$PWD"` > subjectname
  echo `basename "$PWD"` > subjectname
Using this method, the second command will always be the same for any participant.
Using this method, the second command will always be the same for any participant.
=== Batch Subjectname Setup===
The above techniques can be used in a shell script, iterating over a set of subject folder names:
'''setsubjectnames.sh'''
#!/bin/bash
#given a list of subject folders in $SUBJECTS_DIR, venture into each
#folder and ensure that an initialized subjectname file exists
for sub in "$@"
do
        cd ${SUBJECTS_DIR}/${sub}
        echo `basename "$PWD"` > subjectname
done
If your list of subjects can be found in $SUBJECTS_DIR/subjects, then you could easily ensure the file is present for all participants thus:
cd $SUBJECTS_DIR
setsubjectnames.sh `cat subjects`


== Run preproc-sess ==
== Run preproc-sess ==
This command does motion correction, spatial smoothing (to reduce noise, thereby increasing the signal-to-noise ratio), and slice-time correction on your functional (BOLD) data. It can be run on a batch of participants or on individuals.
This command does motion correction, spatial smoothing (to reduce noise, thereby increasing the signal-to-noise ratio), and slice-time correction on your functional (BOLD) data. It can be run on a batch of participants or on individuals. This command takes ~25-35 minutes per person for 6 runs worth of data, depending on the number of surfaces processed (lh, rh, mni305).


=== Batch processing ===
=== Batch processing ===
Line 25: Line 45:
  preproc-sess -sf <sessidfile> -df <srchdirfile> [options]
  preproc-sess -sf <sessidfile> -df <srchdirfile> [options]
e.g.,
e.g.,
  preproc-sess -sf chris_subjects.txt -surface self lhrh -fwhm 6 -per-run -fsd bold
  preproc-sess -sf <span style="color:red">chris_subjects.txt</span> -surface fsaverage lhrh -mni305 -per-run -fsd bold <span style="color:red">-sliceorder up -fwhm 4</span>
If the <code>-df</code> switch isn't provided (it doesn't seem to be mandatory) then it uses a default directory (possibly $SUBJECTS_DIR, or possibly your current working directory) to start in.
If the <code>-df</code> switch isn't provided (it doesn't seem to be mandatory) then it uses a default directory (possibly $SUBJECTS_DIR, or possibly your current working directory) to start in. The parameters in red in the example code above are those that may or may not differ between research project. The parameters in black are almost always going to be kept as-is.


=== Individuals ===
=== Individuals ===
Preprocessing individuals works pretty much the same, except you do not require a sessid file. Instead, you provide it with the name of one of the session folders, and use the <code>-s</code> switch, rather than the <code>-sf</code> switch:
Preprocessing individuals works pretty much the same, except you do not require a sessid file. Instead, you provide it with the name of one of the session folders, and use the <code>-s</code> switch, rather than the <code>-sf</code> switch. For example:
  preproc-sess -per-run -fwhm 6 -surface self lhrh -fsd bold -s FS_T1_501
  preproc-sess -s <span style="color:red">FS_T1_501</span> -per-run -surface fsaverage lhrh -mni305 -fsd bold <span style="color:red">-fwhm 4 -sliceorder up</span>
This processes subject FS_T1_501 using the same parameters as used for the batch processing example above. Again, the parameters in red may or may not differ for your project, but the text in black will be the same.


== Other options ==
== Other options ==
=== Check Registration ===
=== Check Registration ===
Before you proceed with any functional analyses, you should ensure that the bold data has been coregistered to the T1 anatomical data. You can use tkregister-sess to view a representative volume from each run, with the white matter surface contours mapped on to the volume:
Before you proceed with any functional analyses, you should ensure that the bold data has been coregistered to the T1 anatomical data. You can use tkregister-sess to view a representative volume from each run, with the white matter surface contours mapped on to the volume:
cd $SUBJECTS_DIR
  SUBJECT_ID=FS_T1_501
  SUBJECT_ID=FS_T1_501
  tkregister-sess -s ${SUBJECT_ID} -inorm -per-run
  tkregister-sess -s ${SUBJECT_ID} -inorm -per-run
If the contours are out of alignment with the functional data, you can use this interface to manually perform affine (i.e., rigid-body) transformations so that they line up. More information on this can be found [https://surfer.nmr.mgh.harvard.edu/fswiki/FsFastTutorialV4.5 here] and [https://surfer.nmr.mgh.harvard.edu/fswiki/FsTutorial/ManualRegistration here].
If the contours are out of alignment with the functional data, you can use this interface to manually perform affine (i.e., rigid-body) transformations so that they line up. More information on this can be found [https://surfer.nmr.mgh.harvard.edu/fswiki/FsFastTutorialV4.5 here] and [https://surfer.nmr.mgh.harvard.edu/fswiki/FsTutorial/ManualRegistration here].
Note: You must be in your subjects directory for this to work.


=== Slice time correction ===
=== Slice time correction ===
Slice time correction adjusts the functional data to account for the fact that, when you acquire an entire 3D volume as a series of 2-D planes (slices) over the span of 1 TR (commonly 2 seconds), then the very last slice you acquire is going to be delayed 2 seconds relative to the very first slice you acquire. There is a <code>-stc</code> switch that you can provide telling Freesurfer what order the slices were acquired, which the software uses to algorithmically adjust the signals for each slice to account for the fact that they occurred earlier vs. later within that 1 TR time window. Using this switch requires you to know something about the scanner. It is prudent to check with the scanner technician to verify the slice acquisition order so that you can provide the correct value to go along with the '''-stc''' switch. If you omit the '''-stc''' switch, slice time correction is not performed on the data.
Slice time correction adjusts the functional data to account for the fact that, when you acquire an entire 3D volume as a series of 2-D planes (slices) over the span of 1 TR (commonly 2 seconds), then the very last slice you acquire is going to be delayed 2 seconds relative to the very first slice you acquire. There is a <code>-sliceorder</code> switch that you can provide telling Freesurfer what order the slices were acquired, which the software uses to algorithmically adjust the signals for each slice to account for the fact that they occurred earlier vs. later within that 1 TR time window. Using this switch requires you to know something about the scanner. It is prudent to check with the scanner technician to verify the slice acquisition order so that you can provide the correct value to go along with the '''-sliceorder''' switch. If you omit the '''-sliceorder''' switch, slice time correction is not performed on the data.
 
Slice timing correction corrects each voxel's time-series for the fact that later processing assumes that all slices were acquired exactly half-way through the relevant volume's acquisition time (TR), whereas in fact each slice is taken at slightly different times. Slice timing correction works by using (Hanning-windowed) sinc interpolation to shift each time-series by an appropriate fraction of a TR relative to the middle of the TR period. It is necessary to know in what order the slices were acquired and set the appropriate option here. If slices were acquired from the bottom of the brain to the top select Regular up. If slices were acquired from the top of the brain to the bottom select Regular down. If the slices were acquired with interleaved order (0, 2, 4 ... 1, 3, 5 ...) then choose the Interleaved option. If slices were not acquired in regular order you will need to use a slice order file or a slice timings file. If a slice order file is to be used, create a text file with a single number on each line, where the first line states which slice was acquired first, the second line states which slice was acquired second, etc. The first slice is numbered 1 not 0. If a slice timings file is to be used, put one value (ie for each slice) on each line of a text file. The units are in TRs, with 0 corresponding to no shift. Therefore a sensible range of values will be between -0.5 and 0.5.
 
At the CRTC 3T magnet (as of November 2016), data are collected in sequential ascending order (i.e, from the chin to the crown of the head).


At the CRTC 3T magnet (as of November 2016), data are collected in sequential ascending order (i.e, from the chin to the crown of the head), corresponding to <code>'''-stc down'''</code> (note: initially I had indicated that the code was ''-stc up'', but after looking at some basic task vs. rest maps (which should show some robust blobs) it's pretty clear that down is correct).
Data collected at the DENT Neurologic Institute are reportedly also acquired sequentially from the chin to the crown of the head. From the description above, both of these acquisition timings should correspond to '''-sliceorder up'''


== Problems ==
== Problems ==
=== Unsupported slice timing 5 ===
=== Header Problems 1: Misstated volume count in the header ===
Running preproc-sess on an existing dataset, I ran into
Sigh of relief as I figured this out, so we didn't actually lose all of our DENT data.
 
It is more likely than not that the information about the number of volumes indicated in the header does not match the actual number of volumes recorded in the data for the SEMCAT experiment, which is self-paced. The MRI at DENT is configured to run for 256 volumes (512 seconds) per run. In practice, people typically finish before this. However, the .NII file headers are having the number of volumes listed as 256, and not the actual number of volumes acquired. When you go to run preproc-sess with these data, you will probably get an uninformative error right away when <code>mri_convert</code> tries to open up the data.
 
Fortunately, the problem is easily fixed (though discovering that it is easily fixed was not easy). Since this is 99.9% likely to affect any given functional run from a self-paced study (e.g., the SEMCAT experiment) you should probably get in the habit of opening up each of your functional volumes with fsledithd:
fsledithd f.nii
 
Don't do anything else. Just open it and then save it without making any modifications. When you return to the terminal you will see a message along the lines of:
 
++ WARNING: nifti_read_buffer(run4.nii):
  data bytes needed = 60817408
  data bytes input  = 52379648
  number missing    = 8437760 (set to 0)
 
What happened here is that fsledithd saw that there are supposed to be 256 volumes of data, but a smaller amount of data was found. So it went ahead and filled in all the missing bytes with 0. Now your f.nii file will be bigger, with all the missing volumes at the end filled with zeros.
 
=== Header Problems 2: Unsupported slice timing 5 ===
Running preproc-sess on an existing dataset from the Booth lab, I ran into
  nifti1Read(): unsupported slice timing pattern 5 in /home/chris/ubfs/cpmcnorg/openfmri/booth/FS_T2_501/bold/029/f.nii
  nifti1Read(): unsupported slice timing pattern 5 in /home/chris/ubfs/cpmcnorg/openfmri/booth/FS_T2_501/bold/029/f.nii
Some of the .nii files have invalid information in the header. In the case of the <code>slice_code</code> field, valid values are 0,1,2,3, or 4:
Some of the .nii files have invalid information in the header. In the case of the <code>slice_code</code> field, valid values are 0,1,2,3, or 4:
Line 62: Line 107:
=== ERROR: 3dvolreg.afni & libxp.so.6 ===
=== ERROR: 3dvolreg.afni & libxp.so.6 ===


This error occurs on Ubuntu 16.04 due to the lack of libXp packages. To fix this, we simply need to install an older version of the needed packages. Get the correct package [https://packages.debian.org/jessie/amd64/libxp6/download here]. Download this file (somewhere, probably Downloads is easiest). Open the terminal and cd to the directory you downloaded the file in, then run
This error occurs on Ubuntu 16.04 due to the lack of libXp packages. FreeSurfer piggybacks on some AFNI software that was built using outdated code: [https://neurostars.org/t/freesurfer-3dvolreg-afni-error-while-loading-shared-libraries-libxp-so-6/5893/12 as you can read about here].
  sudo dpkg --install <name of file>
To fix this, we can either:
# install an older version of the needed packages[https://packages.debian.org/jessie/amd64/libxp6/download here]
#* sudo dpkg --install <name of file> (not recommended because library is deprecated and unsupported)
# or symlink libXp.so.6 to libXpm4.so.11, which is included in the libXpm package
If you choose the latter option, you will find libXpm4 in <code>/usr/lib/x86_64-linux-gnu/</code>
 
Note: if you're having this problem, you may also find yourself missing libpng12
wget http://se.archive.ubuntu.com/ubuntu/pool/main/libp/libpng/libpng12-0_1.2.54-1ubuntu1_amd64.deb
  sudo dpkg --install libpng12-0_1.2.54-1ubuntu1_amd64.deb
 
You will need to get one of the End Bosses to run this as an admin.
 


You may need to get one of the End Bosses to run this as an admin.
[[Category: FreeSurfer]]

Latest revision as of 12:30, 11 May 2022

You can preprocess multiple participants in a batch or individual participants, depending on how you invoke the preproc-sess command. This took ~40 min to run one participant on Brocasarea.

Before you start

Check down at the Problems section at the bottom of this page. In particular, there is an additional step you will have to do to work with data collected from the DENT Institute (Header Problems 1).

Ensure your BOLD timing is correct!

There is a chance that the data collected at UB does not have the correct header information. In particular, the TR information may be inaccurately set to 1.0 (we have never run an experiment with a 1-second TR). Both the LDT and the Semantic Categories experiment used a 2.047 TR, and so you should consult the Freesurfer_BOLD_files page for information on how to correct these files -- if the timing is incorrect, then the slice timing calculations will be incorrect. Discover this too late (like I just did), and you'll have to run preprocessing all over again!

Create a subjectname file

The first thing you will need to do is create a file called subjectname in the folders for each of the participants you will be preprocessing. This file contains only a single line, indicating the Freesurfer subjectid by which that subject is known. For example, for the subject whose data is in $SUBJECTS_DIR/FS_T1_501/, there should be a file called subjectname that contains just a single line:

FS_T1_501

If such a file does not exist, create it. There are a couple of ways to do this. First:

cd $SUBJECTS_DIR/FS_T1_501
echo FS_T1_501 > subjectname

Alternatively, if you want to be fancy and not have to type out the name of the participant, you can use the output of the 'basename' linux command to generate the name of your directory, which should match your participant name:

cd $SUBJECTS_DIR/FS_T1_501
echo `basename "$PWD"` > subjectname

Using this method, the second command will always be the same for any participant.

Batch Subjectname Setup

The above techniques can be used in a shell script, iterating over a set of subject folder names:

setsubjectnames.sh

#!/bin/bash
#given a list of subject folders in $SUBJECTS_DIR, venture into each
#folder and ensure that an initialized subjectname file exists

for sub in "$@"
do
       cd ${SUBJECTS_DIR}/${sub}
       echo `basename "$PWD"` > subjectname
done

If your list of subjects can be found in $SUBJECTS_DIR/subjects, then you could easily ensure the file is present for all participants thus:

cd $SUBJECTS_DIR
setsubjectnames.sh `cat subjects`

Run preproc-sess

This command does motion correction, spatial smoothing (to reduce noise, thereby increasing the signal-to-noise ratio), and slice-time correction on your functional (BOLD) data. It can be run on a batch of participants or on individuals. This command takes ~25-35 minutes per person for 6 runs worth of data, depending on the number of surfaces processed (lh, rh, mni305).

Batch processing

Batch processing requires a sessid file containing the session names (i.e., the names of all the Freesurfer folders) for all your participants. The sessid file can be called anything you wish, and is simply a plain text file with one folder name on each line. For example:

chris_subjects.txt:
 FS_T1_501
 FS_T2_501
 FS_T1_505
 FS_T2_505

When doing a batch, you call using the -sf switch:

preproc-sess -sf <sessidfile> -df <srchdirfile> [options]

e.g.,

preproc-sess -sf chris_subjects.txt -surface fsaverage lhrh -mni305 -per-run -fsd bold -sliceorder up -fwhm 4

If the -df switch isn't provided (it doesn't seem to be mandatory) then it uses a default directory (possibly $SUBJECTS_DIR, or possibly your current working directory) to start in. The parameters in red in the example code above are those that may or may not differ between research project. The parameters in black are almost always going to be kept as-is.

Individuals

Preprocessing individuals works pretty much the same, except you do not require a sessid file. Instead, you provide it with the name of one of the session folders, and use the -s switch, rather than the -sf switch. For example:

preproc-sess -s FS_T1_501 -per-run -surface fsaverage lhrh -mni305 -fsd bold -fwhm 4 -sliceorder up 

This processes subject FS_T1_501 using the same parameters as used for the batch processing example above. Again, the parameters in red may or may not differ for your project, but the text in black will be the same.

Other options

Check Registration

Before you proceed with any functional analyses, you should ensure that the bold data has been coregistered to the T1 anatomical data. You can use tkregister-sess to view a representative volume from each run, with the white matter surface contours mapped on to the volume:

cd $SUBJECTS_DIR
SUBJECT_ID=FS_T1_501
tkregister-sess -s ${SUBJECT_ID} -inorm -per-run

If the contours are out of alignment with the functional data, you can use this interface to manually perform affine (i.e., rigid-body) transformations so that they line up. More information on this can be found here and here.

Note: You must be in your subjects directory for this to work.

Slice time correction

Slice time correction adjusts the functional data to account for the fact that, when you acquire an entire 3D volume as a series of 2-D planes (slices) over the span of 1 TR (commonly 2 seconds), then the very last slice you acquire is going to be delayed 2 seconds relative to the very first slice you acquire. There is a -sliceorder switch that you can provide telling Freesurfer what order the slices were acquired, which the software uses to algorithmically adjust the signals for each slice to account for the fact that they occurred earlier vs. later within that 1 TR time window. Using this switch requires you to know something about the scanner. It is prudent to check with the scanner technician to verify the slice acquisition order so that you can provide the correct value to go along with the -sliceorder switch. If you omit the -sliceorder switch, slice time correction is not performed on the data.

Slice timing correction corrects each voxel's time-series for the fact that later processing assumes that all slices were acquired exactly half-way through the relevant volume's acquisition time (TR), whereas in fact each slice is taken at slightly different times. Slice timing correction works by using (Hanning-windowed) sinc interpolation to shift each time-series by an appropriate fraction of a TR relative to the middle of the TR period. It is necessary to know in what order the slices were acquired and set the appropriate option here. If slices were acquired from the bottom of the brain to the top select Regular up. If slices were acquired from the top of the brain to the bottom select Regular down. If the slices were acquired with interleaved order (0, 2, 4 ... 1, 3, 5 ...) then choose the Interleaved option. If slices were not acquired in regular order you will need to use a slice order file or a slice timings file. If a slice order file is to be used, create a text file with a single number on each line, where the first line states which slice was acquired first, the second line states which slice was acquired second, etc. The first slice is numbered 1 not 0. If a slice timings file is to be used, put one value (ie for each slice) on each line of a text file. The units are in TRs, with 0 corresponding to no shift. Therefore a sensible range of values will be between -0.5 and 0.5.

At the CRTC 3T magnet (as of November 2016), data are collected in sequential ascending order (i.e, from the chin to the crown of the head).

Data collected at the DENT Neurologic Institute are reportedly also acquired sequentially from the chin to the crown of the head. From the description above, both of these acquisition timings should correspond to -sliceorder up

Problems

Header Problems 1: Misstated volume count in the header

Sigh of relief as I figured this out, so we didn't actually lose all of our DENT data.

It is more likely than not that the information about the number of volumes indicated in the header does not match the actual number of volumes recorded in the data for the SEMCAT experiment, which is self-paced. The MRI at DENT is configured to run for 256 volumes (512 seconds) per run. In practice, people typically finish before this. However, the .NII file headers are having the number of volumes listed as 256, and not the actual number of volumes acquired. When you go to run preproc-sess with these data, you will probably get an uninformative error right away when mri_convert tries to open up the data.

Fortunately, the problem is easily fixed (though discovering that it is easily fixed was not easy). Since this is 99.9% likely to affect any given functional run from a self-paced study (e.g., the SEMCAT experiment) you should probably get in the habit of opening up each of your functional volumes with fsledithd:

fsledithd f.nii

Don't do anything else. Just open it and then save it without making any modifications. When you return to the terminal you will see a message along the lines of:

++ WARNING: nifti_read_buffer(run4.nii):
  data bytes needed = 60817408
  data bytes input  = 52379648
  number missing    = 8437760 (set to 0)

What happened here is that fsledithd saw that there are supposed to be 256 volumes of data, but a smaller amount of data was found. So it went ahead and filled in all the missing bytes with 0. Now your f.nii file will be bigger, with all the missing volumes at the end filled with zeros.

Header Problems 2: Unsupported slice timing 5

Running preproc-sess on an existing dataset from the Booth lab, I ran into

nifti1Read(): unsupported slice timing pattern 5 in /home/chris/ubfs/cpmcnorg/openfmri/booth/FS_T2_501/bold/029/f.nii

Some of the .nii files have invalid information in the header. In the case of the slice_code field, valid values are 0,1,2,3, or 4:

#define NIFTI_SLICE_UNKNOWN  0
#define NIFTI_SLICE_SEQ_INC  1
#define NIFTI_SLICE_SEQ_DEC  2
#define NIFTI_SLICE_ALT_INC  3
#define NIFTI_SLICE_ALT_DEC  4

According to the NifTI header documentation, if the slice_code field nonzero, AND if slice_dim is nonzero, AND if slice_duration is positive, this value indicates the timing pattern of the slice acquisition (sequential or interleaving slices; incremental or decremental). Because this value is sometimes '0' in some of our older data, but '5' in some of the newer data, my working theory is that one of the pieces of software used to convert from DICOM to NifTI format (MRI Convert, a Windows program) inserts nonsensical default values in some of the header fields that are not used by all fMRI analysis packages. In any case, the set_TR script now sets the slice_time field to 0 at the same time that it modifies the TR field. Another program, fsledithd can be used to modify some (but not all) of the header fields, including the slice_code field:

fsledithd f.nii

This command would open up a default command-line text editor (e.g., nano) where fields and their values are listed, one per line. You would locate the line for slice_code and modify it so that it reads:

slice_code = '0'

Then save the changes and quit.

ERROR: 3dvolreg.afni & libxp.so.6

This error occurs on Ubuntu 16.04 due to the lack of libXp packages. FreeSurfer piggybacks on some AFNI software that was built using outdated code: as you can read about here. To fix this, we can either:

  1. install an older version of the needed packages: here
    • sudo dpkg --install <name of file> (not recommended because library is deprecated and unsupported)
  2. or symlink libXp.so.6 to libXpm4.so.11, which is included in the libXpm package

If you choose the latter option, you will find libXpm4 in /usr/lib/x86_64-linux-gnu/

Note: if you're having this problem, you may also find yourself missing libpng12

wget http://se.archive.ubuntu.com/ubuntu/pool/main/libp/libpng/libpng12-0_1.2.54-1ubuntu1_amd64.deb
sudo dpkg --install libpng12-0_1.2.54-1ubuntu1_amd64.deb

You will need to get one of the End Bosses to run this as an admin.