CCN Wiki - User contributions [en]

Main Page

2022-11-14T20:35:58Z

Chris: /* Experiment A to Z (not necessarily in alphabetical order) */

Detrending FreeSurfer Data

2022-09-27T20:20:12Z

Chris: /* Scripting */

Over the course of a run, there can be a linear drift in the signal in different regions of the brain. There are many possible causes for this that have nothing to do with any interesting aspect of your data -- in other words, this linear drift is a nuisance artifact. The second step is to remove this signal drift from the data because it can introduce spurious correlations between two unrelated time series. You can see this for yourself in a quick experiment you could whip up in Excel: take two vectors of 100 randomly generated numbers (e.g., randbetween(1,99)). They should be uncorrelated. Now add 1, 2, 3, ... , 99, 100 to the values in each vector. This simulates a linear trend in the data. You shouldn't be surprised to find that the two vectors are now highly and positively correlated!

A script has been written called detrend.sh that removes the linear trend in your BOLD data. The latest version of this script can be found in /usr/local/sbin (which should be in your $PATH):

'''Update:''' The script has been modified to also regress out motion parameters from the mcprextreg files. This modification is not (yet) reflected in the code below.
==The Script ==
The most recent version of the BASH script (detrend.sh) is below:
=== detrend.sh ===
#!/bin/bash
USAGE="Usage: detrend.sh filepattern surf sub1 ... subN"

if [ "$#" == "0" ]; then
echo "$USAGE"
exit 1
fi

#first parameter is the filepattern for the .nii.gz time series to be detrended, up to the surface indicator
#e.g., fmcpr.siemens.sm6.fsaverage.?h.nii.gz would use fmcpr.siemens.sm6 as the filepattern, fsaverage as the surf
filepat="$1"
shift

#second parameter should be specified either as self or fsaverage
surf="$1"
shift

#after the shift commands, all the arguments are shifted down two places and the first
#2 arguments (the filepattern and surface) fall off the list.
#The remaining arguments should be subject_ids
subs=( "$@" );
hemis=( "lh" "rh" );

for sub in "${subs[@]}"; do
source_dir=${SUBJECTS_DIR}/${sub}/bold
echo ${source_dir}
if [ ! -d ${source_dir} ]; then
#The subject_id does not exist
echo "${source_dir} does not exist!"
else
cd ${source_dir}
readarray -t runs < runs
for r in "${runs[@]}"; do
if [ -n "${r}" ]; then
#the -n test makes sure that the run number is not an empty string
#caused by a trailing newline in the runs file
for hemi in "${hemis[@]}"; do
cd ${source_dir}/${r}
pwd
#subject_id does exist. Detrend
if [ "${surf}" = "self" ]; then

SURFTOUSE=${sub}
else
SURFTOUSE=fsaverage
fi
mri_glmfit --y ${source_dir}/${r}/${filepat}.${surf}.${hemi}.nii.gz \
--glmdir ${source_dir}/${r}/${hemi}.detrend \
--qa --save-yhat --eres-save \
--surf ${SURFTOUSE} ${hemi}
mv ${source_dir}/${r}/${hemi}.detrend/eres.mgh ${source_dir}/${r}/${filepat}.${surf}.${hemi}.mgh
done
#now detrend the mni305 file
mri_glmfit --y ${source_dir}/${r}/${filepat}.mni305.2mm.nii.gz \
--glmdir ${source_dir}/${r}/mni305.detrend \
--qa --save-yhat --eres-save
mv ${source_dir}/${r}/mni305.detrend/eres.mgh ${source_dir}/${r}/${filepat}.${surf}.mni305.2mm.mgh
fi
done
fi
done

== Running the Script ==
===Before you run these scripts===
Before running either of the scripts on this page, you will need to create a text file called 'runs' in the bold/ directory for each subject's dataset, e.g.,
*FS_T1_501/
**bold/
***runs
***005/
***006/

The <code>runs</code> file simply lists each run folder on its own line:
005
006
The detrend.sh script uses this file to determine the folders containing the data to be detrended. If this file doesn't already exist, you can manually generate it in any text editor (e.g., <code>nano runs</code> or <code>gedit runs</code>), but the quickest method takes advantage of the fact that the run folders all start with 0, and uses common command-line utilities:

SUBJECT=FS_501
cd ${SUBJECTS_DIR}/${SUBJECT}/bold
#if there are fewer than 10 runs of BOLD data, then the run directories will probably have 2 leading zeros
ls -1 | grep "^00*" > runs
===Example script===
Assuming all your subject folders have the same run folders to detrend, you would detrend multiple subjects using detrend.sh, specifying a file pattern for the source data (i.e., the name of the preprocessed files generated by FS-FAST, omitting anything after the '?h' hemisphere identifier), followed by a list of subject IDs:
#A SPECIFIC EXAMPLE: (note these parameters may differ '''substantially''' from what you would be typing in)
detrend.sh fmcpr.sm6 self FS_T1_501 FS_T2_501 FS_T1_505 FS_T2_505
#For a more generalizable example of how you should call this function, see the section below using variables
The gist is that it calls the mri_glmfit function and saves the residuals after the linear trend has been removed from the data. Multiple files are generated in ?h.detrend/ directories in each run directory. The detrended data is subsequently copied back to the run directory as a new file called ${filepat}.?h.mgh, where ${filepat} is whatever file pattern you provided to the script (note that the source data are .nii.gz files, whereas the detrended data are .mgh files).

==White Matter and CSF Signal==
It is among best practices in functional connectivity studies to regress out WM and CSF signals from your time series before computing time series correlations, at least for resting state studies (Muschelli et al (2014)). We can add WM and CSF signal to the set of nuisance regressors when detrending our data (note that the procedure below removes the mean WM + CSF as a single component; it can be modified to remove WM and CSF independently):

===Create a configuration file===
An answer to a question I posted to the FreeSurfer mailing list pointed me to [http://surfer.nmr.mgh.harvard.edu/fswiki/FsFastFunctionalConnectivityWalkthrough a walkthrough] for functional connectivity analyses in FreeSurfer is the lead that I'm following. We will be using fcseed-sess to pull out our mean signal from seed regions. In our case, our seed regions will be from the white matter and csf masks. It's a 2-step process. The first step is to create a configuration file:
fcseed-config -cfg wmcsf.cfg -wm -vcsf -fsd bold -mean
This assumes that the BOLD data are in ${SUBJECTS_DIR}/${SUBID}/'''bold''' (-fsd), which is likely going to be the case if you've been following the FreeSurfer pipeline on this wiki. I ran the above step in ${SUBJECTS_DIR}.

===Computing WM and CSF Regressor Values===
Now you will have a configuration file, '''wmcsf.cfg''' that you will pass to the next step that performs the math on the fmri data:
fcseed-sess -s ${SUBID} -cfg wmcsf.cfg
or
fcseed-sess -sf ''subjects'' -cfg wmcsf.cfg

This will use the config file just created to extract a seed regressor for ${SUBID} (or for all subjects listed in your ''subjects'' file if you use the -sf switch to run in batch mode). You'll find a file called '''wmcsf''' in each of your BOLD/run directories. This file is a plaintext file with one column of values that you can use as a nuisance regressor. The detrend script could be modified to first run fcseed-sess to generate this value to include in the nuisance regressor matrix.

If you wanted to remove WM and CSF as two separate components, you would need to generate two separate .cfg files, and then run fcseed-sess twice.

===Scripting===
These steps have been combined with the detrend.sh script above to remove linear trends, motion parameters and wm+csf signal (as a single component). You will find detrend_wmcsf.sh in the ubfs/Scripts/Shell folder. However the script is reproduced below:

'''Note: I think there's a bug that prevents the script from working correctly when passed more than 1 subject at a time.'''

====detrend_wmcsf.sh====
#!/bin/bash
USAGE="Usage: detrend_wmcsf.sh filepattern surf SUBJECT"

if [ "$#" == "0" ]; then
echo "$USAGE"
exit 1
fi
#first parameter is the filepattern for the .nii.gz time series to be detrended, up to the surface indicator
#e.g., fmcpr.siemens.sm6.fsaverage.?h.nii.gz would use fmcpr.siemens.sm6 as the filepattern, fsaverage as the surf
filepat="$1"
shift
#second parameter should be specified either as self or fsaverage
surf="$1"
shift

#after the shift command, all the arguments are shifted down one place and the first argument (the filepattern)
#falls off the list. The remaining arguments should be subject_ids
subs=( "$@" );
hemis=( "lh" "rh" );

#Make sure we're in SUBJECTS_DIR to start so that we can make some assumptions about relative file locations
cd ${SUBJECTS_DIR}

#first, create configuration file for the wm & csf regression
echo "Making fcseed config file..."
fcseed-config -cfg wmcsf.cfg -wm -vcsf -fsd bold -mean
echo "done"

echo "Processing subjects..."
for sub in "${subs[@]}"
do
source_dir=${SUBJECTS_DIR}/${sub}/bold
echo ${source_dir}
if [ ! -d ${source_dir} ]; then
#The subject_id does not exist
echo "${source_dir} does not exist!"
else
#The subject bold directory exists. Proceed.
fcseed-sess -s ${sub} -cfg ${SUBJECTS_DIR}/wmcsf.cfg
echo "Completed wm & csf nuisance regressor calculation for ${sub}"
cd ${source_dir}

########readarray -t runlist < runs ### This is fine and all on Linux but MacOS does not have readarray

#This is a MacOS compatible alternative to readarray that also works on Linux
while IFS= read r; do
runlist+=($r)
done < runs
for r in "${runlist[@]}"; do
if [ -n "${r}" ]; then
#the -n test makes sure that the run number is not an empty string
#caused by a trailing newline in the runs file
cd ${source_dir}/${r}
pwd
#subject_id does exist. Detrend
if [ "${surf}" = "self" ]; then
SURFTOUSE=${sub}
else
SURFTOUSE=fsaverage
fi

#this will generate the QA matrix in lh.detrend if needed:
if [ ! -f ${source_dir}/${r}/lh.detrend/Xg.dat ]; then
mri_glmfit --y ${source_dir}/${r}/${filepat}.${surf}.lh.nii.gz \
--glmdir ${source_dir}/${r}/lh.detrend \
--qa \
--surf ${SURFTOUSE} lh
fi

#copy and merge the QA, WMCSF and MC matrices
echo "Generating nuisance regressor matrix for ${hemi} ${r}"
paste ${source_dir}/${r}/lh.detrend/Xg.dat wmcsf mcprextreg > nuisance

for hemi in "${hemis[@]}"; do
#regress out nuisance
mri_glmfit --y ${source_dir}/${r}/${filepat}.${surf}.${hemi}.nii.gz \
--glmdir ${source_dir}/${r}/${hemi}.detrend \
--X nuisance \
--no-contrasts-ok --save-yhat --eres-save \
--surf ${SURFTOUSE} ${hemi}
#move and rename detrended files
mv ${source_dir}/${r}/${hemi}.detrend/eres.mgh ${source_dir}/${r}/${filepat}.${surf}.${hemi}.mgh
done

#now detrend the mni305 file
mri_glmfit --y ${source_dir}/${r}/${filepat}.mni305.2mm.nii.gz \
--glmdir ${source_dir}/${r}/mni305.detrend \
--X nuisance --no-contrasts-ok --save-yhat --eres-save
mv ${source_dir}/${r}/mni305.detrend/eres.mgh ${source_dir}/${r}/${filepat}.${surf}.mni305.2mm.mgh
fi
done
fi
done

== Using variables ==
As I mentioned in the comments in the sample code snippet above, what you type depends entirely on the filenames, which in turn depend entirely on how the data were preprocessed. You can use environment variables to help walk you through figuring out the correct file patterns. Also, a handy shortcut exists if you happen to have a <code>subjects</code> file in <code>$SUBJECTS_DIR</code>. Putting these two techniques together:
FILEPATTERN=fmcpr #the preprocessed files will almost always be called '''fmcpr'''
SMOOTHING=".sm4" #how much smoothing did you use when you ran preproc-sess?
SLICETIME=".up" #[".up" | ".down" | ".siemens" | OMIT ]
SURFACE=fsaverage #[self | fsaverage]

detrend.sh ${FILEPATTERN}${SLICETIME}${SMOOTHING} $SURFACE `cat subjects`
This will execute the detrend.sh script on all the subjects listed in the subjects text file, using the fsaverage surface. The assemblage of variables will look for files named ''fmcpr.up.sm4.*''
[[Category: Time Series]]

Detrending FreeSurfer Data

2022-09-27T20:19:14Z

Chris: /* detrend_wmcsf.sh */

Over the course of a run, there can be a linear drift in the signal in different regions of the brain. There are many possible causes for this that have nothing to do with any interesting aspect of your data -- in other words, this linear drift is a nuisance artifact. The second step is to remove this signal drift from the data because it can introduce spurious correlations between two unrelated time series. You can see this for yourself in a quick experiment you could whip up in Excel: take two vectors of 100 randomly generated numbers (e.g., randbetween(1,99)). They should be uncorrelated. Now add 1, 2, 3, ... , 99, 100 to the values in each vector. This simulates a linear trend in the data. You shouldn't be surprised to find that the two vectors are now highly and positively correlated!

A script has been written called detrend.sh that removes the linear trend in your BOLD data. The latest version of this script can be found in /usr/local/sbin (which should be in your $PATH):

'''Update:''' The script has been modified to also regress out motion parameters from the mcprextreg files. This modification is not (yet) reflected in the code below.
==The Script ==
The most recent version of the BASH script (detrend.sh) is below:
=== detrend.sh ===
#!/bin/bash
USAGE="Usage: detrend.sh filepattern surf sub1 ... subN"

if [ "$#" == "0" ]; then
echo "$USAGE"
exit 1
fi

#first parameter is the filepattern for the .nii.gz time series to be detrended, up to the surface indicator
#e.g., fmcpr.siemens.sm6.fsaverage.?h.nii.gz would use fmcpr.siemens.sm6 as the filepattern, fsaverage as the surf
filepat="$1"
shift

#second parameter should be specified either as self or fsaverage
surf="$1"
shift

#after the shift commands, all the arguments are shifted down two places and the first
#2 arguments (the filepattern and surface) fall off the list.
#The remaining arguments should be subject_ids
subs=( "$@" );
hemis=( "lh" "rh" );

for sub in "${subs[@]}"; do
source_dir=${SUBJECTS_DIR}/${sub}/bold
echo ${source_dir}
if [ ! -d ${source_dir} ]; then
#The subject_id does not exist
echo "${source_dir} does not exist!"
else
cd ${source_dir}
readarray -t runs < runs
for r in "${runs[@]}"; do
if [ -n "${r}" ]; then
#the -n test makes sure that the run number is not an empty string
#caused by a trailing newline in the runs file
for hemi in "${hemis[@]}"; do
cd ${source_dir}/${r}
pwd
#subject_id does exist. Detrend
if [ "${surf}" = "self" ]; then

SURFTOUSE=${sub}
else
SURFTOUSE=fsaverage
fi
mri_glmfit --y ${source_dir}/${r}/${filepat}.${surf}.${hemi}.nii.gz \
--glmdir ${source_dir}/${r}/${hemi}.detrend \
--qa --save-yhat --eres-save \
--surf ${SURFTOUSE} ${hemi}
mv ${source_dir}/${r}/${hemi}.detrend/eres.mgh ${source_dir}/${r}/${filepat}.${surf}.${hemi}.mgh
done
#now detrend the mni305 file
mri_glmfit --y ${source_dir}/${r}/${filepat}.mni305.2mm.nii.gz \
--glmdir ${source_dir}/${r}/mni305.detrend \
--qa --save-yhat --eres-save
mv ${source_dir}/${r}/mni305.detrend/eres.mgh ${source_dir}/${r}/${filepat}.${surf}.mni305.2mm.mgh
fi
done
fi
done

== Running the Script ==
===Before you run these scripts===
Before running either of the scripts on this page, you will need to create a text file called 'runs' in the bold/ directory for each subject's dataset, e.g.,
*FS_T1_501/
**bold/
***runs
***005/
***006/

The <code>runs</code> file simply lists each run folder on its own line:
005
006
The detrend.sh script uses this file to determine the folders containing the data to be detrended. If this file doesn't already exist, you can manually generate it in any text editor (e.g., <code>nano runs</code> or <code>gedit runs</code>), but the quickest method takes advantage of the fact that the run folders all start with 0, and uses common command-line utilities:

SUBJECT=FS_501
cd ${SUBJECTS_DIR}/${SUBJECT}/bold
#if there are fewer than 10 runs of BOLD data, then the run directories will probably have 2 leading zeros
ls -1 | grep "^00*" > runs
===Example script===
Assuming all your subject folders have the same run folders to detrend, you would detrend multiple subjects using detrend.sh, specifying a file pattern for the source data (i.e., the name of the preprocessed files generated by FS-FAST, omitting anything after the '?h' hemisphere identifier), followed by a list of subject IDs:
#A SPECIFIC EXAMPLE: (note these parameters may differ '''substantially''' from what you would be typing in)
detrend.sh fmcpr.sm6 self FS_T1_501 FS_T2_501 FS_T1_505 FS_T2_505
#For a more generalizable example of how you should call this function, see the section below using variables
The gist is that it calls the mri_glmfit function and saves the residuals after the linear trend has been removed from the data. Multiple files are generated in ?h.detrend/ directories in each run directory. The detrended data is subsequently copied back to the run directory as a new file called ${filepat}.?h.mgh, where ${filepat} is whatever file pattern you provided to the script (note that the source data are .nii.gz files, whereas the detrended data are .mgh files).

==White Matter and CSF Signal==
It is among best practices in functional connectivity studies to regress out WM and CSF signals from your time series before computing time series correlations, at least for resting state studies (Muschelli et al (2014)). We can add WM and CSF signal to the set of nuisance regressors when detrending our data (note that the procedure below removes the mean WM + CSF as a single component; it can be modified to remove WM and CSF independently):

===Create a configuration file===
An answer to a question I posted to the FreeSurfer mailing list pointed me to [http://surfer.nmr.mgh.harvard.edu/fswiki/FsFastFunctionalConnectivityWalkthrough a walkthrough] for functional connectivity analyses in FreeSurfer is the lead that I'm following. We will be using fcseed-sess to pull out our mean signal from seed regions. In our case, our seed regions will be from the white matter and csf masks. It's a 2-step process. The first step is to create a configuration file:
fcseed-config -cfg wmcsf.cfg -wm -vcsf -fsd bold -mean
This assumes that the BOLD data are in ${SUBJECTS_DIR}/${SUBID}/'''bold''' (-fsd), which is likely going to be the case if you've been following the FreeSurfer pipeline on this wiki. I ran the above step in ${SUBJECTS_DIR}.

===Computing WM and CSF Regressor Values===
Now you will have a configuration file, '''wmcsf.cfg''' that you will pass to the next step that performs the math on the fmri data:
fcseed-sess -s ${SUBID} -cfg wmcsf.cfg
or
fcseed-sess -sf ''subjects'' -cfg wmcsf.cfg

This will use the config file just created to extract a seed regressor for ${SUBID} (or for all subjects listed in your ''subjects'' file if you use the -sf switch to run in batch mode). You'll find a file called '''wmcsf''' in each of your BOLD/run directories. This file is a plaintext file with one column of values that you can use as a nuisance regressor. The detrend script could be modified to first run fcseed-sess to generate this value to include in the nuisance regressor matrix.

If you wanted to remove WM and CSF as two separate components, you would need to generate two separate .cfg files, and then run fcseed-sess twice.

===Scripting===
These steps have been combined with the detrend.sh script above to remove linear trends, motion parameters and wm+csf signal (as a single component). You will find detrend_wmcsf.sh in the ubfs/Scripts/Shell folder. However the script is reproduced below:
====detrend_wmcsf.sh====
#!/bin/bash
USAGE="Usage: detrend_wmcsf.sh filepattern surf SUBJECT"

if [ "$#" == "0" ]; then
echo "$USAGE"
exit 1
fi
#first parameter is the filepattern for the .nii.gz time series to be detrended, up to the surface indicator
#e.g., fmcpr.siemens.sm6.fsaverage.?h.nii.gz would use fmcpr.siemens.sm6 as the filepattern, fsaverage as the surf
filepat="$1"
shift
#second parameter should be specified either as self or fsaverage
surf="$1"
shift

#after the shift command, all the arguments are shifted down one place and the first argument (the filepattern)
#falls off the list. The remaining arguments should be subject_ids
subs=( "$@" );
hemis=( "lh" "rh" );

#Make sure we're in SUBJECTS_DIR to start so that we can make some assumptions about relative file locations
cd ${SUBJECTS_DIR}

#first, create configuration file for the wm & csf regression
echo "Making fcseed config file..."
fcseed-config -cfg wmcsf.cfg -wm -vcsf -fsd bold -mean
echo "done"

echo "Processing subjects..."
for sub in "${subs[@]}"
do
source_dir=${SUBJECTS_DIR}/${sub}/bold
echo ${source_dir}
if [ ! -d ${source_dir} ]; then
#The subject_id does not exist
echo "${source_dir} does not exist!"
else
#The subject bold directory exists. Proceed.
fcseed-sess -s ${sub} -cfg ${SUBJECTS_DIR}/wmcsf.cfg
echo "Completed wm & csf nuisance regressor calculation for ${sub}"
cd ${source_dir}

########readarray -t runlist < runs ### This is fine and all on Linux but MacOS does not have readarray

#This is a MacOS compatible alternative to readarray that also works on Linux
while IFS= read r; do
runlist+=($r)
done < runs
for r in "${runlist[@]}"; do
if [ -n "${r}" ]; then
#the -n test makes sure that the run number is not an empty string
#caused by a trailing newline in the runs file
cd ${source_dir}/${r}
pwd
#subject_id does exist. Detrend
if [ "${surf}" = "self" ]; then
SURFTOUSE=${sub}
else
SURFTOUSE=fsaverage
fi

#this will generate the QA matrix in lh.detrend if needed:
if [ ! -f ${source_dir}/${r}/lh.detrend/Xg.dat ]; then
mri_glmfit --y ${source_dir}/${r}/${filepat}.${surf}.lh.nii.gz \
--glmdir ${source_dir}/${r}/lh.detrend \
--qa \
--surf ${SURFTOUSE} lh
fi

#copy and merge the QA, WMCSF and MC matrices
echo "Generating nuisance regressor matrix for ${hemi} ${r}"
paste ${source_dir}/${r}/lh.detrend/Xg.dat wmcsf mcprextreg > nuisance

for hemi in "${hemis[@]}"; do
#regress out nuisance
mri_glmfit --y ${source_dir}/${r}/${filepat}.${surf}.${hemi}.nii.gz \
--glmdir ${source_dir}/${r}/${hemi}.detrend \
--X nuisance \
--no-contrasts-ok --save-yhat --eres-save \
--surf ${SURFTOUSE} ${hemi}
#move and rename detrended files
mv ${source_dir}/${r}/${hemi}.detrend/eres.mgh ${source_dir}/${r}/${filepat}.${surf}.${hemi}.mgh
done

#now detrend the mni305 file
mri_glmfit --y ${source_dir}/${r}/${filepat}.mni305.2mm.nii.gz \
--glmdir ${source_dir}/${r}/mni305.detrend \
--X nuisance --no-contrasts-ok --save-yhat --eres-save
mv ${source_dir}/${r}/mni305.detrend/eres.mgh ${source_dir}/${r}/${filepat}.${surf}.mni305.2mm.mgh
fi
done
fi
done

== Using variables ==
As I mentioned in the comments in the sample code snippet above, what you type depends entirely on the filenames, which in turn depend entirely on how the data were preprocessed. You can use environment variables to help walk you through figuring out the correct file patterns. Also, a handy shortcut exists if you happen to have a <code>subjects</code> file in <code>$SUBJECTS_DIR</code>. Putting these two techniques together:
FILEPATTERN=fmcpr #the preprocessed files will almost always be called '''fmcpr'''
SMOOTHING=".sm4" #how much smoothing did you use when you ran preproc-sess?
SLICETIME=".up" #[".up" | ".down" | ".siemens" | OMIT ]
SURFACE=fsaverage #[self | fsaverage]

detrend.sh ${FILEPATTERN}${SLICETIME}${SMOOTHING} $SURFACE `cat subjects`
This will execute the detrend.sh script on all the subjects listed in the subjects text file, using the fsaverage surface. The assemblage of variables will look for files named ''fmcpr.up.sm4.*''
[[Category: Time Series]]

Detrending FreeSurfer Data

2022-09-27T19:59:47Z

Chris: /* Running the Script */

Over the course of a run, there can be a linear drift in the signal in different regions of the brain. There are many possible causes for this that have nothing to do with any interesting aspect of your data -- in other words, this linear drift is a nuisance artifact. The second step is to remove this signal drift from the data because it can introduce spurious correlations between two unrelated time series. You can see this for yourself in a quick experiment you could whip up in Excel: take two vectors of 100 randomly generated numbers (e.g., randbetween(1,99)). They should be uncorrelated. Now add 1, 2, 3, ... , 99, 100 to the values in each vector. This simulates a linear trend in the data. You shouldn't be surprised to find that the two vectors are now highly and positively correlated!

A script has been written called detrend.sh that removes the linear trend in your BOLD data. The latest version of this script can be found in /usr/local/sbin (which should be in your $PATH):

'''Update:''' The script has been modified to also regress out motion parameters from the mcprextreg files. This modification is not (yet) reflected in the code below.
==The Script ==
The most recent version of the BASH script (detrend.sh) is below:
=== detrend.sh ===
#!/bin/bash
USAGE="Usage: detrend.sh filepattern surf sub1 ... subN"

if [ "$#" == "0" ]; then
echo "$USAGE"
exit 1
fi

#first parameter is the filepattern for the .nii.gz time series to be detrended, up to the surface indicator
#e.g., fmcpr.siemens.sm6.fsaverage.?h.nii.gz would use fmcpr.siemens.sm6 as the filepattern, fsaverage as the surf
filepat="$1"
shift

#second parameter should be specified either as self or fsaverage
surf="$1"
shift

#after the shift commands, all the arguments are shifted down two places and the first
#2 arguments (the filepattern and surface) fall off the list.
#The remaining arguments should be subject_ids
subs=( "$@" );
hemis=( "lh" "rh" );

for sub in "${subs[@]}"; do
source_dir=${SUBJECTS_DIR}/${sub}/bold
echo ${source_dir}
if [ ! -d ${source_dir} ]; then
#The subject_id does not exist
echo "${source_dir} does not exist!"
else
cd ${source_dir}
readarray -t runs < runs
for r in "${runs[@]}"; do
if [ -n "${r}" ]; then
#the -n test makes sure that the run number is not an empty string
#caused by a trailing newline in the runs file
for hemi in "${hemis[@]}"; do
cd ${source_dir}/${r}
pwd
#subject_id does exist. Detrend
if [ "${surf}" = "self" ]; then

SURFTOUSE=${sub}
else
SURFTOUSE=fsaverage
fi
mri_glmfit --y ${source_dir}/${r}/${filepat}.${surf}.${hemi}.nii.gz \
--glmdir ${source_dir}/${r}/${hemi}.detrend \
--qa --save-yhat --eres-save \
--surf ${SURFTOUSE} ${hemi}
mv ${source_dir}/${r}/${hemi}.detrend/eres.mgh ${source_dir}/${r}/${filepat}.${surf}.${hemi}.mgh
done
#now detrend the mni305 file
mri_glmfit --y ${source_dir}/${r}/${filepat}.mni305.2mm.nii.gz \
--glmdir ${source_dir}/${r}/mni305.detrend \
--qa --save-yhat --eres-save
mv ${source_dir}/${r}/mni305.detrend/eres.mgh ${source_dir}/${r}/${filepat}.${surf}.mni305.2mm.mgh
fi
done
fi
done

== Running the Script ==
===Before you run these scripts===
Before running either of the scripts on this page, you will need to create a text file called 'runs' in the bold/ directory for each subject's dataset, e.g.,
*FS_T1_501/
**bold/
***runs
***005/
***006/

The <code>runs</code> file simply lists each run folder on its own line:
005
006
The detrend.sh script uses this file to determine the folders containing the data to be detrended. If this file doesn't already exist, you can manually generate it in any text editor (e.g., <code>nano runs</code> or <code>gedit runs</code>), but the quickest method takes advantage of the fact that the run folders all start with 0, and uses common command-line utilities:

SUBJECT=FS_501
cd ${SUBJECTS_DIR}/${SUBJECT}/bold
#if there are fewer than 10 runs of BOLD data, then the run directories will probably have 2 leading zeros
ls -1 | grep "^00*" > runs
===Example script===
Assuming all your subject folders have the same run folders to detrend, you would detrend multiple subjects using detrend.sh, specifying a file pattern for the source data (i.e., the name of the preprocessed files generated by FS-FAST, omitting anything after the '?h' hemisphere identifier), followed by a list of subject IDs:
#A SPECIFIC EXAMPLE: (note these parameters may differ '''substantially''' from what you would be typing in)
detrend.sh fmcpr.sm6 self FS_T1_501 FS_T2_501 FS_T1_505 FS_T2_505
#For a more generalizable example of how you should call this function, see the section below using variables
The gist is that it calls the mri_glmfit function and saves the residuals after the linear trend has been removed from the data. Multiple files are generated in ?h.detrend/ directories in each run directory. The detrended data is subsequently copied back to the run directory as a new file called ${filepat}.?h.mgh, where ${filepat} is whatever file pattern you provided to the script (note that the source data are .nii.gz files, whereas the detrended data are .mgh files).

==White Matter and CSF Signal==
It is among best practices in functional connectivity studies to regress out WM and CSF signals from your time series before computing time series correlations, at least for resting state studies (Muschelli et al (2014)). We can add WM and CSF signal to the set of nuisance regressors when detrending our data (note that the procedure below removes the mean WM + CSF as a single component; it can be modified to remove WM and CSF independently):

===Create a configuration file===
An answer to a question I posted to the FreeSurfer mailing list pointed me to [http://surfer.nmr.mgh.harvard.edu/fswiki/FsFastFunctionalConnectivityWalkthrough a walkthrough] for functional connectivity analyses in FreeSurfer is the lead that I'm following. We will be using fcseed-sess to pull out our mean signal from seed regions. In our case, our seed regions will be from the white matter and csf masks. It's a 2-step process. The first step is to create a configuration file:
fcseed-config -cfg wmcsf.cfg -wm -vcsf -fsd bold -mean
This assumes that the BOLD data are in ${SUBJECTS_DIR}/${SUBID}/'''bold''' (-fsd), which is likely going to be the case if you've been following the FreeSurfer pipeline on this wiki. I ran the above step in ${SUBJECTS_DIR}.

===Computing WM and CSF Regressor Values===
Now you will have a configuration file, '''wmcsf.cfg''' that you will pass to the next step that performs the math on the fmri data:
fcseed-sess -s ${SUBID} -cfg wmcsf.cfg
or
fcseed-sess -sf ''subjects'' -cfg wmcsf.cfg

This will use the config file just created to extract a seed regressor for ${SUBID} (or for all subjects listed in your ''subjects'' file if you use the -sf switch to run in batch mode). You'll find a file called '''wmcsf''' in each of your BOLD/run directories. This file is a plaintext file with one column of values that you can use as a nuisance regressor. The detrend script could be modified to first run fcseed-sess to generate this value to include in the nuisance regressor matrix.

If you wanted to remove WM and CSF as two separate components, you would need to generate two separate .cfg files, and then run fcseed-sess twice.

===Scripting===
These steps have been combined with the detrend.sh script above to remove linear trends, motion parameters and wm+csf signal (as a single component). You will find detrend_wmcsf.sh in the ubfs/Scripts/Shell folder. However the script is reproduced below:
====detrend_wmcsf.sh====
#!/bin/bash
USAGE="Usage: detrend_wmcsf.sh filepattern surf sub1 ... subN"

if [ "$#" == "0" ]; then
echo "$USAGE"
exit 1
fi
#first parameter is the filepattern for the .nii.gz time series to be detrended, up to the surface indicator
#e.g., fmcpr.siemens.sm6.fsaverage.?h.nii.gz would use fmcpr.siemens.sm6 as the filepattern, fsaverage as the surf
filepat="$1"
shift
#second parameter should be specified either as self or fsaverage
surf="$1"
shift

#after the shift command, all the arguments are shifted down one place and the first argument (the filepattern)
#falls off the list. The remaining arguments should be subject_ids
subs=( "$@" );
hemis=( "lh" "rh" );

#Make sure we're in SUBJECTS_DIR to start so that we can make some assumptions about relative file locations
cd ${SUBJECTS_DIR}

#first, create configuration file for the wm & csf regression
echo "Making fcseed config file..."
fcseed-config -cfg wmcsf.cfg -wm -vcsf -fsd bold -mean
echo "done"

echo "Processing subjects..."
for sub in "${subs[@]}"
do
source_dir=${SUBJECTS_DIR}/${sub}/bold
echo ${source_dir}
if [ ! -d ${source_dir} ]; then
#The subject_id does not exist
echo "${source_dir} does not exist!"
else
#The subject bold directory exists. Proceed.
fcseed-sess -s ${sub} -cfg ${SUBJECTS_DIR}/wmcsf.cfg
echo "Completed wm & csf nuisance regressor calculation for ${sub}"
cd ${source_dir}

########readarray -t runlist < runs ### This is fine and all on Linux but MacOS does not have readarray

#This is a MacOS compatible alternative to readarray that also works on Linux
while IFS= read r; do
runlist+=($r)
done < runs
for r in "${runlist[@]}"; do
if [ -n "${r}" ]; then
#the -n test makes sure that the run number is not an empty string
#caused by a trailing newline in the runs file
cd ${source_dir}/${r}
pwd
#subject_id does exist. Detrend
if [ "${surf}" = "self" ]; then
SURFTOUSE=${sub}
else
SURFTOUSE=fsaverage
fi

#this will generate the QA matrix in lh.detrend if needed:
if [ ! -f ${source_dir}/${r}/lh.detrend/Xg.dat ]; then
mri_glmfit --y ${source_dir}/${r}/${filepat}.${surf}.lh.nii.gz \
--glmdir ${source_dir}/${r}/lh.detrend \
--qa \
--surf ${SURFTOUSE} lh
fi

#copy and merge the QA, WMCSF and MC matrices
echo "Generating nuisance regressor matrix for ${hemi} ${r}"
paste ${source_dir}/${r}/lh.detrend/Xg.dat wmcsf mcprextreg > nuisance

for hemi in "${hemis[@]}"; do
#regress out nuisance
mri_glmfit --y ${source_dir}/${r}/${filepat}.${surf}.${hemi}.nii.gz \
--glmdir ${source_dir}/${r}/${hemi}.detrend \
--X nuisance \
--no-contrasts-ok --save-yhat --eres-save \
--surf ${SURFTOUSE} ${hemi}
#move and rename detrended files
mv ${source_dir}/${r}/${hemi}.detrend/eres.mgh ${source_dir}/${r}/${filepat}.${surf}.${hemi}.mgh
done

#now detrend the mni305 file
mri_glmfit --y ${source_dir}/${r}/${filepat}.mni305.2mm.nii.gz \
--glmdir ${source_dir}/${r}/mni305.detrend \
--X nuisance --no-contrasts-ok --save-yhat --eres-save
mv ${source_dir}/${r}/mni305.detrend/eres.mgh ${source_dir}/${r}/${filepat}.${surf}.mni305.2mm.mgh
fi
done
fi
done

== Using variables ==
As I mentioned in the comments in the sample code snippet above, what you type depends entirely on the filenames, which in turn depend entirely on how the data were preprocessed. You can use environment variables to help walk you through figuring out the correct file patterns. Also, a handy shortcut exists if you happen to have a <code>subjects</code> file in <code>$SUBJECTS_DIR</code>. Putting these two techniques together:
FILEPATTERN=fmcpr #the preprocessed files will almost always be called '''fmcpr'''
SMOOTHING=".sm4" #how much smoothing did you use when you ran preproc-sess?
SLICETIME=".up" #[".up" | ".down" | ".siemens" | OMIT ]
SURFACE=fsaverage #[self | fsaverage]

detrend.sh ${FILEPATTERN}${SLICETIME}${SMOOTHING} $SURFACE `cat subjects`
This will execute the detrend.sh script on all the subjects listed in the subjects text file, using the fsaverage surface. The assemblage of variables will look for files named ''fmcpr.up.sm4.*''
[[Category: Time Series]]

BASH Tricks

2022-09-27T19:57:26Z

Chris: /* Make a series of numbered directories */

==Text File Batch==
Suppose we have a list of items to process -- like all the entries in the subjects file, for example. We want to use each line in the file in a for-loop:
while read s; do
echo "$s";
done <subjects

== How many lines in my text file? ==
Totally useful when you have some kind of training file with many rows and columns:
FILENAME=myfile.csv
nl ${FILENAME} | awk '{ print $1 }'

== I want to drop the first line of my text file ==
<code>tail</code> echoes the last n lines (default: 10) of a text file to stdout. Using the -n flag flips it around so that it echoes back ''all up to the last'' n lines of the file. So -n +2 will echo back the file up to the 2nd line of the file (i.e., dropping the first line). We can pipe this to a temp file (so we don't write out an empty file), and then rename:
tail -n +2 "$FILE" > "$FILE.tmp" && mv "$FILE.tmp" "$FILE"
'''PROTIP:''' I'm pretty sure the same trick applies using the <code>head</code> command to drop the ''last'' n lines from a file.

== Make a list of directory names ==
We often organize subject data so that each subject gets their own directory. Freesurfer uses a '''subjects''' file when batch processing. Rather than manually type out each folder name into a text file, it can be generated in one line of code:
ls -1 -d */ | sed "s,/$,," > subjects
This lists in 1 column all the directories (-1 -d) and uses ''sed'' to snip off the trailing forward slashes in the directory names

==What directories are in one directory but not the other?==
Scenario: We had a directory, let's call it ALLSUBJECTS, that had a bunch of subject directories named '''NDARINVxxxxxxxx'''. Some of them had a full dataset, but many of them did not. Sophia made a directory called '''gooddata''' that contained only the subset of folders that had full datasets. What's the fastest way to figure out who has incomplete data? Look for folders appearing in '''ALLSUBJECTS''' that don't appear in '''gooddata'''.

cd ALLSUBJECTS
#next command lists only directories (-d), in a single column (-1), sorts the list (sort),
#makes sure it only lists folder names starting with "ND" ( grep "^ND"), and then uses
#sed to strip the trailing backslash
ls -d -1 */ | grep "^ND" | sort | sed 's/\///g' >> ../allsubs.txt
cd ../gooddata
ls -d -1 */ | grep "^ND" | sort | sed 's/\///g' >> ../goodsubs.txt
#next line finds lines appearing in allsubs that do not appear in goodsubs:
comm -23 allsubs.txt goodsubs.txt

== Making a <code>tar</code> archive containing only the minimal set of structural files for FreeSurfer ==
FreeSurfer makes a zillion files during the recon-all step. I have no idea what most of them are there for. Which do we need? The absolute minimal list is a work in progress, but I have made a file called '''fsaverage.required''' (copied to <code>/ubfs/caset/cpmcnorg/Scripts/fsaverage.required</code>) based on the contents of the fsaverage template subject directory contents. I dropped the obvious directories (e.g., mri 2mm), so what's left should hopefully be close to the minimal required set for getting things done with a subject. The idea is to reduce the number of superfluous files that you store or copy over the network so that we don't waste as much time and disk space with useless nonsense.

So here's what you do (note the text in red will vary - don't just blindly copy the code snippets below and expect them to work; that's how Chernobyl happens! You need to understand what you're doing):

#Copy '''fsaverage.required''' to <code>$SUBJECTS_DIR</code>
#Inspect fsaverage.required to make sure that it has any idiosyncratic files that you might wish to include
#*e.g., the original version only includes f.nii.gz files. If you want to also grab all your preprocessed .mgz files, then you'll want to include *.mgz up at the top. Save any changes.
#Navigate to a subject directory: <code>cd FS_sub-001</code>
#The following command will use the files listed in <code>$SUBJECTS_DIR/fsaverage.required</code> to find and archive the desired files for this subject:
#*<code>tar -czvf sub-001.minimal.tgz `find . | grep -G -f ${SUBJECTS_DIR}/fsaverage.required`</code>
#When you're done, you'll have the bare-bones minimum files to permit FS-FAST analyses of your BOLD data for your subject.
#You can copy the .tgz files to an external drive or over the network. Be sure to unpack the .tgz archive in an empty subject directory
#*e.g.:
mkdir ~/new_project/ #starting a new project directory - in this case on the same computer, but it could be anywhere
cd new_project #enter the new project directory
mkdir FS_sub-001 #making an empty subject directory for the files we're about to unpack
cd FS_sub-001 #navigate into the new empty subject directory
#next line copies the minimal file archive from the source directory into the new empty subject directory
cp ${SUBJECTS_DIR}/FS_sub-001/sub-001.minimal.tgz ./
#next line unzips the file archive into the empty directory
tar -xzvf sub-001.minimal.tgz

With the minimal set of structural files, you should be able to unzip the surface and T1 anatomical files and inspect for reconstruction accuracy, or add BOLD files from elsewhere (the BOLD files are what ''really'' does you in, and I've developed a similar procedure to grab your blob analysis files)

== Making a <code>tar</code> archive containing only the minimal set of GLM Analysis Files for FreeSurfer ==
After running a first level GLM analysis (a "blob" analysis) using <code>selxavg3-sess</code>, each of your subject/bold directories will contain an analysis directory for each of the surfaces you included in your analysis (typically for lh and rh, and possibly also for mni305). Assuming the analyses were done in fsaverage template space (and there's no good reason anymore why they ''wouldn't'' be), then if you would like to download the bare minimum set of files required to inspect the subject-level analyses, then you can do so with the following script:

#!/bin/bash
#usage: ./zip1stla.sh SUBJECT_ID ANALYSIS_DIR_1 [ANALYSIS_DIR_2 ... etc]
#this first step is going to be to enforce that this only works when SUBJECTS_DIR is set
cd ${SUBJECTS_DIR}
#first param is subject id
SUB=${1}
shift
#remaining params are analysis directories
DIRS="$@"
#we're going to clone the analysis directory structure
cd ${SUB}/bold
mkdir --parents ${SUB}/bold

#iterate through analysis directories
for DIR in "${DIRS[@]}";
do
cp -r ${DIR} ${SUB}/bold/
done
#zip up our cloned directory structure
tar -czvf ${SUBJECTS_DIR}/${SUB}.1stla.tgz ${SUB}
#delete the clone
rm -rf ${SUB}
#go back to where we started
cd ${SUBJECTS_DIR}

If you were to copy/paste the above script to a file named '''zip1stla.sh''' and make it executable (<code>chmod ug+x zip1stla.sh</code>) then you would run it this way:
#suppose my analysis directories are called FAM.sm6.lh, FAM.sm6.rh and FAM.sm6.mni
zip1stla.sh FS_SUB01 FAM.sm6.lh FAM.sm6.rh FAM.sm6.mni
This will create a file called FS_SUB01.1stla.tgz. When you unzip the file, it will create a subject folder with the following structure:
*FS_SUB01
**bold
***FAM.sm6.lh
****{some files}
***FAM.sm6.rh
****{some files}
***FAM.sm6.mni
****{some files}
No other files will be included in the archive, which keeps the archive size to a minimum. If FS_SUB01 already exists, then the contents of this archive will be added to the existing directory. This can be useful if you previously used the method described above to archive a minimal set of FreeSurfer structural files. Note that the FreeSurfer structural files are not needed to view the first level GLM data if you ran the analysis in fsaverage space, because these data are mapped to the fsaverage template, which you will already have on your local machine if you have FreeSurfer installed.

== Make a series of numbered directories ==
FreeSurfer BOLD data goes in a series of directories, numbered 001, 002, ... , 0nn. A one-liner of code to create these directories in the command line:
for i in $(seq -f "%03g" 1 6); do mkdir ${i}; done
#this will create directories 001 to 006. Obviously, if you need more directories, change the second value from 6 to something else
''Protip:'' If you want to also make the <code>runs</code> file that some of our scripts use at the same time, the above snippet can be modified:
for i in $(seq -f "%03g" 1 6); do mkdir ${i}; echo ${i} >> runs; done

=== Save list of numbered directories to file ===
Another protip: If you already had a set of numbered directories and want to save them to a list (e.g., a "runs" file):
while read s; do
ls "$SUBJECTS_DIR/$s/bold" | grep "^00*" > $SUBJECTS_DIR/$s/bold/runs
done <subjects

== Restart Window Manager ==
This has happened a couple times before: you step away from the computer for awhile (maybe even overnight) and when you come back, you find it is locked up and completely unresponsive. The nuclear option is to reboot the whole machine:
sudo shutdown -r now #Sad for anyone running autorecon or a neural network
Unfortunately, that will stop anything that might be running in the background. A less severe solution might be to just restart the window manager. To do this you will need to ssh into the locked-up computer from a different computer, and then restart the lightdm process. This will require superuser privileges.
ssh ''hostname''
Then after you have connected to the frozen computer:
sudo restart lightdm
Any processes that were dependent on the window manager will be terminated (e.g., so if you had been in the middle of editing labels in tksurfer, you will find that tksurfer has been shutdown and you will need to start over), however anything that was running in the background (e.g., autorecon) should be unaffected.

==Renaming Multiple Files==
=== Rename Using <code>rename</code>===
A '''perl''' command, called <code>rename</code> might be available on your *nix system:
rename [OPTIONS] perlexpr files
Among useful options are the <code>-n</code> flag, which just reports what all the file renames would be, but doesn't actually execute them.
A handy application of '''rename''' is to hide files and/or directories. Files with names beginning with a dot are hidden by default and don't show up in directory listings. This can be a handy way of excluding chunks of data from your scripts.
====Use-Case: Hiding Session 2 Data====
In our Multisensory Imagery experiment, we collect 6 runs at time points 1 and 2. If we wish to be able to analyze all the data, these would be stored together as runs 001 to 012. Suppose we wish to temporarily hide the second time point data:
rename -n 's/01/\._01/' `find ./ -type d -name "01*"`
This would find all the directories ("-type -d") named 01*, then it would show you how it would rename them. If everything looked right, you would execute the same command again, but omit the -n flag so that the renaming actually takes place.
Note that this example only gets the 010, 011 and 012 directories. You would do something similar for directories 00[6-9].

====Use-Case: Unhiding Directories====
This one is easier, since all the hidden directories start with "._" using the approach described above:
rename 's/\._//' `find ./ -type d -name "._0*"`
In case you're curious about the syntax of the perl expression, you might want to [https://www.regular-expressions.info/quickstart.html read up a bit about regular expressions], but in this case, 's/\._//' indicates we are doing a substitution that will replace every instance of ._ with an empty string (//).
The extra back-slash in front of the period is an ''escape character'', which is needed because otherwise the dot (period) will be interpreted as a special character.

== Rename Using <code>mv</code>==
If you don't have access to the rename command (Mac OSX), you can fake it:
PREFIX=LO
for file in `find . -name "*.txt"`; do mv ${file##*/} ${PREFIX}_${file##*/}; done
Source: [https://stackoverflow.com/questions/2664740/extract-file-basename-without-path-and-extension-in-bash]
== Related Trick: Collecting and Renaming Multiple Files in Subdirectories ==
Use case: I ran a bunch of model simulations. Each batch of simulations produced a series of 8 Keras files named '''model_0x.h5''', and stored in directories named '''batch_##/'''. 10 batches of simulations produced 80 model files, except that they all had the same names. I wanted to run some tests on the complete set, so I needed to aggregate all the files in a single directory, but rename them from 01 to 80:

for run in $(seq 1 10)
do
r=`printf "%02d" $run`
echo "Gathering run $r files"
for m in {1..8}
do
basemodel=`printf "%02d" $m`
blockstart=$(( ($run-1)*8 ))
newmodel=$(( $blockstart+$m ))
cp batch_$r/model_$basemodel.h5 ./model_$newmodel.h5
done
done

== <code>sed</code> Tricks ==
===Replacing Text in Multiple Files===
sed -i 's/oldtext/newtext/g' *.ext

===Remove punctuation and convert to lowercase===
$FILENAME=file.txt
sed 's/[[:punct:]]//g' $FILENAME | sed $'s/\t//g' | tr '[:upper:]' '[:lower:]' > lowercase.$FILENAME

==Archiving Specific Files in a Directory Tree==
The <code>tar</code> has an <code>--include</code> switch which will archive ''only'' matching file patterns, however it appears that this filtering [https://stackoverflow.com/questions/5747755/tar-with-include-pattern breaks] when trying to archive files in subdirectories. Fortunately, the person who posed the question on StackExchange already had a workaround that works fine (it's just ugly):

find ./ -name "*.wav.txt" -print0 | tar -cvzf ~/adhd.tgz --null -T -

No idea what the -T does, nor what the trailing - does, but there you have it. This works. Just replace your file pattern with whatever it is you're filtering out, and of course specify an appropriate tgz archive name.

== mysql on the terminal ==
So I learned tonight how to export query results to a text file from the shell interface. Note that MySQL server is running with the --secure-file-priv option enabled, so you can't just willy-nilly write files wherever you want. However /var/lib/mysql-files/ is fair game, so for example:
select * from conceptstats inner join concepts on conceptstats.concid=concepts.concid where pid=183 and norm=1 into outfile '/var/lib/mysql-files/0183.txt'

Freesurfer Subparcellation

2022-09-27T18:30:24Z

Chris:

Freesurfer annotations and labels describe swaths of cortical regions, defined anatomically during the recon-all process, or functionally or statistically (e.g., significant GLM clusters). Tools exist to further subdivide these regions to the desired level of granularity.
= Using AnchorBar =
AnchorBar is a set of Python tools developed in our lab that maintain a database of a set of annotations and supports set operations on the labels associated with one or more annotations. For example, an atlas-based annotation can be intersected with a functional annotation (e.g., from a group-level GLM) to subdivide large functional clusters along anatomical boundaries.

= Using mris_divide_parcellation =
This program divides one or more parcellations into divisions
perpendicular to the long axis of the label. The number of divisions
can be specified in one of two ways, depending upon the nature of the
fourth argument.

First, a splitfile can be specified as the fourth argument. The
splitfile is a text file with two columns. The first column is the
name of the label in the source annotation, and the second column is
the number of units that label should be divided into. The names of
the labels depends upon the source parcellation. For aparc.annot and
aparc.a2005.annot, the names can be found in
$FREESURFER_HOME/FreeSurferColorLUT.txt. For aparc.annot, the labels
are between the ranges of 1000-1034. For aparc.a2005s.annot, the
labels are between the ranges of 1100-1181. The name for the label is
the name of the segmentation without the 'ctx-lh'. Note that the name
included in the splitfile does not indicate the hemisphere. For
example, 1023 is 'ctx-lh-posteriorcingulate'. You should put
'posteriorcingulate' in the splitfile. Eg, to divide it into three
segments, the following line should appear in the splitfile:

posteriorcingulate 3

Only labels that should be split need be specified in the splitfile.

The second method is to specify an area threshold (in mm^2) as the
fourth argument, in which case each label is divided until each
subdivision is smaller than the area threshold.

The output label name will be the original name with _divN appended,
where N is the division number. N will go from 2 to the number of
divisions. The first division has the same name as the original label.

== Sample Split File (Functional Clusters) ==
In this example, I have carried out a one-sample group mean (OSGM) analysis and did a cluster-size threshold. This saved a number of clusters in two .annot files (one in each hemisphere). There are also some ''*.cluster.summary'' files in this directory. Reading these files, I see the smallest clusters are about 120mm2. I would like to subdivide the larger clusters so that they are all between about 100-150mm2. I load the corresponding .annot file in Freesurfer just to verify the cluster names (the summary file lists 10 clusters, but the names do not follow a consistent naming convention). From the file, I see that clusters 1,2,3,4,6 and 8 need to be subdivided. Simple division by the desired cluster size tells me the number of divisions I want for each cluster:

echo cluster-001 6 > rsplittable.txt
echo cluster-002 7 >> rsplittable.txt
echo cluster-003 5 >> rsplittable.txt
echo cluster-004 5 >> rsplittable.txt
echo cluster-006 2 >> rsplittable.txt
echo cluster-008 2 >> rsplittable.txt

Repeat the process for the lh clusters, creating an lsplittable.txt file.

The group analysis used the fsaverage surface. Since mris_divide_parcellation assumes we're subdividing a particular subject's annotation, it makes sense to copy the source .annot files into the fsaverage label/ folder (giving the appropriate ?h prefix)

cp cache.th30.abs.sig.ocn.annot \
$SUBJECTS_DIR\fsaverage\label\rh.cache.th30.abs.sig.ocn.annot

Then call the program thus (assuming that rsplittable.txt is in your current working directory):

mris_divide_parcellation fsaverage rh \
cache.th30.abs.sig.ocn.annot rsplittable.txt \
functional_subclusters

Program output:

reading colortable from annotation file...
colortable with 11 entries read (originally none)
interpreting 4th command line arg as split file name
dividing cluster-001 (1) into 6 parts
dividing cluster-002 (2) into 7 parts
dividing cluster-003 (3) into 5 parts
dividing cluster-004 (4) into 5 parts
dividing cluster-006 (6) into 2 parts
dividing cluster-008 (8) into 2 parts
allocating new colortable with 31 additional units...
saving annotation to functional_subclusters

Repeat for lh.

Note: the first time I ran this, I have no idea where the rh.functional_subclusters.annot file went! Running the lh and re-running the rh subparcellation generated the expected files in the fsaverage/label directory.

== Check Subparcellation ==
Presumably the python wrapper does this iteratively because all subdivisions are done along the same axis. As I discovered, a large number of subdivisions may be realized as a series of thin bands (below). In this case, it might be best to subdivide larger regions over a series of calls, as I describe below.

[[File:Striped_Parcellation.png]]

=== Iterative Subparcellation ===
If banding among your subdivisions is a problem, here's an example of the iterative approach using region size, rather than a split file:

First, break down clusters into 400mm2 chunks:
mris_divide_parcellation fsaverage lh \
cache.th30.abs.sig.ocn.annot
400 400functional_subclusters

Next, break down the 400mm2 chunks into sub-200mm2 regions:

mris_divide_parcellation fsaverage lh \
400functional_subclusters.annot \
200 200functional_subclusters

Because each call to <code>mris_divide_parcellation</code> bisects the region along it's longest axis, successive function calls at each iteration produce increasingly less elongated subparcellations.

== Transfer Subparcellation to Other Subjects ==
Once satisfied with the source subparcellation, the scheme ''can'' be mapped to other surfaces using mri_surf2surf if, for example, you are working in '''self''' space (this needs to be done individually for the lh and rh surfaces -- below is an example command for the left):

mri_surf2surf \
--srcsubject fsaverage \
--trgsubject FS_T1_501 \
--hemi lh \
--sval-annot $SUBJECTS_DIR/fsaverage/label/lh.200functional_subclusters.annot \
--tval $SUBJECTS_DIR/FS_T1_501/label/lh.200functional_subclusters.annot

''Pro Tip: Use a global variable (e.g. SUBN) and use $SUBN in place of your subject in trgsubject and tval. Better yet, use the clustmap.sh shell script.''

You may want to pull out overlapping voxels for T1 and T2 data (like a venn diagram) before mapping the surfaces to each subject, [http://ccnlab.psy.buffalo.edu/wiki/index.php/Working_with_ROIs_(Freesurfer) here].

'''Note: If you are working in ''fsaverage'' space for all participants, you only have to subparcellate your annotations once -- for the fsaverage surface.''' Because the functional analyses are invariably carried out in fsaverage space in order to compute group-level statistics, you will seldom have to transfer a subparcellation to a specific participant's surface space.

[[Category: FreeSurfer]]
[[Category: Network Analyses]]

AnchorBar

2022-09-14T15:14:54Z

Chris:

AnchorBar (aka "Analyzer") is a Python program developed to maintain and compute set operations on FreeSurfer annotation files. It's kind of like SPM's MarsBar plugin, but for FreeSurfer.
=Requirements=
AnchorBar has a few dependencies that might not be part of your base Python install:
*hashlib
*nibabel
Attempting to run the scripts will fail and report missing libraries. They can be installed using pip:
pip install nibabel
The utility is (currently) comprised of 3 scripts. Syntax help for each of these scripts can be obtained using the <code>-h</code> switch:
$ AnchorBar_tools.py -h
usage: AnchorBar_tools.py [-h] [--list] [--labels annot_id] [--db DB] [--rename annot_id new_shortname]
[--relabel annot_id label_id new_label_name] [--abbrev annot_id label_id abbrev_label_name]

=Querying Available Annotations=
AnchorBar_tools.py provides tools to list, rename and relabel annotations. Each annotation has a unique ID and descriptors indicating the hemisphere, a shortname used for naming labels generated from set operations, and a path indicating the file path of the source .annot file. The default set of annotations stored in '''annot.db''' includes the 10 pairs of lhrh annotations from fsaverage/label:
AnchorBar_tools.py --db annot.db --list
The labels associated with a given annotation can be found using the <code>--labels</code> switch:
#List all the labels in the lh.aparc_2009s.annot FreeSurfer annotation
AnchorBar_tools.py --db annot.db --labels 2

=Adding Annotations=
The most common use-case for adding annotations is when working with functional ROIs from group analyses. AnchorBar_init.py is the tool that imports new .annot files into the database for later operations. Imported annotations are associated with a unique identifier based on the labels and vertices that make up an annotation. This allows the scripts to recognize and ignore duplicate annotations that might have been given different names, but also to differentiate distinct annotations that happen to have the same name (as is common when working with functional ROIs).

The import process infers the hemisphere from the filename. Functional cluster annotations are unlikely to have an lh/rh prefix, and so will require renaming prior to importing (I like to stick them in the project copy of the fsaverage folder):
cp $SUBJECTS_DIR/RFX/myanalysis.lh/a_vs_b/glm.wls/osgm/cache.th30.abs.sig.ocn.annot $SUBJECTS_DIR/fsaverage/label/lh.a_vs_b.cache.th30.abs.annot
cp $SUBJECTS_DIR/RFX/myanalysis.rh/a_vs_b/glm.wls/osgm/cache.th30.abs.sig.ocn.annot $SUBJECTS_DIR/fsaverage/label/rh.a_vs_b.cache.th30.abs.annot

To add one or more new annotations (that you've renamed and saved to the fsaverage/label folder):
AnchorBar_init.py --db annot.db --annot $SUBJECTS_DIR/fsaverage/label/*.th30.abs.annot
Of course, the particular filename pattern in red in the above command may differ from what's shown here, depending on your file naming convention, but you get the idea. When this code executes, it will import all the files that end in "th30.abs.annot" that it finds. This is handy for importing annotations for multiple hemispheres and/or for multiple contrasts.

==Renaming Imported Annotations==
Imported annotations will be given a shortname that corresponds to the filename of the source annotation. These names can be unwieldy, so AnchorBar_tools.py allows you to assign a new shortname to stored annotations. First, list the annotations and note their shortnames and id values. Use the id when selecting the annotation for renaming. For example:
./AnchorBar_tools.py --db annot.db --list

*** Annotation List ***
id LR shortname path
1 lh aparc.a2005s /usr/local/freesurfer/subjects/fsaverage/label
2 lh aparc.a2009s /usr/local/freesurfer/subjects/fsaverage/label
...
25 lh a_v_b.th30.abs /home/projects/mystudy/fsaverage/label
26 rh a_v_b.th30.abs /home/projects/mystudy/fsaverage/label

If we wanted to remove the "th30.abs" part of the shortname for the annotations 25 and 26:
./AnchorBar_tools.py --db annot.db --rename 26 AvsB
./AnchorBar_tools.py --db annot.db --rename 25 AvsB

Now inspect the annotation database to see the shortname has been updated with the values you just provided:
./AnchorBar_tools.py --db annot.db --list

*** Annotation List ***
id LR shortname path
1 lh aparc.a2005s /usr/local/freesurfer/subjects/fsaverage/label
2 lh aparc.a2009s /usr/local/freesurfer/subjects/fsaverage/label
...
25 lh AvsB /home/projects/mystudy/fsaverage/label
26 rh AvsB /home/projects/mystudy/fsaverage/label

=Set Operations on Annotations=
Note that all set operations require pairs of annotations from the same hemisphere. If unsure, list the annotations and take careful note of the hemisphere noted for the ids of the annotations you're interested in. Using annotations from opposite hemispheres will result in an error message and produce no new output.

==Intersection of Two Annotations==
Example: We have a functional contrast (A vs Rest) and wish to divide the large clusters along Brodmann Area boundaries. By intersecting the clusters (which only cover some of the brain surface) with one of the anatomical atlas annotations (which cover all of the brain surface), we assign a new label to each vertex according to which cluster and which anatomical label it corresponds to. Vertices not belonging to functional clusters will be dropped. The end result is a new annotation where the clusters are subdivided along anatomical boundaries.

We first list the annotations in the database to find the corresponding id numbers. A vs Rest are annotations 23 (lh) and 24 (rh), and the Brodmann annotations are 5 (lh) and 15 (rh). We intersect the two corresponding annotations thus:
AnchorBar_sets.py --db annot.db --intersect 5 23
AnchorBar_sets.py --db annot.db --intersect 15 24

==Merging (Union) of Two Annotations==
Example: We have two tasks, A and B, and each task condition has been contrasted with rest (A vs Rest; B vs Rest). We wish to identify left-hemisphere regions that are involved in either task, so we compute the union. This functionality has been added to AnchorBar
#Assumes that lh.AnnotA and lh.AnnotB correspond to the annotations derived from the two task contrasts,
#A vs Rest and B vs Rest, respectively, and that these annotations are numbered 23 and 25
AnchorBar_sets.py --db annot.db --union 23 25

AnchorBar

2022-09-14T15:10:39Z

Chris: /* Merging (Union) of Two Annotations */

AnchorBar (aka "Analyzer") is a Python program developed to maintain and compute set operations on FreeSurfer annotation files. It's kind of like SPM's MarsBar plugin, but for FreeSurfer.
=Requirements=
AnchorBar has a few dependencies that might not be part of your base Python install:
*hashlib
*nibabel
Attempting to run the scripts will fail and report missing libraries. They can be installed using pip:
pip install nibabel
The utility is (currently) comprised of 3 scripts. Syntax help for each of these scripts can be obtained using the <code>-h</code> switch:
$ AnchorBar_tools.py -h
usage: AnchorBar_tools.py [-h] [--list] [--labels annot_id] [--db DB] [--rename annot_id new_shortname]
[--relabel annot_id label_id new_label_name] [--abbrev annot_id label_id abbrev_label_name]

=Querying Available Annotations=
AnchorBar_tools.py provides tools to list, rename and relabel annotations. Each annotation has a unique ID and descriptors indicating the hemisphere, a shortname used for naming labels generated from set operations, and a path indicating the file path of the source .annot file. The default set of annotations stored in '''annot.db''' includes the 10 pairs of lhrh annotations from fsaverage/label:
AnchorBar_tools.py --db annot.db --list

=Adding Annotations=
The most common use-case for adding annotations is when working with functional ROIs from group analyses. AnchorBar_init.py is the tool that imports new .annot files into the database for later operations. Imported annotations are associated with a unique identifier based on the labels and vertices that make up an annotation. This allows the scripts to recognize and ignore duplicate annotations that might have been given different names, but also to differentiate distinct annotations that happen to have the same name (as is common when working with functional ROIs).

The import process infers the hemisphere from the filename. Functional cluster annotations are unlikely to have an lh/rh prefix, and so will require renaming prior to importing (I like to stick them in the project copy of the fsaverage folder):
cp $SUBJECTS_DIR/RFX/myanalysis.lh/a_vs_b/glm.wls/osgm/cache.th30.abs.sig.ocn.annot $SUBJECTS_DIR/fsaverage/label/lh.a_vs_b.cache.th30.abs.annot
cp $SUBJECTS_DIR/RFX/myanalysis.rh/a_vs_b/glm.wls/osgm/cache.th30.abs.sig.ocn.annot $SUBJECTS_DIR/fsaverage/label/rh.a_vs_b.cache.th30.abs.annot

To add one or more new annotations (that you've renamed and saved to the fsaverage/label folder):
AnchorBar_init.py --db annot.db --annot $SUBJECTS_DIR/fsaverage/label/*.th30.abs.annot
Of course, the particular filename pattern in red in the above command may differ from what's shown here, depending on your file naming convention, but you get the idea. When this code executes, it will import all the files that end in "th30.abs.annot" that it finds. This is handy for importing annotations for multiple hemispheres and/or for multiple contrasts.

==Renaming Imported Annotations==
Imported annotations will be given a shortname that corresponds to the filename of the source annotation. These names can be unwieldy, so AnchorBar_tools.py allows you to assign a new shortname to stored annotations. First, list the annotations and note their shortnames and id values. Use the id when selecting the annotation for renaming. For example:
./AnchorBar_tools.py --db annot.db --list

*** Annotation List ***
id LR shortname path
1 lh aparc.a2005s /usr/local/freesurfer/subjects/fsaverage/label
2 lh aparc.a2009s /usr/local/freesurfer/subjects/fsaverage/label
...
25 lh a_v_b.th30.abs /home/projects/mystudy/fsaverage/label
26 rh a_v_b.th30.abs /home/projects/mystudy/fsaverage/label

If we wanted to remove the "th30.abs" part of the shortname for the annotations 25 and 26:
./AnchorBar_tools.py --db annot.db --rename 26 AvsB
./AnchorBar_tools.py --db annot.db --rename 25 AvsB

Now inspect the annotation database to see the shortname has been updated with the values you just provided:
./AnchorBar_tools.py --db annot.db --list

*** Annotation List ***
id LR shortname path
1 lh aparc.a2005s /usr/local/freesurfer/subjects/fsaverage/label
2 lh aparc.a2009s /usr/local/freesurfer/subjects/fsaverage/label
...
25 lh AvsB /home/projects/mystudy/fsaverage/label
26 rh AvsB /home/projects/mystudy/fsaverage/label

=Set Operations on Annotations=
Note that all set operations require pairs of annotations from the same hemisphere. If unsure, list the annotations and take careful note of the hemisphere noted for the ids of the annotations you're interested in. Using annotations from opposite hemispheres will result in an error message and produce no new output.

==Intersection of Two Annotations==
Example: We have a functional contrast (A vs Rest) and wish to divide the large clusters along Brodmann Area boundaries. By intersecting the clusters (which only cover some of the brain surface) with one of the anatomical atlas annotations (which cover all of the brain surface), we assign a new label to each vertex according to which cluster and which anatomical label it corresponds to. Vertices not belonging to functional clusters will be dropped. The end result is a new annotation where the clusters are subdivided along anatomical boundaries.

We first list the annotations in the database to find the corresponding id numbers. A vs Rest are annotations 23 (lh) and 24 (rh), and the Brodmann annotations are 5 (lh) and 15 (rh). We intersect the two corresponding annotations thus:
AnchorBar_sets.py --db annot.db --intersect 5 23
AnchorBar_sets.py --db annot.db --intersect 15 24

==Merging (Union) of Two Annotations==
Example: We have two tasks, A and B, and each task condition has been contrasted with rest (A vs Rest; B vs Rest). We wish to identify left-hemisphere regions that are involved in either task, so we compute the union. This functionality has been added to AnchorBar
#Assumes that lh.AnnotA and lh.AnnotB correspond to the annotations derived from the two task contrasts,
#A vs Rest and B vs Rest, respectively, and that these annotations are numbered 23 and 25
AnchorBar_sets.py --db annot.db --union 23 25

AnchorBar

2022-09-14T15:10:20Z

Chris: /* Merging (Union) of Two Annotations */

AnchorBar (aka "Analyzer") is a Python program developed to maintain and compute set operations on FreeSurfer annotation files. It's kind of like SPM's MarsBar plugin, but for FreeSurfer.
=Requirements=
AnchorBar has a few dependencies that might not be part of your base Python install:
*hashlib
*nibabel
Attempting to run the scripts will fail and report missing libraries. They can be installed using pip:
pip install nibabel
The utility is (currently) comprised of 3 scripts. Syntax help for each of these scripts can be obtained using the <code>-h</code> switch:
$ AnchorBar_tools.py -h
usage: AnchorBar_tools.py [-h] [--list] [--labels annot_id] [--db DB] [--rename annot_id new_shortname]
[--relabel annot_id label_id new_label_name] [--abbrev annot_id label_id abbrev_label_name]

=Querying Available Annotations=
AnchorBar_tools.py provides tools to list, rename and relabel annotations. Each annotation has a unique ID and descriptors indicating the hemisphere, a shortname used for naming labels generated from set operations, and a path indicating the file path of the source .annot file. The default set of annotations stored in '''annot.db''' includes the 10 pairs of lhrh annotations from fsaverage/label:
AnchorBar_tools.py --db annot.db --list

=Adding Annotations=
The most common use-case for adding annotations is when working with functional ROIs from group analyses. AnchorBar_init.py is the tool that imports new .annot files into the database for later operations. Imported annotations are associated with a unique identifier based on the labels and vertices that make up an annotation. This allows the scripts to recognize and ignore duplicate annotations that might have been given different names, but also to differentiate distinct annotations that happen to have the same name (as is common when working with functional ROIs).

The import process infers the hemisphere from the filename. Functional cluster annotations are unlikely to have an lh/rh prefix, and so will require renaming prior to importing (I like to stick them in the project copy of the fsaverage folder):
cp $SUBJECTS_DIR/RFX/myanalysis.lh/a_vs_b/glm.wls/osgm/cache.th30.abs.sig.ocn.annot $SUBJECTS_DIR/fsaverage/label/lh.a_vs_b.cache.th30.abs.annot
cp $SUBJECTS_DIR/RFX/myanalysis.rh/a_vs_b/glm.wls/osgm/cache.th30.abs.sig.ocn.annot $SUBJECTS_DIR/fsaverage/label/rh.a_vs_b.cache.th30.abs.annot

To add one or more new annotations (that you've renamed and saved to the fsaverage/label folder):
AnchorBar_init.py --db annot.db --annot $SUBJECTS_DIR/fsaverage/label/*.th30.abs.annot
Of course, the particular filename pattern in red in the above command may differ from what's shown here, depending on your file naming convention, but you get the idea. When this code executes, it will import all the files that end in "th30.abs.annot" that it finds. This is handy for importing annotations for multiple hemispheres and/or for multiple contrasts.

==Renaming Imported Annotations==
Imported annotations will be given a shortname that corresponds to the filename of the source annotation. These names can be unwieldy, so AnchorBar_tools.py allows you to assign a new shortname to stored annotations. First, list the annotations and note their shortnames and id values. Use the id when selecting the annotation for renaming. For example:
./AnchorBar_tools.py --db annot.db --list

*** Annotation List ***
id LR shortname path
1 lh aparc.a2005s /usr/local/freesurfer/subjects/fsaverage/label
2 lh aparc.a2009s /usr/local/freesurfer/subjects/fsaverage/label
...
25 lh a_v_b.th30.abs /home/projects/mystudy/fsaverage/label
26 rh a_v_b.th30.abs /home/projects/mystudy/fsaverage/label

If we wanted to remove the "th30.abs" part of the shortname for the annotations 25 and 26:
./AnchorBar_tools.py --db annot.db --rename 26 AvsB
./AnchorBar_tools.py --db annot.db --rename 25 AvsB

Now inspect the annotation database to see the shortname has been updated with the values you just provided:
./AnchorBar_tools.py --db annot.db --list

*** Annotation List ***
id LR shortname path
1 lh aparc.a2005s /usr/local/freesurfer/subjects/fsaverage/label
2 lh aparc.a2009s /usr/local/freesurfer/subjects/fsaverage/label
...
25 lh AvsB /home/projects/mystudy/fsaverage/label
26 rh AvsB /home/projects/mystudy/fsaverage/label

=Set Operations on Annotations=
Note that all set operations require pairs of annotations from the same hemisphere. If unsure, list the annotations and take careful note of the hemisphere noted for the ids of the annotations you're interested in. Using annotations from opposite hemispheres will result in an error message and produce no new output.

==Intersection of Two Annotations==
Example: We have a functional contrast (A vs Rest) and wish to divide the large clusters along Brodmann Area boundaries. By intersecting the clusters (which only cover some of the brain surface) with one of the anatomical atlas annotations (which cover all of the brain surface), we assign a new label to each vertex according to which cluster and which anatomical label it corresponds to. Vertices not belonging to functional clusters will be dropped. The end result is a new annotation where the clusters are subdivided along anatomical boundaries.

We first list the annotations in the database to find the corresponding id numbers. A vs Rest are annotations 23 (lh) and 24 (rh), and the Brodmann annotations are 5 (lh) and 15 (rh). We intersect the two corresponding annotations thus:
AnchorBar_sets.py --db annot.db --intersect 5 23
AnchorBar_sets.py --db annot.db --intersect 15 24

==Merging (Union) of Two Annotations==
Example: We have two tasks, A and B, and each task condition has been contrasted with rest (A vs Rest; B vs Rest). We wish to identify left-hemisphere regions that are involved in either task, so we compute the union. This functionality has been added to AnchorBar
#Assumes that lh.AnnotA and lh.AnnotB correspond to the annotations derived from the two task contrasts, A vs Rest and B vs Rest, respectively
#and that these annotations are numbered 23 and 25
AnchorBar_sets.py --db annot.db --union 23 25

AnchorBar

2022-09-14T15:09:52Z

Chris: /* Merging (Union) of Two Annotations */

AnchorBar (aka "Analyzer") is a Python program developed to maintain and compute set operations on FreeSurfer annotation files. It's kind of like SPM's MarsBar plugin, but for FreeSurfer.
=Requirements=
AnchorBar has a few dependencies that might not be part of your base Python install:
*hashlib
*nibabel
Attempting to run the scripts will fail and report missing libraries. They can be installed using pip:
pip install nibabel
The utility is (currently) comprised of 3 scripts. Syntax help for each of these scripts can be obtained using the <code>-h</code> switch:
$ AnchorBar_tools.py -h
usage: AnchorBar_tools.py [-h] [--list] [--labels annot_id] [--db DB] [--rename annot_id new_shortname]
[--relabel annot_id label_id new_label_name] [--abbrev annot_id label_id abbrev_label_name]

=Querying Available Annotations=
AnchorBar_tools.py provides tools to list, rename and relabel annotations. Each annotation has a unique ID and descriptors indicating the hemisphere, a shortname used for naming labels generated from set operations, and a path indicating the file path of the source .annot file. The default set of annotations stored in '''annot.db''' includes the 10 pairs of lhrh annotations from fsaverage/label:
AnchorBar_tools.py --db annot.db --list

=Adding Annotations=
The most common use-case for adding annotations is when working with functional ROIs from group analyses. AnchorBar_init.py is the tool that imports new .annot files into the database for later operations. Imported annotations are associated with a unique identifier based on the labels and vertices that make up an annotation. This allows the scripts to recognize and ignore duplicate annotations that might have been given different names, but also to differentiate distinct annotations that happen to have the same name (as is common when working with functional ROIs).

The import process infers the hemisphere from the filename. Functional cluster annotations are unlikely to have an lh/rh prefix, and so will require renaming prior to importing (I like to stick them in the project copy of the fsaverage folder):
cp $SUBJECTS_DIR/RFX/myanalysis.lh/a_vs_b/glm.wls/osgm/cache.th30.abs.sig.ocn.annot $SUBJECTS_DIR/fsaverage/label/lh.a_vs_b.cache.th30.abs.annot
cp $SUBJECTS_DIR/RFX/myanalysis.rh/a_vs_b/glm.wls/osgm/cache.th30.abs.sig.ocn.annot $SUBJECTS_DIR/fsaverage/label/rh.a_vs_b.cache.th30.abs.annot

To add one or more new annotations (that you've renamed and saved to the fsaverage/label folder):
AnchorBar_init.py --db annot.db --annot $SUBJECTS_DIR/fsaverage/label/*.th30.abs.annot
Of course, the particular filename pattern in red in the above command may differ from what's shown here, depending on your file naming convention, but you get the idea. When this code executes, it will import all the files that end in "th30.abs.annot" that it finds. This is handy for importing annotations for multiple hemispheres and/or for multiple contrasts.

==Renaming Imported Annotations==
Imported annotations will be given a shortname that corresponds to the filename of the source annotation. These names can be unwieldy, so AnchorBar_tools.py allows you to assign a new shortname to stored annotations. First, list the annotations and note their shortnames and id values. Use the id when selecting the annotation for renaming. For example:
./AnchorBar_tools.py --db annot.db --list

*** Annotation List ***
id LR shortname path
1 lh aparc.a2005s /usr/local/freesurfer/subjects/fsaverage/label
2 lh aparc.a2009s /usr/local/freesurfer/subjects/fsaverage/label
...
25 lh a_v_b.th30.abs /home/projects/mystudy/fsaverage/label
26 rh a_v_b.th30.abs /home/projects/mystudy/fsaverage/label

If we wanted to remove the "th30.abs" part of the shortname for the annotations 25 and 26:
./AnchorBar_tools.py --db annot.db --rename 26 AvsB
./AnchorBar_tools.py --db annot.db --rename 25 AvsB

Now inspect the annotation database to see the shortname has been updated with the values you just provided:
./AnchorBar_tools.py --db annot.db --list

*** Annotation List ***
id LR shortname path
1 lh aparc.a2005s /usr/local/freesurfer/subjects/fsaverage/label
2 lh aparc.a2009s /usr/local/freesurfer/subjects/fsaverage/label
...
25 lh AvsB /home/projects/mystudy/fsaverage/label
26 rh AvsB /home/projects/mystudy/fsaverage/label

=Set Operations on Annotations=
Note that all set operations require pairs of annotations from the same hemisphere. If unsure, list the annotations and take careful note of the hemisphere noted for the ids of the annotations you're interested in. Using annotations from opposite hemispheres will result in an error message and produce no new output.

==Intersection of Two Annotations==
Example: We have a functional contrast (A vs Rest) and wish to divide the large clusters along Brodmann Area boundaries. By intersecting the clusters (which only cover some of the brain surface) with one of the anatomical atlas annotations (which cover all of the brain surface), we assign a new label to each vertex according to which cluster and which anatomical label it corresponds to. Vertices not belonging to functional clusters will be dropped. The end result is a new annotation where the clusters are subdivided along anatomical boundaries.

We first list the annotations in the database to find the corresponding id numbers. A vs Rest are annotations 23 (lh) and 24 (rh), and the Brodmann annotations are 5 (lh) and 15 (rh). We intersect the two corresponding annotations thus:
AnchorBar_sets.py --db annot.db --intersect 5 23
AnchorBar_sets.py --db annot.db --intersect 15 24

==Merging (Union) of Two Annotations==
Example: We have two tasks, A and B, and each task condition has been contrasted with rest (A vs Rest; B vs Rest). We wish to identify left-hemisphere regions that are involved in either task, so we compute the union. This functionality has been added to AnchorBar
#Assumes that lh.AnnotA and lh.AnnotB correspond to the annotations derived from the two task contrasts, A and B, respectively
#and that these annotations are numbered 23 and 25
AnchorBar_sets.py --db annot.db --union 23 25

AnchorBar

2022-09-14T15:09:04Z

Chris: /* Merging (Union) of Two Annotations */

AnchorBar (aka "Analyzer") is a Python program developed to maintain and compute set operations on FreeSurfer annotation files. It's kind of like SPM's MarsBar plugin, but for FreeSurfer.
=Requirements=
AnchorBar has a few dependencies that might not be part of your base Python install:
*hashlib
*nibabel
Attempting to run the scripts will fail and report missing libraries. They can be installed using pip:
pip install nibabel
The utility is (currently) comprised of 3 scripts. Syntax help for each of these scripts can be obtained using the <code>-h</code> switch:
$ AnchorBar_tools.py -h
usage: AnchorBar_tools.py [-h] [--list] [--labels annot_id] [--db DB] [--rename annot_id new_shortname]
[--relabel annot_id label_id new_label_name] [--abbrev annot_id label_id abbrev_label_name]

=Querying Available Annotations=
AnchorBar_tools.py provides tools to list, rename and relabel annotations. Each annotation has a unique ID and descriptors indicating the hemisphere, a shortname used for naming labels generated from set operations, and a path indicating the file path of the source .annot file. The default set of annotations stored in '''annot.db''' includes the 10 pairs of lhrh annotations from fsaverage/label:
AnchorBar_tools.py --db annot.db --list

=Adding Annotations=
The most common use-case for adding annotations is when working with functional ROIs from group analyses. AnchorBar_init.py is the tool that imports new .annot files into the database for later operations. Imported annotations are associated with a unique identifier based on the labels and vertices that make up an annotation. This allows the scripts to recognize and ignore duplicate annotations that might have been given different names, but also to differentiate distinct annotations that happen to have the same name (as is common when working with functional ROIs).

The import process infers the hemisphere from the filename. Functional cluster annotations are unlikely to have an lh/rh prefix, and so will require renaming prior to importing (I like to stick them in the project copy of the fsaverage folder):
cp $SUBJECTS_DIR/RFX/myanalysis.lh/a_vs_b/glm.wls/osgm/cache.th30.abs.sig.ocn.annot $SUBJECTS_DIR/fsaverage/label/lh.a_vs_b.cache.th30.abs.annot
cp $SUBJECTS_DIR/RFX/myanalysis.rh/a_vs_b/glm.wls/osgm/cache.th30.abs.sig.ocn.annot $SUBJECTS_DIR/fsaverage/label/rh.a_vs_b.cache.th30.abs.annot

To add one or more new annotations (that you've renamed and saved to the fsaverage/label folder):
AnchorBar_init.py --db annot.db --annot $SUBJECTS_DIR/fsaverage/label/*.th30.abs.annot
Of course, the particular filename pattern in red in the above command may differ from what's shown here, depending on your file naming convention, but you get the idea. When this code executes, it will import all the files that end in "th30.abs.annot" that it finds. This is handy for importing annotations for multiple hemispheres and/or for multiple contrasts.

==Renaming Imported Annotations==
Imported annotations will be given a shortname that corresponds to the filename of the source annotation. These names can be unwieldy, so AnchorBar_tools.py allows you to assign a new shortname to stored annotations. First, list the annotations and note their shortnames and id values. Use the id when selecting the annotation for renaming. For example:
./AnchorBar_tools.py --db annot.db --list

*** Annotation List ***
id LR shortname path
1 lh aparc.a2005s /usr/local/freesurfer/subjects/fsaverage/label
2 lh aparc.a2009s /usr/local/freesurfer/subjects/fsaverage/label
...
25 lh a_v_b.th30.abs /home/projects/mystudy/fsaverage/label
26 rh a_v_b.th30.abs /home/projects/mystudy/fsaverage/label

If we wanted to remove the "th30.abs" part of the shortname for the annotations 25 and 26:
./AnchorBar_tools.py --db annot.db --rename 26 AvsB
./AnchorBar_tools.py --db annot.db --rename 25 AvsB

Now inspect the annotation database to see the shortname has been updated with the values you just provided:
./AnchorBar_tools.py --db annot.db --list

*** Annotation List ***
id LR shortname path
1 lh aparc.a2005s /usr/local/freesurfer/subjects/fsaverage/label
2 lh aparc.a2009s /usr/local/freesurfer/subjects/fsaverage/label
...
25 lh AvsB /home/projects/mystudy/fsaverage/label
26 rh AvsB /home/projects/mystudy/fsaverage/label

=Set Operations on Annotations=
Note that all set operations require pairs of annotations from the same hemisphere. If unsure, list the annotations and take careful note of the hemisphere noted for the ids of the annotations you're interested in. Using annotations from opposite hemispheres will result in an error message and produce no new output.

==Intersection of Two Annotations==
Example: We have a functional contrast (A vs Rest) and wish to divide the large clusters along Brodmann Area boundaries. By intersecting the clusters (which only cover some of the brain surface) with one of the anatomical atlas annotations (which cover all of the brain surface), we assign a new label to each vertex according to which cluster and which anatomical label it corresponds to. Vertices not belonging to functional clusters will be dropped. The end result is a new annotation where the clusters are subdivided along anatomical boundaries.

We first list the annotations in the database to find the corresponding id numbers. A vs Rest are annotations 23 (lh) and 24 (rh), and the Brodmann annotations are 5 (lh) and 15 (rh). We intersect the two corresponding annotations thus:
AnchorBar_sets.py --db annot.db --intersect 5 23
AnchorBar_sets.py --db annot.db --intersect 15 24

==Merging (Union) of Two Annotations==
Example: We have two tasks, A and B, and each task condition has been contrasted with rest (A vs Rest; B vs Rest). We wish to identify left-hemisphere regions that are involved in either task, so we compute the union. This functionality has been added to AnchorBar
#Assumes that lh.AnnotA and lh.AnnotB correspond to the annotations derived from the two task contrasts, A and B, respectively
#and that these annotations are numbered 14 and 15
AnchorBar_sets.py --db annot.db --union 14 15

AnchorBar

2022-08-17T21:48:03Z

Chris: /* Set Operations on Annotations */

AnchorBar (aka "Analyzer") is a Python program developed to maintain and compute set operations on FreeSurfer annotation files. It's kind of like SPM's MarsBar plugin, but for FreeSurfer.
=Requirements=
AnchorBar has a few dependencies that might not be part of your base Python install:
*hashlib
*nibabel
Attempting to run the scripts will fail and report missing libraries. They can be installed using pip:
pip install nibabel
The utility is (currently) comprised of 3 scripts. Syntax help for each of these scripts can be obtained using the <code>-h</code> switch:
$ AnchorBar_tools.py -h
usage: AnchorBar_tools.py [-h] [--list] [--labels annot_id] [--db DB] [--rename annot_id new_shortname]
[--relabel annot_id label_id new_label_name] [--abbrev annot_id label_id abbrev_label_name]

=Querying Available Annotations=
AnchorBar_tools.py provides tools to list, rename and relabel annotations. Each annotation has a unique ID and descriptors indicating the hemisphere, a shortname used for naming labels generated from set operations, and a path indicating the file path of the source .annot file. The default set of annotations stored in '''annot.db''' includes the 10 pairs of lhrh annotations from fsaverage/label:
AnchorBar_tools.py --db annot.db --list

=Adding Annotations=
The most common use-case for adding annotations is when working with functional ROIs from group analyses. AnchorBar_init.py is the tool that imports new .annot files into the database for later operations. Imported annotations are associated with a unique identifier based on the labels and vertices that make up an annotation. This allows the scripts to recognize and ignore duplicate annotations that might have been given different names, but also to differentiate distinct annotations that happen to have the same name (as is common when working with functional ROIs).

The import process infers the hemisphere from the filename. Functional cluster annotations are unlikely to have an lh/rh prefix, and so will require renaming prior to importing (I like to stick them in the project copy of the fsaverage folder):
cp $SUBJECTS_DIR/RFX/myanalysis.lh/a_vs_b/glm.wls/osgm/cache.th30.abs.sig.ocn.annot $SUBJECTS_DIR/fsaverage/label/lh.a_vs_b.cache.th30.abs.annot
cp $SUBJECTS_DIR/RFX/myanalysis.rh/a_vs_b/glm.wls/osgm/cache.th30.abs.sig.ocn.annot $SUBJECTS_DIR/fsaverage/label/rh.a_vs_b.cache.th30.abs.annot

To add one or more new annotations (that you've renamed and saved to the fsaverage/label folder):
AnchorBar_init.py --db annot.db --annot $SUBJECTS_DIR/fsaverage/label/*.th30.abs.annot
Of course, the particular filename pattern in red in the above command may differ from what's shown here, depending on your file naming convention, but you get the idea. When this code executes, it will import all the files that end in "th30.abs.annot" that it finds. This is handy for importing annotations for multiple hemispheres and/or for multiple contrasts.

==Renaming Imported Annotations==
Imported annotations will be given a shortname that corresponds to the filename of the source annotation. These names can be unwieldy, so AnchorBar_tools.py allows you to assign a new shortname to stored annotations. First, list the annotations and note their shortnames and id values. Use the id when selecting the annotation for renaming. For example:
./AnchorBar_tools.py --db annot.db --list

*** Annotation List ***
id LR shortname path
1 lh aparc.a2005s /usr/local/freesurfer/subjects/fsaverage/label
2 lh aparc.a2009s /usr/local/freesurfer/subjects/fsaverage/label
...
25 lh a_v_b.th30.abs /home/projects/mystudy/fsaverage/label
26 rh a_v_b.th30.abs /home/projects/mystudy/fsaverage/label

If we wanted to remove the "th30.abs" part of the shortname for the annotations 25 and 26:
./AnchorBar_tools.py --db annot.db --rename 26 AvsB
./AnchorBar_tools.py --db annot.db --rename 25 AvsB

Now inspect the annotation database to see the shortname has been updated with the values you just provided:
./AnchorBar_tools.py --db annot.db --list

*** Annotation List ***
id LR shortname path
1 lh aparc.a2005s /usr/local/freesurfer/subjects/fsaverage/label
2 lh aparc.a2009s /usr/local/freesurfer/subjects/fsaverage/label
...
25 lh AvsB /home/projects/mystudy/fsaverage/label
26 rh AvsB /home/projects/mystudy/fsaverage/label

=Set Operations on Annotations=
Note that all set operations require pairs of annotations from the same hemisphere. If unsure, list the annotations and take careful note of the hemisphere noted for the ids of the annotations you're interested in. Using annotations from opposite hemispheres will result in an error message and produce no new output.

==Intersection of Two Annotations==
Example: We have a functional contrast (A vs Rest) and wish to divide the large clusters along Brodmann Area boundaries. By intersecting the clusters (which only cover some of the brain surface) with one of the anatomical atlas annotations (which cover all of the brain surface), we assign a new label to each vertex according to which cluster and which anatomical label it corresponds to. Vertices not belonging to functional clusters will be dropped. The end result is a new annotation where the clusters are subdivided along anatomical boundaries.

We first list the annotations in the database to find the corresponding id numbers. A vs Rest are annotations 23 (lh) and 24 (rh), and the Brodmann annotations are 5 (lh) and 15 (rh). We intersect the two corresponding annotations thus:
AnchorBar_sets.py --db annot.db --intersect 5 23
AnchorBar_sets.py --db annot.db --intersect 15 24

==Merging (Union) of Two Annotations==
Example: We have two tasks, A and B, and each task condition has been contrasted with rest (A vs Rest; B vs Rest). We wish to identify regions that are involved in either task, so we compute the union.
Unfortunately, it looks like I have to write this functionality!

AnchorBar

2022-08-17T21:44:13Z

Chris: /* Intersection of Two Annotations */

AnchorBar (aka "Analyzer") is a Python program developed to maintain and compute set operations on FreeSurfer annotation files. It's kind of like SPM's MarsBar plugin, but for FreeSurfer.
=Requirements=
AnchorBar has a few dependencies that might not be part of your base Python install:
*hashlib
*nibabel
Attempting to run the scripts will fail and report missing libraries. They can be installed using pip:
pip install nibabel
The utility is (currently) comprised of 3 scripts. Syntax help for each of these scripts can be obtained using the <code>-h</code> switch:
$ AnchorBar_tools.py -h
usage: AnchorBar_tools.py [-h] [--list] [--labels annot_id] [--db DB] [--rename annot_id new_shortname]
[--relabel annot_id label_id new_label_name] [--abbrev annot_id label_id abbrev_label_name]

=Querying Available Annotations=
AnchorBar_tools.py provides tools to list, rename and relabel annotations. Each annotation has a unique ID and descriptors indicating the hemisphere, a shortname used for naming labels generated from set operations, and a path indicating the file path of the source .annot file. The default set of annotations stored in '''annot.db''' includes the 10 pairs of lhrh annotations from fsaverage/label:
AnchorBar_tools.py --db annot.db --list

=Adding Annotations=
The most common use-case for adding annotations is when working with functional ROIs from group analyses. AnchorBar_init.py is the tool that imports new .annot files into the database for later operations. Imported annotations are associated with a unique identifier based on the labels and vertices that make up an annotation. This allows the scripts to recognize and ignore duplicate annotations that might have been given different names, but also to differentiate distinct annotations that happen to have the same name (as is common when working with functional ROIs).

The import process infers the hemisphere from the filename. Functional cluster annotations are unlikely to have an lh/rh prefix, and so will require renaming prior to importing (I like to stick them in the project copy of the fsaverage folder):
cp $SUBJECTS_DIR/RFX/myanalysis.lh/a_vs_b/glm.wls/osgm/cache.th30.abs.sig.ocn.annot $SUBJECTS_DIR/fsaverage/label/lh.a_vs_b.cache.th30.abs.annot
cp $SUBJECTS_DIR/RFX/myanalysis.rh/a_vs_b/glm.wls/osgm/cache.th30.abs.sig.ocn.annot $SUBJECTS_DIR/fsaverage/label/rh.a_vs_b.cache.th30.abs.annot

To add one or more new annotations (that you've renamed and saved to the fsaverage/label folder):
AnchorBar_init.py --db annot.db --annot $SUBJECTS_DIR/fsaverage/label/*.th30.abs.annot
Of course, the particular filename pattern in red in the above command may differ from what's shown here, depending on your file naming convention, but you get the idea. When this code executes, it will import all the files that end in "th30.abs.annot" that it finds. This is handy for importing annotations for multiple hemispheres and/or for multiple contrasts.

==Renaming Imported Annotations==
Imported annotations will be given a shortname that corresponds to the filename of the source annotation. These names can be unwieldy, so AnchorBar_tools.py allows you to assign a new shortname to stored annotations. First, list the annotations and note their shortnames and id values. Use the id when selecting the annotation for renaming. For example:
./AnchorBar_tools.py --db annot.db --list

*** Annotation List ***
id LR shortname path
1 lh aparc.a2005s /usr/local/freesurfer/subjects/fsaverage/label
2 lh aparc.a2009s /usr/local/freesurfer/subjects/fsaverage/label
...
25 lh a_v_b.th30.abs /home/projects/mystudy/fsaverage/label
26 rh a_v_b.th30.abs /home/projects/mystudy/fsaverage/label

If we wanted to remove the "th30.abs" part of the shortname for the annotations 25 and 26:
./AnchorBar_tools.py --db annot.db --rename 26 AvsB
./AnchorBar_tools.py --db annot.db --rename 25 AvsB

Now inspect the annotation database to see the shortname has been updated with the values you just provided:
./AnchorBar_tools.py --db annot.db --list

*** Annotation List ***
id LR shortname path
1 lh aparc.a2005s /usr/local/freesurfer/subjects/fsaverage/label
2 lh aparc.a2009s /usr/local/freesurfer/subjects/fsaverage/label
...
25 lh AvsB /home/projects/mystudy/fsaverage/label
26 rh AvsB /home/projects/mystudy/fsaverage/label

=Set Operations on Annotations=
==Intersection of Two Annotations==
Example: We have a functional contrast (A vs Rest) and wish to divide the large clusters along Brodmann Area boundaries. By intersecting the clusters (which only cover some of the brain surface) with one of the anatomical atlas annotations (which cover all of the brain surface), we assign a new label to each vertex according to which cluster and which anatomical label it corresponds to. Vertices not belonging to functional clusters will be dropped. The end result is a new annotation where the clusters are subdivided along anatomical boundaries.

We first list the annotations in the database to find the corresponding id numbers. A vs Rest are annotations 23 (lh) and 24 (rh), and the Brodmann annotations are 5 (lh) and 15 (rh). We intersect the two corresponding annotations thus:
AnchorBar_sets.py --db annot.db --intersect 5 23
AnchorBar_sets.py --db annot.db --intersect 15 24

==Merging (Union) of Two Annotations==
Example: We have two tasks, A and B, and each task condition has been contrasted with rest (A vs Rest; B vs Rest). We wish to identify regions that are involved in either task, so we compute the union.
Unfortunately, it looks like I have to write this functionality!

AnchorBar

2022-08-17T21:37:16Z

Chris: /* Intersection of Two Annotations */

AnchorBar (aka "Analyzer") is a Python program developed to maintain and compute set operations on FreeSurfer annotation files. It's kind of like SPM's MarsBar plugin, but for FreeSurfer.
=Requirements=
AnchorBar has a few dependencies that might not be part of your base Python install:
*hashlib
*nibabel
Attempting to run the scripts will fail and report missing libraries. They can be installed using pip:
pip install nibabel
The utility is (currently) comprised of 3 scripts. Syntax help for each of these scripts can be obtained using the <code>-h</code> switch:
$ AnchorBar_tools.py -h
usage: AnchorBar_tools.py [-h] [--list] [--labels annot_id] [--db DB] [--rename annot_id new_shortname]
[--relabel annot_id label_id new_label_name] [--abbrev annot_id label_id abbrev_label_name]

=Querying Available Annotations=
AnchorBar_tools.py provides tools to list, rename and relabel annotations. Each annotation has a unique ID and descriptors indicating the hemisphere, a shortname used for naming labels generated from set operations, and a path indicating the file path of the source .annot file. The default set of annotations stored in '''annot.db''' includes the 10 pairs of lhrh annotations from fsaverage/label:
AnchorBar_tools.py --db annot.db --list

=Adding Annotations=
The most common use-case for adding annotations is when working with functional ROIs from group analyses. AnchorBar_init.py is the tool that imports new .annot files into the database for later operations. Imported annotations are associated with a unique identifier based on the labels and vertices that make up an annotation. This allows the scripts to recognize and ignore duplicate annotations that might have been given different names, but also to differentiate distinct annotations that happen to have the same name (as is common when working with functional ROIs).

The import process infers the hemisphere from the filename. Functional cluster annotations are unlikely to have an lh/rh prefix, and so will require renaming prior to importing (I like to stick them in the project copy of the fsaverage folder):
cp $SUBJECTS_DIR/RFX/myanalysis.lh/a_vs_b/glm.wls/osgm/cache.th30.abs.sig.ocn.annot $SUBJECTS_DIR/fsaverage/label/lh.a_vs_b.cache.th30.abs.annot
cp $SUBJECTS_DIR/RFX/myanalysis.rh/a_vs_b/glm.wls/osgm/cache.th30.abs.sig.ocn.annot $SUBJECTS_DIR/fsaverage/label/rh.a_vs_b.cache.th30.abs.annot

To add one or more new annotations (that you've renamed and saved to the fsaverage/label folder):
AnchorBar_init.py --db annot.db --annot $SUBJECTS_DIR/fsaverage/label/*.th30.abs.annot
Of course, the particular filename pattern in red in the above command may differ from what's shown here, depending on your file naming convention, but you get the idea. When this code executes, it will import all the files that end in "th30.abs.annot" that it finds. This is handy for importing annotations for multiple hemispheres and/or for multiple contrasts.

==Renaming Imported Annotations==
Imported annotations will be given a shortname that corresponds to the filename of the source annotation. These names can be unwieldy, so AnchorBar_tools.py allows you to assign a new shortname to stored annotations. First, list the annotations and note their shortnames and id values. Use the id when selecting the annotation for renaming. For example:
./AnchorBar_tools.py --db annot.db --list

*** Annotation List ***
id LR shortname path
1 lh aparc.a2005s /usr/local/freesurfer/subjects/fsaverage/label
2 lh aparc.a2009s /usr/local/freesurfer/subjects/fsaverage/label
...
25 lh a_v_b.th30.abs /home/projects/mystudy/fsaverage/label
26 rh a_v_b.th30.abs /home/projects/mystudy/fsaverage/label

If we wanted to remove the "th30.abs" part of the shortname for the annotations 25 and 26:
./AnchorBar_tools.py --db annot.db --rename 26 AvsB
./AnchorBar_tools.py --db annot.db --rename 25 AvsB

Now inspect the annotation database to see the shortname has been updated with the values you just provided:
./AnchorBar_tools.py --db annot.db --list

*** Annotation List ***
id LR shortname path
1 lh aparc.a2005s /usr/local/freesurfer/subjects/fsaverage/label
2 lh aparc.a2009s /usr/local/freesurfer/subjects/fsaverage/label
...
25 lh AvsB /home/projects/mystudy/fsaverage/label
26 rh AvsB /home/projects/mystudy/fsaverage/label

=Set Operations on Annotations=
==Intersection of Two Annotations==
Example: We have a functional contrast (A vs Rest) and wish to divide the large clusters along Brodmann Area boundaries. By intersecting the clusters (which only cover some of the brain surface) with one of the anatomical atlas annotations (which cover all of the brain surface), we assign a new label to each vertex according to which cluster and which anatomical label it corresponds to. Vertices not belonging to functional clusters will be dropped. The end result is a new annotation where the clusters are subdivided along anatomical boundaries.
We first list the annotations in the database to find the corresponding id numbers. A vs Rest are annotations 23 (lh) and 24 (rh), and the Brodmann annotations are 5 (lh) and 15 (rh). We intersect the two corresponding annotations thus:
AnchorBar_sets.py --db annot.db --intersect 5 23
AnchorBar_sets.py --db annot.db --intersect 15 24

==Merging (Union) of Two Annotations==
Example: We have two tasks, A and B, and each task condition has been contrasted with rest (A vs Rest; B vs Rest). We wish to identify regions that are involved in either task, so we compute the union.
Unfortunately, it looks like I have to write this functionality!

AnchorBar

2022-08-17T21:36:58Z

Chris:

AnchorBar (aka "Analyzer") is a Python program developed to maintain and compute set operations on FreeSurfer annotation files. It's kind of like SPM's MarsBar plugin, but for FreeSurfer.
=Requirements=
AnchorBar has a few dependencies that might not be part of your base Python install:
*hashlib
*nibabel
Attempting to run the scripts will fail and report missing libraries. They can be installed using pip:
pip install nibabel
The utility is (currently) comprised of 3 scripts. Syntax help for each of these scripts can be obtained using the <code>-h</code> switch:
$ AnchorBar_tools.py -h
usage: AnchorBar_tools.py [-h] [--list] [--labels annot_id] [--db DB] [--rename annot_id new_shortname]
[--relabel annot_id label_id new_label_name] [--abbrev annot_id label_id abbrev_label_name]

=Querying Available Annotations=
AnchorBar_tools.py provides tools to list, rename and relabel annotations. Each annotation has a unique ID and descriptors indicating the hemisphere, a shortname used for naming labels generated from set operations, and a path indicating the file path of the source .annot file. The default set of annotations stored in '''annot.db''' includes the 10 pairs of lhrh annotations from fsaverage/label:
AnchorBar_tools.py --db annot.db --list

=Adding Annotations=
The most common use-case for adding annotations is when working with functional ROIs from group analyses. AnchorBar_init.py is the tool that imports new .annot files into the database for later operations. Imported annotations are associated with a unique identifier based on the labels and vertices that make up an annotation. This allows the scripts to recognize and ignore duplicate annotations that might have been given different names, but also to differentiate distinct annotations that happen to have the same name (as is common when working with functional ROIs).

The import process infers the hemisphere from the filename. Functional cluster annotations are unlikely to have an lh/rh prefix, and so will require renaming prior to importing (I like to stick them in the project copy of the fsaverage folder):
cp $SUBJECTS_DIR/RFX/myanalysis.lh/a_vs_b/glm.wls/osgm/cache.th30.abs.sig.ocn.annot $SUBJECTS_DIR/fsaverage/label/lh.a_vs_b.cache.th30.abs.annot
cp $SUBJECTS_DIR/RFX/myanalysis.rh/a_vs_b/glm.wls/osgm/cache.th30.abs.sig.ocn.annot $SUBJECTS_DIR/fsaverage/label/rh.a_vs_b.cache.th30.abs.annot

To add one or more new annotations (that you've renamed and saved to the fsaverage/label folder):
AnchorBar_init.py --db annot.db --annot $SUBJECTS_DIR/fsaverage/label/*.th30.abs.annot
Of course, the particular filename pattern in red in the above command may differ from what's shown here, depending on your file naming convention, but you get the idea. When this code executes, it will import all the files that end in "th30.abs.annot" that it finds. This is handy for importing annotations for multiple hemispheres and/or for multiple contrasts.

==Renaming Imported Annotations==
Imported annotations will be given a shortname that corresponds to the filename of the source annotation. These names can be unwieldy, so AnchorBar_tools.py allows you to assign a new shortname to stored annotations. First, list the annotations and note their shortnames and id values. Use the id when selecting the annotation for renaming. For example:
./AnchorBar_tools.py --db annot.db --list

*** Annotation List ***
id LR shortname path
1 lh aparc.a2005s /usr/local/freesurfer/subjects/fsaverage/label
2 lh aparc.a2009s /usr/local/freesurfer/subjects/fsaverage/label
...
25 lh a_v_b.th30.abs /home/projects/mystudy/fsaverage/label
26 rh a_v_b.th30.abs /home/projects/mystudy/fsaverage/label

If we wanted to remove the "th30.abs" part of the shortname for the annotations 25 and 26:
./AnchorBar_tools.py --db annot.db --rename 26 AvsB
./AnchorBar_tools.py --db annot.db --rename 25 AvsB

Now inspect the annotation database to see the shortname has been updated with the values you just provided:
./AnchorBar_tools.py --db annot.db --list

*** Annotation List ***
id LR shortname path
1 lh aparc.a2005s /usr/local/freesurfer/subjects/fsaverage/label
2 lh aparc.a2009s /usr/local/freesurfer/subjects/fsaverage/label
...
25 lh AvsB /home/projects/mystudy/fsaverage/label
26 rh AvsB /home/projects/mystudy/fsaverage/label

=Set Operations on Annotations=
==Intersection of Two Annotations==
Example: We have a functional contrast (A vs Rest) and wish to divide the large clusters along Brodmann Area boundaries. By intersecting the clusters (which only cover some of the brain surface) with one of the anatomical atlas annotations (which cover all of the brain surface), we assign a new label to each vertex according to which cluster and which anatomical label it corresponds to. Vertices not belonging to functional clusters will be dropped. The end result is a new annotation where the clusters are subdivided along anatomical boundaries.
We first list the annotations in the database to find the corresponding id numbers. A vs Rest are annotations 23 (lh) and 24 (rh), and the Brodmann annotations are 5 (lh) and 15 (rh). We intersect the two corresponding annotations thus:
AnchorBar_sets.py --db annot.db --intersect 5 23
AnchorBar_sets.py --db annot.db --intersect 15 24

==Merging (Union) of Two Annotations==
Example: We have two tasks, A and B, and each task condition has been contrasted with rest (A vs Rest; B vs Rest). We wish to identify regions that are involved in either task, so we compute the union.
Unfortunately, it looks like I have to write this functionality!

AnchorBar

2022-08-17T21:19:30Z

Chris: /* Requirements */

AnchorBar

2022-08-17T21:14:05Z

Chris: /* Renaming Imported Annotations */

AnchorBar

2022-08-17T21:12:45Z

Chris: /* Adding Annotations */

AnchorBar

2022-08-17T20:53:08Z

Chris: /* Adding Annotations */

AnchorBar

2022-08-17T20:46:02Z

Chris:

AnchorBar

2022-08-17T20:29:28Z

Chris: Created page with "AnchorBar (aka "Analyzer") is a Python program developed to maintain and compute set operations on FreeSurfer annotation files. It's kind of like SPM's MarsBar plugin, but for FreeSurfer. =Requirements= AnchorBar has a few dependencies that must be met: #argparse #sqlite3 #hashlib #nibabel #random #numpy Attempting to run the scripts will fail and report missing libraries. They can be installed using pip: pip install nibabel The utility is (currently) comprised of 3 scr..."

Working with ROIs (Freesurfer)

2022-08-17T20:13:04Z

Chris: /* AnchorBar */

This page seriously needs to be reorganized. It documents some procedures that were developed before we figured out better ways to do some things, and so it may be more confusing than helpful.

=Working From Group-Level Results=
Group-level contrast maps, generated by [[Mri_glmfit]] are well-suited for generating a set of ROIs that are appropriate for use across the set of participants entered into the group-level analysis. They have the useful property of being spatially consistent across all participants. There are a couple things to note, however. The first is that there must first be a sufficient number of participants to produce statistically significant contrast clusters. The second is that the group-level analysis must be done in fsaverage space. If the data you wish to explore is in native '''self''' space, you would need to map the ROIs back to a subject's native surface space. Note that this is not a strong consideration, given that you would have had to have preprocessed and analyzed individuals in fsaverage space in order to produce group-level contrast maps, so I cannot imagine a situation that would absolutely ''require'' you to map ROIs to native space.

=Working From Existing Labels=
The other primary source of ROIs come from anatomical labels provided in the <code>fsaverage/label/</code> folder. Because the folder in the FreeSurfer install directory is owned by root, you won't be able to add new labels to this folder. Instead, you should make a copy of the fsaverage folder in your project directory:
cp -r $FREESURFER_HOME/subjects/fsaverage $SUBJECTS_DIR

=AnchorBar=
All the procedures and code lower down on the page have been replaced with a handy Python tool called [[AnchorBar]] (aka "Analyzer"), which maintains a sqlite3 database of annotations in template space. The program allows the user to add new annotations to the database and compute set operations on the stored annotations to create new annotation schemes; for example dividing functional clusters along anatomical boundaries.

==Adding Annotations to the AnchorBar database==
The initial AnchorBar annotation database is called annot.db, and has been populated with standard FreeSurfer annotations.

See also: [[Working with Subcortical ROIs (Freesurfer)]]

= Old Information =
The information below this point is accurate, but is largely obsolete. I couldn't bring myself to just delete it all because you never know when it'll be handy to know some of this stuff.

The .label files created during autorecon3, or by saving an overlay are plaintext lists of vertices. It should be straightforward to find the intersection or union of the vertices in two or more .label files using common Unix shell commands.

== .label Syntax ==
Below is the first few lines of the lh.banksts.label file for a participant. This file was produced during the Lausanne parcellation procedure.
#!ascii label , from subject FS_0043 vox2ras=TkReg
1457
32992 -47.867 -62.891 33.007 0.0000000000
32993 -48.881 -62.907 32.963 0.0000000000
34014 -47.298 -61.696 33.538 0.0000000000
34015 -47.972 -61.259 33.648 0.0000000000
34024 -47.634 -62.264 33.251 0.0000000000
34025 -48.286 -61.775 33.303 0.0000000000
...etc

The first line is a header line. The second line indicates the number of vertices in the label file. The remaining lines indicate the vertex number, x,y,z coordinates of the vertex, and I have no idea what that last column indicates but it's not terribly important for our purposes.

==Working From Individual-Level Results==
Running selxavg3-sess will produce some number of statistical contrast maps that can be loaded as an overlay in tksurfer. Significant clusters can be turned into ROIs for that particular individual in a series of steps. This has the advantage of being applicable to a single dataset -- there is no need to wait until sufficient fMRI data has been collected to produce significant group-level contrasts. The downside is that this is process has many steps.

=== Load Overlay ===
The first step is to load up the participant's surface data and the desired overlay. Care must be taken in selecting a contrast overlay that is appropriate for the type of analysis that you eventually plan to do. As a rule of thumb, the overlay contrast should be orthogonal (i.e., completely independent) of the contrast that you are ultimately interested in. For example, if your goal is to compare mean signal strength for low- vs. high-familiar trials, it would be ''inappropriate'' to use the contrast of high-familiar.vs.low-familiar to generate ROIs from which to calculate the mean signal strength. The reason for this is that this contrast will be selecting voxels that have already been found to demonstrate a significant difference. Naturally, any analysis of familiarity effects on these voxels will be ''biased''! A more appropriate contrast to use would be something like task.vs.rest, which isn't biased towards either high or low-familiarity items, but is instead just showing the voxels that were more highly active during the task trials (which presumably contain equal numbers of high and low familiarity items).

After the overlay is loaded, you may want to play around with the [https://surfer.nmr.mgh.harvard.edu/fswiki/TkSurferGuide/TkSurferWorkingWithData/TkSurferOverlay overlay configuration parameters]: for example you might choose to ''truncate'' your results to only show positive t-values, or modify your significance threshold.

===Save Clusters as Labels===
Now comes the somewhat tedious part of individually converting each cluster into a .label file. This simple but repetitive process is described [https://surfer.nmr.mgh.harvard.edu/fswiki/CreatingROIs here] and you can get pretty quick at it:
#If you wish to subdivide a large cluster, first create one or more paths that completely cross (e.g., bisect) the cluster.
#Use the cursor to select a point anywhere in the desired cluster
#Click the ''Custom Fill'' button. A new window will popup:
#*Choose '''Up to functional values below threshold'''
#*Optionally choose '''Up to and including paths''' if had subdivided the cluster. You will have to repeat these steps on the remainder of the cluster
#*The region will be colored in white before reverting to a yellow outlined region
#On the Tools window, go to '''File > Label > Save Selected Label''
#*A box will open with a path to save the ROI in the subject's label directory by default
#*Give it a meaningful name that includes the hemisphere and uses the .label filename extension (e.g. lh.TASK_001.label)
#*Click OK

=== Merge .label Files ===
If you wish to work with the resulting .label files individually, your work is done. However, if you wish to merge multiple clusters into a single .annot file, you have to first create a color table which is called by <code>mris_label2annot</code>.
=== Create a CTAB File ===
Existing instructions for ctab files are a little hard to follow, so I'll do my best. The CTAB files are plaintext files with 6 columns and 1 row for each label. The structure of the columns is as follows:
{|
|Index
|Label
|R
|G
|B
|A
|}

You might find it easiest if left and right hemispheres have their own CTAB files. These two files can be identical if you want, but, having two files allows you to have different numbers of labels associated with each hemispheres without encountering problems in subsequent steps. The indices for each hemisphere should be numbered independently. For example, if there are 11 labels in the lh and 8 in the rh, then one ctab file should have indices 1 to 11 and the other should have indices 1 to 8.

The label names should correspond to the ?h.*.label files you created. For example, suppose you created the following 5 label files in the previous step:
lh.TASK_001.label
lh.TASK_002.label
rh.TASK_003.label
rh.TASK_004.label
rh.TASK_005.label
The lh ctab file should contain the following labels:
0 unknown 128 128 128 0
1 TASK_001 110 112 100 0
2 TASK_002 128 140 130 0
and the rh ctab file should contain the following labels:
0 unknown 128 128 128 0
1 TASK_003 110 112 100 0
2 TASK_004 190 140 105 0
3 TASK_005 200 150 100 0

Note that as explained above, the ctab indices (column 1) were independent. Sets of RGB values can be generated according to your whims, or you can use an external website to produce a set of n distinguishable colors. For example, I went to http://tools.medialab.sciences-po.fr/iwanthue/ to generate 34 distinct colors. Note that the final column, A, refers to the alpha transparency value, and is always set to 0 as far as I can tell.

===Call mris_label2annot===
The last step uses the information in the ctab files to generate the .annot files from the appropriate sets of .label files. The syntax is:
mris_label2annot --s ${SUBJECT_ID} \
--h [lh | rh] \
--ctab [lh | rh].${CTAB_FILENAME} \
--a ${PREFIX} \
--ldir ${SUBJECTS_DIR}/${SUBJECT_ID}/label
This will parse the indicated ctab file to identify the labels indicated therein. The identified labels will then be assigned the colors indicated in that ctab file and written to the .annot file specified by the ${PREFIX} associated with the --a argument. It is assumed that all the .label files you created earlier were saved to ${SUBJECTS_DIR}/${SUBJECT_ID}/label. If this is different, you should specify the actual file location with the --ldir argument.

Note that the above command is the syntax used when you have an ''unknown'' label and corresponding entry in your CTAB file. If you do not wish to include the unknown label, rather than assign index 0 to the unknown label, start your CTAB index numbering at 0 for the first real label, and use the <code>--no-unknown</code> command-line argument when calling <code>mris_label2annot</code>. I didn't remember this being a problem the first time I tried this, but when I just went through this process again, it seemed that the CTAB index needed to start at 0, but I couldn't have index 0 associated with unknown if there was not an ''?h.unknown.label'' file, even when using the --no-unknown switch. When I edited the ctab files to have index 0, 1, ... , n-1 corresponding with label_001, label_002, ... , label_n, then everything seemed to work fine.

==Preparation of .label Files==
The group analysis used the fsaverage surface, and since mris_divide_parcellation (which you may also be using, or use later) assumes we're subdividing a particular subject's annotation, it makes sense to copy the source .annot files into the fsaverage label/ folder (giving the appropriate ?h prefix). The group analysis annot file will be found in the contrast directory for a particular analysis. The example below describes working with the .annot file for the TASK contrast in the group analysis, found under '''$SUBJECTS_DIR/RFX'''
HEMI=rh
cd $SUBJECTS_DIR/RFX/analysis.${HEMI}/task/glm.wls/osgm
ANNOT_FILE=cache.th30.abs.sign.ocn.annot
cp ${ANNOT_FILE} $SUBJECTS_DIR\fsaverage\label\${HEMI}.${ANNOT_FILE}

Note that fsaverage may not be writable if it is a symbolic link to $FREESURFER_DIR/subjects/fsaverage. If this is the case, you will have to make a copy of fsaverage that you own:
#first unlink the symbolic link to fsaverage
cd $SUBJECTS_DIR
unlink fsaverage
#now make a new empty fsaverage directory. You will own it and so should be able to read/write anything it contains
mkdir fsaverage
#finally, copy everything in the ''real'' fsaverage directory to the directory that you own:
cp -r $FREESURFER_DIR/subjects/fsaverage/* fsaverage

===Breaking Up .annot into multiple .label Files ===
The steps described below do not work on .annot files because they are not plaintext and easily manipulated. However, [https://surfer.nmr.mgh.harvard.edu/fswiki/mri_annotation2label mri_annotation2label] will convert a .annot file to a series of .label files:
BASE_LABEL_NAME=TD_W
ANNOT=lh.cache.th30.abs.sign.ocn.annot
HEMI=lh
SURF=orig
mri_annotation2label --annotation ${ANNOT} --hemi ${HEMI} --subject fsaverage --surf ${SURF} --labelbase ${HEMI}.${BASE_LABEL_NAME}
Alternatively, you might want to dump all your .label files into a single output directory. Instead of supplying a labelbase, you specify an output dir (''outdir''):
mri_annotation2label --subject fsaverage --hemi lh --outdir LDT.lh --annotation lh.ldt_hi.annot
This command created a new subdirectory (LDT.lh), and broke up the ld.ldt_hi.annot into its constituent label files, placing them in the subdirectory.

'''Note: when working with functional contrast map annotation files, the output files will be called ?h.cluster-nnn.label. If you convert multiple .annot files for the same hemisphere (e.g., from two separate contrasts) with the intention of later merging them, you will run the risk of overwriting earlier .label files unless you take care to rename them as you go.'''

=Set Operations on .label Files=
==Intersecting .label Files (Intersection)==
Given what we know about the contents of a label file, the unix <code>comm</code> command should suffice to find the common entries between two label files, which is the set intersection of vertices appearing in each.

[https://www.mail-archive.com/freesurfer@nmr.mgh.harvard.edu/msg11344.html Relevant thread here]

==Merging .label Files (Union)==
A union operation will find all vertices appearing in ''either'' .label file. You will want to make sure that only unique values are retained (i.e., you don't want to list the same vertex multiple times in the output .label file).

The easiest way to accomplish this is FreeSurfer's <code>mri_mergelabels</code> tool:
mri_mergelabels -i label1 -i label2 ... -o outputlabel
or
mri_mergelabels -d <dirname> -o outputlabel
This FreeSurfer tool merges two or more label files. It does this by catting the label files together (after removing the first two lines). It inserts a new header (the first two lines). The number of entries in the new file (ie, the number on the second line), is computed by summing those from the input files. When you pass a directory name with the -d flag, it will merge all labels in a directory. This is handy when you want to merge many labels created from multiple .annot files.

For example, if I have followed the steps above to create a series of ?h.CONTRAST_A-0??.label and ?h.CONTRAST_B-0??.label files, then I can find the union of CONTRAST_A and CONTRAST_B by creating a lh/ and rh/ subdirectory and moving the label files to the appropriate directories. Then:
mri_mergelabels -d lh -o lh.AB_UNION.label
mri_mergelabels -d rh -o rh.AB_UNION.label
The above two commands will generate my lh and rh label files in the current working directory (assumed to be ${SUBJECTS_DIR}/${SUBJECT}/label; ${SUBJECT} is likely to be fsaverage if you are making a functional mask from group-level analyses in fsaverage space).
===You're Going to Want to Turn These into an .annot File===
After you've merged multiple .label files into a single .label file, all the vertices will be assigned to the same label. You will probably want to go through and relabel each of the distinct regions (e.g., by cluster) in FreeSurfer. After you've done that, save (export) each of the relabeled regions as separate .label files (e.g., CLUST_01, CLUST_02, ... etc.) in a working directory.

After you do, you need to create or snag a CLUT file.

Finally, you use mris_label2annot to merge your united labels, along with the CLUT into a single .annot file.

=ROI 'VennDiagram' Matlab Function=
ROIVennDiagram.m uses intersect and setdiff functions to extract intersecting and unique vertices. Variable names suggest that it's used for T1 and T2 data, but you can use it for any paired data. The label files generated above (using mri_annotation2label) need to be copied into a lh and rh directory within a parent dir. A simple modification would allow these files to remain in your fsaverage or other subject's dir, if you wanted to clutter those up.

function ROIvenndiagram = ROIvenndiagram(hemi)
%% function ROIvenndiagram = ROIvenndiagram('hemi');
%Pull out intersecting and different vertices for FS label files,
%and store in one of 3 text files (T1, T1T2, T2)
%
%Currently takes one string input indicating lh or rh.
%
%You'll need to adjust the file names on lines 42 and 43 to match your
%file names.
%
%Note - The headers in the output text files have quotes around them, as
%the headers begin with matlab unfriendly characters (#). So, you'll need to
%go into those 3 output text files and delete the quotes.
%
%Intended for use in parent dir containing lh and rh dirs, which
%contain label files.
% _____________________________________
% G.J. Smith
% gjsmith4@buffalo.edu
% _____________________________________
parentdir=pwd;
hemidir = sprintf('%s/%s', pwd, hemi);
cd(hemidir);

%% Pre-initialize array for cating labels
T1Agg = cell(1,5);
T2Agg = cell(1,5);

% Begin cycle through label files
for labeln = 1:3000 %arbirary number higher than any label number
%% Apply appropriate number of 0's before number - eg. 001, 010, 100
numcorrection=num2str(labeln);
if labeln < 10
labelnum=strcat('00',numcorrection);
elseif labeln < 100
labelnum=strcat('0',numcorrection);
else
labelnum=strcat(numcorrection);
end

%% T1 and T2 label file names -- Edit these to match your file naming convention
T1_filename = sprintf('%s.T1-%s.label', hemi, labelnum);
T2_filename = sprintf('%s.T2-%s.label', hemi, labelnum);

%% Build one large aggregate file with all vertices from all label files
header = 2; % lines 1 and 2 of label file stored seperately
delimiter = ' ';
if exist([pwd filesep T1_filename], 'file')
%load data
T1file = importdata(T1_filename,delimiter,header);
%Store in one array
T1Agg = vertcat(T1Agg, num2cell(T1file.data));
end
if exist([pwd filesep T2_filename], 'file')
%load data
T2file = importdata(T2_filename,delimiter,header);
%Store in one array
T2Agg = vertcat(T2Agg, num2cell(T2file.data));
end
end

%% Check that label files were found
exist T1file;
if ans == 1
display('Labels found, pulling out intersects and setdiffs');
else
display('No label files found, check your dir or file names');
return;
end

%% Find intersection and differences
T1Agg = cell2mat(T1Agg);
T2Agg = cell2mat(T2Agg);
%compare
%union = union(T1file.data(:,1), T2file.data(:,1));
intersection = intersect(T1Agg, T2Agg, 'rows'); %'rows' works with same # of columns
T1_diffs = setdiff(T1Agg, T2Agg, 'rows');
T2_diffs = setdiff(T2Agg, T1Agg, 'rows');

%% Build new label files
%T1
T1_differences = cell(50000, 5); %hard coded arbitrarily high number of rows for pre-init
T1_differences(1,1) = T1file.textdata(1,1); % header
T1_differences{2,1} = length(T1_diffs); % header line 2, new number of rows
T1_differences(3:length(T1_diffs) + 2,:) = num2cell(T1_diffs); % data

%T2
T2_differences = cell(50000, 5); %hard coded arbitrarily high number of rows for pre-init
T2_differences(1,1) = T2file.textdata(1,1); % header
T2_differences{2,1} = length(T2_diffs); % header line 2, new number of rows
T2_differences(3:length(T2_diffs) + 2,:) = num2cell(T2_diffs); % data

%Match
Match_labelfile = cell(50000, 5); %hard coded arbitrarily high number of rows for pre-init
Match_labelfile(1,1) = T1file.textdata(1,1); % header T1 and T2 for same subject is the same
Match_labelfile{2,1} = length(intersection); % header line 2, new number of rows
Match_labelfile(3:length(intersection) + 2, :) = num2cell(intersection); % data

%% Remove blank rows from oversized pre-alo arrays
T1_differences(all(cellfun(@isempty,T1_differences),2),:) = [];
T2_differences(all(cellfun(@isempty,T2_differences),2),:) = [];
Match_labelfile(all(cellfun(@isempty,Match_labelfile),2),:) = [];

%% Write files
cd(parentdir);
T1_fname = sprintf('%s.T1.label.txt', hemi);
writetable(cell2table(T1_differences), T1_fname,'Delimiter',' ','WriteVariableNames',false);

%T2
T2_fname = sprintf('%s.T2.label.txt', hemi);
writetable(cell2table(T2_differences), T2_fname,'Delimiter',' ','WriteVariableNames',false);

%Match
Match_fname = sprintf('%s.T1T2.label.txt', hemi);
writetable(cell2table(Match_labelfile), Match_fname,'Delimiter',' ','WriteVariableNames',false);

end

==Visualize and Cluster (Intersected Label File)==
*in progress*
Open tksurfer.
tksurfer fsaverage ?h inflated

Import ?h.aparc.annot files for a reference

Load the intersected label file.

Use cut line to make cluster boundaries (If necessary, otherwise see below).

Erase aparc.annot labels that overlap with a desired label.

Select area you want to cluster and use custom fill to create new label. Up to and including path, up to other labels, and up to unlabeled, filling from last clicked vertex. If you left adjacent aparc.annot labels these can be used as boundaries.

Save each label file using save selected label. -- must premake text files as cant specify file names. -- must be a better way of doing this, or getting export annotation to work.

Pro tip: Colour your label file red, when creating new labels, they will be blue. (see below)

==CTAB and Annot==
Now you'll need to create a ctab file for the labels to make a .annot file.

If you don't have a ton of labels, this can easily be done by hand by copying and editing the 'FreeSurferColorLUT.txt' file found in /usr/local/freesurfer, and simply renaming the regions(label names) to match your labelling convention. Then save it as a .annot.ctab file, e.g. 'RH.annot.ctab'.

Then you'll want to use the mris_label2annot command found [https://surfer.nmr.mgh.harvard.edu/fswiki/mris_label2annot here]. There are 2 ways to do this. The first is to pass it each of the label files using the--l switch, and a ctab fie. Example:

mris_label2annot --l rh.label-001.txt --l rh.label-002.txt --l rh.label-003.txt --l rh.label-004.txt --l rh.label-005.txt --l rh.label-006.txt --l rh.label-007.txt --l rh.label-008.txt --l rh.label-009.txt --l rh.label-010.txt --l rh.label-011.txt --l rh.label-012.txt --l rh.label-013.txt --l rh.label-014.txt --l rh.label-015.txt --l rh.label-016.txt --l rh.label-017.txt --l rh.label-018.txt --l rh.label-019.txt --l rh.label-020.txt --l rh.label-021.txt --l rh.label-022.txt --l rh.label-023.txt --l rh.label-024.txt --l rh.label-025.txt --l rh.label-026.txt --ctab RH.annot.ctab --s fsaverage --h rh --annot T1T2intersected

That's unwieldy (and yet, that's how I first did this). The easier way is to throw all the label files into a single directory, and pass the directory and a ctab file.

Now that you have .annot files for your intersecting vertices, you can [http://ccnlab.psy.buffalo.edu/wiki/index.php/Freesurfer_Subparcellation#Transfer_Subparcellation_to_Other_Subjects, subparcellate] and/or continue with network analyses.

==Using R to Work With Colortables==
This is kind of a stub topic, but there is an R library when trying to extract a CLUT from an existing .annot file. I introduce the [https://rdrr.io/cran/freesurferformats/ freesurferformats R package]. In particular, there is a function to extract the color lookup table from a .annot file:
> library("freesurferformats")
> annotfile="/path/to/subject/label/lh.myannot.annot"
> annot = read.fs.annot(annotfile);
> colortable = colortable.from.annot(annot);
> head(colortable);

If you install this library, you can use [[Extracting CLUT from .annot Files Using R|this handy R script]] that I wrote that can be run from the terminal to extract a CLUT from a .annot file.

[[Category: FreeSurfer]]
[[Category: ROI]]

Working with ROIs (Freesurfer)

2022-08-17T20:08:21Z

Chris:

This page seriously needs to be reorganized. It documents some procedures that were developed before we figured out better ways to do some things, and so it may be more confusing than helpful.

=Working From Group-Level Results=
Group-level contrast maps, generated by [[Mri_glmfit]] are well-suited for generating a set of ROIs that are appropriate for use across the set of participants entered into the group-level analysis. They have the useful property of being spatially consistent across all participants. There are a couple things to note, however. The first is that there must first be a sufficient number of participants to produce statistically significant contrast clusters. The second is that the group-level analysis must be done in fsaverage space. If the data you wish to explore is in native '''self''' space, you would need to map the ROIs back to a subject's native surface space. Note that this is not a strong consideration, given that you would have had to have preprocessed and analyzed individuals in fsaverage space in order to produce group-level contrast maps, so I cannot imagine a situation that would absolutely ''require'' you to map ROIs to native space.

=Working From Existing Labels=
The other primary source of ROIs come from anatomical labels provided in the <code>fsaverage/label/</code> folder. Because the folder in the FreeSurfer install directory is owned by root, you won't be able to add new labels to this folder. Instead, you should make a copy of the fsaverage folder in your project directory:
cp -r $FREESURFER_HOME/subjects/fsaverage $SUBJECTS_DIR

=AnchorBar=
All the procedures and code lower down on the page have been replaced with a handy Python tool called AnchorBar (aka "Analyzer"), which maintains a sqlite3 database of annotations in template space. The program allows the user to add new annotations to the database and compute set operations on the stored annotations to create new annotation schemes; for example dividing functional clusters along anatomical boundaries.

==Adding Annotations to the AnchorBar database==
The initial AnchorBar annotation database is called annot.db, and has been populated with standard FreeSurfer annotations.

See also: [[Working with Subcortical ROIs (Freesurfer)]]
= Old Information =
The information below this point is accurate, but is largely obsolete. I couldn't bring myself to just delete it all because you never know when it'll be handy to know some of this stuff.

The .label files created during autorecon3, or by saving an overlay are plaintext lists of vertices. It should be straightforward to find the intersection or union of the vertices in two or more .label files using common Unix shell commands.

== .label Syntax ==
Below is the first few lines of the lh.banksts.label file for a participant. This file was produced during the Lausanne parcellation procedure.
#!ascii label , from subject FS_0043 vox2ras=TkReg
1457
32992 -47.867 -62.891 33.007 0.0000000000
32993 -48.881 -62.907 32.963 0.0000000000
34014 -47.298 -61.696 33.538 0.0000000000
34015 -47.972 -61.259 33.648 0.0000000000
34024 -47.634 -62.264 33.251 0.0000000000
34025 -48.286 -61.775 33.303 0.0000000000
...etc

The first line is a header line. The second line indicates the number of vertices in the label file. The remaining lines indicate the vertex number, x,y,z coordinates of the vertex, and I have no idea what that last column indicates but it's not terribly important for our purposes.

==Working From Individual-Level Results==
Running selxavg3-sess will produce some number of statistical contrast maps that can be loaded as an overlay in tksurfer. Significant clusters can be turned into ROIs for that particular individual in a series of steps. This has the advantage of being applicable to a single dataset -- there is no need to wait until sufficient fMRI data has been collected to produce significant group-level contrasts. The downside is that this is process has many steps.

=== Load Overlay ===
The first step is to load up the participant's surface data and the desired overlay. Care must be taken in selecting a contrast overlay that is appropriate for the type of analysis that you eventually plan to do. As a rule of thumb, the overlay contrast should be orthogonal (i.e., completely independent) of the contrast that you are ultimately interested in. For example, if your goal is to compare mean signal strength for low- vs. high-familiar trials, it would be ''inappropriate'' to use the contrast of high-familiar.vs.low-familiar to generate ROIs from which to calculate the mean signal strength. The reason for this is that this contrast will be selecting voxels that have already been found to demonstrate a significant difference. Naturally, any analysis of familiarity effects on these voxels will be ''biased''! A more appropriate contrast to use would be something like task.vs.rest, which isn't biased towards either high or low-familiarity items, but is instead just showing the voxels that were more highly active during the task trials (which presumably contain equal numbers of high and low familiarity items).

After the overlay is loaded, you may want to play around with the [https://surfer.nmr.mgh.harvard.edu/fswiki/TkSurferGuide/TkSurferWorkingWithData/TkSurferOverlay overlay configuration parameters]: for example you might choose to ''truncate'' your results to only show positive t-values, or modify your significance threshold.

===Save Clusters as Labels===
Now comes the somewhat tedious part of individually converting each cluster into a .label file. This simple but repetitive process is described [https://surfer.nmr.mgh.harvard.edu/fswiki/CreatingROIs here] and you can get pretty quick at it:
#If you wish to subdivide a large cluster, first create one or more paths that completely cross (e.g., bisect) the cluster.
#Use the cursor to select a point anywhere in the desired cluster
#Click the ''Custom Fill'' button. A new window will popup:
#*Choose '''Up to functional values below threshold'''
#*Optionally choose '''Up to and including paths''' if had subdivided the cluster. You will have to repeat these steps on the remainder of the cluster
#*The region will be colored in white before reverting to a yellow outlined region
#On the Tools window, go to '''File > Label > Save Selected Label''
#*A box will open with a path to save the ROI in the subject's label directory by default
#*Give it a meaningful name that includes the hemisphere and uses the .label filename extension (e.g. lh.TASK_001.label)
#*Click OK

=== Merge .label Files ===
If you wish to work with the resulting .label files individually, your work is done. However, if you wish to merge multiple clusters into a single .annot file, you have to first create a color table which is called by <code>mris_label2annot</code>.
=== Create a CTAB File ===
Existing instructions for ctab files are a little hard to follow, so I'll do my best. The CTAB files are plaintext files with 6 columns and 1 row for each label. The structure of the columns is as follows:
{|
|Index
|Label
|R
|G
|B
|A
|}

You might find it easiest if left and right hemispheres have their own CTAB files. These two files can be identical if you want, but, having two files allows you to have different numbers of labels associated with each hemispheres without encountering problems in subsequent steps. The indices for each hemisphere should be numbered independently. For example, if there are 11 labels in the lh and 8 in the rh, then one ctab file should have indices 1 to 11 and the other should have indices 1 to 8.

The label names should correspond to the ?h.*.label files you created. For example, suppose you created the following 5 label files in the previous step:
lh.TASK_001.label
lh.TASK_002.label
rh.TASK_003.label
rh.TASK_004.label
rh.TASK_005.label
The lh ctab file should contain the following labels:
0 unknown 128 128 128 0
1 TASK_001 110 112 100 0
2 TASK_002 128 140 130 0
and the rh ctab file should contain the following labels:
0 unknown 128 128 128 0
1 TASK_003 110 112 100 0
2 TASK_004 190 140 105 0
3 TASK_005 200 150 100 0

Note that as explained above, the ctab indices (column 1) were independent. Sets of RGB values can be generated according to your whims, or you can use an external website to produce a set of n distinguishable colors. For example, I went to http://tools.medialab.sciences-po.fr/iwanthue/ to generate 34 distinct colors. Note that the final column, A, refers to the alpha transparency value, and is always set to 0 as far as I can tell.

===Call mris_label2annot===
The last step uses the information in the ctab files to generate the .annot files from the appropriate sets of .label files. The syntax is:
mris_label2annot --s ${SUBJECT_ID} \
--h [lh | rh] \
--ctab [lh | rh].${CTAB_FILENAME} \
--a ${PREFIX} \
--ldir ${SUBJECTS_DIR}/${SUBJECT_ID}/label
This will parse the indicated ctab file to identify the labels indicated therein. The identified labels will then be assigned the colors indicated in that ctab file and written to the .annot file specified by the ${PREFIX} associated with the --a argument. It is assumed that all the .label files you created earlier were saved to ${SUBJECTS_DIR}/${SUBJECT_ID}/label. If this is different, you should specify the actual file location with the --ldir argument.

Note that the above command is the syntax used when you have an ''unknown'' label and corresponding entry in your CTAB file. If you do not wish to include the unknown label, rather than assign index 0 to the unknown label, start your CTAB index numbering at 0 for the first real label, and use the <code>--no-unknown</code> command-line argument when calling <code>mris_label2annot</code>. I didn't remember this being a problem the first time I tried this, but when I just went through this process again, it seemed that the CTAB index needed to start at 0, but I couldn't have index 0 associated with unknown if there was not an ''?h.unknown.label'' file, even when using the --no-unknown switch. When I edited the ctab files to have index 0, 1, ... , n-1 corresponding with label_001, label_002, ... , label_n, then everything seemed to work fine.

==Preparation of .label Files==
The group analysis used the fsaverage surface, and since mris_divide_parcellation (which you may also be using, or use later) assumes we're subdividing a particular subject's annotation, it makes sense to copy the source .annot files into the fsaverage label/ folder (giving the appropriate ?h prefix). The group analysis annot file will be found in the contrast directory for a particular analysis. The example below describes working with the .annot file for the TASK contrast in the group analysis, found under '''$SUBJECTS_DIR/RFX'''
HEMI=rh
cd $SUBJECTS_DIR/RFX/analysis.${HEMI}/task/glm.wls/osgm
ANNOT_FILE=cache.th30.abs.sign.ocn.annot
cp ${ANNOT_FILE} $SUBJECTS_DIR\fsaverage\label\${HEMI}.${ANNOT_FILE}

Note that fsaverage may not be writable if it is a symbolic link to $FREESURFER_DIR/subjects/fsaverage. If this is the case, you will have to make a copy of fsaverage that you own:
#first unlink the symbolic link to fsaverage
cd $SUBJECTS_DIR
unlink fsaverage
#now make a new empty fsaverage directory. You will own it and so should be able to read/write anything it contains
mkdir fsaverage
#finally, copy everything in the ''real'' fsaverage directory to the directory that you own:
cp -r $FREESURFER_DIR/subjects/fsaverage/* fsaverage

===Breaking Up .annot into multiple .label Files ===
The steps described below do not work on .annot files because they are not plaintext and easily manipulated. However, [https://surfer.nmr.mgh.harvard.edu/fswiki/mri_annotation2label mri_annotation2label] will convert a .annot file to a series of .label files:
BASE_LABEL_NAME=TD_W
ANNOT=lh.cache.th30.abs.sign.ocn.annot
HEMI=lh
SURF=orig
mri_annotation2label --annotation ${ANNOT} --hemi ${HEMI} --subject fsaverage --surf ${SURF} --labelbase ${HEMI}.${BASE_LABEL_NAME}
Alternatively, you might want to dump all your .label files into a single output directory. Instead of supplying a labelbase, you specify an output dir (''outdir''):
mri_annotation2label --subject fsaverage --hemi lh --outdir LDT.lh --annotation lh.ldt_hi.annot
This command created a new subdirectory (LDT.lh), and broke up the ld.ldt_hi.annot into its constituent label files, placing them in the subdirectory.

'''Note: when working with functional contrast map annotation files, the output files will be called ?h.cluster-nnn.label. If you convert multiple .annot files for the same hemisphere (e.g., from two separate contrasts) with the intention of later merging them, you will run the risk of overwriting earlier .label files unless you take care to rename them as you go.'''

=Set Operations on .label Files=
==Intersecting .label Files (Intersection)==
Given what we know about the contents of a label file, the unix <code>comm</code> command should suffice to find the common entries between two label files, which is the set intersection of vertices appearing in each.

[https://www.mail-archive.com/freesurfer@nmr.mgh.harvard.edu/msg11344.html Relevant thread here]

==Merging .label Files (Union)==
A union operation will find all vertices appearing in ''either'' .label file. You will want to make sure that only unique values are retained (i.e., you don't want to list the same vertex multiple times in the output .label file).

The easiest way to accomplish this is FreeSurfer's <code>mri_mergelabels</code> tool:
mri_mergelabels -i label1 -i label2 ... -o outputlabel
or
mri_mergelabels -d <dirname> -o outputlabel
This FreeSurfer tool merges two or more label files. It does this by catting the label files together (after removing the first two lines). It inserts a new header (the first two lines). The number of entries in the new file (ie, the number on the second line), is computed by summing those from the input files. When you pass a directory name with the -d flag, it will merge all labels in a directory. This is handy when you want to merge many labels created from multiple .annot files.

For example, if I have followed the steps above to create a series of ?h.CONTRAST_A-0??.label and ?h.CONTRAST_B-0??.label files, then I can find the union of CONTRAST_A and CONTRAST_B by creating a lh/ and rh/ subdirectory and moving the label files to the appropriate directories. Then:
mri_mergelabels -d lh -o lh.AB_UNION.label
mri_mergelabels -d rh -o rh.AB_UNION.label
The above two commands will generate my lh and rh label files in the current working directory (assumed to be ${SUBJECTS_DIR}/${SUBJECT}/label; ${SUBJECT} is likely to be fsaverage if you are making a functional mask from group-level analyses in fsaverage space).
===You're Going to Want to Turn These into an .annot File===
After you've merged multiple .label files into a single .label file, all the vertices will be assigned to the same label. You will probably want to go through and relabel each of the distinct regions (e.g., by cluster) in FreeSurfer. After you've done that, save (export) each of the relabeled regions as separate .label files (e.g., CLUST_01, CLUST_02, ... etc.) in a working directory.

After you do, you need to create or snag a CLUT file.

Finally, you use mris_label2annot to merge your united labels, along with the CLUT into a single .annot file.

=ROI 'VennDiagram' Matlab Function=
ROIVennDiagram.m uses intersect and setdiff functions to extract intersecting and unique vertices. Variable names suggest that it's used for T1 and T2 data, but you can use it for any paired data. The label files generated above (using mri_annotation2label) need to be copied into a lh and rh directory within a parent dir. A simple modification would allow these files to remain in your fsaverage or other subject's dir, if you wanted to clutter those up.

function ROIvenndiagram = ROIvenndiagram(hemi)
%% function ROIvenndiagram = ROIvenndiagram('hemi');
%Pull out intersecting and different vertices for FS label files,
%and store in one of 3 text files (T1, T1T2, T2)
%
%Currently takes one string input indicating lh or rh.
%
%You'll need to adjust the file names on lines 42 and 43 to match your
%file names.
%
%Note - The headers in the output text files have quotes around them, as
%the headers begin with matlab unfriendly characters (#). So, you'll need to
%go into those 3 output text files and delete the quotes.
%
%Intended for use in parent dir containing lh and rh dirs, which
%contain label files.
% _____________________________________
% G.J. Smith
% gjsmith4@buffalo.edu
% _____________________________________
parentdir=pwd;
hemidir = sprintf('%s/%s', pwd, hemi);
cd(hemidir);

%% Pre-initialize array for cating labels
T1Agg = cell(1,5);
T2Agg = cell(1,5);

% Begin cycle through label files
for labeln = 1:3000 %arbirary number higher than any label number
%% Apply appropriate number of 0's before number - eg. 001, 010, 100
numcorrection=num2str(labeln);
if labeln < 10
labelnum=strcat('00',numcorrection);
elseif labeln < 100
labelnum=strcat('0',numcorrection);
else
labelnum=strcat(numcorrection);
end

%% T1 and T2 label file names -- Edit these to match your file naming convention
T1_filename = sprintf('%s.T1-%s.label', hemi, labelnum);
T2_filename = sprintf('%s.T2-%s.label', hemi, labelnum);

%% Build one large aggregate file with all vertices from all label files
header = 2; % lines 1 and 2 of label file stored seperately
delimiter = ' ';
if exist([pwd filesep T1_filename], 'file')
%load data
T1file = importdata(T1_filename,delimiter,header);
%Store in one array
T1Agg = vertcat(T1Agg, num2cell(T1file.data));
end
if exist([pwd filesep T2_filename], 'file')
%load data
T2file = importdata(T2_filename,delimiter,header);
%Store in one array
T2Agg = vertcat(T2Agg, num2cell(T2file.data));
end
end

%% Check that label files were found
exist T1file;
if ans == 1
display('Labels found, pulling out intersects and setdiffs');
else
display('No label files found, check your dir or file names');
return;
end

%% Find intersection and differences
T1Agg = cell2mat(T1Agg);
T2Agg = cell2mat(T2Agg);
%compare
%union = union(T1file.data(:,1), T2file.data(:,1));
intersection = intersect(T1Agg, T2Agg, 'rows'); %'rows' works with same # of columns
T1_diffs = setdiff(T1Agg, T2Agg, 'rows');
T2_diffs = setdiff(T2Agg, T1Agg, 'rows');

%% Build new label files
%T1
T1_differences = cell(50000, 5); %hard coded arbitrarily high number of rows for pre-init
T1_differences(1,1) = T1file.textdata(1,1); % header
T1_differences{2,1} = length(T1_diffs); % header line 2, new number of rows
T1_differences(3:length(T1_diffs) + 2,:) = num2cell(T1_diffs); % data

%T2
T2_differences = cell(50000, 5); %hard coded arbitrarily high number of rows for pre-init
T2_differences(1,1) = T2file.textdata(1,1); % header
T2_differences{2,1} = length(T2_diffs); % header line 2, new number of rows
T2_differences(3:length(T2_diffs) + 2,:) = num2cell(T2_diffs); % data

%Match
Match_labelfile = cell(50000, 5); %hard coded arbitrarily high number of rows for pre-init
Match_labelfile(1,1) = T1file.textdata(1,1); % header T1 and T2 for same subject is the same
Match_labelfile{2,1} = length(intersection); % header line 2, new number of rows
Match_labelfile(3:length(intersection) + 2, :) = num2cell(intersection); % data

%% Remove blank rows from oversized pre-alo arrays
T1_differences(all(cellfun(@isempty,T1_differences),2),:) = [];
T2_differences(all(cellfun(@isempty,T2_differences),2),:) = [];
Match_labelfile(all(cellfun(@isempty,Match_labelfile),2),:) = [];

%% Write files
cd(parentdir);
T1_fname = sprintf('%s.T1.label.txt', hemi);
writetable(cell2table(T1_differences), T1_fname,'Delimiter',' ','WriteVariableNames',false);

%T2
T2_fname = sprintf('%s.T2.label.txt', hemi);
writetable(cell2table(T2_differences), T2_fname,'Delimiter',' ','WriteVariableNames',false);

%Match
Match_fname = sprintf('%s.T1T2.label.txt', hemi);
writetable(cell2table(Match_labelfile), Match_fname,'Delimiter',' ','WriteVariableNames',false);

end

==Visualize and Cluster (Intersected Label File)==
*in progress*
Open tksurfer.
tksurfer fsaverage ?h inflated

Import ?h.aparc.annot files for a reference

Load the intersected label file.

Use cut line to make cluster boundaries (If necessary, otherwise see below).

Erase aparc.annot labels that overlap with a desired label.

Select area you want to cluster and use custom fill to create new label. Up to and including path, up to other labels, and up to unlabeled, filling from last clicked vertex. If you left adjacent aparc.annot labels these can be used as boundaries.

Save each label file using save selected label. -- must premake text files as cant specify file names. -- must be a better way of doing this, or getting export annotation to work.

Pro tip: Colour your label file red, when creating new labels, they will be blue. (see below)

==CTAB and Annot==
Now you'll need to create a ctab file for the labels to make a .annot file.

If you don't have a ton of labels, this can easily be done by hand by copying and editing the 'FreeSurferColorLUT.txt' file found in /usr/local/freesurfer, and simply renaming the regions(label names) to match your labelling convention. Then save it as a .annot.ctab file, e.g. 'RH.annot.ctab'.

Then you'll want to use the mris_label2annot command found [https://surfer.nmr.mgh.harvard.edu/fswiki/mris_label2annot here]. There are 2 ways to do this. The first is to pass it each of the label files using the--l switch, and a ctab fie. Example:

mris_label2annot --l rh.label-001.txt --l rh.label-002.txt --l rh.label-003.txt --l rh.label-004.txt --l rh.label-005.txt --l rh.label-006.txt --l rh.label-007.txt --l rh.label-008.txt --l rh.label-009.txt --l rh.label-010.txt --l rh.label-011.txt --l rh.label-012.txt --l rh.label-013.txt --l rh.label-014.txt --l rh.label-015.txt --l rh.label-016.txt --l rh.label-017.txt --l rh.label-018.txt --l rh.label-019.txt --l rh.label-020.txt --l rh.label-021.txt --l rh.label-022.txt --l rh.label-023.txt --l rh.label-024.txt --l rh.label-025.txt --l rh.label-026.txt --ctab RH.annot.ctab --s fsaverage --h rh --annot T1T2intersected

That's unwieldy (and yet, that's how I first did this). The easier way is to throw all the label files into a single directory, and pass the directory and a ctab file.

Now that you have .annot files for your intersecting vertices, you can [http://ccnlab.psy.buffalo.edu/wiki/index.php/Freesurfer_Subparcellation#Transfer_Subparcellation_to_Other_Subjects, subparcellate] and/or continue with network analyses.

==Using R to Work With Colortables==
This is kind of a stub topic, but there is an R library when trying to extract a CLUT from an existing .annot file. I introduce the [https://rdrr.io/cran/freesurferformats/ freesurferformats R package]. In particular, there is a function to extract the color lookup table from a .annot file:
> library("freesurferformats")
> annotfile="/path/to/subject/label/lh.myannot.annot"
> annot = read.fs.annot(annotfile);
> colortable = colortable.from.annot(annot);
> head(colortable);

If you install this library, you can use [[Extracting CLUT from .annot Files Using R|this handy R script]] that I wrote that can be run from the terminal to extract a CLUT from a .annot file.

[[Category: FreeSurfer]]
[[Category: ROI]]

ML Environment

2022-07-26T20:44:15Z

Chris: /* Using the CCN Lab ML Environment */

The lab primarily uses a Linux environment. We have several workstations running the latest version of [https://ubuntu.com/download Ubuntu]. Our machine learning (ML) research is primarily carried out using Google's [https://www.tensorflow.org/ TensorFlow] libraries, written for Python. Workstations with powerful GPUs can use CUDA for GPU acceleration.
=Using the CCN Lab ML Environment=
To avoid clashes between different releases of Python, Tensorflow and whatever libraries are required for a particular ML application, we'll be using Virtual Environments. We have a Python 2.9 virtual environment, which is like a sandbox that all our ML libraries can be installed into and toggled on/off without disrupting anything else. '''There is a one-time setup procedure every individual user will have to complete:'''

#ssh or open a terminal on ws03
#run the following command: <code>conda init bash</code>
#log out, log back in. Now your shell prompt will look like: <code>(base) user@host:~$</code>
#*To toggle it on: <code>conda activate /data/tf</code>
#*To toggle it off: <code>conda deactivate</code>
#*To list all other environments that you can activate <code>conda info --envs</code>
#*To list all the packages installed in your current environment: <code>conda list</code>
#If <code>'''conda list'''</code> output doesn't include a bunch of tensorflow libraries, install it using pip: <code>pip install tensorflow</code>
#*Other libraries may have to be installed separately by each user using pip. For example: <code>pip install sklearn</code>

=Setting up your own ML environment=
Below are the specs and walkthrough for setting up a modest ML programming environment.

==Hardware==
There are no particular hardware requirements for running TensorFlow. Obviously, more disk space and RAM are desirable, as is a [https://developer.nvidia.com/cuda-gpus CUDA-enabled GPU].

==Operating System==
The closed Apple ecosystem might make MacOS a good choice of platform because a Mac is a Mac is a Mac. However, I'm not Mr. Moneybags over here, and Apple drops support for older hardware after a time. You couldn't pay me to use Windows for anything but Office applications, so that leaves Linux. The TensorFlow installation directions assume Ubuntu 16.04 or higher, and though I have dabbled with Debian Linux for a home media server, I typically use Ubuntu. These instructions assume Ubuntu 22.04 LTS.

==Python Version==
Ubuntu 22.04 has retired Python 2.x, and uses Python 3.9 by default, which works nicely with TensorFlow (I understand that getting TensorFlow to work with Python 3.10+ includes some challenges). These instructions assume Python 3.9. If you want to use other versions of Python for other applications, I recommend using virtual environments to manage and switch between Python versions. The pip installation instructions use miniconda to create a Python 3.9 virtual environment.

==Setup==
===TensorFlow Installation===
First off, you can use Conda, but don't use Conda to install TensorFlow. Instead, follow the [https://www.tensorflow.org/install/pip pip install instructions] published by the TensorFlow people. The steps are broadly described below:
====Create a virtual environment====
The first step is to install miniconda if it isn't already installed. Then, create a new Python 3.9 virtual environment:
conda create --name tf python=3.9
Then activate your new environment:
conda activate tf

====Update pip and install tensorflow====
You'll need to update pip first:
pip install --upgrade pip
Then pip install tensorflow:
pip install tensorflow

===Spyder Installation===
Anthony got me using the Spyder IDE for Python programming. Problem is, as of today (July 6, 2022), it's buggy on Ubuntu 22.04. You can install it using pip, as documented [https://bugs.launchpad.net/ubuntu/+source/spyder/+bug/1968479 here]:
pip install -U spyder
However, when I uninstalled the apt package dependencies (there are LOTS), the pip installation no longer worked, so I reinstalled the apt package as well. The apt package binary still won't launch, but at least the pip version will have everything it needs. It's possible that having the apt package installed prior to the pip installation caused pip to not bother installing some critical components. If that's the case, then it may not be strictly necessary to have both the apt and pip versions installed.

Note that pip will have installed spyder into a specific virtual environment (e.g., tf). You won't be able to launch spyder without first activating that virtual environment. After installation, you can launch it from a terminal window:
(base) user@host:~ conda activate tf
(tf) user@host:~ spyder

ML Environment

2022-07-26T20:42:00Z

Chris: /* Using the CCN Lab ML Environment */

The lab primarily uses a Linux environment. We have several workstations running the latest version of [https://ubuntu.com/download Ubuntu]. Our machine learning (ML) research is primarily carried out using Google's [https://www.tensorflow.org/ TensorFlow] libraries, written for Python. Workstations with powerful GPUs can use CUDA for GPU acceleration.
=Using the CCN Lab ML Environment=
To avoid clashes between different releases of Python, Tensorflow and whatever libraries are required for a particular ML application, we'll be using Virtual Environments. We have a Python 2.9 virtual environment, which is like a sandbox that all our ML libraries can be installed into and toggled on/off without disrupting anything else. '''There is a one-time setup procedure every individual user will have to complete:'''

#ssh or open a terminal on ws03
#run the following command: <code>conda init bash</code>
#log out, log back in. Now your shell prompt will look like: <code>(base) user@host:~$</code>
#*To toggle it on: <code>conda activate /data/tf</code>
#*To toggle it off: <code>conda deactivate</code>
#*To list all other environments that you can activate <code>conda info --envs</code>
#*To list all the packages installed in your current environment: <code>conda list</code>
#If <code>'''conda list'''</code> doesn't include a bunch of tensorflow libraries, install it using pip: <code>pip install tensorflow</code>

=Setting up your own ML environment=
Below are the specs and walkthrough for setting up a modest ML programming environment.

==Hardware==
There are no particular hardware requirements for running TensorFlow. Obviously, more disk space and RAM are desirable, as is a [https://developer.nvidia.com/cuda-gpus CUDA-enabled GPU].

==Operating System==
The closed Apple ecosystem might make MacOS a good choice of platform because a Mac is a Mac is a Mac. However, I'm not Mr. Moneybags over here, and Apple drops support for older hardware after a time. You couldn't pay me to use Windows for anything but Office applications, so that leaves Linux. The TensorFlow installation directions assume Ubuntu 16.04 or higher, and though I have dabbled with Debian Linux for a home media server, I typically use Ubuntu. These instructions assume Ubuntu 22.04 LTS.

==Python Version==
Ubuntu 22.04 has retired Python 2.x, and uses Python 3.9 by default, which works nicely with TensorFlow (I understand that getting TensorFlow to work with Python 3.10+ includes some challenges). These instructions assume Python 3.9. If you want to use other versions of Python for other applications, I recommend using virtual environments to manage and switch between Python versions. The pip installation instructions use miniconda to create a Python 3.9 virtual environment.

==Setup==
===TensorFlow Installation===
First off, you can use Conda, but don't use Conda to install TensorFlow. Instead, follow the [https://www.tensorflow.org/install/pip pip install instructions] published by the TensorFlow people. The steps are broadly described below:
====Create a virtual environment====
The first step is to install miniconda if it isn't already installed. Then, create a new Python 3.9 virtual environment:
conda create --name tf python=3.9
Then activate your new environment:
conda activate tf

====Update pip and install tensorflow====
You'll need to update pip first:
pip install --upgrade pip
Then pip install tensorflow:
pip install tensorflow

===Spyder Installation===
Anthony got me using the Spyder IDE for Python programming. Problem is, as of today (July 6, 2022), it's buggy on Ubuntu 22.04. You can install it using pip, as documented [https://bugs.launchpad.net/ubuntu/+source/spyder/+bug/1968479 here]:
pip install -U spyder
However, when I uninstalled the apt package dependencies (there are LOTS), the pip installation no longer worked, so I reinstalled the apt package as well. The apt package binary still won't launch, but at least the pip version will have everything it needs. It's possible that having the apt package installed prior to the pip installation caused pip to not bother installing some critical components. If that's the case, then it may not be strictly necessary to have both the apt and pip versions installed.

Note that pip will have installed spyder into a specific virtual environment (e.g., tf). You won't be able to launch spyder without first activating that virtual environment. After installation, you can launch it from a terminal window:
(base) user@host:~ conda activate tf
(tf) user@host:~ spyder

ML Environment

2022-07-26T20:38:59Z

Chris: /* Using the CCN Lab ML Environment */

The lab primarily uses a Linux environment. We have several workstations running the latest version of [https://ubuntu.com/download Ubuntu]. Our machine learning (ML) research is primarily carried out using Google's [https://www.tensorflow.org/ TensorFlow] libraries, written for Python. Workstations with powerful GPUs can use CUDA for GPU acceleration.
=Using the CCN Lab ML Environment=
To avoid clashes between different releases of Python, Tensorflow and whatever libraries are required for a particular ML application, we'll be using Virtual Environments. We have a Python 2.9 virtual environment, which is like a sandbox that all our ML libraries can be installed into and toggled on/off without disrupting anything else. '''There is a one-time setup procedure every individual user will have to complete:'''

#ssh or open a terminal on ws03
#run the following command: <code>conda init bash</code>
#log out, log back in. Now your shell prompt will look like: <code>(base) user@host:~$</code>
#*To toggle it on: <code>conda activate /data/tf</code>
#*To toggle it off: <code>conda deactivate</code>
#*To list all other environments that you can activate <code>conda info --envs</code>
#*To list all the packages installed in your current environment: <code>conda list</code>
#It appears everyone has to install tensorflow individually using pip <code>pip install tensorflow</code>

=Setting up your own ML environment=
Below are the specs and walkthrough for setting up a modest ML programming environment.

==Hardware==
There are no particular hardware requirements for running TensorFlow. Obviously, more disk space and RAM are desirable, as is a [https://developer.nvidia.com/cuda-gpus CUDA-enabled GPU].

==Operating System==
The closed Apple ecosystem might make MacOS a good choice of platform because a Mac is a Mac is a Mac. However, I'm not Mr. Moneybags over here, and Apple drops support for older hardware after a time. You couldn't pay me to use Windows for anything but Office applications, so that leaves Linux. The TensorFlow installation directions assume Ubuntu 16.04 or higher, and though I have dabbled with Debian Linux for a home media server, I typically use Ubuntu. These instructions assume Ubuntu 22.04 LTS.

==Python Version==
Ubuntu 22.04 has retired Python 2.x, and uses Python 3.9 by default, which works nicely with TensorFlow (I understand that getting TensorFlow to work with Python 3.10+ includes some challenges). These instructions assume Python 3.9. If you want to use other versions of Python for other applications, I recommend using virtual environments to manage and switch between Python versions. The pip installation instructions use miniconda to create a Python 3.9 virtual environment.

==Setup==
===TensorFlow Installation===
First off, you can use Conda, but don't use Conda to install TensorFlow. Instead, follow the [https://www.tensorflow.org/install/pip pip install instructions] published by the TensorFlow people. The steps are broadly described below:
====Create a virtual environment====
The first step is to install miniconda if it isn't already installed. Then, create a new Python 3.9 virtual environment:
conda create --name tf python=3.9
Then activate your new environment:
conda activate tf

====Update pip and install tensorflow====
You'll need to update pip first:
pip install --upgrade pip
Then pip install tensorflow:
pip install tensorflow

===Spyder Installation===
Anthony got me using the Spyder IDE for Python programming. Problem is, as of today (July 6, 2022), it's buggy on Ubuntu 22.04. You can install it using pip, as documented [https://bugs.launchpad.net/ubuntu/+source/spyder/+bug/1968479 here]:
pip install -U spyder
However, when I uninstalled the apt package dependencies (there are LOTS), the pip installation no longer worked, so I reinstalled the apt package as well. The apt package binary still won't launch, but at least the pip version will have everything it needs. It's possible that having the apt package installed prior to the pip installation caused pip to not bother installing some critical components. If that's the case, then it may not be strictly necessary to have both the apt and pip versions installed.

Note that pip will have installed spyder into a specific virtual environment (e.g., tf). You won't be able to launch spyder without first activating that virtual environment. After installation, you can launch it from a terminal window:
(base) user@host:~ conda activate tf
(tf) user@host:~ spyder

Main Page

2022-07-26T20:37:04Z

Chris: /* Technobabble */

Welcome to the CCN Lab Wiki.

[[File:cpicard.jpg]]

== Terminal Commands & Other Lab Stuff ==
* [[New Research Staff]]
* [[Lab Roles]]
* [[Mess Ups | Times when we messed up]]
* [[Highlight Reel | Times when we rocked it]]
* [[Participant Screening]]
* [[3D Brain Models]]

== Technobabble ==
* [[ML Environment | Running a ML Workstation ]]
* [[ubmount | UBFS and ubmount]]
* [[SSH | Using SSH]]
* [[Synching Scripts]]
* [[Connecting to CCR]]
* [[Lab Email]]
* [[Excel Formulas]]
* [[Troubleshooting]]
* [[BASH Tricks]]
* [[FreeSurfer on Windows]]

== Data Analysis ==
* [https://wiki.cam.ac.uk/bmuwiki/FMRI Data Quality]
* [[Downloading CRTC Data]]
* [[Behavioral Analyses]]
* [[FreeSurfer | FreeSurfer Pipeline]]
* [[SPM | SPM Pipeline]]
* [[Time Series Analysis]]
* [[Network Analyses]]
*[[Self-organized-mapping(SOM)]]
* [[Data Simulation]]

== Manuscript Preparation ==
* [//ccnlab.psy.buffalo.edu/wiki/index.php?title=Rendering_MRIcron Brain Rendering in MRIcron (SPM)]
* [//ccnlab.psy.buffalo.edu/wiki/index.php?title=Producing_Tables_of_Coordinates_(SPM) Producing Tables of Coordinates (SPM)]
* [[Annotation Coordinates| Extracting XYZ coordinates of .annot labels]]
* [http://colorbrewer2.org/ Color schemes (e.g., color keys for graphs, experiment conditions in fMRI renderings, etc.)]
** Above link yoinked from [http://sites.bu.edu/cnrlab/lab-resources/ BU's CNR Lab]
* [[Acquisition_Parameters | Acquisition Parameters at CTRC]]
* [[Brain Net Viewer]]
* [[Manuscript Formatting]]

== Experiment A to Z (not necessarily in alphabetical order) ==
* [[Project-Specific Documentation]]
* [//ccnlab.psy.buffalo.edu/wiki/index.php/Category:MATLAB_functions MATLAB Functions]
* [//ccnlab.psy.buffalo.edu/wiki/index.php?title=Participant_Triage Participant Triage]
* [//ccnlab.psy.buffalo.edu/wiki/index.php?title=Pre-fMRI_Scanning_Protocol Pre-fMRI Scanning Protocol]
* [//ccnlab.psy.buffalo.edu/wiki/index.php?title=Participant_Instructions Participant Instructions]
* [//ccnlab.psy.buffalo.edu/wiki/index.php?title=MRI_Prep MRI Prep (Prior to scan date)]
* [//ccnlab.psy.buffalo.edu/wiki/index.php?title=MRI_Setup MRI Setup]
* [//ccnlab.psy.buffalo.edu/wiki/index.php?title=MIKENET MIKENET Neural Network C Library Setup]
* [//ccnlab.psy.buffalo.edu/wiki/index.php?title=TensorFlow TensorFlow OpenSource Python Setup]
* [//ccnlab.psy.buffalo.edu/wiki/index.php?title=Reading_Experiment_IDs IDs for Reading Experiment BOLD]
* [//ccnlab.psy.buffalo.edu/wiki/index.php/CCR Center for Computational Research (CCR)]
* [[Neural Networks in Python]]
* [https://openneuro.org/datasets/ds001246/versions/1.1.0 Horikawa dataset]
** <code>aws s3 sync --no-sign-request s3://openneuro.org/ds001246 ds001246-download/</code>
* [https://nda.nih.gov/edit_collection.html?id=2155 MTA dataset]
* [[Wall of Ideas]]

== Informational ==
* [//ccnlab.psy.buffalo.edu/wiki/index.php/Developmental_Neuroscience Developmental Neuroscience]
* [[NSF Proposal Submission Walkthrough for n00bs]]
* [[Recipes]]

== MediaWiki - Guides for Getting started ==

* [//www.mediawiki.org/wiki/Help:Formatting Useful Formatting Guide]
* [//www.mediawiki.org/wiki/Special:MyLanguage/Manual:Configuration_settings Configuration settings list]
* [//www.mediawiki.org/wiki/Special:MyLanguage/Manual:FAQ MediaWiki FAQ]
* [https://lists.wikimedia.org/mailman/listinfo/mediawiki-announce MediaWiki release mailing list]

Consult the [//meta.wikimedia.org/wiki/Help:Contents User's Guide] for information on using the wiki software.

ML Environment

2022-07-26T17:32:44Z

Chris: /* Using the CCN Lab ML Environment */

The lab primarily uses a Linux environment. We have several workstations running the latest version of [https://ubuntu.com/download Ubuntu]. Our machine learning (ML) research is primarily carried out using Google's [https://www.tensorflow.org/ TensorFlow] libraries, written for Python. Workstations with powerful GPUs can use CUDA for GPU acceleration.
=Using the CCN Lab ML Environment=
To avoid clashes between different releases of Python, Tensorflow and whatever libraries are required for a particular ML application, we'll be using Virtual Environments. We have a Python 2.9 virtual environment, which is like a sandbox that all our ML libraries can be installed into and toggled on/off without disrupting anything else. '''There is a one-time setup procedure every individual user will have to complete:'''

#ssh or open a terminal on ws03
#run the following command: <code>conda init bash</code>
#log out, log back in. Now your shell prompt will look like: <code>(base) user@host:~$</code>

Now you can toggle the base and tensorflow python environments, and any other environments that might get added down the road.
#To toggle it on: <code>conda activate /data/tf</code>
#To toggle it off: <code>conda deactivate</code>
#To list all other environments that you can activate <code>conda info --envs</code>
#To list all the packages installed in your current environment: <code>conda list</code>

=Setting up your own ML environment=
Below are the specs and walkthrough for setting up a modest ML programming environment.

==Hardware==
There are no particular hardware requirements for running TensorFlow. Obviously, more disk space and RAM are desirable, as is a [https://developer.nvidia.com/cuda-gpus CUDA-enabled GPU].

==Operating System==
The closed Apple ecosystem might make MacOS a good choice of platform because a Mac is a Mac is a Mac. However, I'm not Mr. Moneybags over here, and Apple drops support for older hardware after a time. You couldn't pay me to use Windows for anything but Office applications, so that leaves Linux. The TensorFlow installation directions assume Ubuntu 16.04 or higher, and though I have dabbled with Debian Linux for a home media server, I typically use Ubuntu. These instructions assume Ubuntu 22.04 LTS.

==Python Version==
Ubuntu 22.04 has retired Python 2.x, and uses Python 3.9 by default, which works nicely with TensorFlow (I understand that getting TensorFlow to work with Python 3.10+ includes some challenges). These instructions assume Python 3.9. If you want to use other versions of Python for other applications, I recommend using virtual environments to manage and switch between Python versions. The pip installation instructions use miniconda to create a Python 3.9 virtual environment.

==Setup==
===TensorFlow Installation===
First off, you can use Conda, but don't use Conda to install TensorFlow. Instead, follow the [https://www.tensorflow.org/install/pip pip install instructions] published by the TensorFlow people. The steps are broadly described below:
====Create a virtual environment====
The first step is to install miniconda if it isn't already installed. Then, create a new Python 3.9 virtual environment:
conda create --name tf python=3.9
Then activate your new environment:
conda activate tf

====Update pip and install tensorflow====
You'll need to update pip first:
pip install --upgrade pip
Then pip install tensorflow:
pip install tensorflow

===Spyder Installation===
Anthony got me using the Spyder IDE for Python programming. Problem is, as of today (July 6, 2022), it's buggy on Ubuntu 22.04. You can install it using pip, as documented [https://bugs.launchpad.net/ubuntu/+source/spyder/+bug/1968479 here]:
pip install -U spyder
However, when I uninstalled the apt package dependencies (there are LOTS), the pip installation no longer worked, so I reinstalled the apt package as well. The apt package binary still won't launch, but at least the pip version will have everything it needs. It's possible that having the apt package installed prior to the pip installation caused pip to not bother installing some critical components. If that's the case, then it may not be strictly necessary to have both the apt and pip versions installed.

Note that pip will have installed spyder into a specific virtual environment (e.g., tf). You won't be able to launch spyder without first activating that virtual environment. After installation, you can launch it from a terminal window:
(base) user@host:~ conda activate tf
(tf) user@host:~ spyder

ML Environment

2022-07-26T17:32:22Z

Chris: /* Using the CCN Lab ML Environment */

The lab primarily uses a Linux environment. We have several workstations running the latest version of [https://ubuntu.com/download Ubuntu]. Our machine learning (ML) research is primarily carried out using Google's [https://www.tensorflow.org/ TensorFlow] libraries, written for Python. Workstations with powerful GPUs can use CUDA for GPU acceleration.
=Using the CCN Lab ML Environment=
To avoid clashes between different releases of Python, Tensorflow and whatever libraries are required for a particular ML application, we'll be using Virtual Environments. We have a Python 2.9 virtual environment, which is like a sandbox that all our ML libraries can be installed into and toggled on/off without disrupting anything else. '''There is a one-time setup procedure every individual user will have to complete:'''

#ssh or open a terminal on ws03
#run the following command: <code>conda init bash</code>
#log out, log back in. Now your shell prompt will look like: <code>(base) user@host:~$</code>

Now you can toggle the base and tensorflow python environments, and any other environments that might get added down the road.
#To toggle it on: <code>conda activate /data/tf</code>
#To toggle it off: <code>conda deactivate</conda>
#To list all other environments that you can activate <code>conda info --envs</code>
#To list all the packages installed in your current environment: <code>conda list</code>

=Setting up your own ML environment=
Below are the specs and walkthrough for setting up a modest ML programming environment.

==Hardware==
There are no particular hardware requirements for running TensorFlow. Obviously, more disk space and RAM are desirable, as is a [https://developer.nvidia.com/cuda-gpus CUDA-enabled GPU].

==Operating System==
The closed Apple ecosystem might make MacOS a good choice of platform because a Mac is a Mac is a Mac. However, I'm not Mr. Moneybags over here, and Apple drops support for older hardware after a time. You couldn't pay me to use Windows for anything but Office applications, so that leaves Linux. The TensorFlow installation directions assume Ubuntu 16.04 or higher, and though I have dabbled with Debian Linux for a home media server, I typically use Ubuntu. These instructions assume Ubuntu 22.04 LTS.

==Python Version==
Ubuntu 22.04 has retired Python 2.x, and uses Python 3.9 by default, which works nicely with TensorFlow (I understand that getting TensorFlow to work with Python 3.10+ includes some challenges). These instructions assume Python 3.9. If you want to use other versions of Python for other applications, I recommend using virtual environments to manage and switch between Python versions. The pip installation instructions use miniconda to create a Python 3.9 virtual environment.

==Setup==
===TensorFlow Installation===
First off, you can use Conda, but don't use Conda to install TensorFlow. Instead, follow the [https://www.tensorflow.org/install/pip pip install instructions] published by the TensorFlow people. The steps are broadly described below:
====Create a virtual environment====
The first step is to install miniconda if it isn't already installed. Then, create a new Python 3.9 virtual environment:
conda create --name tf python=3.9
Then activate your new environment:
conda activate tf

====Update pip and install tensorflow====
You'll need to update pip first:
pip install --upgrade pip
Then pip install tensorflow:
pip install tensorflow

===Spyder Installation===
Anthony got me using the Spyder IDE for Python programming. Problem is, as of today (July 6, 2022), it's buggy on Ubuntu 22.04. You can install it using pip, as documented [https://bugs.launchpad.net/ubuntu/+source/spyder/+bug/1968479 here]:
pip install -U spyder
However, when I uninstalled the apt package dependencies (there are LOTS), the pip installation no longer worked, so I reinstalled the apt package as well. The apt package binary still won't launch, but at least the pip version will have everything it needs. It's possible that having the apt package installed prior to the pip installation caused pip to not bother installing some critical components. If that's the case, then it may not be strictly necessary to have both the apt and pip versions installed.

Note that pip will have installed spyder into a specific virtual environment (e.g., tf). You won't be able to launch spyder without first activating that virtual environment. After installation, you can launch it from a terminal window:
(base) user@host:~ conda activate tf
(tf) user@host:~ spyder

ML Environment

2022-07-26T17:31:48Z

Chris: /* Using the CCN Lab ML Environment */

The lab primarily uses a Linux environment. We have several workstations running the latest version of [https://ubuntu.com/download Ubuntu]. Our machine learning (ML) research is primarily carried out using Google's [https://www.tensorflow.org/ TensorFlow] libraries, written for Python. Workstations with powerful GPUs can use CUDA for GPU acceleration.
=Using the CCN Lab ML Environment=
To avoid clashes between different releases of Python, Tensorflow and whatever libraries are required for a particular ML application, we'll be using Virtual Environments. We have a Python 2.9 virtual environment, which is like a sandbox that all our ML libraries can be installed into and toggled on/off without disrupting anything else. '''There is a one-time setup procedure every individual user will have to complete:'''

#ssh or open a terminal on ws03
#run the following command: <code>conda init bash</code>
#log out, log back in. Now your shell prompt will look like: <code>(base) user@host:~$</code>

Now you can toggle the base and tensorflow python environments, and any other environments that might get added down the road.
#To toggle it on: <code>conda activate /data/tf</code>
#To toggle it off: <code>conda deactivate</conda>
#To list all other environments that you can activate <source>conda info --envs</source>
#To list all the packages installed in your current environment: <code>conda list</code>

=Setting up your own ML environment=
Below are the specs and walkthrough for setting up a modest ML programming environment.

==Hardware==
There are no particular hardware requirements for running TensorFlow. Obviously, more disk space and RAM are desirable, as is a [https://developer.nvidia.com/cuda-gpus CUDA-enabled GPU].

==Operating System==
The closed Apple ecosystem might make MacOS a good choice of platform because a Mac is a Mac is a Mac. However, I'm not Mr. Moneybags over here, and Apple drops support for older hardware after a time. You couldn't pay me to use Windows for anything but Office applications, so that leaves Linux. The TensorFlow installation directions assume Ubuntu 16.04 or higher, and though I have dabbled with Debian Linux for a home media server, I typically use Ubuntu. These instructions assume Ubuntu 22.04 LTS.

==Python Version==
Ubuntu 22.04 has retired Python 2.x, and uses Python 3.9 by default, which works nicely with TensorFlow (I understand that getting TensorFlow to work with Python 3.10+ includes some challenges). These instructions assume Python 3.9. If you want to use other versions of Python for other applications, I recommend using virtual environments to manage and switch between Python versions. The pip installation instructions use miniconda to create a Python 3.9 virtual environment.

==Setup==
===TensorFlow Installation===
First off, you can use Conda, but don't use Conda to install TensorFlow. Instead, follow the [https://www.tensorflow.org/install/pip pip install instructions] published by the TensorFlow people. The steps are broadly described below:
====Create a virtual environment====
The first step is to install miniconda if it isn't already installed. Then, create a new Python 3.9 virtual environment:
conda create --name tf python=3.9
Then activate your new environment:
conda activate tf

====Update pip and install tensorflow====
You'll need to update pip first:
pip install --upgrade pip
Then pip install tensorflow:
pip install tensorflow

===Spyder Installation===
Anthony got me using the Spyder IDE for Python programming. Problem is, as of today (July 6, 2022), it's buggy on Ubuntu 22.04. You can install it using pip, as documented [https://bugs.launchpad.net/ubuntu/+source/spyder/+bug/1968479 here]:
pip install -U spyder
However, when I uninstalled the apt package dependencies (there are LOTS), the pip installation no longer worked, so I reinstalled the apt package as well. The apt package binary still won't launch, but at least the pip version will have everything it needs. It's possible that having the apt package installed prior to the pip installation caused pip to not bother installing some critical components. If that's the case, then it may not be strictly necessary to have both the apt and pip versions installed.

Note that pip will have installed spyder into a specific virtual environment (e.g., tf). You won't be able to launch spyder without first activating that virtual environment. After installation, you can launch it from a terminal window:
(base) user@host:~ conda activate tf
(tf) user@host:~ spyder

ML Environment

2022-07-26T17:30:53Z

Chris: /* Using the CCN Lab ML Environment */

The lab primarily uses a Linux environment. We have several workstations running the latest version of [https://ubuntu.com/download Ubuntu]. Our machine learning (ML) research is primarily carried out using Google's [https://www.tensorflow.org/ TensorFlow] libraries, written for Python. Workstations with powerful GPUs can use CUDA for GPU acceleration.
=Using the CCN Lab ML Environment=
To avoid clashes between different releases of Python, Tensorflow and whatever libraries are required for a particular ML application, we'll be using Virtual Environments. We have a Python 2.9 virtual environment, which is like a sandbox that all our ML libraries can be installed into and toggled on/off without disrupting anything else. There is a one-time setup procedure every individual user will have to complete:

#ssh or open a terminal on ws03
#run the following command: <code>conda init bash</code>
#log out, log back in. Now your shell prompt will look like: <code>(base) user@host:~$</code>
Now you can toggle the base and tensorflow python environments.
#To toggle it on: <code>conda activate /data/tf</code>
#To toggle it off: <code>conda deactivate</conda>
#To list all other environments that you can activate <source>conda info --envs</source>
#To list all the packages installed in your current environment: <code>conda list</code>

=Setting up your own ML environment=
Below are the specs and walkthrough for setting up a modest ML programming environment.

==Hardware==
There are no particular hardware requirements for running TensorFlow. Obviously, more disk space and RAM are desirable, as is a [https://developer.nvidia.com/cuda-gpus CUDA-enabled GPU].

==Operating System==
The closed Apple ecosystem might make MacOS a good choice of platform because a Mac is a Mac is a Mac. However, I'm not Mr. Moneybags over here, and Apple drops support for older hardware after a time. You couldn't pay me to use Windows for anything but Office applications, so that leaves Linux. The TensorFlow installation directions assume Ubuntu 16.04 or higher, and though I have dabbled with Debian Linux for a home media server, I typically use Ubuntu. These instructions assume Ubuntu 22.04 LTS.

==Python Version==
Ubuntu 22.04 has retired Python 2.x, and uses Python 3.9 by default, which works nicely with TensorFlow (I understand that getting TensorFlow to work with Python 3.10+ includes some challenges). These instructions assume Python 3.9. If you want to use other versions of Python for other applications, I recommend using virtual environments to manage and switch between Python versions. The pip installation instructions use miniconda to create a Python 3.9 virtual environment.

==Setup==
===TensorFlow Installation===
First off, you can use Conda, but don't use Conda to install TensorFlow. Instead, follow the [https://www.tensorflow.org/install/pip pip install instructions] published by the TensorFlow people. The steps are broadly described below:
====Create a virtual environment====
The first step is to install miniconda if it isn't already installed. Then, create a new Python 3.9 virtual environment:
conda create --name tf python=3.9
Then activate your new environment:
conda activate tf

====Update pip and install tensorflow====
You'll need to update pip first:
pip install --upgrade pip
Then pip install tensorflow:
pip install tensorflow

===Spyder Installation===
Anthony got me using the Spyder IDE for Python programming. Problem is, as of today (July 6, 2022), it's buggy on Ubuntu 22.04. You can install it using pip, as documented [https://bugs.launchpad.net/ubuntu/+source/spyder/+bug/1968479 here]:
pip install -U spyder
However, when I uninstalled the apt package dependencies (there are LOTS), the pip installation no longer worked, so I reinstalled the apt package as well. The apt package binary still won't launch, but at least the pip version will have everything it needs. It's possible that having the apt package installed prior to the pip installation caused pip to not bother installing some critical components. If that's the case, then it may not be strictly necessary to have both the apt and pip versions installed.

Note that pip will have installed spyder into a specific virtual environment (e.g., tf). You won't be able to launch spyder without first activating that virtual environment. After installation, you can launch it from a terminal window:
(base) user@host:~ conda activate tf
(tf) user@host:~ spyder

ML Environment

2022-07-26T17:13:21Z

Chris: /* Using the CCN Lab ML Environment */

The lab primarily uses a Linux environment. We have several workstations running the latest version of [https://ubuntu.com/download Ubuntu]. Our machine learning (ML) research is primarily carried out using Google's [https://www.tensorflow.org/ TensorFlow] libraries, written for Python. Workstations with powerful GPUs can use CUDA for GPU acceleration.
=Using the CCN Lab ML Environment=
To avoid clashes between different releases of Python, Tensorflow and whatever libraries are required for a particular ML application, we'll be using Virtual Environments. We have a Python 2.9 virtual environment, which is like a sandbox that all our ML libraries can be installed into and toggled on/off without disrupting anything else. There is a one-time setup procedure every individual user will have to complete:

#ssh or open a terminal on ws03
#run the following command: <code>conda init bash</code>
#log out, log back in. Now your shell prompt will look like: <code>(base) user@host:~$</code>
#I have created the tf environment in /data/tf. To activate it: <code>conda activate /data/tf</code>

=Setting up your own ML environment=
Below are the specs and walkthrough for setting up a modest ML programming environment.

==Hardware==
There are no particular hardware requirements for running TensorFlow. Obviously, more disk space and RAM are desirable, as is a [https://developer.nvidia.com/cuda-gpus CUDA-enabled GPU].

==Operating System==
The closed Apple ecosystem might make MacOS a good choice of platform because a Mac is a Mac is a Mac. However, I'm not Mr. Moneybags over here, and Apple drops support for older hardware after a time. You couldn't pay me to use Windows for anything but Office applications, so that leaves Linux. The TensorFlow installation directions assume Ubuntu 16.04 or higher, and though I have dabbled with Debian Linux for a home media server, I typically use Ubuntu. These instructions assume Ubuntu 22.04 LTS.

==Python Version==
Ubuntu 22.04 has retired Python 2.x, and uses Python 3.9 by default, which works nicely with TensorFlow (I understand that getting TensorFlow to work with Python 3.10+ includes some challenges). These instructions assume Python 3.9. If you want to use other versions of Python for other applications, I recommend using virtual environments to manage and switch between Python versions. The pip installation instructions use miniconda to create a Python 3.9 virtual environment.

==Setup==
===TensorFlow Installation===
First off, you can use Conda, but don't use Conda to install TensorFlow. Instead, follow the [https://www.tensorflow.org/install/pip pip install instructions] published by the TensorFlow people. The steps are broadly described below:
====Create a virtual environment====
The first step is to install miniconda if it isn't already installed. Then, create a new Python 3.9 virtual environment:
conda create --name tf python=3.9
Then activate your new environment:
conda activate tf

====Update pip and install tensorflow====
You'll need to update pip first:
pip install --upgrade pip
Then pip install tensorflow:
pip install tensorflow

===Spyder Installation===
Anthony got me using the Spyder IDE for Python programming. Problem is, as of today (July 6, 2022), it's buggy on Ubuntu 22.04. You can install it using pip, as documented [https://bugs.launchpad.net/ubuntu/+source/spyder/+bug/1968479 here]:
pip install -U spyder
However, when I uninstalled the apt package dependencies (there are LOTS), the pip installation no longer worked, so I reinstalled the apt package as well. The apt package binary still won't launch, but at least the pip version will have everything it needs. It's possible that having the apt package installed prior to the pip installation caused pip to not bother installing some critical components. If that's the case, then it may not be strictly necessary to have both the apt and pip versions installed.

Note that pip will have installed spyder into a specific virtual environment (e.g., tf). You won't be able to launch spyder without first activating that virtual environment. After installation, you can launch it from a terminal window:
(base) user@host:~ conda activate tf
(tf) user@host:~ spyder

ML Environment

2022-07-26T17:12:46Z

Chris: /* Using the CCN Lab ML Environment */

The lab primarily uses a Linux environment. We have several workstations running the latest version of [https://ubuntu.com/download Ubuntu]. Our machine learning (ML) research is primarily carried out using Google's [https://www.tensorflow.org/ TensorFlow] libraries, written for Python. Workstations with powerful GPUs can use CUDA for GPU acceleration.
=Using the CCN Lab ML Environment=
To avoid clashes between different releases of Python, Tensorflow and whatever libraries are required for a particular ML application, we'll be using Virtual Environments. We have a Python 2.9 virtual environment, which is like a sandbox that all our ML libraries can be installed into and toggled on/off without disrupting anything else. There is a one-time setup procedure every individual user will have to complete:

#ssh or open a terminal on ws03
#run the following command: <code>conda init bash</code>
#log out, log back in. Now your shell prompt will look like:
(base) user@host:~$
#I have created the tf environment in /data/tf. To activate it:
conda activate /data/tf

=Setting up your own ML environment=
Below are the specs and walkthrough for setting up a modest ML programming environment.

==Hardware==
There are no particular hardware requirements for running TensorFlow. Obviously, more disk space and RAM are desirable, as is a [https://developer.nvidia.com/cuda-gpus CUDA-enabled GPU].

==Operating System==
The closed Apple ecosystem might make MacOS a good choice of platform because a Mac is a Mac is a Mac. However, I'm not Mr. Moneybags over here, and Apple drops support for older hardware after a time. You couldn't pay me to use Windows for anything but Office applications, so that leaves Linux. The TensorFlow installation directions assume Ubuntu 16.04 or higher, and though I have dabbled with Debian Linux for a home media server, I typically use Ubuntu. These instructions assume Ubuntu 22.04 LTS.

==Python Version==
Ubuntu 22.04 has retired Python 2.x, and uses Python 3.9 by default, which works nicely with TensorFlow (I understand that getting TensorFlow to work with Python 3.10+ includes some challenges). These instructions assume Python 3.9. If you want to use other versions of Python for other applications, I recommend using virtual environments to manage and switch between Python versions. The pip installation instructions use miniconda to create a Python 3.9 virtual environment.

==Setup==
===TensorFlow Installation===
First off, you can use Conda, but don't use Conda to install TensorFlow. Instead, follow the [https://www.tensorflow.org/install/pip pip install instructions] published by the TensorFlow people. The steps are broadly described below:
====Create a virtual environment====
The first step is to install miniconda if it isn't already installed. Then, create a new Python 3.9 virtual environment:
conda create --name tf python=3.9
Then activate your new environment:
conda activate tf

====Update pip and install tensorflow====
You'll need to update pip first:
pip install --upgrade pip
Then pip install tensorflow:
pip install tensorflow

===Spyder Installation===
Anthony got me using the Spyder IDE for Python programming. Problem is, as of today (July 6, 2022), it's buggy on Ubuntu 22.04. You can install it using pip, as documented [https://bugs.launchpad.net/ubuntu/+source/spyder/+bug/1968479 here]:
pip install -U spyder
However, when I uninstalled the apt package dependencies (there are LOTS), the pip installation no longer worked, so I reinstalled the apt package as well. The apt package binary still won't launch, but at least the pip version will have everything it needs. It's possible that having the apt package installed prior to the pip installation caused pip to not bother installing some critical components. If that's the case, then it may not be strictly necessary to have both the apt and pip versions installed.

Note that pip will have installed spyder into a specific virtual environment (e.g., tf). You won't be able to launch spyder without first activating that virtual environment. After installation, you can launch it from a terminal window:
(base) user@host:~ conda activate tf
(tf) user@host:~ spyder

ML Environment

2022-07-26T17:12:19Z

Chris: /* Using the CCN Lab ML Environment */

The lab primarily uses a Linux environment. We have several workstations running the latest version of [https://ubuntu.com/download Ubuntu]. Our machine learning (ML) research is primarily carried out using Google's [https://www.tensorflow.org/ TensorFlow] libraries, written for Python. Workstations with powerful GPUs can use CUDA for GPU acceleration.
=Using the CCN Lab ML Environment=
To avoid clashes between different releases of Python, Tensorflow and whatever libraries are required for a particular ML application, we'll be using Virtual Environments. We have a Python 2.9 virtual environment, which is like a sandbox that all our ML libraries can be installed into and toggled on/off without disrupting anything else. There is a one-time setup procedure every individual user will have to complete:

#ssh or open a terminal on ws03
#run the following command:
conda init bash
#log out, log back in. Now your shell prompt will look like:
(base) user@host:~$
#I have created the tf environment in /data/tf. To activate it:
conda activate /data/tf

=Setting up your own ML environment=
Below are the specs and walkthrough for setting up a modest ML programming environment.

==Hardware==
There are no particular hardware requirements for running TensorFlow. Obviously, more disk space and RAM are desirable, as is a [https://developer.nvidia.com/cuda-gpus CUDA-enabled GPU].

==Operating System==
The closed Apple ecosystem might make MacOS a good choice of platform because a Mac is a Mac is a Mac. However, I'm not Mr. Moneybags over here, and Apple drops support for older hardware after a time. You couldn't pay me to use Windows for anything but Office applications, so that leaves Linux. The TensorFlow installation directions assume Ubuntu 16.04 or higher, and though I have dabbled with Debian Linux for a home media server, I typically use Ubuntu. These instructions assume Ubuntu 22.04 LTS.

==Python Version==
Ubuntu 22.04 has retired Python 2.x, and uses Python 3.9 by default, which works nicely with TensorFlow (I understand that getting TensorFlow to work with Python 3.10+ includes some challenges). These instructions assume Python 3.9. If you want to use other versions of Python for other applications, I recommend using virtual environments to manage and switch between Python versions. The pip installation instructions use miniconda to create a Python 3.9 virtual environment.

==Setup==
===TensorFlow Installation===
First off, you can use Conda, but don't use Conda to install TensorFlow. Instead, follow the [https://www.tensorflow.org/install/pip pip install instructions] published by the TensorFlow people. The steps are broadly described below:
====Create a virtual environment====
The first step is to install miniconda if it isn't already installed. Then, create a new Python 3.9 virtual environment:
conda create --name tf python=3.9
Then activate your new environment:
conda activate tf

====Update pip and install tensorflow====
You'll need to update pip first:
pip install --upgrade pip
Then pip install tensorflow:
pip install tensorflow

===Spyder Installation===
Anthony got me using the Spyder IDE for Python programming. Problem is, as of today (July 6, 2022), it's buggy on Ubuntu 22.04. You can install it using pip, as documented [https://bugs.launchpad.net/ubuntu/+source/spyder/+bug/1968479 here]:
pip install -U spyder
However, when I uninstalled the apt package dependencies (there are LOTS), the pip installation no longer worked, so I reinstalled the apt package as well. The apt package binary still won't launch, but at least the pip version will have everything it needs. It's possible that having the apt package installed prior to the pip installation caused pip to not bother installing some critical components. If that's the case, then it may not be strictly necessary to have both the apt and pip versions installed.

Note that pip will have installed spyder into a specific virtual environment (e.g., tf). You won't be able to launch spyder without first activating that virtual environment. After installation, you can launch it from a terminal window:
(base) user@host:~ conda activate tf
(tf) user@host:~ spyder

ML Environment

2022-07-26T17:06:27Z

Chris:

The lab primarily uses a Linux environment. We have several workstations running the latest version of [https://ubuntu.com/download Ubuntu]. Our machine learning (ML) research is primarily carried out using Google's [https://www.tensorflow.org/ TensorFlow] libraries, written for Python. Workstations with powerful GPUs can use CUDA for GPU acceleration.
=Using the CCN Lab ML Environment=
To avoid clashes between different releases of Python, Tensorflow and whatever libraries are required for a particular ML application, we'll be using Virtual Environments. CASET has set up a Python 2.9 environment for us. There is a one-time setup procedure every individual user will have to complete:

#ssh or open a terminal on ws03
#run <code>conda init bash</code>
#log out, log back in. Now your shell prompt will look like:
(base) user@host:~$

=Setting up your own ML environment=
Below are the specs and walkthrough for setting up a modest ML programming environment.

==Hardware==
There are no particular hardware requirements for running TensorFlow. Obviously, more disk space and RAM are desirable, as is a [https://developer.nvidia.com/cuda-gpus CUDA-enabled GPU].

==Operating System==
The closed Apple ecosystem might make MacOS a good choice of platform because a Mac is a Mac is a Mac. However, I'm not Mr. Moneybags over here, and Apple drops support for older hardware after a time. You couldn't pay me to use Windows for anything but Office applications, so that leaves Linux. The TensorFlow installation directions assume Ubuntu 16.04 or higher, and though I have dabbled with Debian Linux for a home media server, I typically use Ubuntu. These instructions assume Ubuntu 22.04 LTS.

==Python Version==
Ubuntu 22.04 has retired Python 2.x, and uses Python 3.9 by default, which works nicely with TensorFlow (I understand that getting TensorFlow to work with Python 3.10+ includes some challenges). These instructions assume Python 3.9. If you want to use other versions of Python for other applications, I recommend using virtual environments to manage and switch between Python versions. The pip installation instructions use miniconda to create a Python 3.9 virtual environment.

==Setup==
===TensorFlow Installation===
First off, you can use Conda, but don't use Conda to install TensorFlow. Instead, follow the [https://www.tensorflow.org/install/pip pip install instructions] published by the TensorFlow people. The steps are broadly described below:
====Create a virtual environment====
The first step is to install miniconda if it isn't already installed. Then, create a new Python 3.9 virtual environment:
conda create --name tf python=3.9
Then activate your new environment:
conda activate tf

====Update pip and install tensorflow====
You'll need to update pip first:
pip install --upgrade pip
Then pip install tensorflow:
pip install tensorflow

===Spyder Installation===
Anthony got me using the Spyder IDE for Python programming. Problem is, as of today (July 6, 2022), it's buggy on Ubuntu 22.04. You can install it using pip, as documented [https://bugs.launchpad.net/ubuntu/+source/spyder/+bug/1968479 here]:
pip install -U spyder
However, when I uninstalled the apt package dependencies (there are LOTS), the pip installation no longer worked, so I reinstalled the apt package as well. The apt package binary still won't launch, but at least the pip version will have everything it needs. It's possible that having the apt package installed prior to the pip installation caused pip to not bother installing some critical components. If that's the case, then it may not be strictly necessary to have both the apt and pip versions installed.

Note that pip will have installed spyder into a specific virtual environment (e.g., tf). You won't be able to launch spyder without first activating that virtual environment. After installation, you can launch it from a terminal window:
(base) user@host:~ conda activate tf
(tf) user@host:~ spyder

Freesurfer Group-Level Analysis (mri glmfit)

2022-07-21T18:58:24Z

Chris: /* Correct for Multiple Comparisons with Permutation Test (MNI305 Space, or Surface-Based) */

This procedure is based on information from [https://surfer.nmr.mgh.harvard.edu/fswiki/FsFastTutorialV5.1/FsFastGroupLevel here]

The group-level (Random-Effects) analysis repeats the contrasts performed for individual subjects on a variance-weighted composite of all your subjects. This will identify voxels for which your contrast effects are consistent across all your participants. For the function MRI group analysis you will need to:

*Concatenate individuals into one file (isxconcat-sess)
*Do not smooth (already smoothed during first-level analysis)
*For each space (lh/rh/mni305):
**Run mri_glmfit using weighted least squares (WLS)
**Correct for multiple comparisons
**Correct for multiple comparisons
*Optionally merge into one volume space

==Before you start==
Ensure that your SUBJECTS_DIR variable is set to the working directory containing all your participants, and that the first-level analyses have been completed for all participants (using selxavg-3)

==Concatenate First-Level Analyses(isxconcat-sess)==
In your '''''$SUBJECTS_DIR''''', there should be a '''''subjects''''' file containing the subjectID for each participant for which you had run selxavg-3. Assume that the analysis directories are called ''my_analysis.*'' ([[Configure_mkanalysis-sess | mkanalysis-sess]]), and that the contrast is called ''conditionA_vs_conditionB'' ([[Configure_mkcontrast-sess | mkcontrast-sess]]). You might find it helpful to use environment variables and run the <code>isxconcat-sess</code> script as follows:
ANALYSIS=my_analysis.sm4
CONTRAST1=conditionA_vs_conditionB
#left hemisphere
isxconcat-sess -sf subjects -analysis ${ANALYSIS}.lh -contrast $CONTRAST1 -o RFX
#right hemisphere
isxconcat-sess -sf subjects -analysis ${ANALYSIS}.rh -contrast $CONTRAST1 -o RFX
#subcortical
isxconcat-sess -sf subjects -analysis ${ANALYSIS}.mni -contrast $CONTRAST1 -o RFX

When finished, a new directory will exist, '''''$SUBJECTS_DIR/RFX''''' and contain sub-folders for each of your analyses (in this case, left- and right-hemisphere and MNI305 analysis). Note that your analysis directory does not have to be called '''RFX''' (e.g., if multiple people are working in the same subjects folder, you might have an output folder named something like '''RFX_CHRIS''', or whatever suits you).

If you have multiple contrasts, you can just create additional CONTRAST variables and rerun the same scripts, replacing ${CONTRAST1} with alternative contrasts.

Note that if you have already run isxconcat-sess in the past, you may get a warning because an earlier group-level directory will already exist. You should probably just delete the old directory and rerun isxconcatenate-sess.
===Protip for Looping Through Contrasts===
Contrasts will have a corresponding .config file in one or more of the analysis directories found in $SUBJECTS_DIR. If you plan to automate this process using a script that loops over analysis folders and contrasts, the following code snippet will create a text file containing the names of the contrasts with .config files:
cd $SUBJECTS_DIR/my_analysis.lh
ls -1 | grep config | sed 's/\.config//g' > ../list_of_contrasts
#now $SUBJECTS_DIR has a text file called list_of_contrasts for use in a loop

==Run mri_glmfit==
'''PAY ATTENTION:''' You will need to run mri_glmfit '''for each contrast''' peformed '''for each of the analyses'''! (but see the section below about scripting)

In the above example, we have 1 contrast (conditionA_vs_conditionB) to be carried out for the my_analysis.lh, my_analysis.rh and my_analysis.mni305 analyses directories. Of course, if we had also contrasted other conditions, then you will have to run mri_glmfit many more times.
===One-Sample Group Mean===
The 1-sample group mean (OSGM) tests whether the contrast A-B differs significantly from zero. The example that follows demonstrates how you would run the conditionA_vs_conditionB OSGM analysis for each of the analysis directories as in my example scenario:
ANALYSIS=my_analysis
CONTRAST1=conditionA_vs_conditionB

#contrast 1 in left hemisphere
cd ${SUBJECTS_DIR}/RFX/${ANALYSIS}.lh/${CONTRAST1}
mri_glmfit --y ces.nii.gz --wls cesvar.nii.gz --osgm --surface fsaverage lh --glmdir glm.wls --nii.gz

#contrast 1 in right hemisphere
cd ${SUBJECTS_DIR}/RFX/${ANALYSIS}.rh/${CONTRAST1}
mri_glmfit --y ces.nii.gz --wls cesvar.nii.gz --osgm --surface fsaverage rh --glmdir glm.wls --nii.gz

#contrast 1 in subcortical MNI space - no need to specify the surface
cd ${SUBJECTS_DIR}/RFX/${ANALYSIS}.mni/${CONTRAST1}
mri_glmfit --y ces.nii.gz --osgm --glmdir glm.wls --nii.gz --eres-save

===Correct for Multiple Comparisons (Surface-Based)===
After you have run the voxel-wise analysis, you need to correct the contrast maps for multiple comparisons. This is because the previous step runs the analysis on each of many thousand voxels independently, which mathematically implies that ''alpha'' of those significant voxels are actually Type I errors. Again, you will have to perform this step for each analysis directory. This takes very little time to run for the surface-based analyses, which can use precomputed values.

An explanation of the parameters: <code>--glmdir glm.wls</code> refers to the directory you specified when you ran the mri_glmfit command. <code>--cache 3 pos</code> indicates that you want to use the pre-cached simulation with a voxel-wise threshold of 10E-3 (i.e., 0.001) and use positive contrast values. <code>--cwpvalthresh .025</code> indicates you will retain clusters with a size-corrected p-value of ''P''<.025 (see the '''Notes''' below).
*my_analysis.lh
**<code>cd ''$SUBJECTS_DIR/RFX/my_analysis.lh/conditionA_vs_conditionB/''</code>
**mri_glmfit-sim --glmdir glm.wls --cache 3 pos --cwpvalthresh .025
*my_analysis.rh
**<code>cd ''$SUBJECTS_DIR/RFX/my_analysis.rh/conditionA_vs_conditionB/''</code>
**mri_glmfit-sim --glmdir glm.wls --cache 3 pos --cwpvalthresh .025

Running this step saves thresholded files in the directory indicated by the <code>--glmdir</code> switch (in the above example, files were saved in glm.wls). There, you will find files named ''cache.*.nii.gz''. You can then load those files as overlays to see which clusters survived the size thresholding.

===Correct for Multiple Comparisons with Permutation Test (MNI305 Space, or Surface-Based)===
The above examples use cached Monte Carlo simulations for speed. The help text for mri_glmfit-sim gives the syntax for running your own sims, which take much longer to execute, but are required for MNI305 space:
mri_glmfit-sim --glmdir glmdir \
--sim nulltype nsim threshold csdbase \
--sim-sign sign
e.g. Monte Carlo correction for surface space (10K iterations, voxel-wise p<.005, positive differences only, no correction for multiple spaces):
mri_glmfit-sim --glmdir glm.wls \
--sim mc-z 10000 2.3 mc-z.th005.pos --sim-sign pos

e.g. permutation correction for MNI space (10K iterations, voxel-wise p<.01, absolute value differences, cluster-wise pfamilywise=.05 is corrected for lh, rh and mni spaces):
mri_glmfit-sim --glmdir glm.wls --sim perm 10000 2 perm10k.th01.abs \
--sim-sign abs --cwp 0.05 --3spaces

e.g. splitting permutation correction for MNI spaces into 4 parallel jobs for speed (each job will do 2500/10k sims); using -log thresh notation in filenames:
mri_glmfit-sim --glmdir glm.wls --sim perm 10000 3 perm10k.th30.abs --sim-sign abs --cwp 0.05 --3spaces --bg 4

Options for sim type (listed fastest to slowest) are perm, mc-z, and mc-full.

====Notes====
The choice of '''cwpvalthresh''' was computed by dividing the nominal desired p-value (0.05) by the number of spaces entailed by the simulation. In the above example if we are only looking at the lh and rh surfaces we divide 0.05/2 = 0.025. If you were looking at the lh, rh and MNI305 space, you would have to divide by 3 to maintain the same FWE: 0.05/3=0.0167.

As indicated above, the uncorrected voxel threshold (p=.001) was used to enable the use of pre-cached simulated values with the <code>--cache</code> switch. Though .1, .01, and .001 are easy enough to compute (1, 2, 3, respectively), other values can be computed by noting that this value corresponds to -1 × the base-10 logarithm of the uncorrected p-value. For example, to use cluster-size correction with an uncorrected voxel-wise p-value of 0.005, a spreadsheet or calculator can compute -1×(log100.005)=2.301. Or for a p-value of 0.05: -1×(log100.05)=1.301 . Rather than specify positive values, you can also use negative values (<code>neg</code>) or absolute values (<code>abs</code>, which I think may be the suggested default?).

====Scripting====
Because mri_glmfit has to be run for each combination of analysis directory (*.lh, *.rh, possibly *.mni305) and contrast, the number of commands you need to run multiplies pretty quickly. You can check out the [[OSGM Script]] page for an example of how to automate these calls to mri_glmfit using a shell script.

==Check It Out==
This little snippet shows you how to inspect the results of the MNI305 first level analysis.
===MNI305===
RFXDIR=RFX
ANALYSIS=FAM.sm4
CONTRAST=task
#assuming you used the conventions described above, the glm results can be found
#in the directory '''glm.wls''', and the osgm results in a subdirectory called '''osgm'''
THEDIR=${SUBJECTS_DIR}/${RFXDIR}/${ANALYSIS}.mni305/${CONTRAST}/glm.wls/osgm

tkmeditfv fsaverage orig.mgz -ov ${THEDIR}/perm.abs.01.sig.cluster.nii.gz \
-seg ${THEDIR}/perm.abs.01.sig.ocn.nii.gz \
${THEDIR}/perm.abs.01.sig.ocn.lut

==What Next?==
The cluster-level correction step will create a series of files in each of the glm directories. They will be automatically given names that describe the parameters used. For example, the above commands generated files named ''cache.th30.abs.sig.*''. One of these files is a .annot file, where each cluster is a label in the file. These labels make convenient ''[[Working_with_ROIs_(Freesurfer) | functional ROIs]]'', wouldn't you say?

I haven't done this yet, but it seems that one thing you might want to do is map the labels in the .annot file generated by the group mri_glmfit-sim to each participant using <code>mri_surf2surf</code>. A script called <code>cpannot.sh</code> that simplifies this has been written and copied to the UBFS/Scripts/Shell/ folder. This script is invoked thus:
cpannot.sh $SOURCESUBJECT $TARGETSUBJECT $ANNOTFILE

e.g.:

cpannot.sh fsaverage FS_0183 lausanne.annot

Before you do this, you can check out the details of the cluster statistics, which include things like the associated anatomical labels for each cluster, and also the surface area of the cluster. The clusters can be further subdivided to make them more similar in size by <code>[[Freesurfer Subparcellation | mris_divide_parcellation]]</code>. With a series of roughly equal-sized ROIs, they can be mapped from the fsaverage surface to each participant to extract time series or other information from that individual.

[[Category:FreeSurfer]]
[[Category:Functional Analyses]]

Freesurfer Group-Level Analysis (mri glmfit)

2022-07-21T18:53:27Z

Chris: /* Correct for Multiple Comparisons with Permutation Test (MNI305 Space, or Surface-Based) */

This procedure is based on information from [https://surfer.nmr.mgh.harvard.edu/fswiki/FsFastTutorialV5.1/FsFastGroupLevel here]

The group-level (Random-Effects) analysis repeats the contrasts performed for individual subjects on a variance-weighted composite of all your subjects. This will identify voxels for which your contrast effects are consistent across all your participants. For the function MRI group analysis you will need to:

*Concatenate individuals into one file (isxconcat-sess)
*Do not smooth (already smoothed during first-level analysis)
*For each space (lh/rh/mni305):
**Run mri_glmfit using weighted least squares (WLS)
**Correct for multiple comparisons
**Correct for multiple comparisons
*Optionally merge into one volume space

==Before you start==
Ensure that your SUBJECTS_DIR variable is set to the working directory containing all your participants, and that the first-level analyses have been completed for all participants (using selxavg-3)

==Concatenate First-Level Analyses(isxconcat-sess)==
In your '''''$SUBJECTS_DIR''''', there should be a '''''subjects''''' file containing the subjectID for each participant for which you had run selxavg-3. Assume that the analysis directories are called ''my_analysis.*'' ([[Configure_mkanalysis-sess | mkanalysis-sess]]), and that the contrast is called ''conditionA_vs_conditionB'' ([[Configure_mkcontrast-sess | mkcontrast-sess]]). You might find it helpful to use environment variables and run the <code>isxconcat-sess</code> script as follows:
ANALYSIS=my_analysis.sm4
CONTRAST1=conditionA_vs_conditionB
#left hemisphere
isxconcat-sess -sf subjects -analysis ${ANALYSIS}.lh -contrast $CONTRAST1 -o RFX
#right hemisphere
isxconcat-sess -sf subjects -analysis ${ANALYSIS}.rh -contrast $CONTRAST1 -o RFX
#subcortical
isxconcat-sess -sf subjects -analysis ${ANALYSIS}.mni -contrast $CONTRAST1 -o RFX

When finished, a new directory will exist, '''''$SUBJECTS_DIR/RFX''''' and contain sub-folders for each of your analyses (in this case, left- and right-hemisphere and MNI305 analysis). Note that your analysis directory does not have to be called '''RFX''' (e.g., if multiple people are working in the same subjects folder, you might have an output folder named something like '''RFX_CHRIS''', or whatever suits you).

If you have multiple contrasts, you can just create additional CONTRAST variables and rerun the same scripts, replacing ${CONTRAST1} with alternative contrasts.

Note that if you have already run isxconcat-sess in the past, you may get a warning because an earlier group-level directory will already exist. You should probably just delete the old directory and rerun isxconcatenate-sess.
===Protip for Looping Through Contrasts===
Contrasts will have a corresponding .config file in one or more of the analysis directories found in $SUBJECTS_DIR. If you plan to automate this process using a script that loops over analysis folders and contrasts, the following code snippet will create a text file containing the names of the contrasts with .config files:
cd $SUBJECTS_DIR/my_analysis.lh
ls -1 | grep config | sed 's/\.config//g' > ../list_of_contrasts
#now $SUBJECTS_DIR has a text file called list_of_contrasts for use in a loop

==Run mri_glmfit==
'''PAY ATTENTION:''' You will need to run mri_glmfit '''for each contrast''' peformed '''for each of the analyses'''! (but see the section below about scripting)

In the above example, we have 1 contrast (conditionA_vs_conditionB) to be carried out for the my_analysis.lh, my_analysis.rh and my_analysis.mni305 analyses directories. Of course, if we had also contrasted other conditions, then you will have to run mri_glmfit many more times.
===One-Sample Group Mean===
The 1-sample group mean (OSGM) tests whether the contrast A-B differs significantly from zero. The example that follows demonstrates how you would run the conditionA_vs_conditionB OSGM analysis for each of the analysis directories as in my example scenario:
ANALYSIS=my_analysis
CONTRAST1=conditionA_vs_conditionB

#contrast 1 in left hemisphere
cd ${SUBJECTS_DIR}/RFX/${ANALYSIS}.lh/${CONTRAST1}
mri_glmfit --y ces.nii.gz --wls cesvar.nii.gz --osgm --surface fsaverage lh --glmdir glm.wls --nii.gz

#contrast 1 in right hemisphere
cd ${SUBJECTS_DIR}/RFX/${ANALYSIS}.rh/${CONTRAST1}
mri_glmfit --y ces.nii.gz --wls cesvar.nii.gz --osgm --surface fsaverage rh --glmdir glm.wls --nii.gz

#contrast 1 in subcortical MNI space - no need to specify the surface
cd ${SUBJECTS_DIR}/RFX/${ANALYSIS}.mni/${CONTRAST1}
mri_glmfit --y ces.nii.gz --osgm --glmdir glm.wls --nii.gz --eres-save

===Correct for Multiple Comparisons (Surface-Based)===
After you have run the voxel-wise analysis, you need to correct the contrast maps for multiple comparisons. This is because the previous step runs the analysis on each of many thousand voxels independently, which mathematically implies that ''alpha'' of those significant voxels are actually Type I errors. Again, you will have to perform this step for each analysis directory. This takes very little time to run for the surface-based analyses, which can use precomputed values.

An explanation of the parameters: <code>--glmdir glm.wls</code> refers to the directory you specified when you ran the mri_glmfit command. <code>--cache 3 pos</code> indicates that you want to use the pre-cached simulation with a voxel-wise threshold of 10E-3 (i.e., 0.001) and use positive contrast values. <code>--cwpvalthresh .025</code> indicates you will retain clusters with a size-corrected p-value of ''P''<.025 (see the '''Notes''' below).
*my_analysis.lh
**<code>cd ''$SUBJECTS_DIR/RFX/my_analysis.lh/conditionA_vs_conditionB/''</code>
**mri_glmfit-sim --glmdir glm.wls --cache 3 pos --cwpvalthresh .025
*my_analysis.rh
**<code>cd ''$SUBJECTS_DIR/RFX/my_analysis.rh/conditionA_vs_conditionB/''</code>
**mri_glmfit-sim --glmdir glm.wls --cache 3 pos --cwpvalthresh .025

Running this step saves thresholded files in the directory indicated by the <code>--glmdir</code> switch (in the above example, files were saved in glm.wls). There, you will find files named ''cache.*.nii.gz''. You can then load those files as overlays to see which clusters survived the size thresholding.

===Correct for Multiple Comparisons with Permutation Test (MNI305 Space, or Surface-Based)===
The above examples use cached Monte Carlo simulations for speed. The help text for mri_glmfit-sim gives the syntax for running your own sims, which take much longer to execute, but are required for MNI305 space:
mri_glmfit-sim --glmdir glmdir \
--sim nulltype nsim threshold csdbase \
--sim-sign sign
e.g. Monte Carlo correction for surface space (10K iterations, voxel-wise p<.005, positive differences only, no correction for multiple spaces):
mri_glmfit-sim --glmdir glm.wls \
--sim mc-z 10000 2.3 mc-z.pos.th005 --sim-sign pos

e.g. permutation correction for MNI space (10K iterations, voxel-wise p<.01, absolute value differences, cluster-wise pfamilywise=.05 is corrected for lh, rh and mni spaces):
mri_glmfit-sim --glmdir glm.wls --sim perm 10000 2 perm.abs.th01 \
--sim-sign abs --cwp 0.05 --3spaces

e.g. splitting permutation correction for MNI spaces into 4 parallel jobs for speed (each job will do 2500/10k sims); using -log thresh notation in filenames:
mri_glmfit-sim --glmdir glm.wls --sim perm 10000 3 perm10k.th30 --sim-sign abs --cwp 0.05 --3spaces --bg 4

Options for sim type (listed fastest to slowest) are perm, mc-z, and mc-full.

====Notes====
The choice of '''cwpvalthresh''' was computed by dividing the nominal desired p-value (0.05) by the number of spaces entailed by the simulation. In the above example if we are only looking at the lh and rh surfaces we divide 0.05/2 = 0.025. If you were looking at the lh, rh and MNI305 space, you would have to divide by 3 to maintain the same FWE: 0.05/3=0.0167.

As indicated above, the uncorrected voxel threshold (p=.001) was used to enable the use of pre-cached simulated values with the <code>--cache</code> switch. Though .1, .01, and .001 are easy enough to compute (1, 2, 3, respectively), other values can be computed by noting that this value corresponds to -1 × the base-10 logarithm of the uncorrected p-value. For example, to use cluster-size correction with an uncorrected voxel-wise p-value of 0.005, a spreadsheet or calculator can compute -1×(log100.005)=2.301. Or for a p-value of 0.05: -1×(log100.05)=1.301 . Rather than specify positive values, you can also use negative values (<code>neg</code>) or absolute values (<code>abs</code>, which I think may be the suggested default?).

====Scripting====
Because mri_glmfit has to be run for each combination of analysis directory (*.lh, *.rh, possibly *.mni305) and contrast, the number of commands you need to run multiplies pretty quickly. You can check out the [[OSGM Script]] page for an example of how to automate these calls to mri_glmfit using a shell script.

==Check It Out==
This little snippet shows you how to inspect the results of the MNI305 first level analysis.
===MNI305===
RFXDIR=RFX
ANALYSIS=FAM.sm4
CONTRAST=task
#assuming you used the conventions described above, the glm results can be found
#in the directory '''glm.wls''', and the osgm results in a subdirectory called '''osgm'''
THEDIR=${SUBJECTS_DIR}/${RFXDIR}/${ANALYSIS}.mni305/${CONTRAST}/glm.wls/osgm

tkmeditfv fsaverage orig.mgz -ov ${THEDIR}/perm.abs.01.sig.cluster.nii.gz \
-seg ${THEDIR}/perm.abs.01.sig.ocn.nii.gz \
${THEDIR}/perm.abs.01.sig.ocn.lut

==What Next?==
The cluster-level correction step will create a series of files in each of the glm directories. They will be automatically given names that describe the parameters used. For example, the above commands generated files named ''cache.th30.abs.sig.*''. One of these files is a .annot file, where each cluster is a label in the file. These labels make convenient ''[[Working_with_ROIs_(Freesurfer) | functional ROIs]]'', wouldn't you say?

I haven't done this yet, but it seems that one thing you might want to do is map the labels in the .annot file generated by the group mri_glmfit-sim to each participant using <code>mri_surf2surf</code>. A script called <code>cpannot.sh</code> that simplifies this has been written and copied to the UBFS/Scripts/Shell/ folder. This script is invoked thus:
cpannot.sh $SOURCESUBJECT $TARGETSUBJECT $ANNOTFILE

e.g.:

cpannot.sh fsaverage FS_0183 lausanne.annot

Before you do this, you can check out the details of the cluster statistics, which include things like the associated anatomical labels for each cluster, and also the surface area of the cluster. The clusters can be further subdivided to make them more similar in size by <code>[[Freesurfer Subparcellation | mris_divide_parcellation]]</code>. With a series of roughly equal-sized ROIs, they can be mapped from the fsaverage surface to each participant to extract time series or other information from that individual.

[[Category:FreeSurfer]]
[[Category:Functional Analyses]]

Freesurfer Group-Level Analysis (mri glmfit)

2022-07-21T18:52:21Z

Chris: /* Correct for Multiple Comparisons with Permutation Test (MNI305 Space, or Surface-Based) */

This procedure is based on information from [https://surfer.nmr.mgh.harvard.edu/fswiki/FsFastTutorialV5.1/FsFastGroupLevel here]

The group-level (Random-Effects) analysis repeats the contrasts performed for individual subjects on a variance-weighted composite of all your subjects. This will identify voxels for which your contrast effects are consistent across all your participants. For the function MRI group analysis you will need to:

*Concatenate individuals into one file (isxconcat-sess)
*Do not smooth (already smoothed during first-level analysis)
*For each space (lh/rh/mni305):
**Run mri_glmfit using weighted least squares (WLS)
**Correct for multiple comparisons
**Correct for multiple comparisons
*Optionally merge into one volume space

==Before you start==
Ensure that your SUBJECTS_DIR variable is set to the working directory containing all your participants, and that the first-level analyses have been completed for all participants (using selxavg-3)

==Concatenate First-Level Analyses(isxconcat-sess)==
In your '''''$SUBJECTS_DIR''''', there should be a '''''subjects''''' file containing the subjectID for each participant for which you had run selxavg-3. Assume that the analysis directories are called ''my_analysis.*'' ([[Configure_mkanalysis-sess | mkanalysis-sess]]), and that the contrast is called ''conditionA_vs_conditionB'' ([[Configure_mkcontrast-sess | mkcontrast-sess]]). You might find it helpful to use environment variables and run the <code>isxconcat-sess</code> script as follows:
ANALYSIS=my_analysis.sm4
CONTRAST1=conditionA_vs_conditionB
#left hemisphere
isxconcat-sess -sf subjects -analysis ${ANALYSIS}.lh -contrast $CONTRAST1 -o RFX
#right hemisphere
isxconcat-sess -sf subjects -analysis ${ANALYSIS}.rh -contrast $CONTRAST1 -o RFX
#subcortical
isxconcat-sess -sf subjects -analysis ${ANALYSIS}.mni -contrast $CONTRAST1 -o RFX

When finished, a new directory will exist, '''''$SUBJECTS_DIR/RFX''''' and contain sub-folders for each of your analyses (in this case, left- and right-hemisphere and MNI305 analysis). Note that your analysis directory does not have to be called '''RFX''' (e.g., if multiple people are working in the same subjects folder, you might have an output folder named something like '''RFX_CHRIS''', or whatever suits you).

If you have multiple contrasts, you can just create additional CONTRAST variables and rerun the same scripts, replacing ${CONTRAST1} with alternative contrasts.

Note that if you have already run isxconcat-sess in the past, you may get a warning because an earlier group-level directory will already exist. You should probably just delete the old directory and rerun isxconcatenate-sess.
===Protip for Looping Through Contrasts===
Contrasts will have a corresponding .config file in one or more of the analysis directories found in $SUBJECTS_DIR. If you plan to automate this process using a script that loops over analysis folders and contrasts, the following code snippet will create a text file containing the names of the contrasts with .config files:
cd $SUBJECTS_DIR/my_analysis.lh
ls -1 | grep config | sed 's/\.config//g' > ../list_of_contrasts
#now $SUBJECTS_DIR has a text file called list_of_contrasts for use in a loop

==Run mri_glmfit==
'''PAY ATTENTION:''' You will need to run mri_glmfit '''for each contrast''' peformed '''for each of the analyses'''! (but see the section below about scripting)

In the above example, we have 1 contrast (conditionA_vs_conditionB) to be carried out for the my_analysis.lh, my_analysis.rh and my_analysis.mni305 analyses directories. Of course, if we had also contrasted other conditions, then you will have to run mri_glmfit many more times.
===One-Sample Group Mean===
The 1-sample group mean (OSGM) tests whether the contrast A-B differs significantly from zero. The example that follows demonstrates how you would run the conditionA_vs_conditionB OSGM analysis for each of the analysis directories as in my example scenario:
ANALYSIS=my_analysis
CONTRAST1=conditionA_vs_conditionB

#contrast 1 in left hemisphere
cd ${SUBJECTS_DIR}/RFX/${ANALYSIS}.lh/${CONTRAST1}
mri_glmfit --y ces.nii.gz --wls cesvar.nii.gz --osgm --surface fsaverage lh --glmdir glm.wls --nii.gz

#contrast 1 in right hemisphere
cd ${SUBJECTS_DIR}/RFX/${ANALYSIS}.rh/${CONTRAST1}
mri_glmfit --y ces.nii.gz --wls cesvar.nii.gz --osgm --surface fsaverage rh --glmdir glm.wls --nii.gz

#contrast 1 in subcortical MNI space - no need to specify the surface
cd ${SUBJECTS_DIR}/RFX/${ANALYSIS}.mni/${CONTRAST1}
mri_glmfit --y ces.nii.gz --osgm --glmdir glm.wls --nii.gz --eres-save

===Correct for Multiple Comparisons (Surface-Based)===
After you have run the voxel-wise analysis, you need to correct the contrast maps for multiple comparisons. This is because the previous step runs the analysis on each of many thousand voxels independently, which mathematically implies that ''alpha'' of those significant voxels are actually Type I errors. Again, you will have to perform this step for each analysis directory. This takes very little time to run for the surface-based analyses, which can use precomputed values.

An explanation of the parameters: <code>--glmdir glm.wls</code> refers to the directory you specified when you ran the mri_glmfit command. <code>--cache 3 pos</code> indicates that you want to use the pre-cached simulation with a voxel-wise threshold of 10E-3 (i.e., 0.001) and use positive contrast values. <code>--cwpvalthresh .025</code> indicates you will retain clusters with a size-corrected p-value of ''P''<.025 (see the '''Notes''' below).
*my_analysis.lh
**<code>cd ''$SUBJECTS_DIR/RFX/my_analysis.lh/conditionA_vs_conditionB/''</code>
**mri_glmfit-sim --glmdir glm.wls --cache 3 pos --cwpvalthresh .025
*my_analysis.rh
**<code>cd ''$SUBJECTS_DIR/RFX/my_analysis.rh/conditionA_vs_conditionB/''</code>
**mri_glmfit-sim --glmdir glm.wls --cache 3 pos --cwpvalthresh .025

Running this step saves thresholded files in the directory indicated by the <code>--glmdir</code> switch (in the above example, files were saved in glm.wls). There, you will find files named ''cache.*.nii.gz''. You can then load those files as overlays to see which clusters survived the size thresholding.

===Correct for Multiple Comparisons with Permutation Test (MNI305 Space, or Surface-Based)===
The above examples use cached Monte Carlo simulations for speed. The help text for mri_glmfit-sim gives the syntax for running your own sims, which take much longer to execute, but are required for MNI305 space:
mri_glmfit-sim --glmdir glmdir \
--sim nulltype nsim threshold csdbase \
--sim-sign sign
e.g. Monte Carlo correction for surface space (10K iterations, voxel-wise p<.005, positive differences only, no correction for multiple spaces):
mri_glmfit-sim --glmdir glm.wls \
--sim mc-z 10000 2.3 mc-z.pos.th005 --sim-sign pos

e.g. permutation correction for MNI space (10K iterations, voxel-wise p<.01, absolute value differences, pfamilywise=.05 is corrected for 3 spaces):
mri_glmfit-sim --glmdir glm.wls --sim perm 10000 2 perm.abs.th01 \
--sim-sign abs --cwp 0.05 --3spaces

e.g. splitting permutation correction for MNI spaces into 4 parallel jobs for speed (each job will do 2500/10k sims); using -log thresh notation in filenames:
mri_glmfit-sim --glmdir glm.wls --sim perm 10000 3 perm10k.th30 --sim-sign abs --cwp 0.05 --3spaces --bg 4

Options for sim type (listed fastest to slowest) are perm, mc-z, and mc-full.

====Notes====
The choice of '''cwpvalthresh''' was computed by dividing the nominal desired p-value (0.05) by the number of spaces entailed by the simulation. In the above example if we are only looking at the lh and rh surfaces we divide 0.05/2 = 0.025. If you were looking at the lh, rh and MNI305 space, you would have to divide by 3 to maintain the same FWE: 0.05/3=0.0167.

As indicated above, the uncorrected voxel threshold (p=.001) was used to enable the use of pre-cached simulated values with the <code>--cache</code> switch. Though .1, .01, and .001 are easy enough to compute (1, 2, 3, respectively), other values can be computed by noting that this value corresponds to -1 × the base-10 logarithm of the uncorrected p-value. For example, to use cluster-size correction with an uncorrected voxel-wise p-value of 0.005, a spreadsheet or calculator can compute -1×(log100.005)=2.301. Or for a p-value of 0.05: -1×(log100.05)=1.301 . Rather than specify positive values, you can also use negative values (<code>neg</code>) or absolute values (<code>abs</code>, which I think may be the suggested default?).

====Scripting====
Because mri_glmfit has to be run for each combination of analysis directory (*.lh, *.rh, possibly *.mni305) and contrast, the number of commands you need to run multiplies pretty quickly. You can check out the [[OSGM Script]] page for an example of how to automate these calls to mri_glmfit using a shell script.

==Check It Out==
This little snippet shows you how to inspect the results of the MNI305 first level analysis.
===MNI305===
RFXDIR=RFX
ANALYSIS=FAM.sm4
CONTRAST=task
#assuming you used the conventions described above, the glm results can be found
#in the directory '''glm.wls''', and the osgm results in a subdirectory called '''osgm'''
THEDIR=${SUBJECTS_DIR}/${RFXDIR}/${ANALYSIS}.mni305/${CONTRAST}/glm.wls/osgm

tkmeditfv fsaverage orig.mgz -ov ${THEDIR}/perm.abs.01.sig.cluster.nii.gz \
-seg ${THEDIR}/perm.abs.01.sig.ocn.nii.gz \
${THEDIR}/perm.abs.01.sig.ocn.lut

==What Next?==
The cluster-level correction step will create a series of files in each of the glm directories. They will be automatically given names that describe the parameters used. For example, the above commands generated files named ''cache.th30.abs.sig.*''. One of these files is a .annot file, where each cluster is a label in the file. These labels make convenient ''[[Working_with_ROIs_(Freesurfer) | functional ROIs]]'', wouldn't you say?

I haven't done this yet, but it seems that one thing you might want to do is map the labels in the .annot file generated by the group mri_glmfit-sim to each participant using <code>mri_surf2surf</code>. A script called <code>cpannot.sh</code> that simplifies this has been written and copied to the UBFS/Scripts/Shell/ folder. This script is invoked thus:
cpannot.sh $SOURCESUBJECT $TARGETSUBJECT $ANNOTFILE

e.g.:

cpannot.sh fsaverage FS_0183 lausanne.annot

Before you do this, you can check out the details of the cluster statistics, which include things like the associated anatomical labels for each cluster, and also the surface area of the cluster. The clusters can be further subdivided to make them more similar in size by <code>[[Freesurfer Subparcellation | mris_divide_parcellation]]</code>. With a series of roughly equal-sized ROIs, they can be mapped from the fsaverage surface to each participant to extract time series or other information from that individual.

[[Category:FreeSurfer]]
[[Category:Functional Analyses]]

Freesurfer Group-Level Analysis (mri glmfit)

2022-07-21T18:50:29Z

Chris: /* Correct for Multiple Comparisons with Permutation Test (MNI305 Space, or Surface-Based) */

This procedure is based on information from [https://surfer.nmr.mgh.harvard.edu/fswiki/FsFastTutorialV5.1/FsFastGroupLevel here]

The group-level (Random-Effects) analysis repeats the contrasts performed for individual subjects on a variance-weighted composite of all your subjects. This will identify voxels for which your contrast effects are consistent across all your participants. For the function MRI group analysis you will need to:

*Concatenate individuals into one file (isxconcat-sess)
*Do not smooth (already smoothed during first-level analysis)
*For each space (lh/rh/mni305):
**Run mri_glmfit using weighted least squares (WLS)
**Correct for multiple comparisons
**Correct for multiple comparisons
*Optionally merge into one volume space

==Before you start==
Ensure that your SUBJECTS_DIR variable is set to the working directory containing all your participants, and that the first-level analyses have been completed for all participants (using selxavg-3)

==Concatenate First-Level Analyses(isxconcat-sess)==
In your '''''$SUBJECTS_DIR''''', there should be a '''''subjects''''' file containing the subjectID for each participant for which you had run selxavg-3. Assume that the analysis directories are called ''my_analysis.*'' ([[Configure_mkanalysis-sess | mkanalysis-sess]]), and that the contrast is called ''conditionA_vs_conditionB'' ([[Configure_mkcontrast-sess | mkcontrast-sess]]). You might find it helpful to use environment variables and run the <code>isxconcat-sess</code> script as follows:
ANALYSIS=my_analysis.sm4
CONTRAST1=conditionA_vs_conditionB
#left hemisphere
isxconcat-sess -sf subjects -analysis ${ANALYSIS}.lh -contrast $CONTRAST1 -o RFX
#right hemisphere
isxconcat-sess -sf subjects -analysis ${ANALYSIS}.rh -contrast $CONTRAST1 -o RFX
#subcortical
isxconcat-sess -sf subjects -analysis ${ANALYSIS}.mni -contrast $CONTRAST1 -o RFX

When finished, a new directory will exist, '''''$SUBJECTS_DIR/RFX''''' and contain sub-folders for each of your analyses (in this case, left- and right-hemisphere and MNI305 analysis). Note that your analysis directory does not have to be called '''RFX''' (e.g., if multiple people are working in the same subjects folder, you might have an output folder named something like '''RFX_CHRIS''', or whatever suits you).

If you have multiple contrasts, you can just create additional CONTRAST variables and rerun the same scripts, replacing ${CONTRAST1} with alternative contrasts.

Note that if you have already run isxconcat-sess in the past, you may get a warning because an earlier group-level directory will already exist. You should probably just delete the old directory and rerun isxconcatenate-sess.
===Protip for Looping Through Contrasts===
Contrasts will have a corresponding .config file in one or more of the analysis directories found in $SUBJECTS_DIR. If you plan to automate this process using a script that loops over analysis folders and contrasts, the following code snippet will create a text file containing the names of the contrasts with .config files:
cd $SUBJECTS_DIR/my_analysis.lh
ls -1 | grep config | sed 's/\.config//g' > ../list_of_contrasts
#now $SUBJECTS_DIR has a text file called list_of_contrasts for use in a loop

==Run mri_glmfit==
'''PAY ATTENTION:''' You will need to run mri_glmfit '''for each contrast''' peformed '''for each of the analyses'''! (but see the section below about scripting)

In the above example, we have 1 contrast (conditionA_vs_conditionB) to be carried out for the my_analysis.lh, my_analysis.rh and my_analysis.mni305 analyses directories. Of course, if we had also contrasted other conditions, then you will have to run mri_glmfit many more times.
===One-Sample Group Mean===
The 1-sample group mean (OSGM) tests whether the contrast A-B differs significantly from zero. The example that follows demonstrates how you would run the conditionA_vs_conditionB OSGM analysis for each of the analysis directories as in my example scenario:
ANALYSIS=my_analysis
CONTRAST1=conditionA_vs_conditionB

#contrast 1 in left hemisphere
cd ${SUBJECTS_DIR}/RFX/${ANALYSIS}.lh/${CONTRAST1}
mri_glmfit --y ces.nii.gz --wls cesvar.nii.gz --osgm --surface fsaverage lh --glmdir glm.wls --nii.gz

#contrast 1 in right hemisphere
cd ${SUBJECTS_DIR}/RFX/${ANALYSIS}.rh/${CONTRAST1}
mri_glmfit --y ces.nii.gz --wls cesvar.nii.gz --osgm --surface fsaverage rh --glmdir glm.wls --nii.gz

#contrast 1 in subcortical MNI space - no need to specify the surface
cd ${SUBJECTS_DIR}/RFX/${ANALYSIS}.mni/${CONTRAST1}
mri_glmfit --y ces.nii.gz --osgm --glmdir glm.wls --nii.gz --eres-save

===Correct for Multiple Comparisons (Surface-Based)===
After you have run the voxel-wise analysis, you need to correct the contrast maps for multiple comparisons. This is because the previous step runs the analysis on each of many thousand voxels independently, which mathematically implies that ''alpha'' of those significant voxels are actually Type I errors. Again, you will have to perform this step for each analysis directory. This takes very little time to run for the surface-based analyses, which can use precomputed values.

An explanation of the parameters: <code>--glmdir glm.wls</code> refers to the directory you specified when you ran the mri_glmfit command. <code>--cache 3 pos</code> indicates that you want to use the pre-cached simulation with a voxel-wise threshold of 10E-3 (i.e., 0.001) and use positive contrast values. <code>--cwpvalthresh .025</code> indicates you will retain clusters with a size-corrected p-value of ''P''<.025 (see the '''Notes''' below).
*my_analysis.lh
**<code>cd ''$SUBJECTS_DIR/RFX/my_analysis.lh/conditionA_vs_conditionB/''</code>
**mri_glmfit-sim --glmdir glm.wls --cache 3 pos --cwpvalthresh .025
*my_analysis.rh
**<code>cd ''$SUBJECTS_DIR/RFX/my_analysis.rh/conditionA_vs_conditionB/''</code>
**mri_glmfit-sim --glmdir glm.wls --cache 3 pos --cwpvalthresh .025

Running this step saves thresholded files in the directory indicated by the <code>--glmdir</code> switch (in the above example, files were saved in glm.wls). There, you will find files named ''cache.*.nii.gz''. You can then load those files as overlays to see which clusters survived the size thresholding.

===Correct for Multiple Comparisons with Permutation Test (MNI305 Space, or Surface-Based)===
The above examples use cached Monte Carlo simulations for speed. The help text for mri_glmfit-sim gives the syntax for running your own sims, which take much longer to execute, but are required for MNI305 space:
mri_glmfit-sim --glmdir glmdir \
--sim nulltype nsim threshold csdbase \
--sim-sign sign
e.g. Monte Carlo correction for surface space (10K iterations, voxel-wise p<.005, positive differences only, no correction for multiple spaces):
mri_glmfit-sim --glmdir glm.wls \
--sim mc-z 10000 2.3 mc-z.pos.th005 --sim-sign pos

e.g. permutation correction for MNI space (10K iterations, voxel-wise p<.01, absolute value differences, correcting for 3 spaces):
mri_glmfit-sim --glmdir glm.wls --sim perm 10000 2 perm.abs.th01 \
--sim-sign abs --cwp 0.05 --3spaces

e.g. splitting permutation correction for MNI spaces into 4 parallel jobs for speed (each job will do 2500/10k sims); using -log thresh notation in filenames:
mri_glmfit-sim --glmdir glm.wls --sim perm 10000 3 perm10k.th30 --sim-sign abs --cwp 0.05 --3spaces --bg 4

Options for sim type (listed fastest to slowest) are perm, mc-z, and mc-full.

====Notes====
The choice of '''cwpvalthresh''' was computed by dividing the nominal desired p-value (0.05) by the number of spaces entailed by the simulation. In the above example if we are only looking at the lh and rh surfaces we divide 0.05/2 = 0.025. If you were looking at the lh, rh and MNI305 space, you would have to divide by 3 to maintain the same FWE: 0.05/3=0.0167.

As indicated above, the uncorrected voxel threshold (p=.001) was used to enable the use of pre-cached simulated values with the <code>--cache</code> switch. Though .1, .01, and .001 are easy enough to compute (1, 2, 3, respectively), other values can be computed by noting that this value corresponds to -1 × the base-10 logarithm of the uncorrected p-value. For example, to use cluster-size correction with an uncorrected voxel-wise p-value of 0.005, a spreadsheet or calculator can compute -1×(log100.005)=2.301. Or for a p-value of 0.05: -1×(log100.05)=1.301 . Rather than specify positive values, you can also use negative values (<code>neg</code>) or absolute values (<code>abs</code>, which I think may be the suggested default?).

====Scripting====
Because mri_glmfit has to be run for each combination of analysis directory (*.lh, *.rh, possibly *.mni305) and contrast, the number of commands you need to run multiplies pretty quickly. You can check out the [[OSGM Script]] page for an example of how to automate these calls to mri_glmfit using a shell script.

==Check It Out==
This little snippet shows you how to inspect the results of the MNI305 first level analysis.
===MNI305===
RFXDIR=RFX
ANALYSIS=FAM.sm4
CONTRAST=task
#assuming you used the conventions described above, the glm results can be found
#in the directory '''glm.wls''', and the osgm results in a subdirectory called '''osgm'''
THEDIR=${SUBJECTS_DIR}/${RFXDIR}/${ANALYSIS}.mni305/${CONTRAST}/glm.wls/osgm

tkmeditfv fsaverage orig.mgz -ov ${THEDIR}/perm.abs.01.sig.cluster.nii.gz \
-seg ${THEDIR}/perm.abs.01.sig.ocn.nii.gz \
${THEDIR}/perm.abs.01.sig.ocn.lut

==What Next?==
The cluster-level correction step will create a series of files in each of the glm directories. They will be automatically given names that describe the parameters used. For example, the above commands generated files named ''cache.th30.abs.sig.*''. One of these files is a .annot file, where each cluster is a label in the file. These labels make convenient ''[[Working_with_ROIs_(Freesurfer) | functional ROIs]]'', wouldn't you say?

I haven't done this yet, but it seems that one thing you might want to do is map the labels in the .annot file generated by the group mri_glmfit-sim to each participant using <code>mri_surf2surf</code>. A script called <code>cpannot.sh</code> that simplifies this has been written and copied to the UBFS/Scripts/Shell/ folder. This script is invoked thus:
cpannot.sh $SOURCESUBJECT $TARGETSUBJECT $ANNOTFILE

e.g.:

cpannot.sh fsaverage FS_0183 lausanne.annot

Before you do this, you can check out the details of the cluster statistics, which include things like the associated anatomical labels for each cluster, and also the surface area of the cluster. The clusters can be further subdivided to make them more similar in size by <code>[[Freesurfer Subparcellation | mris_divide_parcellation]]</code>. With a series of roughly equal-sized ROIs, they can be mapped from the fsaverage surface to each participant to extract time series or other information from that individual.

[[Category:FreeSurfer]]
[[Category:Functional Analyses]]

Freesurfer Group-Level Analysis (mri glmfit)

2022-07-21T18:49:06Z

Chris: /* Correct for Multiple Comparisons with Permutation Test (MNI305 Space, or Surface-Based) */

This procedure is based on information from [https://surfer.nmr.mgh.harvard.edu/fswiki/FsFastTutorialV5.1/FsFastGroupLevel here]

The group-level (Random-Effects) analysis repeats the contrasts performed for individual subjects on a variance-weighted composite of all your subjects. This will identify voxels for which your contrast effects are consistent across all your participants. For the function MRI group analysis you will need to:

*Concatenate individuals into one file (isxconcat-sess)
*Do not smooth (already smoothed during first-level analysis)
*For each space (lh/rh/mni305):
**Run mri_glmfit using weighted least squares (WLS)
**Correct for multiple comparisons
**Correct for multiple comparisons
*Optionally merge into one volume space

==Before you start==
Ensure that your SUBJECTS_DIR variable is set to the working directory containing all your participants, and that the first-level analyses have been completed for all participants (using selxavg-3)

==Concatenate First-Level Analyses(isxconcat-sess)==
In your '''''$SUBJECTS_DIR''''', there should be a '''''subjects''''' file containing the subjectID for each participant for which you had run selxavg-3. Assume that the analysis directories are called ''my_analysis.*'' ([[Configure_mkanalysis-sess | mkanalysis-sess]]), and that the contrast is called ''conditionA_vs_conditionB'' ([[Configure_mkcontrast-sess | mkcontrast-sess]]). You might find it helpful to use environment variables and run the <code>isxconcat-sess</code> script as follows:
ANALYSIS=my_analysis.sm4
CONTRAST1=conditionA_vs_conditionB
#left hemisphere
isxconcat-sess -sf subjects -analysis ${ANALYSIS}.lh -contrast $CONTRAST1 -o RFX
#right hemisphere
isxconcat-sess -sf subjects -analysis ${ANALYSIS}.rh -contrast $CONTRAST1 -o RFX
#subcortical
isxconcat-sess -sf subjects -analysis ${ANALYSIS}.mni -contrast $CONTRAST1 -o RFX

When finished, a new directory will exist, '''''$SUBJECTS_DIR/RFX''''' and contain sub-folders for each of your analyses (in this case, left- and right-hemisphere and MNI305 analysis). Note that your analysis directory does not have to be called '''RFX''' (e.g., if multiple people are working in the same subjects folder, you might have an output folder named something like '''RFX_CHRIS''', or whatever suits you).

If you have multiple contrasts, you can just create additional CONTRAST variables and rerun the same scripts, replacing ${CONTRAST1} with alternative contrasts.

Note that if you have already run isxconcat-sess in the past, you may get a warning because an earlier group-level directory will already exist. You should probably just delete the old directory and rerun isxconcatenate-sess.
===Protip for Looping Through Contrasts===
Contrasts will have a corresponding .config file in one or more of the analysis directories found in $SUBJECTS_DIR. If you plan to automate this process using a script that loops over analysis folders and contrasts, the following code snippet will create a text file containing the names of the contrasts with .config files:
cd $SUBJECTS_DIR/my_analysis.lh
ls -1 | grep config | sed 's/\.config//g' > ../list_of_contrasts
#now $SUBJECTS_DIR has a text file called list_of_contrasts for use in a loop

==Run mri_glmfit==
'''PAY ATTENTION:''' You will need to run mri_glmfit '''for each contrast''' peformed '''for each of the analyses'''! (but see the section below about scripting)

In the above example, we have 1 contrast (conditionA_vs_conditionB) to be carried out for the my_analysis.lh, my_analysis.rh and my_analysis.mni305 analyses directories. Of course, if we had also contrasted other conditions, then you will have to run mri_glmfit many more times.
===One-Sample Group Mean===
The 1-sample group mean (OSGM) tests whether the contrast A-B differs significantly from zero. The example that follows demonstrates how you would run the conditionA_vs_conditionB OSGM analysis for each of the analysis directories as in my example scenario:
ANALYSIS=my_analysis
CONTRAST1=conditionA_vs_conditionB

#contrast 1 in left hemisphere
cd ${SUBJECTS_DIR}/RFX/${ANALYSIS}.lh/${CONTRAST1}
mri_glmfit --y ces.nii.gz --wls cesvar.nii.gz --osgm --surface fsaverage lh --glmdir glm.wls --nii.gz

#contrast 1 in right hemisphere
cd ${SUBJECTS_DIR}/RFX/${ANALYSIS}.rh/${CONTRAST1}
mri_glmfit --y ces.nii.gz --wls cesvar.nii.gz --osgm --surface fsaverage rh --glmdir glm.wls --nii.gz

#contrast 1 in subcortical MNI space - no need to specify the surface
cd ${SUBJECTS_DIR}/RFX/${ANALYSIS}.mni/${CONTRAST1}
mri_glmfit --y ces.nii.gz --osgm --glmdir glm.wls --nii.gz --eres-save

===Correct for Multiple Comparisons (Surface-Based)===
After you have run the voxel-wise analysis, you need to correct the contrast maps for multiple comparisons. This is because the previous step runs the analysis on each of many thousand voxels independently, which mathematically implies that ''alpha'' of those significant voxels are actually Type I errors. Again, you will have to perform this step for each analysis directory. This takes very little time to run for the surface-based analyses, which can use precomputed values.

An explanation of the parameters: <code>--glmdir glm.wls</code> refers to the directory you specified when you ran the mri_glmfit command. <code>--cache 3 pos</code> indicates that you want to use the pre-cached simulation with a voxel-wise threshold of 10E-3 (i.e., 0.001) and use positive contrast values. <code>--cwpvalthresh .025</code> indicates you will retain clusters with a size-corrected p-value of ''P''<.025 (see the '''Notes''' below).
*my_analysis.lh
**<code>cd ''$SUBJECTS_DIR/RFX/my_analysis.lh/conditionA_vs_conditionB/''</code>
**mri_glmfit-sim --glmdir glm.wls --cache 3 pos --cwpvalthresh .025
*my_analysis.rh
**<code>cd ''$SUBJECTS_DIR/RFX/my_analysis.rh/conditionA_vs_conditionB/''</code>
**mri_glmfit-sim --glmdir glm.wls --cache 3 pos --cwpvalthresh .025

Running this step saves thresholded files in the directory indicated by the <code>--glmdir</code> switch (in the above example, files were saved in glm.wls). There, you will find files named ''cache.*.nii.gz''. You can then load those files as overlays to see which clusters survived the size thresholding.

===Correct for Multiple Comparisons with Permutation Test (MNI305 Space, or Surface-Based)===
The above examples use cached Monte Carlo simulations for speed. The help text for mri_glmfit-sim gives the syntax for running your own sims, which take much longer to execute, but are required for MNI305 space:
mri_glmfit-sim --glmdir glmdir \
--sim nulltype nsim threshold csdbase \
--sim-sign sign
e.g. Monte Carlo correction for surface space (10K iterations, voxel-wise p<.005, positive differences only):
mri_glmfit-sim --glmdir glm.wls \
--sim mc-z 10000 2.3 mc-z.pos.th005 --sim-sign pos

e.g. permutation correction for MNI space (10K iterations, voxel-wise p<.01, absolute value differences):
mri_glmfit-sim --glmdir glm.wls --sim perm 10000 2 perm.abs.th01 \
--sim-sign abs --cwp 0.05 --3spaces

e.g. splitting permutation correction for MNI spaces into 4 parallel jobs for speed (each job will do 2500/10k sims); using -log thresh notation in filenames:
mri_glmfit-sim --glmdir glm.wls --sim perm 10000 3 perm10k.th30 --sim-sign abs --cwp 0.05 --3spaces --bg 4

Options for sim type (listed fastest to slowest) are perm, mc-z, and mc-full.

====Notes====
The choice of '''cwpvalthresh''' was computed by dividing the nominal desired p-value (0.05) by the number of spaces entailed by the simulation. In the above example if we are only looking at the lh and rh surfaces we divide 0.05/2 = 0.025. If you were looking at the lh, rh and MNI305 space, you would have to divide by 3 to maintain the same FWE: 0.05/3=0.0167.

As indicated above, the uncorrected voxel threshold (p=.001) was used to enable the use of pre-cached simulated values with the <code>--cache</code> switch. Though .1, .01, and .001 are easy enough to compute (1, 2, 3, respectively), other values can be computed by noting that this value corresponds to -1 × the base-10 logarithm of the uncorrected p-value. For example, to use cluster-size correction with an uncorrected voxel-wise p-value of 0.005, a spreadsheet or calculator can compute -1×(log100.005)=2.301. Or for a p-value of 0.05: -1×(log100.05)=1.301 . Rather than specify positive values, you can also use negative values (<code>neg</code>) or absolute values (<code>abs</code>, which I think may be the suggested default?).

====Scripting====
Because mri_glmfit has to be run for each combination of analysis directory (*.lh, *.rh, possibly *.mni305) and contrast, the number of commands you need to run multiplies pretty quickly. You can check out the [[OSGM Script]] page for an example of how to automate these calls to mri_glmfit using a shell script.

==Check It Out==
This little snippet shows you how to inspect the results of the MNI305 first level analysis.
===MNI305===
RFXDIR=RFX
ANALYSIS=FAM.sm4
CONTRAST=task
#assuming you used the conventions described above, the glm results can be found
#in the directory '''glm.wls''', and the osgm results in a subdirectory called '''osgm'''
THEDIR=${SUBJECTS_DIR}/${RFXDIR}/${ANALYSIS}.mni305/${CONTRAST}/glm.wls/osgm

tkmeditfv fsaverage orig.mgz -ov ${THEDIR}/perm.abs.01.sig.cluster.nii.gz \
-seg ${THEDIR}/perm.abs.01.sig.ocn.nii.gz \
${THEDIR}/perm.abs.01.sig.ocn.lut

==What Next?==
The cluster-level correction step will create a series of files in each of the glm directories. They will be automatically given names that describe the parameters used. For example, the above commands generated files named ''cache.th30.abs.sig.*''. One of these files is a .annot file, where each cluster is a label in the file. These labels make convenient ''[[Working_with_ROIs_(Freesurfer) | functional ROIs]]'', wouldn't you say?

I haven't done this yet, but it seems that one thing you might want to do is map the labels in the .annot file generated by the group mri_glmfit-sim to each participant using <code>mri_surf2surf</code>. A script called <code>cpannot.sh</code> that simplifies this has been written and copied to the UBFS/Scripts/Shell/ folder. This script is invoked thus:
cpannot.sh $SOURCESUBJECT $TARGETSUBJECT $ANNOTFILE

e.g.:

cpannot.sh fsaverage FS_0183 lausanne.annot

Before you do this, you can check out the details of the cluster statistics, which include things like the associated anatomical labels for each cluster, and also the surface area of the cluster. The clusters can be further subdivided to make them more similar in size by <code>[[Freesurfer Subparcellation | mris_divide_parcellation]]</code>. With a series of roughly equal-sized ROIs, they can be mapped from the fsaverage surface to each participant to extract time series or other information from that individual.

[[Category:FreeSurfer]]
[[Category:Functional Analyses]]

OSGM Script

2022-07-21T17:54:08Z

Chris: /* Contrasts */

A BASH script can be written to take the tedium out of running group level analyses for multiple contrasts.

==Example Script==
Below is a script I wrote to run mri_glmfit and mri_glmfit-sim on a set of 3 contrasts. It accomplishes this by executing the commands within a nested loop.

#!/bin/bash
SUBJECTS_DIR=`pwd`
RFXDIR=RFX #This is the name of the directory into which the group-level analyses will go
HEMIS=( lh rh ) #Which hemispheres are you analyzing? Possible choices are [lh | rh | mni305]
ANALYSES=( LDT.fsaverage.sm4.down ) #Name of the analysis directory created by mkanalysis-sess
CONTRASTS=( task w-v-pw hi-v-lo ) #Names of all contrasts you're interested in

for hemi in "${HEMIS[@]}"
do
for anls in "${ANALYSES[@]}"
do
#the analysis directory contains each contrast directory
ANDIR=${SUBJECTS_DIR}/${RFXDIR}/${anls}.${hemi}
#move to analysis directory for the first time
cd ${ANDIR}
for con in "${CONTRASTS[@]}"
do
cd ${con}
pwd
mri_glmfit --y ces.nii.gz --wls cesvar.nii.gz --osgm \
--surface fsaverage ${hemi} --glmdir glm.wls --nii.gz
#correct for multiple comparisons
mri_glmfit-sim --glmdir glm.wls --cache 2 abs --cwpvalthresh .0167
cd ${ANDIR} #go back to the analysis directory
done
done
done

#There appears to be no cached file for multiple comparison correction in
#MNI space, so this loop handles the mni-space analysis
for anls in "${ANALYSES[@]}"
do
#the analysis directory contains each contrast directory
ANDIR=${SUBJECTS_DIR}/${RFXDIR}/${anls}.mni
#move to analysis directory for the first time
cd ${ANDIR}
for con in "${CONTRASTS[@]}"
do
cd ${con}
mri_glmfit --y ces.nii.gz --wls cesvar.nii.gz --osgm \
--glmdir glm.wls --nii.gz
#correct for multiple comparisons
mri_glmfit-sim --glmdir glm.wls --sim perm 1000 2 perm.abs.01 \
--sim-sign abs --cwp 0.05 --3spaces
cd ${ANDIR} #go back to the analysis directory
done
done

This script ran mri_glmfit on the fsaverage surface for each hemisphere/surface appearing in the HEMIS array. The output volumes go into the glmdir (in this case a directory called glm.wls) and will be saved as gzipped .nii files. The script also did cluster size thresholding using a cached Monte Carlo simulation with uncorrected .01 ("cache 2", or p=10E-02) p-values for positive t-scores. The thresholding was corrected by dividing .05 by 2, the total number of surfaces examined. A more clever way would be to determine the cwpvalthresh by calling the Unix ''bc'' function to calculate it according to the size of the HEMIS array.

== What values do I use for ANALYSES and CONTRASTS? ==
The above script runs in nested for-loops that iterate through each entry in the ANALYSES and CONTRASTS arrays. The script comments tell you what these array values represent, but you might not be clear what the correct values should be.

=== Analyses ===
When you ran mkanalysis-sess in your <code>SUBJECTS_DIR</code>, 1, 2 or 3 directories were created, and will be sitting along-side your individual participant directories. These will have names like ''my_analysis.lh'', ''my_analysis.rh'', and ''my_analysis.mni305''. In this case, your ANALYSES array declaration should read like:
ANALYSES=( my_analysis )
Where you omit the part of the directory name that indicates the surface/space (lh/rh/mni305).
It is our usual practice to use analysis folder names that give some indication of the nature of the processing that was done, and the surface that was used (self or fsaverage). In the above script, the analysis folder is called ''LDT.fsaverage.sm4.down'' because it used par-files called LDT, used the fsaverage surface for the first-level analysis, and the data used a 4mm smoothing kernel and were slice-time corrected in the down direction.

=== Contrasts ===
In the analysis directory will be a series of .mat and .config files: one for each contrast you defined using mkcontrast-sess. Your set of contrast names should be the names of each of these contrasts you would like to see at the group level -- just omit the .mat or .config part of the filename.

[[Category: FreeSurfer]]
[[Category: Functional Analyses]]

Freesurfer Group-Level Analysis (mri glmfit)

2022-07-21T17:30:14Z

Chris: /* Correct for Multiple Comparisons (Surface-Based) */

This procedure is based on information from [https://surfer.nmr.mgh.harvard.edu/fswiki/FsFastTutorialV5.1/FsFastGroupLevel here]

The group-level (Random-Effects) analysis repeats the contrasts performed for individual subjects on a variance-weighted composite of all your subjects. This will identify voxels for which your contrast effects are consistent across all your participants. For the function MRI group analysis you will need to:

*Concatenate individuals into one file (isxconcat-sess)
*Do not smooth (already smoothed during first-level analysis)
*For each space (lh/rh/mni305):
**Run mri_glmfit using weighted least squares (WLS)
**Correct for multiple comparisons
**Correct for multiple comparisons
*Optionally merge into one volume space

==Before you start==
Ensure that your SUBJECTS_DIR variable is set to the working directory containing all your participants, and that the first-level analyses have been completed for all participants (using selxavg-3)

==Concatenate First-Level Analyses(isxconcat-sess)==
In your '''''$SUBJECTS_DIR''''', there should be a '''''subjects''''' file containing the subjectID for each participant for which you had run selxavg-3. Assume that the analysis directories are called ''my_analysis.*'' ([[Configure_mkanalysis-sess | mkanalysis-sess]]), and that the contrast is called ''conditionA_vs_conditionB'' ([[Configure_mkcontrast-sess | mkcontrast-sess]]). You might find it helpful to use environment variables and run the <code>isxconcat-sess</code> script as follows:
ANALYSIS=my_analysis.sm4
CONTRAST1=conditionA_vs_conditionB
#left hemisphere
isxconcat-sess -sf subjects -analysis ${ANALYSIS}.lh -contrast $CONTRAST1 -o RFX
#right hemisphere
isxconcat-sess -sf subjects -analysis ${ANALYSIS}.rh -contrast $CONTRAST1 -o RFX
#subcortical
isxconcat-sess -sf subjects -analysis ${ANALYSIS}.mni -contrast $CONTRAST1 -o RFX

When finished, a new directory will exist, '''''$SUBJECTS_DIR/RFX''''' and contain sub-folders for each of your analyses (in this case, left- and right-hemisphere and MNI305 analysis). Note that your analysis directory does not have to be called '''RFX''' (e.g., if multiple people are working in the same subjects folder, you might have an output folder named something like '''RFX_CHRIS''', or whatever suits you).

If you have multiple contrasts, you can just create additional CONTRAST variables and rerun the same scripts, replacing ${CONTRAST1} with alternative contrasts.

Note that if you have already run isxconcat-sess in the past, you may get a warning because an earlier group-level directory will already exist. You should probably just delete the old directory and rerun isxconcatenate-sess.
===Protip for Looping Through Contrasts===
Contrasts will have a corresponding .config file in one or more of the analysis directories found in $SUBJECTS_DIR. If you plan to automate this process using a script that loops over analysis folders and contrasts, the following code snippet will create a text file containing the names of the contrasts with .config files:
cd $SUBJECTS_DIR/my_analysis.lh
ls -1 | grep config | sed 's/\.config//g' > ../list_of_contrasts
#now $SUBJECTS_DIR has a text file called list_of_contrasts for use in a loop

==Run mri_glmfit==
'''PAY ATTENTION:''' You will need to run mri_glmfit '''for each contrast''' peformed '''for each of the analyses'''! (but see the section below about scripting)

In the above example, we have 1 contrast (conditionA_vs_conditionB) to be carried out for the my_analysis.lh, my_analysis.rh and my_analysis.mni305 analyses directories. Of course, if we had also contrasted other conditions, then you will have to run mri_glmfit many more times.
===One-Sample Group Mean===
The 1-sample group mean (OSGM) tests whether the contrast A-B differs significantly from zero. The example that follows demonstrates how you would run the conditionA_vs_conditionB OSGM analysis for each of the analysis directories as in my example scenario:
ANALYSIS=my_analysis
CONTRAST1=conditionA_vs_conditionB

#contrast 1 in left hemisphere
cd ${SUBJECTS_DIR}/RFX/${ANALYSIS}.lh/${CONTRAST1}
mri_glmfit --y ces.nii.gz --wls cesvar.nii.gz --osgm --surface fsaverage lh --glmdir glm.wls --nii.gz

#contrast 1 in right hemisphere
cd ${SUBJECTS_DIR}/RFX/${ANALYSIS}.rh/${CONTRAST1}
mri_glmfit --y ces.nii.gz --wls cesvar.nii.gz --osgm --surface fsaverage rh --glmdir glm.wls --nii.gz

#contrast 1 in subcortical MNI space - no need to specify the surface
cd ${SUBJECTS_DIR}/RFX/${ANALYSIS}.mni/${CONTRAST1}
mri_glmfit --y ces.nii.gz --osgm --glmdir glm.wls --nii.gz --eres-save

===Correct for Multiple Comparisons (Surface-Based)===
After you have run the voxel-wise analysis, you need to correct the contrast maps for multiple comparisons. This is because the previous step runs the analysis on each of many thousand voxels independently, which mathematically implies that ''alpha'' of those significant voxels are actually Type I errors. Again, you will have to perform this step for each analysis directory. This takes very little time to run for the surface-based analyses, which can use precomputed values.

An explanation of the parameters: <code>--glmdir glm.wls</code> refers to the directory you specified when you ran the mri_glmfit command. <code>--cache 3 pos</code> indicates that you want to use the pre-cached simulation with a voxel-wise threshold of 10E-3 (i.e., 0.001) and use positive contrast values. <code>--cwpvalthresh .025</code> indicates you will retain clusters with a size-corrected p-value of ''P''<.025 (see the '''Notes''' below).
*my_analysis.lh
**<code>cd ''$SUBJECTS_DIR/RFX/my_analysis.lh/conditionA_vs_conditionB/''</code>
**mri_glmfit-sim --glmdir glm.wls --cache 3 pos --cwpvalthresh .025
*my_analysis.rh
**<code>cd ''$SUBJECTS_DIR/RFX/my_analysis.rh/conditionA_vs_conditionB/''</code>
**mri_glmfit-sim --glmdir glm.wls --cache 3 pos --cwpvalthresh .025

Running this step saves thresholded files in the directory indicated by the <code>--glmdir</code> switch (in the above example, files were saved in glm.wls). There, you will find files named ''cache.*.nii.gz''. You can then load those files as overlays to see which clusters survived the size thresholding.

===Correct for Multiple Comparisons with Permutation Test (MNI305 Space, or Surface-Based)===
The above examples use cached Monte Carlo simulations for speed. The help text for mri_glmfit-sim gives the syntax for running your own sims, which take much longer to execute, but are required for MNI305 space:
mri_glmfit-sim --glmdir glmdir \
--sim nulltype nsim threshold csdbase \
--sim-sign sign
e.g. Monte Carlo correction for surface space (10K iterations, voxel-wise p<.005, positive differences only):
mri_glmfit-sim --glmdir glm.wls \
--sim mc-z 10000 2.3 mc-z.pos.005 --sim-sign pos

e.g. permutation correction for MNI space (10K iterations, voxel-wise p<.01, absolute value differences):
mri_glmfit-sim --glmdir glm.wls --sim perm 10000 2 perm.abs.01 \
--sim-sign abs --cwp 0.05 --3spaces

Options for sim type (listed fastest to slowest) are perm, mc-z, and mc-full.

====Notes====
The choice of '''cwpvalthresh''' was computed by dividing the nominal desired p-value (0.05) by the number of spaces entailed by the simulation. In the above example if we are only looking at the lh and rh surfaces we divide 0.05/2 = 0.025. If you were looking at the lh, rh and MNI305 space, you would have to divide by 3 to maintain the same FWE: 0.05/3=0.0167.

As indicated above, the uncorrected voxel threshold (p=.001) was used to enable the use of pre-cached simulated values with the <code>--cache</code> switch. Though .1, .01, and .001 are easy enough to compute (1, 2, 3, respectively), other values can be computed by noting that this value corresponds to -1 × the base-10 logarithm of the uncorrected p-value. For example, to use cluster-size correction with an uncorrected voxel-wise p-value of 0.005, a spreadsheet or calculator can compute -1×(log100.005)=2.301. Or for a p-value of 0.05: -1×(log100.05)=1.301 . Rather than specify positive values, you can also use negative values (<code>neg</code>) or absolute values (<code>abs</code>, which I think may be the suggested default?).

====Scripting====
Because mri_glmfit has to be run for each combination of analysis directory (*.lh, *.rh, possibly *.mni305) and contrast, the number of commands you need to run multiplies pretty quickly. You can check out the [[OSGM Script]] page for an example of how to automate these calls to mri_glmfit using a shell script.

==Check It Out==
This little snippet shows you how to inspect the results of the MNI305 first level analysis.
===MNI305===
RFXDIR=RFX
ANALYSIS=FAM.sm4
CONTRAST=task
#assuming you used the conventions described above, the glm results can be found
#in the directory '''glm.wls''', and the osgm results in a subdirectory called '''osgm'''
THEDIR=${SUBJECTS_DIR}/${RFXDIR}/${ANALYSIS}.mni305/${CONTRAST}/glm.wls/osgm

tkmeditfv fsaverage orig.mgz -ov ${THEDIR}/perm.abs.01.sig.cluster.nii.gz \
-seg ${THEDIR}/perm.abs.01.sig.ocn.nii.gz \
${THEDIR}/perm.abs.01.sig.ocn.lut

==What Next?==
The cluster-level correction step will create a series of files in each of the glm directories. They will be automatically given names that describe the parameters used. For example, the above commands generated files named ''cache.th30.abs.sig.*''. One of these files is a .annot file, where each cluster is a label in the file. These labels make convenient ''[[Working_with_ROIs_(Freesurfer) | functional ROIs]]'', wouldn't you say?

I haven't done this yet, but it seems that one thing you might want to do is map the labels in the .annot file generated by the group mri_glmfit-sim to each participant using <code>mri_surf2surf</code>. A script called <code>cpannot.sh</code> that simplifies this has been written and copied to the UBFS/Scripts/Shell/ folder. This script is invoked thus:
cpannot.sh $SOURCESUBJECT $TARGETSUBJECT $ANNOTFILE

e.g.:

cpannot.sh fsaverage FS_0183 lausanne.annot

Before you do this, you can check out the details of the cluster statistics, which include things like the associated anatomical labels for each cluster, and also the surface area of the cluster. The clusters can be further subdivided to make them more similar in size by <code>[[Freesurfer Subparcellation | mris_divide_parcellation]]</code>. With a series of roughly equal-sized ROIs, they can be mapped from the fsaverage surface to each participant to extract time series or other information from that individual.

[[Category:FreeSurfer]]
[[Category:Functional Analyses]]

Freesurfer Group-Level Analysis (mri glmfit)

2022-07-21T16:20:53Z

Chris: /* Concatenate First-Level Analyses(isxconcat-sess) */

This procedure is based on information from [https://surfer.nmr.mgh.harvard.edu/fswiki/FsFastTutorialV5.1/FsFastGroupLevel here]

The group-level (Random-Effects) analysis repeats the contrasts performed for individual subjects on a variance-weighted composite of all your subjects. This will identify voxels for which your contrast effects are consistent across all your participants. For the function MRI group analysis you will need to:

*Concatenate individuals into one file (isxconcat-sess)
*Do not smooth (already smoothed during first-level analysis)
*For each space (lh/rh/mni305):
**Run mri_glmfit using weighted least squares (WLS)
**Correct for multiple comparisons
**Correct for multiple comparisons
*Optionally merge into one volume space

==Before you start==
Ensure that your SUBJECTS_DIR variable is set to the working directory containing all your participants, and that the first-level analyses have been completed for all participants (using selxavg-3)

==Concatenate First-Level Analyses(isxconcat-sess)==
In your '''''$SUBJECTS_DIR''''', there should be a '''''subjects''''' file containing the subjectID for each participant for which you had run selxavg-3. Assume that the analysis directories are called ''my_analysis.*'' ([[Configure_mkanalysis-sess | mkanalysis-sess]]), and that the contrast is called ''conditionA_vs_conditionB'' ([[Configure_mkcontrast-sess | mkcontrast-sess]]). You might find it helpful to use environment variables and run the <code>isxconcat-sess</code> script as follows:
ANALYSIS=my_analysis.sm4
CONTRAST1=conditionA_vs_conditionB
#left hemisphere
isxconcat-sess -sf subjects -analysis ${ANALYSIS}.lh -contrast $CONTRAST1 -o RFX
#right hemisphere
isxconcat-sess -sf subjects -analysis ${ANALYSIS}.rh -contrast $CONTRAST1 -o RFX
#subcortical
isxconcat-sess -sf subjects -analysis ${ANALYSIS}.mni -contrast $CONTRAST1 -o RFX

When finished, a new directory will exist, '''''$SUBJECTS_DIR/RFX''''' and contain sub-folders for each of your analyses (in this case, left- and right-hemisphere and MNI305 analysis). Note that your analysis directory does not have to be called '''RFX''' (e.g., if multiple people are working in the same subjects folder, you might have an output folder named something like '''RFX_CHRIS''', or whatever suits you).

If you have multiple contrasts, you can just create additional CONTRAST variables and rerun the same scripts, replacing ${CONTRAST1} with alternative contrasts.

Note that if you have already run isxconcat-sess in the past, you may get a warning because an earlier group-level directory will already exist. You should probably just delete the old directory and rerun isxconcatenate-sess.
===Protip for Looping Through Contrasts===
Contrasts will have a corresponding .config file in one or more of the analysis directories found in $SUBJECTS_DIR. If you plan to automate this process using a script that loops over analysis folders and contrasts, the following code snippet will create a text file containing the names of the contrasts with .config files:
cd $SUBJECTS_DIR/my_analysis.lh
ls -1 | grep config | sed 's/\.config//g' > ../list_of_contrasts
#now $SUBJECTS_DIR has a text file called list_of_contrasts for use in a loop

==Run mri_glmfit==
'''PAY ATTENTION:''' You will need to run mri_glmfit '''for each contrast''' peformed '''for each of the analyses'''! (but see the section below about scripting)

In the above example, we have 1 contrast (conditionA_vs_conditionB) to be carried out for the my_analysis.lh, my_analysis.rh and my_analysis.mni305 analyses directories. Of course, if we had also contrasted other conditions, then you will have to run mri_glmfit many more times.
===One-Sample Group Mean===
The 1-sample group mean (OSGM) tests whether the contrast A-B differs significantly from zero. The example that follows demonstrates how you would run the conditionA_vs_conditionB OSGM analysis for each of the analysis directories as in my example scenario:
ANALYSIS=my_analysis
CONTRAST1=conditionA_vs_conditionB

#contrast 1 in left hemisphere
cd ${SUBJECTS_DIR}/RFX/${ANALYSIS}.lh/${CONTRAST1}
mri_glmfit --y ces.nii.gz --wls cesvar.nii.gz --osgm --surface fsaverage lh --glmdir glm.wls --nii.gz

#contrast 1 in right hemisphere
cd ${SUBJECTS_DIR}/RFX/${ANALYSIS}.rh/${CONTRAST1}
mri_glmfit --y ces.nii.gz --wls cesvar.nii.gz --osgm --surface fsaverage rh --glmdir glm.wls --nii.gz

#contrast 1 in subcortical MNI space - no need to specify the surface
cd ${SUBJECTS_DIR}/RFX/${ANALYSIS}.mni/${CONTRAST1}
mri_glmfit --y ces.nii.gz --osgm --glmdir glm.wls --nii.gz --eres-save

===Correct for Multiple Comparisons (Surface-Based)===
After you have run the first-level voxel-wise analysis, you need to correct the contrast maps for multiple comparisons. This is because the previous step runs the analysis on each of many thousand voxels independently, which mathematically implies that ''alpha'' of those significant voxels are actually Type I errors. Again, you will have to perform this step for each analysis directory. This takes very little time to run for the surface-based analyses, which can use precomputed values.

An explanation of the parameters: <code>--glmdir glm.wls</code> refers to the directory you specified when you ran the mri_glmfit command. <code>--cache 3 pos</code> indicates that you want to use the pre-cached simulation with a voxel-wise threshold of 10E-3 (i.e., 0.001) and use positive contrast values. <code>--cwpvalthresh .025</code> indicates you will retain clusters with a size-corrected p-value of ''P''<.025 (see the '''Notes''' below).
*my_analysis.lh
**<code>cd ''$SUBJECTS_DIR/RFX/my_analysis.lh/conditionA_vs_conditionB/''</code>
**mri_glmfit-sim --glmdir glm.wls --cache 3 pos --cwpvalthresh .025
*my_analysis.rh
**<code>cd ''$SUBJECTS_DIR/RFX/my_analysis.rh/conditionA_vs_conditionB/''</code>
**mri_glmfit-sim --glmdir glm.wls --cache 3 pos --cwpvalthresh .025

Running this step saves thresholded files in the directory indicated by the <code>--glmdir</code> switch (in the above example, files were saved in glm.wls). There, you will find files named ''cache.*.nii.gz''. You can then load those files as overlays to see which clusters survived the size thresholding.

===Correct for Multiple Comparisons with Permutation Test (MNI305 Space, or Surface-Based)===
The above examples use cached Monte Carlo simulations for speed. The help text for mri_glmfit-sim gives the syntax for running your own sims, which take much longer to execute, but are required for MNI305 space:
mri_glmfit-sim --glmdir glmdir \
--sim nulltype nsim threshold csdbase \
--sim-sign sign
e.g. Monte Carlo correction for surface space (10K iterations, voxel-wise p<.005, positive differences only):
mri_glmfit-sim --glmdir glm.wls \
--sim mc-z 10000 2.3 mc-z.pos.005 --sim-sign pos

e.g. permutation correction for MNI space (10K iterations, voxel-wise p<.01, absolute value differences):
mri_glmfit-sim --glmdir glm.wls --sim perm 10000 2 perm.abs.01 \
--sim-sign abs --cwp 0.05 --3spaces

Options for sim type (listed fastest to slowest) are perm, mc-z, and mc-full.

====Notes====
The choice of '''cwpvalthresh''' was computed by dividing the nominal desired p-value (0.05) by the number of spaces entailed by the simulation. In the above example if we are only looking at the lh and rh surfaces we divide 0.05/2 = 0.025. If you were looking at the lh, rh and MNI305 space, you would have to divide by 3 to maintain the same FWE: 0.05/3=0.0167.

As indicated above, the uncorrected voxel threshold (p=.001) was used to enable the use of pre-cached simulated values with the <code>--cache</code> switch. Though .1, .01, and .001 are easy enough to compute (1, 2, 3, respectively), other values can be computed by noting that this value corresponds to -1 × the base-10 logarithm of the uncorrected p-value. For example, to use cluster-size correction with an uncorrected voxel-wise p-value of 0.005, a spreadsheet or calculator can compute -1×(log100.005)=2.301. Or for a p-value of 0.05: -1×(log100.05)=1.301 . Rather than specify positive values, you can also use negative values (<code>neg</code>) or absolute values (<code>abs</code>, which I think may be the suggested default?).

====Scripting====
Because mri_glmfit has to be run for each combination of analysis directory (*.lh, *.rh, possibly *.mni305) and contrast, the number of commands you need to run multiplies pretty quickly. You can check out the [[OSGM Script]] page for an example of how to automate these calls to mri_glmfit using a shell script.

==Check It Out==
This little snippet shows you how to inspect the results of the MNI305 first level analysis.
===MNI305===
RFXDIR=RFX
ANALYSIS=FAM.sm4
CONTRAST=task
#assuming you used the conventions described above, the glm results can be found
#in the directory '''glm.wls''', and the osgm results in a subdirectory called '''osgm'''
THEDIR=${SUBJECTS_DIR}/${RFXDIR}/${ANALYSIS}.mni305/${CONTRAST}/glm.wls/osgm

tkmeditfv fsaverage orig.mgz -ov ${THEDIR}/perm.abs.01.sig.cluster.nii.gz \
-seg ${THEDIR}/perm.abs.01.sig.ocn.nii.gz \
${THEDIR}/perm.abs.01.sig.ocn.lut

==What Next?==
The cluster-level correction step will create a series of files in each of the glm directories. They will be automatically given names that describe the parameters used. For example, the above commands generated files named ''cache.th30.abs.sig.*''. One of these files is a .annot file, where each cluster is a label in the file. These labels make convenient ''[[Working_with_ROIs_(Freesurfer) | functional ROIs]]'', wouldn't you say?

I haven't done this yet, but it seems that one thing you might want to do is map the labels in the .annot file generated by the group mri_glmfit-sim to each participant using <code>mri_surf2surf</code>. A script called <code>cpannot.sh</code> that simplifies this has been written and copied to the UBFS/Scripts/Shell/ folder. This script is invoked thus:
cpannot.sh $SOURCESUBJECT $TARGETSUBJECT $ANNOTFILE

e.g.:

cpannot.sh fsaverage FS_0183 lausanne.annot

Before you do this, you can check out the details of the cluster statistics, which include things like the associated anatomical labels for each cluster, and also the surface area of the cluster. The clusters can be further subdivided to make them more similar in size by <code>[[Freesurfer Subparcellation | mris_divide_parcellation]]</code>. With a series of roughly equal-sized ROIs, they can be mapped from the fsaverage surface to each participant to extract time series or other information from that individual.

[[Category:FreeSurfer]]
[[Category:Functional Analyses]]

ML Environment

2022-07-07T00:46:44Z

Chris: /* Spyder Installation */

The lab primarily uses a Linux environment. We have several workstations running the latest version of [https://ubuntu.com/download Ubuntu]. Our machine learning (ML) research is primarily carried out using Google's [https://www.tensorflow.org/ TensorFlow] libraries, written for Python. Workstations with powerful GPUs can use CUDA for GPU acceleration.

Below are the specs and walkthrough for setting up a modest ML programming environment.

=Hardware=
There are no particular hardware requirements for running TensorFlow. Obviously, more disk space and RAM are desirable, as is a [https://developer.nvidia.com/cuda-gpus CUDA-enabled GPU].

=Operating System=
The closed Apple ecosystem might make MacOS a good choice of platform because a Mac is a Mac is a Mac. However, I'm not Mr. Moneybags over here, and Apple drops support for older hardware after a time. You couldn't pay me to use Windows for anything but Office applications, so that leaves Linux. The TensorFlow installation directions assume Ubuntu 16.04 or higher, and though I have dabbled with Debian Linux for a home media server, I typically use Ubuntu. These instructions assume Ubuntu 22.04 LTS.

=Python Version=
Ubuntu 22.04 has retired Python 2.x, and uses Python 3.9 by default, which works nicely with TensorFlow (I understand that getting TensorFlow to work with Python 3.10+ includes some challenges). These instructions assume Python 3.9. If you want to use other versions of Python for other applications, I recommend using virtual environments to manage and switch between Python versions. The pip installation instructions use miniconda to create a Python 3.9 virtual environment.

=Setup=
==TensorFlow Installation==
First off, you can use Conda, but don't use Conda to install TensorFlow. Instead, follow the [https://www.tensorflow.org/install/pip pip install instructions] published by the TensorFlow people. The steps are broadly described below:
===Create a virtual environment===
The first step is to install miniconda if it isn't already installed. Then, create a new Python 3.9 virtual environment:
conda create --name tf python=3.9
Then activate your new environment:
conda activate tf

===Update pip and install tensorflow===
You'll need to update pip first:
pip install --upgrade pip
Then pip install tensorflow:
pip install tensorflow

==Spyder Installation==
Anthony got me using the Spyder IDE for Python programming. Problem is, as of today (July 6, 2022), it's buggy on Ubuntu 22.04. You can install it using pip, as documented [https://bugs.launchpad.net/ubuntu/+source/spyder/+bug/1968479 here]:
pip install -U spyder
However, when I uninstalled the apt package dependencies (there are LOTS), the pip installation no longer worked, so I reinstalled the apt package as well. The apt package binary still won't launch, but at least the pip version will have everything it needs. It's possible that having the apt package installed prior to the pip installation caused pip to not bother installing some critical components. If that's the case, then it may not be strictly necessary to have both the apt and pip versions installed.

Note that pip will have installed spyder into a specific virtual environment (e.g., tf). You won't be able to launch spyder without first activating that virtual environment. After installation, you can launch it from a terminal window:
(base) user@host:~ conda activate tf
(tf) user@host:~ spyder

ML Environment

2022-07-07T00:21:44Z

Chris: /* Update pip and install tensorflow */

ML Environment

2022-07-07T00:01:03Z

Chris:

ML Environment

2022-07-06T23:51:39Z

Chris: /* Python Version */

ML Environment

2022-07-06T23:51:11Z

Chris: /* Python Version */

ML Environment

2022-07-06T23:21:37Z

Chris: /* Spyder Installation */

ML Environment

2022-07-06T23:20:52Z

Chris: Created page with "The lab primarily uses a Linux environment. We have several workstations running the latest version of [https://ubuntu.com/download Ubuntu]. Our machine learning (ML) research..."