Par-Files - Revision history

Chris: /* Generating Par Files from our own Self-Paced Experiments */

2020-11-16T18:24:08Z

Generating Par Files from our own Self-Paced Experiments

← Older revision		Revision as of 14:24, 16 November 2020
Line 35:		Line 35:
	A MATLAB function, <code>FSTSExtractor</code>, has been written to extract the schedule of events from the .mat files generated by our PsychToolBox experiment scripts. Note that the structure of these .mat files is particular to our scripts, and that you shouldn't expect this procedure to work on experiment data files generated by any other means. However, the MATLAB code for this script is provided below and can be modified to accommodate other input data.		A MATLAB function, <code>FSTSExtractor</code>, has been written to extract the schedule of events from the .mat files generated by our PsychToolBox experiment scripts. Note that the structure of these .mat files is particular to our scripts, and that you shouldn't expect this procedure to work on experiment data files generated by any other means. However, the MATLAB code for this script is provided below and can be modified to accommodate other input data.

	<span style="color:red">'''note: If you are working with the SemCat experiment, you should be using <code>~~FSSEMEXTRACTOR~~.m</code>.'''</span>		<span style="color:red">'''note: If you are working with the SemCat experiment, you should be using <code>FSSEMExtractor.m</code>.'''</span>

	=== FSTSExtractor.m ===		=== FSTSExtractor.m ===

Chris: /* Generating Par Files from our own Self-Paced Experiments */

2020-11-16T18:23:22Z

Generating Par Files from our own Self-Paced Experiments

← Older revision		Revision as of 14:23, 16 November 2020
Line 35:		Line 35:
	A MATLAB function, <code>FSTSExtractor</code>, has been written to extract the schedule of events from the .mat files generated by our PsychToolBox experiment scripts. Note that the structure of these .mat files is particular to our scripts, and that you shouldn't expect this procedure to work on experiment data files generated by any other means. However, the MATLAB code for this script is provided below and can be modified to accommodate other input data.		A MATLAB function, <code>FSTSExtractor</code>, has been written to extract the schedule of events from the .mat files generated by our PsychToolBox experiment scripts. Note that the structure of these .mat files is particular to our scripts, and that you shouldn't expect this procedure to work on experiment data files generated by any other means. However, the MATLAB code for this script is provided below and can be modified to accommodate other input data.

	'''note: If you are working with the SemCat experiment, you should be using <code>FSSEMEXTRACTOR.m</code>.'''		<span style="color:red">'''note: If you are working with the SemCat experiment, you should be using <code>FSSEMEXTRACTOR.m</code>.'''</span>

	=== FSTSExtractor.m ===		=== FSTSExtractor.m ===

Chris: /* Generating Par Files from our own Self-Paced Experiments */

2020-11-16T18:22:49Z

Generating Par Files from our own Self-Paced Experiments

← Older revision		Revision as of 14:22, 16 November 2020
Line 35:		Line 35:
	A MATLAB function, <code>FSTSExtractor</code>, has been written to extract the schedule of events from the .mat files generated by our PsychToolBox experiment scripts. Note that the structure of these .mat files is particular to our scripts, and that you shouldn't expect this procedure to work on experiment data files generated by any other means. However, the MATLAB code for this script is provided below and can be modified to accommodate other input data.		A MATLAB function, <code>FSTSExtractor</code>, has been written to extract the schedule of events from the .mat files generated by our PsychToolBox experiment scripts. Note that the structure of these .mat files is particular to our scripts, and that you shouldn't expect this procedure to work on experiment data files generated by any other means. However, the MATLAB code for this script is provided below and can be modified to accommodate other input data.

	note: FSSEMEXTRACTOR.m ~~is used for the SemCat experiment~~.		'''note: If you are working with the SemCat experiment, you should be using <code>FSSEMEXTRACTOR.m</code>.'''

	=== FSTSExtractor.m ===		=== FSTSExtractor.m ===
Line 144:		Line 144:

	For the reading development study, collected in the James Booth lab, we use booth.par as a file name convention. For our Lexical Decision Task (LDT), we have been using LDT.par as a file name convention. Using a consistent and appropriate file naming convention will help avoid confusion and ambiguity.		For the reading development study, collected in the James Booth lab, we use booth.par as a file name convention. For our Lexical Decision Task (LDT), we have been using LDT.par as a file name convention. Using a consistent and appropriate file naming convention will help avoid confusion and ambiguity.

	=Generating PAR Files for Archival Data=		=Generating PAR Files for Archival Data=
	We often work with archival data from other experiments that naturally store their experiment meta-data in other formats. The procedures for generating par files for these data can vary wildly, so I will document them by project.		We often work with archival data from other experiments that naturally store their experiment meta-data in other formats. The procedures for generating par files for these data can vary wildly, so I will document them by project.

Chris: /* Generating PAR Files for Archival Data */

2020-04-15T23:09:25Z

Generating PAR Files for Archival Data

← Older revision		Revision as of 19:09, 15 April 2020
Line 170:		Line 170:
	'''Now''' your $PATH will be updated to include ~/bin and you will be able to use any scripts located therein.		'''Now''' your $PATH will be updated to include ~/bin and you will be able to use any scripts located therein.

	==Using the ABCD scripts to generate .par files==		===Using the ABCD scripts to generate .par files===
	If all scripts are in place, .par file generation is quite easy: Assuming you have a set of unpacked ABCD archive data, your project directory will contain a bunch of subject directories:		If all scripts are in place, .par file generation is quite easy: Assuming you have a set of unpacked ABCD archive data, your project directory will contain a bunch of subject directories:
	*ProjectDir/		*ProjectDir/

Chris: /* Running the Installer Script */

2020-04-15T22:56:30Z

Running the Installer Script

← Older revision		Revision as of 18:56, 15 April 2020
Line 166:		Line 166:
	chmod u+x .		chmod u+x .
	./install		./install
	This script will move the MATLAB scripts to ~/Documents/MATLAB (and barf if that directory does not exist), and move the python and bash scripts to ~/bin (creating ~/bin if necessary).		This script will move the MATLAB scripts to ~/Documents/MATLAB (and barf if that directory does not exist), and move the python and bash scripts to ~/bin (creating ~/bin if necessary). Note there is a minor error in the installer script. It reloads the .bashrc file, but that won't update your path. Instead, after you run the installer, you need to reload your .profile file:
			source ~/.profile
			'''Now''' your $PATH will be updated to include ~/bin and you will be able to use any scripts located therein.

	==Using the ABCD scripts to generate .par files==		==Using the ABCD scripts to generate .par files==

Chris: /* Using the ABCD scripts to generate .par files */

2020-04-15T22:32:26Z

Using the ABCD scripts to generate .par files

← Older revision		Revision as of 18:32, 15 April 2020
Line 182:		Line 182:
	The process is driven by the abcd_parfiles.sh script, which requires a subject ID as a parameter. The subject ID is the NDARINV* part of the subject folder names. For example:		The process is driven by the abcd_parfiles.sh script, which requires a subject ID as a parameter. The subject ID is the NDARINV* part of the subject folder names. For example:
	abcd_parfiles.sh NDARINV17K4X0WD		abcd_parfiles.sh NDARINV17K4X0WD
	This ~~launch~~ MATLAB in the terminal window to run the abcd_extract_eprime scripts. Python is used to read in the output files, and then the shell script handles the file organization and folder creation, if necessary. After the script has completed, you will find the .par files in func/<run>/nback.par:		This launches MATLAB in the terminal window to run the abcd_extract_eprime scripts. Python is used to read in the output files, and then the shell script handles the file organization and folder creation, if necessary. After the script has completed, you will find the .par files in func/<run>/nback.par:
	*ProjectDir/		*ProjectDir/
	**NII/		**NII/

Chris: /* Running the Installer Script */

2020-04-15T22:31:25Z

Running the Installer Script

← Older revision		Revision as of 18:31, 15 April 2020
Line 160:		Line 160:
	===Running the Installer Script===		===Running the Installer Script===
	You can find the script archive in /ubfs/caset/cpmcnorg/Scripts/ABCD_SCRIPTS.tgz. Grab the ABCD_SCRIPT.tgz file, unpack it, and run the installer script I wrote.		You can find the script archive in /ubfs/caset/cpmcnorg/Scripts/ABCD_SCRIPTS.tgz. Grab the ABCD_SCRIPT.tgz file, unpack it, and run the installer script I wrote.
	cp /ubfs/caset/cpmcnorg/Scripts/ABCD_SCRIPTS.tgz		cp /ubfs/caset/cpmcnorg/Scripts/ABCD_SCRIPTS.tgz ./
	tar -xzvf ABCD_SCRIPTS.tgz		tar -xzvf ABCD_SCRIPTS.tgz
	cd ABCD_SCRIPTS		cd ABCD_SCRIPTS
			chmod u+x ./install
			chmod u+x .
	./install		./install
	This script will move the MATLAB scripts to ~/Documents/MATLAB (and barf if that directory does not exist), and move the python and bash scripts to ~/bin (creating ~/bin if necessary).		This script will move the MATLAB scripts to ~/Documents/MATLAB (and barf if that directory does not exist), and move the python and bash scripts to ~/bin (creating ~/bin if necessary).

	==Using the ABCD scripts to generate .par files==		==Using the ABCD scripts to generate .par files==
	If all scripts are in place, .par file generation is quite easy: Assuming you have a set of unpacked ABCD archive data, your project directory will contain a bunch of subject directories:		If all scripts are in place, .par file generation is quite easy: Assuming you have a set of unpacked ABCD archive data, your project directory will contain a bunch of subject directories:

Chris at 19:10, 15 April 2020

2020-04-15T19:10:28Z

← Older revision		Revision as of 15:10, 15 April 2020
Line 144:		Line 144:

	For the reading development study, collected in the James Booth lab, we use booth.par as a file name convention. For our Lexical Decision Task (LDT), we have been using LDT.par as a file name convention. Using a consistent and appropriate file naming convention will help avoid confusion and ambiguity.		For the reading development study, collected in the James Booth lab, we use booth.par as a file name convention. For our Lexical Decision Task (LDT), we have been using LDT.par as a file name convention. Using a consistent and appropriate file naming convention will help avoid confusion and ambiguity.
			=Generating PAR Files for Archival Data=
			We often work with archival data from other experiments that naturally store their experiment meta-data in other formats. The procedures for generating par files for these data can vary wildly, so I will document them by project.
			==ABCD Study==
			The ABCD Study includes the run-time data (responses, event timings, etc.) in the same .tgz tarball that contains the raw fMRI DICOM data. Unpacking the tarballs for a participant will produce a complete directory structure for that participant named '''sub-''<SUBJECTID>'''''. Under the subject folder will be a sub-folder for each of the experiment time points (Baseline, 2-year follow-up, etc.). For each time point there is an anat/ subdirectory and a func/ subdirectory. The run-time data for both functional runs can be found in the sub-<SUBJECTID>/<timepoint>/func/*-EventRelatedInformation.txt files. Note there are 2 such files, but these files are ''identical''. Trying to figure out how to parse these files is a nightmare.
			===First: Download abcd_extract_eprime===
			Fortunately someone with knowledge of the file data did the hard work for us. You will need to grab some MATLAB scripts from a GitHub repository: [https://github.com/ABCD-STUDY/abcd_extract_eprime]. From the repository, you can download a .zip file containing all the relevant MATLAB files. Unzip the archive and put all the files into a folder on your MATLAB path (e.g., ~/Documents/MATLAB/). This is important because these scripts need to be in your path in order for them to be visible when we execute them from the command line using a script I have written. These scripts are in a script archive called ABCD_SCRIPTS.tgz that I will be making available on /ubfs/caset/cpmcnorg/Scripts/. Unpack the archive and run the installer script (see below).

			===Second: Grab my Python and BASH Scripts===
			I have written a shell script (abcd_parfiles.sh) and a python script (abcd_to_par.py) to handle all the logic in the event timing parsing. Different datasets acquired with different scanners require different parameters when using the above MATLAB script. The shell script calls the python script, which in turn calls the matlab script. I have copied these scripts to /opt/scripts, which should be in your $PATH, meaning that you should be able to use these scripts without doing anything special. To check, type the following in the command-line terminal:
			which abcd_to_par.py
			You should then see:
			/opt/scripts/abcd_to_par.py
			If, for whatever reason this is not the case, read on ...

			===Running the Installer Script===
			You can find the script archive in /ubfs/caset/cpmcnorg/Scripts/ABCD_SCRIPTS.tgz. Grab the ABCD_SCRIPT.tgz file, unpack it, and run the installer script I wrote.
			cp /ubfs/caset/cpmcnorg/Scripts/ABCD_SCRIPTS.tgz
			tar -xzvf ABCD_SCRIPTS.tgz
			cd ABCD_SCRIPTS
			./install
			This script will move the MATLAB scripts to ~/Documents/MATLAB (and barf if that directory does not exist), and move the python and bash scripts to ~/bin (creating ~/bin if necessary).
			==Using the ABCD scripts to generate .par files==
			If all scripts are in place, .par file generation is quite easy: Assuming you have a set of unpacked ABCD archive data, your project directory will contain a bunch of subject directories:
			*ProjectDir/
			**sub-NDARINV17K4X0WD/
			**sub-NDARINV4JVM9WML/
			**sub-NDARINVJKBDRF1B/
			If you have previously used helper scripts to convert subject DICOM files to NiFTi files, there will also be a NII directory:
			*ProjectDir/
			**NII/
			**sub-NDARINV17K4X0WD/
			**sub-NDARINV4JVM9WML/
			**sub-NDARINVJKBDRF1B/
			The process is driven by the abcd_parfiles.sh script, which requires a subject ID as a parameter. The subject ID is the NDARINV* part of the subject folder names. For example:
			abcd_parfiles.sh NDARINV17K4X0WD
			This launch MATLAB in the terminal window to run the abcd_extract_eprime scripts. Python is used to read in the output files, and then the shell script handles the file organization and folder creation, if necessary. After the script has completed, you will find the .par files in func/<run>/nback.par:
			*ProjectDir/
			**NII/
			***NDARINV17K4X0WD/
			****func/
			*****001/
			******nback.par
			*****002/
			******nback.par
			*****003/
			******nback.par
			*****004/
			******nback.par
			If you have previously converted DICOM to NiFTi files, the run directories will already exist, and you will additionally find the f.nii files along side their respective nback.par files.

	=Aggregating PAR Files=		=Aggregating PAR Files=

Chris: /* Generating Par Files for Self-Paced Experiments */

2020-04-15T17:13:33Z

Generating Par Files for Self-Paced Experiments

← Older revision		Revision as of 13:13, 15 April 2020
Line 32:		Line 32:
	When analyzing data from a self-paced experiment, each participant will have a unique par-file that will have to be generated from his/her MATLAB experimental data file.		When analyzing data from a self-paced experiment, each participant will have a unique par-file that will have to be generated from his/her MATLAB experimental data file.

	== Generating Par Files ~~for~~ Self-Paced Experiments ==		== Generating Par Files from our own Self-Paced Experiments ==
	A MATLAB function, <code>FSTSExtractor</code>, has been written to extract the schedule of events from the .mat files generated by our PsychToolBox experiment scripts. Note that the structure of these .mat files is particular to our scripts, and that you shouldn't expect this procedure to work on experiment data files generated by any other means. However, the MATLAB code for this script is provided below and can be modified to accommodate other input data.		A MATLAB function, <code>FSTSExtractor</code>, has been written to extract the schedule of events from the .mat files generated by our PsychToolBox experiment scripts. Note that the structure of these .mat files is particular to our scripts, and that you shouldn't expect this procedure to work on experiment data files generated by any other means. However, the MATLAB code for this script is provided below and can be modified to accommodate other input data.

Chris: /* Aggregating PAR Files */

2019-07-16T18:07:33Z

Aggregating PAR Files

← Older revision		Revision as of 14:07, 16 July 2019
Line 147:		Line 147:
	=Aggregating PAR Files=		=Aggregating PAR Files=
	FreeSurfer requires the .par files for each functional run to be stored in the run directory along-side the functional data with which they are associated. There may be occasions where you will want to collect all the .par files together. You can use the code below to create a shell script that will aggregate copies of all the par files found in each participant's directories. Because all .par files for analysis must have the same name, we can't simply copy the .par files, or they will overwrite each other. For this reason, the code below will rename the .par files into 001.par, 002.par, etc.		FreeSurfer requires the .par files for each functional run to be stored in the run directory along-side the functional data with which they are associated. There may be occasions where you will want to collect all the .par files together. You can use the code below to create a shell script that will aggregate copies of all the par files found in each participant's directories. Because all .par files for analysis must have the same name, we can't simply copy the .par files, or they will overwrite each other. For this reason, the code below will rename the .par files into 001.par, 002.par, etc.
			'''collectparfiles.sh'''
	#!/bin/bash		#!/bin/bash

Line 165:		Line 165:
	cd ${SUBJECTS_DIR}		cd ${SUBJECTS_DIR}
	done		done
			You would run this script thus:
			collectparfiles.sh FS_*