SSH: Difference between revisions
No edit summary |
(→NOHUP) |
||
(30 intermediate revisions by 7 users not shown) | |||
Line 2: | Line 2: | ||
In order to connect via SSH, you will need an SSH client, and different operating systems will use different SSH clients. Though there are many alternatives, suggested clients are provided below. | In order to connect via SSH, you will need an SSH client, and different operating systems will use different SSH clients. Though there are many alternatives, suggested clients are provided below. | ||
'''IMPORTANT: See NOHUP section below before running commands over a remote connection''' | |||
== Connecting via VPN from outside campus == | |||
Note that if you are outside of the UB network (e.g., working at home) you will have to first be connected via UB's VPN using the [http://www.buffalo.edu/ubit/service-guides/connecting/vpn/computer.html Cisco AnyConnect Client] because these computers will ignore any connection request coming from outside Park Hall. If you have never done this before, follow the link to download, install and configure the client to connect to UB's VPN server. | |||
== Windows == | == Windows == | ||
Line 15: | Line 20: | ||
The important information you need to know are the Host Names (or IP addresses) of the computers you wish to connect to. In our lab, we have two workstations, ''wernickesarea'' and ''brocasarea''. | The important information you need to know are the Host Names (or IP addresses) of the computers you wish to connect to. In our lab, we have two workstations, ''wernickesarea'' and ''brocasarea''. | ||
#To connect to either of them, you need to enter the IP address | #To connect to either of them, you need to enter the IP address or hostname for the computer you want to connect to into the first box (in the image above, replace ''example.com'' with the appropriate IP address or hostname). As of February 2020, two of the machines can be addressed via DNS: | ||
*cast-psy341ws02.caset.buffalo.edu (the machine at the desk in the corner) | |||
*cast-psy341ws03.caset.buffalo.edu (the high-performance machine) | |||
#When you have done so, click the '''Open''' button. | #When you have done so, click the '''Open''' button. | ||
#If this is your first time connecting to a particular computer, you will see a window labeled '''PuTTY Security Alert'''. Just hit the '''Yes''' button. All this window is saying is that the program does not yet have the encryption key being used by the computer you are talking to (this warning is to alert you to the potential for certain types of hacks). | #If this is your first time connecting to a particular computer, you will see a window labeled '''PuTTY Security Alert'''. Just hit the '''Yes''' button. All this window is saying is that the program does not yet have the encryption key being used by the computer you are talking to (this warning is to alert you to the potential for certain types of hacks). | ||
Line 32: | Line 40: | ||
#Open up a terminal window | #Open up a terminal window | ||
#*If you have never done this, you will find it under '''Applications/Utilities/Terminal''' | #*If you have never done this, you will find it under '''Applications/Utilities/Terminal''' | ||
#In your terminal window, type <code>ssh | #In your terminal window, type <code> ssh username@[hostname]</code> | ||
After a successful login, you will find yourself with a terminal connection and in your home directory, just as though you had logged in and launched a terminal window on one of the computers in the lab. | After a successful login, you will find yourself with a terminal connection and in your home directory, just as though you had logged in and launched a terminal window on one of the computers in the lab. | ||
===Connecting with the ssh command using Xwindows=== | ===Connecting with the ssh command using Xwindows=== | ||
The Xorg window server is the system that Unix computers use to draw windows and other graphic elements on the screen. If you have an Xorg client installed on your Mac OSX computer, you can have your local screen accept these graphical instructions from the remote computer. This would let you run remote applications that have a graphical user interface. To use X, simply use the -X command line switch when connecting (that's an upper-case X): | The Xorg window server is the system that Unix computers use to draw windows and other graphic elements on the screen. If you have an Xorg client installed on your Mac OSX computer, you can have your local screen accept these graphical instructions from the remote computer. This would let you run remote applications that have a graphical user interface. To use X, simply use the -X command line switch when connecting (that's an upper-case X), e.g.,: | ||
<code>ssh -X | <code>ssh -X username@cast-psy341ws03.caset.buffalo.edu</code> | ||
====Where can I get an Xorg client?==== | ====Where can I get an Xorg client?==== | ||
The OSX Xorg client is | The OSX Xorg client is called XQuartz (homepage: [http://www.xquartz.org/ www.xquartz.org]). If you go to that website, you can download the .dmg installer. | ||
====Is a connection that uses Xorg slow?==== | ====Is a connection that uses Xorg slow?==== | ||
Yep. | Yep. | ||
Line 47: | Line 54: | ||
But I've done it many times now from my kitchen table to use simple text editors, and it hasn't been too ridiculous. On the other hand, it probably isn't a practical way to edit 3D fMRI data (but it can handle MATLAB scripts running SPM pretty well). | But I've done it many times now from my kitchen table to use simple text editors, and it hasn't been too ridiculous. On the other hand, it probably isn't a practical way to edit 3D fMRI data (but it can handle MATLAB scripts running SPM pretty well). | ||
=== File transfer | ==Linux== | ||
See the instructions for OSX, but if you've committed to Linux, you probably already know how to do this. | |||
== File Transfer == | |||
=== Windows === | |||
[https://winscp.net/download/WinSCP-5.9.2-Portable.zip WinSCP] is recommended as a windows client for making secure file transfer connections with the workstations. The linked-to .zip archive can be downloaded and extracted to your local hard disk. Run the extracted WinSCP.exe program (you may wish to move the extracted files to a new directory, such as C:\Program Files, but that is entirely up to you). | |||
WinSCP connects to the lab computers using the same information used to connect via SSH (as described above): Provide the IP address of the machine to which you wish to connect, and log in with your username and password. After successfully logging in, you will find yourself viewing the list of files in your home directory on that machine (/home/<your_username>). You can upload and/or download files using this interface. | |||
=== Linux / MacOS === | |||
==== SSH ==== | |||
You can use SSH as a handy means of transferring files. SFTP is a Secure File Transfer Protocol over SSH. | You can use SSH as a handy means of transferring files. SFTP is a Secure File Transfer Protocol over SSH. | ||
For example: | |||
<code>sftp username@hostIP</code> | |||
<code>get filename</code> | |||
<code>put filename</code> | |||
This method is very straight forward and recommended. | |||
==SCP== | ==== SCP ==== | ||
'''SCP''' stands for '''S'''ecure '''C'''o'''p'''y, which is a quick and easy way to transfer files to and from multiple machines. Below are examples for transferring files in a variety of situations. Use an IP address for Broca's, Wernicke's, etc. in place of "remotehost" below. If you are using windows SSH, or are already using an SSH connection, the localhost is the host on which you are currently connected. On linux (and probably Mac OSX) the scp command functions independently from ssh. | '''SCP''' stands for '''S'''ecure '''C'''o'''p'''y, which is a quick and easy way to transfer files to and from multiple machines. Below are examples for transferring files in a variety of situations. Use an IP address for Broca's, Wernicke's, etc. in place of "remotehost" below. If you are using windows SSH, or are already using an SSH connection, the localhost is the host on which you are currently connected. On linux (and probably Mac OSX) the scp command functions independently from ssh (like SFTP). | ||
Copying a file from a remote host to a local host: | Copying a file from a remote host to a local host: | ||
Line 74: | Line 98: | ||
Copying multiple files from your home folder on a remote host to your current local directory | Copying multiple files from your home folder on a remote host to your current local directory | ||
<code>scp your_username@remotehost:~/\{filename1,filename2\}</code> | <code>scp your_username@remotehost:~/\{filename1,filename2\}</code> | ||
If you want to transfer a folder, you'll first need to compress it recursively. | |||
With Zip: | |||
<code>zip -r foldername.zip foldername</code> | |||
Or with Tar: | |||
<code>tar -czvf foldername.tar.gz foldername</code> | |||
== NOHUP == | |||
Running programs are terminated by the operating system with a hangup (''SIGHUP'') signal. If you've ever had to stop a hanging program in Windows or OSX, you've killed a process with a HUP signal (or the equivalent). | |||
Unfortunately, when you log out of a terminal, a SIGHUP is processed by the operating system that terminates all the programs that are running in that terminal session. This is a problem if you logged into a computer from your home laptop and are running a script for a long period of time (e.g., many of the FreeSurfer steps take hours to run). If you log out of your SSH session, or if your connection drops for some reason, your program will stop prematurely! | |||
This can be avoided by prefixing your commands with the <code>nohup</code> and running it in the background with the ampersand, like the following example: | |||
<span style="color:red;font-weight:bold;">nohup</span> autorecon1.sh 501 T1_ <span style="color:red;font-weight:bold;">&</span> | |||
<span style="font-weight:bold;font-size:large">Remember: Don't forget to end your command with the ampersand!</span> | |||
All the program output gets sent to a file called ''nohup.out'' in your current working directory, but otherwise the program runs as normal, even if you log out of your SSH session. Note that, because the program output is not echoed to the terminal window, you will not immediately be alerted to any errors that may occur. It is recommended that you echo the contents of the nohup.out file after a moment or two, to make sure that things are running as expected: | |||
cat nohup.out | |||
If you want to run multiple processes simultaneously within the same directory, you'll run into a problem using <code>nohup</code> as it dumps all the output into a single ''nohup.out'' file, which becomes very difficult to sift through and interpret. To have more orderly output files, you can redirect <code>nohup</code> to an ''.out'' file of your choosing using an additional in-line command <code> &> filename.out</code>. Here's an example: | |||
nohup autorecon1.sh 501 T1_ &> 501_T1.out & |
Latest revision as of 21:50, 7 July 2021
SSH stands for Secure SHell. More information about SSH can be found here, but in a nutshell, it is an encrypted connection to a Unix/Linux terminal. Because much of what we do in the lab is done through the terminal, connecting to our lab computers via SSH allows you to get things done without being physically in the lab.
In order to connect via SSH, you will need an SSH client, and different operating systems will use different SSH clients. Though there are many alternatives, suggested clients are provided below.
IMPORTANT: See NOHUP section below before running commands over a remote connection
Connecting via VPN from outside campus
Note that if you are outside of the UB network (e.g., working at home) you will have to first be connected via UB's VPN using the Cisco AnyConnect Client because these computers will ignore any connection request coming from outside Park Hall. If you have never done this before, follow the link to download, install and configure the client to connect to UB's VPN server.
Windows
The suggested SSH client for computers running Windows is called PuTTY , and the latest version of the program can be found here. This program does not need to install a bunch of files or modify your computer in any way in order to run. Simply download putty.exe, copy (or move) this file to whichever directory you prefer (e.g., C:\Windows\Program Files x86\PUTTY\) and launch it.
Note that the fact that this simple program doesn't do a big installation song and dance is a blessing and a curse: It's really compact and doesn't muck up your computer with stuff you don't want. On the other hand, you won't have the benefit of having a handy Start Menu shortcut being created for you. You will either have to find the putty.exe file on your computer each time you want to connect, or else create a menu or desktop shortcut manually.
Connecting in PuTTY
When you launch PuTTY, you will see a configuration window similar to the one below:
The important information you need to know are the Host Names (or IP addresses) of the computers you wish to connect to. In our lab, we have two workstations, wernickesarea and brocasarea.
- To connect to either of them, you need to enter the IP address or hostname for the computer you want to connect to into the first box (in the image above, replace example.com with the appropriate IP address or hostname). As of February 2020, two of the machines can be addressed via DNS:
- cast-psy341ws02.caset.buffalo.edu (the machine at the desk in the corner)
- cast-psy341ws03.caset.buffalo.edu (the high-performance machine)
- When you have done so, click the Open button.
- If this is your first time connecting to a particular computer, you will see a window labeled PuTTY Security Alert. Just hit the Yes button. All this window is saying is that the program does not yet have the encryption key being used by the computer you are talking to (this warning is to alert you to the potential for certain types of hacks).
- Once the SSH Connection is open, you should see a terminal prompt asking for your username (this will likely be your first name)
- Enter your username and hit the Enter key
- Next, you will be asked to enter your password
- Enter your password and hit the Enter key
- Please note that you will NOT see your cursor moving, or any characters typed (such as ******), when typing your password. This is a standard PuTTY security feature.
Assuming your username/password credentials check out, you will now find yourself with a terminal connection and in your home directory, just as though you are logged in and have launched a new terminal window. You can use this terminal window to perform any tasks that do not launch applications that have a graphical user interface. For example, you can use a text editor like nano
to edit and run a script, or check on the progress of some ongoing process.
Mac OSX
The good news is that OSX is Unix/Linux under the hood, and that you already have a command-line SSH client installed. In fact, it's slightly more flexible because you can use it to launch programs that have a graphical interface (although you might find some programs run too slowly over the internet to be very useful).
Connecting with the ssh command
- Open up a terminal window
- If you have never done this, you will find it under Applications/Utilities/Terminal
- In your terminal window, type
ssh username@[hostname]
After a successful login, you will find yourself with a terminal connection and in your home directory, just as though you had logged in and launched a terminal window on one of the computers in the lab.
Connecting with the ssh command using Xwindows
The Xorg window server is the system that Unix computers use to draw windows and other graphic elements on the screen. If you have an Xorg client installed on your Mac OSX computer, you can have your local screen accept these graphical instructions from the remote computer. This would let you run remote applications that have a graphical user interface. To use X, simply use the -X command line switch when connecting (that's an upper-case X), e.g.,:
ssh -X username@cast-psy341ws03.caset.buffalo.edu
Where can I get an Xorg client?
The OSX Xorg client is called XQuartz (homepage: www.xquartz.org). If you go to that website, you can download the .dmg installer.
Is a connection that uses Xorg slow?
Yep.
But I've done it many times now from my kitchen table to use simple text editors, and it hasn't been too ridiculous. On the other hand, it probably isn't a practical way to edit 3D fMRI data (but it can handle MATLAB scripts running SPM pretty well).
Linux
See the instructions for OSX, but if you've committed to Linux, you probably already know how to do this.
File Transfer
Windows
WinSCP is recommended as a windows client for making secure file transfer connections with the workstations. The linked-to .zip archive can be downloaded and extracted to your local hard disk. Run the extracted WinSCP.exe program (you may wish to move the extracted files to a new directory, such as C:\Program Files, but that is entirely up to you).
WinSCP connects to the lab computers using the same information used to connect via SSH (as described above): Provide the IP address of the machine to which you wish to connect, and log in with your username and password. After successfully logging in, you will find yourself viewing the list of files in your home directory on that machine (/home/<your_username>). You can upload and/or download files using this interface.
Linux / MacOS
SSH
You can use SSH as a handy means of transferring files. SFTP is a Secure File Transfer Protocol over SSH.
For example:
sftp username@hostIP
get filename
put filename
This method is very straight forward and recommended.
SCP
SCP stands for Secure Copy, which is a quick and easy way to transfer files to and from multiple machines. Below are examples for transferring files in a variety of situations. Use an IP address for Broca's, Wernicke's, etc. in place of "remotehost" below. If you are using windows SSH, or are already using an SSH connection, the localhost is the host on which you are currently connected. On linux (and probably Mac OSX) the scp command functions independently from ssh (like SFTP).
Copying a file from a remote host to a local host:
scp your_username@remotehost:directory/filename /some/local/directory
Copying a file from a local host to a remote host:
scp directory/filename your_username@remotehost:/some/remote/directory
Copying a file(s) from a local host to your home folder on a remote host:
scp filename1 filename2 your_username@remotehost:~
Copying a file from one remote host to another remote host:
scp your_username@remotehost1:/some/remote/directory/filename \
your_username@remotehost2:/some/remote/directory/
Copying multiple files from a remote host to your current local directory
scp your_username@remotehost:/some/remote/directory/\{filename1,filename2,filename3\}
Copying multiple files from your home folder on a remote host to your current local directory
scp your_username@remotehost:~/\{filename1,filename2\}
If you want to transfer a folder, you'll first need to compress it recursively.
With Zip:
zip -r foldername.zip foldername
Or with Tar:
tar -czvf foldername.tar.gz foldername
NOHUP
Running programs are terminated by the operating system with a hangup (SIGHUP) signal. If you've ever had to stop a hanging program in Windows or OSX, you've killed a process with a HUP signal (or the equivalent).
Unfortunately, when you log out of a terminal, a SIGHUP is processed by the operating system that terminates all the programs that are running in that terminal session. This is a problem if you logged into a computer from your home laptop and are running a script for a long period of time (e.g., many of the FreeSurfer steps take hours to run). If you log out of your SSH session, or if your connection drops for some reason, your program will stop prematurely!
This can be avoided by prefixing your commands with the nohup
and running it in the background with the ampersand, like the following example:
nohup autorecon1.sh 501 T1_ &
Remember: Don't forget to end your command with the ampersand!
All the program output gets sent to a file called nohup.out in your current working directory, but otherwise the program runs as normal, even if you log out of your SSH session. Note that, because the program output is not echoed to the terminal window, you will not immediately be alerted to any errors that may occur. It is recommended that you echo the contents of the nohup.out file after a moment or two, to make sure that things are running as expected:
cat nohup.out
If you want to run multiple processes simultaneously within the same directory, you'll run into a problem using nohup
as it dumps all the output into a single nohup.out file, which becomes very difficult to sift through and interpret. To have more orderly output files, you can redirect nohup
to an .out file of your choosing using an additional in-line command &> filename.out
. Here's an example:
nohup autorecon1.sh 501 T1_ &> 501_T1.out &