Tips for Quick and Easy File Transfer

And we're back! As I mentioned on my Twitter (below), I took a brief hiatus through June to focus on other projects. It was a great month, marked by gaining permission to begin writing my thesis and set the date for my defense/graduation. I am very excited to complete my graduate training and move on to the next stage in my career. But enough about me. My goal for this post is to talk a little about computer file transfer.

Just a blog update. Posting has been slowed by the hectic summer intro, but look forward to weekly posts again soon! http://t.co/DmIMl97hEs

— Geoffrey Hannigan (@iprophage) June 9, 2015

These days it is becoming more and more common to have to work on multiple computer systems. Whether this is between a work computer and personal computer, or between your local computer and a high performance server or super computer, it is important to move easily between systems. Over the years I have used a few different systems for file transfer, and I wanted to share some of the insight I have gained through these experiences.

Cyberduck

Once I started working more heavily in the realm of computational biology, I had to move a lot of files, especially from my local computer to our lab's remote server which was way more powerful. One of the first programs I used to accomplish this task was Cyberduck. This program offer you a nice Graphic User Interface (GUI) for moving files around a remote computer, as well as transferring files, using a point-and-click interface. This is a pretty nice program, but I found it offered less functionality compared to the other programs I am going to talk about next, so I made the switch.

FileZilla

The next GUI I used was FileZilla. This program offers a similar feel to Cyberduck, with more functionality. It gives great control to the user when it comes to moving files between computers, and also has a great system for editing remote text files that is a lot nicer than Cyberduck. I used this for a while after Cyberduck, and found it to be pretty useful. However I finally got to the point where the command line was just easier to use than a GUI, so I made another switch to using some pretty powerful and standard command line tools.

Bash Tools: mv, scp, rsync

After you get used to using the command line, it becomes a lot easier than using a GUI (point-and-click) in a lot of ways. One of the ways this became easier was moving files around computers, and transferring files between computer systems. The tools I am talking about are standard Bash tools, and will be available on any Unix-like operating systems like Linux or Mac OS.

Instead of pointing and clicking to move files around like we are initially taught, it can be easier to use the mv command in the command like. Simply type 'mv <File to move> <File destination>'. This is actually a lot faster than using the mouse because you don't have to worry about clicking through windows and dragging files between them. Simply specify the paths and you are good to go. If you want to generate a copy of the file so that the original also stays put, just use the cp command in the same way.

For transferring files between computer systems, I like to use scp (secure copy). Like the cp command, this allows you to securely copy your files between systems. All you need is the address of the destination computer, as well as the file paths like so: 'scp /path/to/local/file.txt computer@remote.location.edu:/path/to/file/destination/file.txt'. To recursively copy a directory, just add a -r flag after the scp (see the manual for other uses).

Finally, for particularly large file transfers, I like to use the rsync tool. This is similar to scp, but easily picks up transfers where it left off when the connection is interrupted (if you are using scp, you are going to have to start the transfer over). I like to use the following syntax: 'rsync -ravz --progress /path/to/local/file.txt computer@remote.location.edu:/path/to/file/destination/file.txt'. Go ahead and try this out for transferring those large directories with a lot of files.

So in the end, I would suggest FileZilla to get you started. It offers a familiar GUI that especially appeals to beginners who are not yet used to the command line interface. Once you get used to the command line, I definitely suggest utilizing the standard bash tools. They are easy to use and very robust. As always, if you have any questions about using these programs, let me know in the comments below. Furthermore, if you have other tools that you think are awesome, please share! I would love to hear about some cool new programs, and I'm sure other readers will too!

Citations

Cyberduck Logo
FileZilla Logo
Bash Image