Easy FTP Scripting in Linux
Rico Nagtalon May 14th, 2006 Add comment
Ever had the need to transfer files through a cron job in a Linux system? Maybe you need to download some rotated logs for archiving? Or upload some files at a given point in time?
Making a Linux shell script do FTP can actually be quite painless. And it works even if the remote host isn’t *nix (Windows, for example). Using your Linux text editor of choice, copy and paste the following script, and tailor it to your needs. Things that you will need to change are in bold:
#!/bin/bash
# The following line will save the date in a variable.
# It will be in the format YYYYMMDD. Useful for
# handling logs, which usually have the date when they
# are rotated. You can omit this if you won't be
# needing dates in your uploads/downloads.
DATE="$(date +%G%m%d)"
# It's much easier to change to the directory in the
# local filesystem where you want to put your files,
# or where the files you want to upload are
# located, before you connect to FTP.
cd /home/myhome/
ftp -n my.domain.com <<!
quote user my_username
quote pass my_password
binary
[Your FTP commands here]
quit
!Reminder: Be sure to execute binary before uploading or downloading images and other non-text files. You will ruin them otherwise because they’ll be transferred in text / ASCII mode. Text files, in my experience, have been fine with either binary or ASCII mode, so I always just switch to binary so I don’t have to worry about it anymore.
The only FTP commands you will most likely ever need are:
cd [directory_name]- Change to the specified directory.get [remote_file]- This will download a file from the remote host to your local machine. The file, which should be in the directory you are currently at in the remote host, will be downloaded into your local machine, in the directory where you did a change directory before connecting to FTP (see line #13 in the script).put [local_file]- This will upload a file from your local machine to the remote host. This file should exist in the local machine’s directory where you are currently at before connecting to FTP (again, see line #13 in the script). This will be placed in the directory where you are currently at in the remote host.
It’s usually a good idea to save the file with a .sh extension. Linux, for the most part, doesn’t really care what the extensions of files are. They’re mostly there for the users’ convenience. It a lot easier for people to know what the files are by looking at the extensions. A .sh will remind you that it’s a shell script.
Once you save the file, be sure to give it execute permissions. This can be achived using the chmod command. To give the owner (that would be you) execute permissions, execute the following from the Linux command line:
chmod u+rx /path/to/file/your_script.shNow, since you have your username and password in the script, you will want it to be accessible only to you. To make sure that it is readble only to you, execute the following:
chmod go= /path/to/file/your_script.shNote: If you want to run your script manually, be sure to include the path to your script. For example: “/home/user/script.sh”. If you are currently in /home/user/, another way to run it would be to execute “./script.sh”. Many beginners forget to add “./” and are perplexed why their script would not run, even if it has the necessary permissions.
You will want to make sure that the script is doing exactly what you want it to since you will most likely be scheduling this to run automatically. A good way to try out your script is to run your commands one by one manually. To do this, execute the following commands in the Linux command line. Be sure to press enter after each line. Substitute your information for the words in bold:
ftp -n my.domain.com
quote user my_username
quote pass my_password
[Your FTP commands here]
quitSome commands that will make life easier as you fine-tune your script:
pwd- Short for “print working directory”. This will tell you where you are in the remote host.ls- Display all the files in the current directory.
When you’re confident that your script is doing what it should, you’re now ready to schedule it. Marc has an article about using crontab here.
Tip: Remember line #7 in the script, which reads DATE="$(date +%G%m%d)"? If you find yourself needing to download rotated logs from a server, you will most likely have need of that. An example of the usage of the DATE variable is:
get my.log.file.$DATE.tar.gzCustomize the words in bold.
That’s it! You’re all set! Happy FTPing!