LFTP

  1. Intro
  2. Documentation
  3. Getting and Installing
  4. Using
  5. Usage examples
  6. Throughput measurements

Intro

LFTP is an enhanced sftp client that allows faster parallel down and uploads as well as a more convenient interface.

Documentation

LFTP Documentation can be found here http://lftp.yar.ru/

Getting and Installing

Most current Linux distributions offer lftp through their package management. However they carry mostly rather old versions which have annoying bugs.

We suggest to use at least lftp versions above 4.7.x. The LFTP web site offers source packages that enable you to compile the most recent version. Check the README and the INSTALL files in the documentation of lftp.

To install lftp using your distribution package management tool

yum install lftp
sudo apt-get install lftp

Using

Log in with your archive account

lftp -u <abc> sftp://archive-sftp.lsdf.kit.edu
Password: <password>

Usage examples

Upload a local file (lfile) to the archive:

lftp>put lfile

Upload, reverse mirror (-R), a local directory (ldir) to a remote directory (rdir) using 4 processes. Use the --continue (-c) option to allow the mirror operation to continue where it left off in case something happens (useful if you archive thousands of files and directories).

lftp>mirror -c -R--parallel=4 ldir rdir

Download a remote directory using 4 streams for every single file. (This improves download speeds for large i.e. more than 100MB, files):

lftp>mirror -c --use-pget-n=4 rdir ldir

From the command line to copy a whole directory towards the archive and redirection of the output:

lftp -u <username>,<password> -e "mirror --reverse -v <source directory> ./private/<destination>/;quit" sftp://archive-sftp.lsdf.kit.edu 2> error_output.txt 1> stdoout.txt

The advantage of the last command is that the verbose output. The error output and the standard out is in seperate files, as such the error messagesare clearly identified. The standout still allows to see each single file which has been transferred. Please be aware that the password is written in clear text on the command line and can be found in the history, this is just an example and should be avoided.

List a single file in remote directory:

lftp>ls -l | grep "$FILENAME" The workaround with grep is necessary because lftp cannot show a single file with ls when using sftp

Many examples with explantations can be found here:
LFTP use examples

Throughput measurements