Transferring data between RAL and SLAC

If you want to transfer a large amount of data using scp or bbftp you can now use the dedicated transfer machine csfmove01.rl.ac.uk

AFS

For smaller files it is convenient to tranfer the data via AFS. This can be done using the standard copy command which is faster than scp. You just have to ensure you have a SLAC AFS token

[csfc] ~ % klog olaiya@slac.stanford.edu
Password:*****
[csfc] ~ % cp /home/csf/olaiya/file.root /afs/slac.stanford.edu/u/br/olaiya/.

scp

In the case where you want to transfer data between RAL and SLAC and you can't use AFS (i.e the destination disk is on NFS), scp is the preferred command to use:

[csfc] ~ % scp /home/csf/olaiya/file.root olaiya@noric.slac.stanford.edu:.

To copy multiple files from the remote server, you have to put the '*' within speech marks to prevent it being expanded on the command line:

[csfc] ~ % scp "olaiya@noric.slac.stanford.edu:*.root" /home/csf/olaiya/.

For more information type "man scp"

bbftp

When you need to transfer large amounts of data, we recommend you use bbftp. bbftp bbftp is currently blocked at the RAL firewall, unless used from csfmove01.rl.ac.uk, so you have to run the command from this machines. You should find the performance of csfmove01.rl.ac.uk for file transfer to compare very favourably with the othere csf linux machines.

[csfmove01] ~ % bbftp -e "put file.root" -u olaiya -s -p 5 yakut01.slac.stanford.edu

The -s flag remotely starts the bbftpd demon, -p selects how many parallel streams are used during the transfer.

You can transfer multiple files in one command as follows:

[csfmove01] ~ % bbftp -e "mput *.root" -u olaiya -s -p 5 yakut01.slac.stanford.edu

Futhermore you can put your transfer actions in a control file and access it using the -i flag:

[csfmove01] ~ % bbftp -i ctrlfile -u olaiya -s -p 5 yakut01.slac.stanford.edu

Where the ctrlfile contains the following

[csfmove01] ~ % less ctrlfile
lcd /home/csf/olaiya/rootfiles
cd /afs/slac.stanford.edu/g/babar/work/o/olaiya/rootfiles
setoption gzip
mput *root

lcd and cd changes directory on the local and remote hosts respectivley. setoption gzip compresses files before transfer. In the case of ROOT files this does not offer any advantage as ROOT files are already significantly compressed.

For more information type "man bbftp". Additionally yo can go to the bbftp web site

Navigation links: [BaBar UK Home] [Software at RAL] [Objectivity at RAL] [RAL Tier A linux farm]

http://hepunx.rl.ac.uk/BFROOT/datatransfer.html last modified 5th April 2005 by

Emmanuel Olaiya, <E.Olaiya@rl.ac.uk>