Confluence will be unusable 23-July-2024 at 06:00 due to a Crowd upgrade.
...
You can either download a pre-build version of bbcp
or build it yourself from source (or clone the git repo: git clone http://www.slac.stanford.edu/~abh/bbcp/bbcp.git). For more information check the download section in the bbcp
documentation.
...
bbcp is installed in /usr/local/bin/bbcp on the psexport machines. It is a link to the bbcp version in the /reg/common package space:
No Format |
---|
/usr/local/bin/bbcp -> /reg/common/package/bbcp/curr/<arch>/bin/bbcp
|
where <arch> is the architecture and OS version for the system (e.g.: x86_64-rhel5-gcc41-opt for rhel5).
The first test would be to transfer something on the same host. In the following example, you'll be transferring data from the special device (generating zeroes) into the null sink:
Code Block |
---|
% bbcp -P 2 localhost:/dev/zero /dev/null
user@localhost's password:
bbcp: Creating /dev/null/zero
bbcp: At 090413 17:37:56 copy 0% complete; 811858.2 KB/s
bbcp: At 090413 17:37:58 copy 0% complete; 819277.7 KB/s
bbcp: At 090413 17:38:00 copy 0% complete; 827783.2 KB/s
..
|
...
By default bbcp
will use 4 simultaneous streams to send data over the network. You can increase (or decrease) that number by using the "-s" opting and specified the desired number of streams:
Code Block |
---|
% bbcp -P 2 -s 32 localhost:/dev/zero /dev/null
user@localhost's password:
bbcp: Creating /dev/null/zero
bbcp: At 090413 18:12:04 copy 0% complete; 113401.7 KB/s
bbcp: At 090413 18:12:06 copy 0% complete; 113695.8 KB/s
bbcp: At 090413 18:12:08 copy 0% complete; 113981.2 KB/s
..
% bbcp -P 2 -s 1 localhost:/dev/zero /dev/null
user@localhost's password:
bbcp: Creating /dev/null/zero
bbcp: At 090413 18:13:10 copy 0% complete; 99670.7 KB/s
bbcp: At 090413 18:13:12 copy 0% complete; 100668.6 KB/s
bbcp: At 090413 18:13:14 copy 0% complete; 101580.4 KB/s
bbcp: At 090413 18:13:16 copy 0% complete; 101393.9 KB/s
..
|
...
Another parameter worth to consider is so called TCP window size. By default bbcp
would set it to 64 KBytes. The corresponding option for bbcp
is called "-w". In the following example, the desired window size is set to 2 MBytes:
Code Block |
---|
% bbcp -P 2 -w 2M localhost:/dev/zero /dev/null
user@localhost's password:
bbcp: Creating /dev/null/zero
bbcp: At 090413 18:12:04 copy 0% complete; 113401.7 KB/s
bbcp: At 090413 18:12:06 copy 0% complete; 113695.8 KB/s
bbcp: At 090413 18:12:08 copy 0% complete; 113981.2 KB/s
..
|
...
Please use the data mover pool to transfer data to your home institution:
Code Block |
---|
psexport.slac.stanford.edu
|
The following example will copy the source file <srcfilename>
from the LCLS mover node psimport
psexport
to the destination file <dstfilename>
on the local machine, printing the progress every 15 seconds, using 32 streams and a windows size of 2MB:
Code Block |
---|
% bbcp -P 15 -s 32 -w 2M psexport.slac.stanford.edu:<srcfilename> <dstfilename>
|
...
If the bbcp
executable isn't available in the executable search path of user 'user' at remote host 'SomeRemoteHost' then this will be reported as follows:
Code Block |
---|
% bbcp -P 2 user@SomeRemoteHost:/dev/zero /dev/null
bbcp: Command not found.
bbcp: Unable to allocate more than 0 of 4 data streams.
|
...
Sometimes a firewall (either at SLAC or at a remote site) may block certain (ranges of) ports. Unfortunatelly, it's hard to diagnose this problem directly. bbcp
would report this typically as follows:
Code Block |
---|
% bbcp -P 2 user@SomeRemoteHost:/dev/zero /dev/null
bbcp: Accept timed out on port 5031
bbcp: Unable to allocate more than 0 of 4 data streams.
Killed by signal 15.
|
Note, that the actual port number may vary in each specific case because bbcp
would attempt to dynamically allocate the next available (at the remote server machine) port. A workaround for this problem is to use option "-z" which would use reverse connection protocol (i.e., sink to source):
Code Block |
---|
% bbcp -z -P 2 user@SomeRemoteHost:/dev/zero /dev/null
user@SomeRemoteHost's password:
bbcp: Creating /dev/null/zero
bbcp: At 090414 02:59:48 copy 0% complete; 5461.3 KB/s
bbcp: At 090414 02:59:50 copy 0% complete; 13639.7 KB/s
..
|
One cane can find useful tips on how to tun up TCP parameters on machines involved into the data transfer to get the highest data transfer rate with bbcp
:
...