...

The first test would be to transfer something on the same host. In the following example, you'll be transferring data from the special device (generating zeroes) into the null sink:

%  bbcp -P 2 localhost:/dev/zero /dev/null
user@localhost's password:
bbcp: Creating /dev/null/zero
bbcp: At 090413 17:37:56 copy 0% complete; 811858.2 KB/s
bbcp: At 090413 17:37:58 copy 0% complete; 819277.7 KB/s
bbcp: At 090413 17:38:00 copy 0% complete; 827783.2 KB/s
..

The "-P" option would tell the tool to print the statistics every 2 seconds. The tool may ask you for a a user for his/her password on the "local" machine because it will be making SSH login (similar to 'ssh user@localhost') and launching the bbcp executable on the server side. By defdault

Transfering data in multiple streams

By default BBCP will use 4 simultaneous streams to send data over the network. You can increase (or decrease) that number by using the "-s" opting and specified the desired number of streams:

% bbcp -P 2 -s 32 localhost:/dev/zero /dev/null
user@localhost's password:
bbcp: Creating /dev/null/zero
bbcp: At 090413 18:12:04 copy 0% complete; 113401.7 KB/s
bbcp: At 090413 18:12:06 copy 0% complete; 113695.8 KB/s
bbcp: At 090413 18:12:08 copy 0% complete; 113981.2 KB/s
..
% bbcp -P 2 -s 1 localhost:/dev/zero /dev/null
user@localhost's password:
bbcp: Creating /dev/null/zero
bbcp: At 090413 18:13:10 copy 0% complete; 99670.7 KB/s
bbcp: At 090413 18:13:12 copy 0% complete; 100668.6 KB/s
bbcp: At 090413 18:13:14 copy 0% complete; 101580.4 KB/s
bbcp: At 090413 18:13:16 copy 0% complete; 101393.9 KB/s
..

The optimal number of streams depends on many factors. For instance, there is no improvement to have more than 1 stream when transferring data over LAN. And that is exactly the situation which is seen in the last example. For WAN it would always make a sense to incresea the number of streams above the default number of 4. In our experience we could get the best (in a specific setup of transferring data between SLAC and IN2P3/Lyon/France) with 32 streams. In the next example, this is also combined with that case the transfer speed was nearing 38 MBytes/sec.

Increasing the TCP window size

Another parameter worth to consider is to so called TCP Window size, which is 64 KBytes by default.

3 Troubleshooting

No bbcp executable available at a remote site

If the bbcp executable isn't available in the binary search path of user 'user' at remote host 'SomeRemoteHost' then this will be reported as follows:

% bbcp -P 2 user@SomeRemoteHost:/dev/zero /dev/null
bbcp: Command not found.
bbcp: Unable to allocate more than 0 of 4 data streams.

...

The server's ports are blocked by a firewal

Sometimes a firewall (either at SLAC or at a remote site) may block certain (ranges of) ports. Unfortunatelly, it's hard to diagnose this problem directly. BBCP would report this typically as follows:

...

% bbcp -z -P 2 user@SomeRemoteHost:/dev/zero /dev/null
user@SomeRemoteHost's password:
bbcp: Creating /dev/null/zero
bbcp: At 090414 02:59:48 copy 0% complete; 5461.3 KB/s
bbcp: At 090414 02:59:50 copy 0% complete; 13639.7 KB/s
..

4 Tuneup

5 Usefull links

• http://pcbunn.cithep.caltech.edu/bbcp/using_bbcp.htm - the link provides usefull tips on tuning up TCP parameters on machines involved into the data transfer.