High Speed WAN Rsync now possible!!!

[Erich Weiler]

Folks-

Just wanted to plug a totally awesome software package from a group I 
know: UDR (UDT Enabled Rsync).

For those not familiar with UDT, it is a low level network protocol 
based on UDP that allows for high speed transfers over high latency WAN 
networks:

http://udt.sourceforge.net/

For a while the UDT API was available, developed by the Laboratory for 
Advanced Computing at the University of Chicago, but there were little 
development around it for actually developing a software suite to allow 
for high speed WAN transfers, such as can be achieved by Aspera, 
GridFTP, FDT, and a couple others.  The problem with those often is:

Aspera: Great but *crazy* expensive
GridFTP: Not bad but non-trivial to set up
FDT: Java (Eh...)

But the awesome thing here is that UDR is a lightweight wrapper for 
rsync that allows for rsync functionality, but uses UDT as the 
underlying protocol for transfer (no rsh or ssh).  It authenticates over 
ssh and then transfers the data over UDT streams.  And supports encryption.

Right now it is kind of in beta but we've been using it for a while and 
it's very stable.  It has been tested on Linux, BSD and OSX.  It may 
compile on other platforms but not much testing has been done on those. 
  Written in C++.

https://github.com/LabAdvComp/UDR

You can clone the repo with 'git clone' and build the code.  It compiles 
very easily and only requires the OpenSSL library as a dependency.

As an example:  We have been trying to transfer data from California to 
our servers in Germany for a while, and have only be getting 10Mb/s. 
With UDR we get 700Mb/s.  Not bad.

There are details on the GitHub page.  Check it out!!

-erich