very big rsync only worked partially what are size limitations?

mlaks at mlaks at
Sun Dec 19 02:50:53 GMT 2004

Hi, I am very grateful for rsync!!!

Two days ago i started a backup of a 
630 GB directory on a 700GB raid which has 13,945 subdirectories, to another 
server. It copied over 525GB worth of data, and I see that there are 12,627 
directories on the destination. The command I used was 
rsync -a -e ssh /big/dir/  

I thought maybe it ran out of memory or something - yes there are thousands of 
files in the subdirectories - ie some of the directories may have as many as 
as a couple of hundred or so files in sub sub directories.

I then tried again and started the rsync script on the directory again, and it 
ended without copying over any more stuff, as far as I can see  - size wise 
and number of subdirectories.  :(.

Then I just wanted to get it all over with: so I  just wrote a perl script to 
rsync the remainder of the directories one at a time, and that is running now 
and I will see what happens.

So my question, what are the size limitations on this sort of stupid original 
way of doing it?    My machine has 1GB of RAM and I have 2GB of swap space. 
Of course I am also running Postgresql too which is a bit of a memory hog.

Thanks! and I am very happy to have met rsync!
Thank you all!

More information about the rsync mailing list