Best organizing hundreds of thousands files for rsync and find
cristi at imagis.ro
Wed Mar 27 23:30:57 MDT 2013
I need to organize about 100 millions small files (and the number grows
up) on a server which should be copied to other server.
I am wondering how many files are recommended to be kept into a folder
for optimal performance? As well, if I have a folder with only
subfolders (not files) what number of subfolders are recommended to have?
As well, the question could be for "find" command, not just for for
rsyncas I am doing some cleanups using find (or for - find).
I made a mistake before and I increased a lot the number of
subfoldersfolders (having just few files within them) and rsync
performance was decreasing considerably. Was a mistake which I will try
So now as the number of files is increasing constantly I need to find
out a solution on long term to correct the current issues.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the rsync