Building hash table times for large files
Rob Bosch
robbosch at msn.com
Fri Nov 2 17:30:24 GMT 2007
I'm running pre4 on a 77GB file. It seems like the hash table is taking a
long time to be built. I'm not sure what is involved in this step but as an
example the following is logged during a run:
send_files(11, priv1.edb)
send_files mapped priv1.edb of size 79187419136
calling match_sums priv1.edb
<f.st...... priv1.edb
hash search b=131072 len=79187419136
built hash table for entries 0 - 52427
built hash table for entries 52428 - 104855
built hash table for entries 104856 - 157283
built hash table for entries 157284 - 209711
The first two hash table entries (0 - 52427 and 52428 - 104855) took only a
few minutes. The next two entries written to the log file have taken
several hours.
Does this make sense? Should the entries take so long to be built?
Rob
More information about the rsync
mailing list