memory requirements was RE: out of memory in build_hash_table

[Granzow, Doug (NCI)]

>The key growth factor for the file construction is going to be
>per-file information that's about 56 bytes (last I looked) per file.
>However, the kicker is that the file storage block gets resized
>exponentially as you have more files.  So for 385,000 files it'll
>actually be a 512,000 file block of about 30MB.  (So yeah, I suppose
>an ~50 byte file chunk in memory growing as a power of 2 might average
>out close to 100 bytes/file as an estimate :-))

Hmm... I have a filesystem that contains 3,098,119 files.  That's 3,098,119
* 56 bytes or 173,494,664 bytes (about 165 MB).  Allowing for the
exponential resizing we end up with space for 4,096,000 files * 56 bytes =
218 MB.  But 'top' tells me the rsync running on this filesystem is taking
up 646 MB, about 3 times what it should.

Are there other factors that affect how much memory rsync takes up?  I only
ask because I would certainly prefer it used 218 MB instead of 646. :)

Doug