[Bug 10678] performance problem with lots of hard links?

[samba-bugs at samba.org]

https://bugzilla.samba.org/show_bug.cgi?id=10678

--- Comment #1 from Kevin Korb <rsync at sanitarium.net> 2014-06-27 15:42:27 UTC ---
This is a common problem.  The thing is that rsync has to stat() every single
one of those hard links and remember an inode>filename mapping for each.  If
you have lots of backups this can mean hundreds of millions of calls to stat()
and many megabytes of memory to hold the table.

I highly recommend using lvm2 (I am assuming Linux here) storage for rsync
backups.  This way if you need to move your backups to new storage you can do
so with pvmove instead of rsync.  That isn't going to help you right now but if
you move to lvm2 storage now you won't have this problem the next time you need
to upgrade your storage.

BTW, I have had to do this before and I have never once completed it.  Every
time a few days into the process it was decided to abandon the old backups and
only copy the most recent backups so that backing up could resume.  pvmove otoh
could do this in a couple of hours and not interfere with usage while it is
running.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.