Huge directory tree: Get files to sync via tools like sysdig

[Karl O. Pinc]

On Fri, 10 Feb 2017 12:38:32 +1300
Henri Shustak <henri.shustak at gmail.com> wrote:

> As Ben mentioned, ZFS snapshots is one possible approach. Another
> approach is to have a faster storage system. I have seen considerable
> speed improvements with rsync on similar data sets by say upgrading
> the storage sub system.

Another possibility could be to use lvm and lvmcache to throw a ssd in
front of the spinning disks.  This would only improve things if
you didn't otherwise fill up the cache with data -- you want
the cache to contain inodes.  So this might work only if your
ssd cache was larger than whatever amount of data you typically
write between rsync runs, plus enough to hold all the inodes
in your rsync-ed fs.

I've not tried this.  I'm not even certain it's a good idea.  It's
just a thought.

Regards,

Karl <kop at meme.com>
Free Software:  "You don't pay back, you pay forward."
                 -- Robert A. Heinlein