Huge directory tree: Get files to sync via tools like sysdig

Thomas Güttler guettliml at
Thu Feb 9 09:05:53 UTC 2017


we have a huge directory tree.

  * 17M files (number of files)
  * 2.2TBytes of data.
  * Only 0.1% changes per day

Current pain: rsyncs directory tree traversal needs to long to discover the changed files. Only few files change.

I discovered the tool sysdig which could be used to monitor the files which were changed.

Then we could feed the list of changed files to rsync and avoid the long directory traversal of rsync.

Has someone experience with collecting the changed files with a third party tool which detects which
files were changed?

  Thomas Güttler

Thomas Guettler

More information about the rsync mailing list