Syncing from two sources to one target and deleting files not in either source

Joe josephj at main.nc.us
Tue Jul 21 15:29:36 UTC 2020


If there isn't a huge list of files, you could use a bash script:

use ls or find to create a list of files on server 1 followed by a list
of files on server 2
run it through sort -u to eliminate duplicates
use ls or find to make a list of files on server 3
use diff to get lines only in server 3 (see
https://unix.stackexchange.com/questions/56625/diff-show-modified-line-from-right-file-only)
run the resulting list through xargs rm.

If there are (or could be) any funny file names, use find instead of ls
and use the terminated by null options on find and xargs. Do dry runs
first with echo instead of rm!

Joe

On 7/21/20 7:14 AM, @lbutlr via rsync wrote:
> Given three servers where Server 1 has several GB of files and Server 2 has several GB of files and Server 3 has all the files from both Server 1 and Server 2 in a single directory, how can I remove files on Server 3 that do not exist on either Server 1 or Server 2?
>
> For example, it's say Server1:/home/user1 and Server2:home/user1 have rsync jobs that sync to Server3:/backups/users/user1 and most of the files exist on both server1 and server2, then server3 contains the newest version of each file, but it also contains files that do not exist on either of Server1 nd Server2. This means that if Server1 needs to restore /home/user1/folder it will not only get all the files that should be in folder on Server2, but also all the files that USED to be in folder on either server1 or server2.
>
> I suspect the answer to this is "this isn't a task for rsync" and if so, what might it be a task for?
>
>
>
>



More information about the rsync mailing list