rsync --delete with empty source folder for fast snapshot deletion: Permissions of hardlinked files are changed to 644. Workaround?
Kevin Korb
kmk at sanitarium.net
Fri Sep 22 00:08:05 UTC 2023
I had intended to come back to this but because I didn't really think I
had anything to add to the discussion I put it at a low enough priority
that I forgot about it. But I saw your bug report and was surprised to
see that I was already unhelpful on this topic but because that original
poster didn't have access to do an rm while you are trying to get a
performance boost.
I have heard in the past that rsyncing an empty dir over a tree to
delete the tree is faster than an rm -rf but I can't say I have ever
benchmarked it to get any actual numbers. But now that I am hearing
that rsync actually adds a bunch of pointless chmods to the process. Is
it still faster given this problem? If so maybe we should be trying to
investigate why rm is so slow. Otherwise, I would agree that an
optimization to not do a bunch of pointless chmods is a good idea. I
would suspect that they are only there to prevent permission denied
errors perhaps on some obscure filesystem (I tested on vfat since it is
so non-compatible but read only files unlink fine there).
On 9/18/23 19:42, rsync--- via rsync wrote:
> Context
> -------
>
> I am one of the active developers of the open source application "Back in Time"
> which uses "rsync" as backend and I want to fix an open issue:
>
> "Back in Time"-Bug:
> https://github.com/bit-team/backintime/issues/994#issuecomment-1724211507
>
> "Back in Time" uses "--link-dest" to reduce traffic and storage by hardlinking
> unchanged files in the backups/snapshots and tries to keep the permissions
> by using "--perms --group --owner" by default.
>
> Older snapshots are then deleted according to a schedule.
>
>
>
> Issue
> -----
>
> When deleting a complete snapshot folder with rsync using an empty source folder
> (which is a best practice for faster deletion than "rm -f") the permissions of hardlinked
> files are changed from eg. 444/-r--r--r-- to 644/-rw-r--r-- for all deleted files
> with existing hardlinks:
>
> mkdir empty
> rsync -a --delete -s empty/ snapshot1/
>
> This distorts the backup history.
>
> There is a corrsponding 6-year-old issue in Bugzilla incl. a patch but the issue is still unfixed:
>
> https://bugzilla.samba.org/show_bug.cgi?id=12806
>
>
>
> Questions
> ---------
>
> 1. Is there a reliable workaround ("--super" is proposed in the issue but may probably not always work)?
> 2. If a rsync developer is reading this: Is there any chance to fix this?
>
>
>
> Steps to reproduce
> ------------------
>
> I have attached a bash script "setup.sh" as txt file which makes the issue fully reproducible (on Ubuntu 22.04).
>
>
--
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
Kevin Korb Phone: (407) 252-6853
Systems Administrator Internet:
FutureQuest, Inc. Kevin at FutureQuest.net (work)
Orlando, Florida kmk at sanitarium.net (personal)
Web page: https://sanitarium.net/
PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
More information about the rsync
mailing list