rsync many processes and slow backup
kmk at sanitarium.net
Fri Jul 12 13:25:13 UTC 2019
In addition to the cron problems that others have pointed out there are
#1 rsync's -c is almost always a bad idea and is extremely slow
#2 if rsync isn't networking then -z is a waste of time
On 7/12/19 4:56 AM, Hannes Hutmacher via rsync wrote:
> Hi all! :-)
> I have a small rsync script to sync my data to a usb-disk. It works
> fine, when I start it in console. I get 3 rsync processes (look in top)
> and the backup takes ~25 min. But, when I add the script in cron to
> start it at 1am at night it takes 7 - 9 hours and I see up to
> 180 processes. When I look in top I see a hight load of 60 - 80 and 40 -
> 60 waits. Why? Can someone explain why it takes so long when it starts
> with cron?
> This is my rsync command: rsync -azc --delete "$QUELLORDNER" "$ZIELORDNER"
> This is the entry in cron (crontab -e): * 2 * * *
> Data to sync: 18 Gb, 185.000 files.
> When I look in the log files I see errors like this:
> rename "/media/usb/sicherung/var/lib/fail2ban/.fail2ban.sqlite3.JCzY1c"
> -> "var/lib/fail2ban/fail2ban.sqlite3": No such file or directory (2)
> rsync error: some files/attrs were not transferred (see previousrsync error:
> some files/attrs were not transferred (see previous errors) (code 23) at
> main.c(1196) [sender=3.1.2]
> directory (2)
> Can you help me to solve this problem?
> Hannes Hutmacher
Kevin Korb Phone: (407) 252-6853
Systems Administrator Internet:
FutureQuest, Inc. Kevin at FutureQuest.net (work)
Orlando, Florida kmk at sanitarium.net (personal)
Web page: https://sanitarium.net/
PGP public key available on web site.
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 195 bytes
Desc: OpenPGP digital signature
More information about the rsync