rsync many processes and slow backup

[Dave Gordon]

>>> Hi all! :-)
>>>
>>> I have a small rsync script to sync my data to a usb-disk. It works
>>> fine, when I start it in console. I get 3 rsync processes (look in
>>> top) and the backup takes ~25 min. But, when I add the script in
>>> cron to start it at 1am at night it takes 7 - 9 hours and I see up to
>>> 180 processes. When I look in top I see a hight load of 60 - 80 and 40
>>> - 60 waits. Why? Can someone explain why it takes so long when it
>>> starts with cron?
>>>
>>> This is my rsync command:  rsync -azc --delete "$QUELLORDNER"
>>> "$ZIELORDNER" 
>>> This is the entry in cron (crontab -e): * 2 * * *
>>> /root/backupscript/backup.sh
>>> Data to sync: 18 Gb, 185.000 files.
>>>
>>> When I look in the log files I see errors like this: 
>>>
>>> rsync:
>>> rename "/media/usb/sicherung/var/lib/fail2ban/.fail2ban.sqlite3.JCzY1c"
>>> -> "var/lib/fail2ban/fail2ban.sqlite3": No such file or directory (2) 
>>>
>>> rsync error: some files/attrs were not transferred (see previousrsync
>>> error:
>>>
>>> some files/attrs were not transferred (see previous errors) (code 23) at
>>>
>>> main.c(1196) [sender=3.1.2]
>>>
>>> directory (2)
>>>
>>> Can you help me to solve this problem?
>>>
>>> regards,
>>> Hannes Hutmacher
>>>
Hi,

I think your crontab entry is wrong. As shown,

* 2 * * * /root/backupscript/backup.sh

it will start the script *once per minute* between 02:00 and 02:59 --
hence the 180 processes that you see! You probably meant

12 2 * * * /root/backupscript/backup.sh

(start once per day at 02:12) or some such thing.

HTH,
Dave

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.samba.org/pipermail/rsync/attachments/20191015/fee3dc3c/attachment.htm>