rsync many processes and slow backup

[Joe]

On 7/12/19 4:56 AM, Hannes Hutmacher via rsync wrote:
> Hi all! :-)
>  
> I have a small rsync script to sync my data to a usb-disk. It works
> fine, when I start it in console. I get 3 rsync processes (look in
> top) and the backup takes ~25 min. But, when I add the script in
> cron to start it at 1am at night it takes 7 - 9 hours and I see up to
> 180 processes. When I look in top I see a hight load of 60 - 80 and 40
> - 60 waits. Why? Can someone explain why it takes so long when it
> starts with cron?
>  
> This is my rsync command:  rsync -azc --delete "$QUELLORDNER"
> "$ZIELORDNER" 
> This is the entry in cron (crontab -e): * 2 * * *
> /root/backupscript/backup.sh
> Data to sync: 18 Gb, 185.000 files.
>  
> When I look in the log files I see errors like this: 
>  
> rsync:
> rename "/media/usb/sicherung/var/lib/fail2ban/.fail2ban.sqlite3.JCzY1c"
> -> "var/lib/fail2ban/fail2ban.sqlite3": No such file or directory (2) 
>
> rsync error: some files/attrs were not transferred (see previousrsync
> error:
>
> some files/attrs were not transferred (see previous errors) (code 23) at
>
> main.c(1196) [sender=3.1.2]
>
> directory (2)
>
>  
>
> Can you help me to solve this problem?
>
>  
>
> regards,
>
> Hannes Hutmacher
>
>
A couple of questions: (I am not an expert user.)

Does your manual job run as the same user (presumably root) as does your
cron job?

Are you backing up any temporary files that you might be better off
excluding?

Is anything else running at the same time as your cron job which may be
creating and destroying files in the backup source or target? (E.g. Your
firewall?)

I'm not very familiar with fuzzy searching, but using that in places
like /var seems odd.

One of the constant refrains on this list is that using checksums is
almost never a good idea. In combination with fuzzy, it seems even more
tenuous.

Joe