Aw: Re: rsync 3.0.9 segmentation fault

Thu Mar 19 13:13:59 MDT 2015

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Using --no-inc-recursive would increase memory usage not reduce it.
That option forces rsync to keep the entire tree in memory.

On 03/19/2015 03:10 PM, devzero at web.de wrote:
> Hi Aron,
> 
> i hope it`s ok for you if i bring this back on-list. Your issue and
> the way or possible fix to resolve it may be interesting for others
> too (that may include future searches etc)
> 
> so with 3.1.1 we are a step further....
> 
> i don`t really have a clue what`s happening here but my next step
> would be taking a closer look on how the memory usage of rsync on
> the client and server grows.
> 
> you could log it like this: while true;do ps -eo
> vsz,rss,sz,rsync|grep cron;sleep 10;done >logfile
> 
> does it grow continuously? does the oom situation reproducibly
> happen at a certain size ? what`s the client and server platform? 
> how many files? (-> https://rsync.samba.org/FAQ.html#5 ! )
> 
> regards roland
> 
> *Gesendet:* Donnerstag, 19. März 2015 um 12:24 Uhr *Von:* "Aron
> Rotteveel" <rotteveel.aron at gmail.com> *An:* devzero at web.de 
> *Betreff:* Re: rsync 3.0.9 segmentation fault In addition to my
> last message:
> 
> * Client (sender) has 16GB's or RAM, of which only 6.5GB is used 
> during peak. * I tried using --no-inc-recursive, but it does not
> solve the issue.
> 
> What currrently is puzzling me is the question of why I am
> receiving these errors when my server seems to have plenty of
> memory to spare.
> 
> -- Best regards / Met vriendelijke groet,
> 
> Aron Rotteveel
> 
> 2015-03-19 11:52 GMT+01:00 Aron Rotteveel
> <rotteveel.aron at gmail.com>:
> 
> Hi Roland,
> 
> I just upgrade both the client and host to 3.1.1 and seem to
> memory related issues now:
> 
> ERROR: out of memory in make_file [sender] rsync error: error
> allocating core memory buffers (code 22) at util2.c(102)
> [sender=3.1.1] [sender] _exit_cleanup(code=22, file=util2.c,
> line=102): about to call exit(22) [Receiver] _exit_cleanup(code=22,
> file=io.c, line=1633): about to call exit(22) 
> ----------------------------------------------------------------------------
>
> 
rsnapshot encountered an error! The program was invoked with these
> options: /usr/bin/rsnapshot -c \ 
> /home/remotebackup/hosts/<redacted>/rsnapshot.conf sync 
> ----------------------------------------------------------------------------
>
>  -- Best regards / Met vriendelijke groet,
> 
> Aron Rotteveel
> 
> 2015-03-18 23:43 GMT+01:00 <devzero at web.de>:
> 
> Hi,
> 
> rsync 3.0.9 is quite ancient, more than 3 years old. A lot of bugs
> have been fixed since then.
> 
> Is there a chance to update to the latest rsync version and retry
> with that ?
> 
> regards Roland
> 
> *Gesendet:* Dienstag, 17. März 2015 um 11:51 Uhr *Von:* "Aron
> Rotteveel" <rotteveel.aron at gmail.com> *An:* rsync at lists.samba.org 
> *Betreff:* rsync 3.0.9 segmentation fault Hi,
> 
> I am experiencing segfaults when transferring files via rsync 
> though sudo. Setup:
> 
> - Backupserver initiates the rsync command with --delete -vvv 
> --no-inc-recursive --numeric-ids --delete-excluded --relative 
> --rsync-path=/home/backupuser/rsync-wrapper.sh - rsync-wrapper.sh
> (on the client) contains /usr/bin/sudo /usr/bin/rsync "$@"; - user
> "backupuser" has sudo access to the rsync command - Both host and
> client are running 3.0.9
> 
> The transfer starts and some files are actually transferred. Once a
> certain file is reached (plain PHP file, no special characters or
> any other peculiarities) it segfaults.
> 
> rsync host output:
> 
> [sender] 
> make_file(<redacted>/libraries/phputf8/mbstring/strlen.php,*,2) 
> rsync: connection unexpectedly closed (51261222 bytes received so
> far) [Receiver] rsync error: unexplained error (code 139) at
> io.c(605) [Receiver=3.0.9] [Receiver] _exit_cleanup(code=12,
> file=io.c, line=605): about to call exit(139) 
> ----------------------------------------------------------------------------
>
> 
rsnapshot encountered an error! The program was invoked with
> these options: /usr/bin/rsnapshot -c \ 
> /home/remotebackup/hosts/<redacted>/rsnapshot.conf sync 
> ----------------------------------------------------------------------------
>
> 
ERROR: /usr/bin/rsync returned 139 while processing
> backupuser@<redacted>:/backup/
> 
> Client output when using gdb to debug the coredump:
> 
> warning: no loadable sections found in added symbol-file 
> system-supplied DSO at 0x7fff015fd000 Core was generated by
> `/usr/bin/rsync --server --sender -vvvlogDtprRe.Lsf --numeric-ids .
> /backup'. Program terminated with signal 11, Segmentation fault. #0
> 0x00000035cda7b441 in memcpy () from /lib64/libc.so.6
> 
> Any help would be greatly appreciated. Please let me know if 
> additional info is required to properly debug this issue. -- Best
> regards / Met vriendelijke groet,
> 
> Aron Rotteveel -- Please use reply-all for most replies to avoid
> omitting the mailing list. To unsubscribe or change options: 
> https://lists.samba.org/mailman/listinfo/rsync Before posting, 
> read: http://www.catb.org/~esr/faqs/smart-questions.html
> 
> 
> 

- -- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
	Orlando, Florida		kmk at sanitarium.net (personal)
	Web page:			http://www.sanitarium.net/
	PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iEYEARECAAYFAlULH/cACgkQVKC1jlbQAQcs2wCfWcwjC5Ljed22UAwjkrcf1p8s
vyIAoJ8AkX6OqxdNJFZjkNMjccMFc65Y
=PP+Q
-----END PGP SIGNATURE-----