rsync 3.0.9 segmentation fault

Fri Mar 27 05:29:15 MDT 2015

Yes, I removed "--no-inc-recursive", without success.

--
Best regards / Met vriendelijke groet,

Aron Rotteveel

2015-03-27 12:24 GMT+01:00 Kevin Korb <kmk at sanitarium.net>:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Have you tried removing --no-inc-recursive yet?
>
> On 03/27/2015 07:19 AM, Aron Rotteveel wrote:
> > Hi Roland,
> >
> > Thanks for the reply. Memory usage on both machines seem fine. The
> > server has 4GB's of RAM, of which about 3GB is used during the file
> > list build and about 1.5GB is used during the actual transfer. The
> > client has 16GB of RAM with a peak usage of 8.5GB.
> >
> > I just tried three transfers in a row and it consistently breaks at
> > a certain point, after which I get the "ERROR: out of memory in
> > flist_expand [sender]" error. There is not much special to mention
> > regarding the file on which it breaks: it's a 22KB JPEG file with
> > no special attributes.
> >
> > The backup server is running Debian 7.8, the client runs on CentOS
> > 5.11.
> >
> > A `find . | wc -l` in the backup directory results in 7434013
> > files.
> >
> > -- Best regards / Met vriendelijke groet,
> >
> > Aron Rotteveel
> >
> > 2015-03-19 20:10 GMT+01:00 <devzero at web.de
> > <mailto:devzero at web.de>>:
> >
> > Hi Aron,
> >
> > i hope it`s ok for you if i bring this back on-list. Your issue
> > and the way or possible fix to resolve it may be interesting for
> > others too (that may include future searches etc)
> >
> > so with 3.1.1 we are a step further....
> >
> > i don`t really have a clue what`s happening here but my next step
> > would be taking a closer look on how the memory usage of rsync on
> > the client and server grows.
> >
> > you could log it like this: while true;do ps -eo
> > vsz,rss,sz,rsync|grep cron;sleep 10;done >logfile
> >
> > does it grow continuously? does the oom situation reproducibly
> > happen at a certain size ? what`s the client and server platform?
> > how many files? (-> https://rsync.samba.org/FAQ.html#5 ! )
> >
> > regards roland
> >
> > *Gesendet:* Donnerstag, 19. März 2015 um 12:24 Uhr *Von:* "Aron
> > Rotteveel" <rotteveel.aron at gmail.com
> > <mailto:rotteveel.aron at gmail.com>> *An:* devzero at web.de
> > <mailto:devzero at web.de> *Betreff:* Re: rsync 3.0.9 segmentation
> > fault In addition to my last message:
> >
> > * Client (sender) has 16GB's or RAM, of which only 6.5GB is used
> > during peak. * I tried using --no-inc-recursive, but it does not
> > solve the issue.
> >
> > What currrently is puzzling me is the question of why I am
> > receiving these errors when my server seems to have plenty of
> > memory to spare.
> >
> > -- Best regards / Met vriendelijke groet,
> >
> > Aron Rotteveel
> >
> > 2015-03-19 11:52 GMT+01:00 Aron Rotteveel
> > <rotteveel.aron at gmail.com>:
> >
> > Hi Roland,
> >
> > I just upgrade both the client and host to 3.1.1 and seem to memory
> > related issues now:
> >
> > ERROR: out of memory in make_file [sender] rsync error: error
> > allocating core memory buffers (code 22) at util2.c(102)
> > [sender=3.1.1] [sender] _exit_cleanup(code=22, file=util2.c,
> > line=102): about to call exit(22) [Receiver] _exit_cleanup(code=22,
> > file=io.c, line=1633): about to call exit(22)
> >
> ----------------------------------------------------------------------------
> >
> >
> rsnapshot encountered an error! The program was invoked with
> > these options: /usr/bin/rsnapshot -c \
> > /home/remotebackup/hosts/<redacted>/rsnapshot.conf sync
> >
> ----------------------------------------------------------------------------
> >
> >  -- Best regards / Met vriendelijke groet,
> >
> > Aron Rotteveel
> >
> > 2015-03-18 23:43 GMT+01:00 <devzero at web.de>:
> >
> > Hi,
> >
> > rsync 3.0.9 is quite ancient, more than 3 years old. A lot of bugs
> > have been fixed since then.
> >
> > Is there a chance to update to the latest rsync version and retry
> > with that ?
> >
> > regards Roland
> >
> > *Gesendet:* Dienstag, 17. März 2015 um 11:51 Uhr *Von:* "Aron
> > Rotteveel" <rotteveel.aron at gmail.com> *An:* rsync at lists.samba.org
> > *Betreff:* rsync 3.0.9 segmentation fault Hi,
> >
> > I am experiencing segfaults when transferring files via rsync
> > though sudo. Setup:
> >
> > - Backupserver initiates the rsync command with --delete -vvv
> > --no-inc-recursive --numeric-ids --delete-excluded --relative
> > --rsync-path=/home/backupuser/rsync-wrapper.sh - rsync-wrapper.sh
> > (on the client) contains /usr/bin/sudo /usr/bin/rsync "$@"; - user
> > "backupuser" has sudo access to the rsync command - Both host and
> > client are running 3.0.9
> >
> > The transfer starts and some files are actually transferred. Once a
> > certain file is reached (plain PHP file, no special characters or
> > any other peculiarities) it segfaults.
> >
> > rsync host output:
> >
> > [sender]
> > make_file(<redacted>/libraries/phputf8/mbstring/strlen.php,*,2)
> > rsync: connection unexpectedly closed (51261222 bytes received so
> > far) [Receiver] rsync error: unexplained error (code 139) at
> > io.c(605) [Receiver=3.0.9] [Receiver] _exit_cleanup(code=12,
> > file=io.c, line=605): about to call exit(139)
> >
> ----------------------------------------------------------------------------
> >
> >
> rsnapshot encountered an error! The program was invoked with
> > these options: /usr/bin/rsnapshot -c \
> > /home/remotebackup/hosts/<redacted>/rsnapshot.conf sync
> >
> ----------------------------------------------------------------------------
> >
> >
> ERROR: /usr/bin/rsync returned 139 while processing
> > backupuser@<redacted>:/backup/
> >
> > Client output when using gdb to debug the coredump:
> >
> > warning: no loadable sections found in added symbol-file
> > system-supplied DSO at 0x7fff015fd000 Core was generated by
> > `/usr/bin/rsync --server --sender -vvvlogDtprRe.Lsf --numeric-ids .
> > /backup'. Program terminated with signal 11, Segmentation fault. #0
> > 0x00000035cda7b441 in memcpy () from /lib64/libc.so.6
> >
> > Any help would be greatly appreciated. Please let me know if
> > additional info is required to properly debug this issue. -- Best
> > regards / Met vriendelijke groet,
> >
> > Aron Rotteveel -- Please use reply-all for most replies to avoid
> > omitting the mailing list. To unsubscribe or change options:
> > https://lists.samba.org/mailman/listinfo/rsync Before posting,
> > read: http://www.catb.org/~esr/faqs/smart-questions.html
> >
> >
> >
> >
>
> - --
> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
>         Kevin Korb                      Phone:    (407) 252-6853
>         Systems Administrator           Internet:
>         FutureQuest, Inc.               Kevin at FutureQuest.net  (work)
>         Orlando, Florida                kmk at sanitarium.net (personal)
>         Web page:                       http://www.sanitarium.net/
>         PGP public key available on web site.
> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2
>
> iEYEARECAAYFAlUVPe8ACgkQVKC1jlbQAQdJfQCdFpMSsACi2df9P8eJZQAi4qeB
> xA8Amwbpj9PGpzeQfrd4VnAytU9dN1uc
> =1Ayf
> -----END PGP SIGNATURE-----
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.samba.org/pipermail/rsync/attachments/20150327/87ec0246/attachment.html>