rsync 3.0.9 segmentation fault

Tue Apr 7 01:52:16 MDT 2015

Anyone have any other ideas I could try to debug this issue? :)

--
Best regards / Met vriendelijke groet,

Aron Rotteveel

2015-03-27 16:02 GMT+01:00 Aron Rotteveel <rotteveel.aron at gmail.com>:

> Hi Kevin,
>
> Just did: same result.
>
> --
> Best regards / Met vriendelijke groet,
>
> Aron Rotteveel
>
> 2015-03-27 14:32 GMT+01:00 Kevin Korb <kmk at sanitarium.net>:
>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Try it without any --delete options.
>>
>> On 03/27/2015 09:31 AM, Aron Rotteveel wrote:
>> > I am now running with --delete --numeric-ids --relative but the
>> > problem still persists.
>> >
>> > -- Best regards / Met vriendelijke groet,
>> >
>> > Aron Rotteveel
>> >
>> > 2015-03-27 14:22 GMT+01:00 Kevin Korb <kmk at sanitarium.net
>> > <mailto:kmk at sanitarium.net>>:
>> >
>> > Try also removing --delete-excluded.  Without those two options
>> > there should be no reason for rsync to require gigs of RAM.  Well,
>> > unless the other system has rsync 2.x.
>> >
>> > On 03/27/2015 07:29 AM, Aron Rotteveel wrote:
>> >> Yes, I removed "--no-inc-recursive", without success.
>> >
>> >> -- Best regards / Met vriendelijke groet,
>> >
>> >> Aron Rotteveel
>> >
>> >> 2015-03-27 12:24 GMT+01:00 Kevin Korb <kmk at sanitarium.net
>> >> <mailto:kmk at sanitarium.net> <mailto:kmk at sanitarium.net
>> >> <mailto:kmk at sanitarium.net>>>:
>> >
>> >> Have you tried removing --no-inc-recursive yet?
>> >
>> >> On 03/27/2015 07:19 AM, Aron Rotteveel wrote:
>> >>> Hi Roland,
>> >
>> >>> Thanks for the reply. Memory usage on both machines seem fine.
>> >>> The server has 4GB's of RAM, of which about 3GB is used during
>> >>> the file list build and about 1.5GB is used during the actual
>> >>> transfer. The client has 16GB of RAM with a peak usage of
>> >>> 8.5GB.
>> >
>> >>> I just tried three transfers in a row and it consistently
>> >>> breaks at a certain point, after which I get the "ERROR: out of
>> >>> memory in flist_expand [sender]" error. There is not much
>> >>> special to mention regarding the file on which it breaks: it's
>> >>> a 22KB JPEG file with no special attributes.
>> >
>> >>> The backup server is running Debian 7.8, the client runs on
>> >>> CentOS 5.11.
>> >
>> >>> A `find . | wc -l` in the backup directory results in 7434013
>> >>> files.
>> >
>> >>> -- Best regards / Met vriendelijke groet,
>> >
>> >>> Aron Rotteveel
>> >
>> >>> 2015-03-19 20:10 GMT+01:00 <devzero at web.de
>> >>> <mailto:devzero at web.de> <mailto:devzero at web.de
>> >>> <mailto:devzero at web.de>>
>> > <mailto:devzero at web.de <mailto:devzero at web.de>
>> >>> <mailto:devzero at web.de <mailto:devzero at web.de>>>>:
>> >
>> >>> Hi Aron,
>> >
>> >>> i hope it`s ok for you if i bring this back on-list. Your
>> >>> issue and the way or possible fix to resolve it may be
>> >>> interesting for others too (that may include future searches
>> >>> etc)
>> >
>> >>> so with 3.1.1 we are a step further....
>> >
>> >>> i don`t really have a clue what`s happening here but my next
>> >>> step would be taking a closer look on how the memory usage of
>> >>> rsync on the client and server grows.
>> >
>> >>> you could log it like this: while true;do ps -eo
>> >>> vsz,rss,sz,rsync|grep cron;sleep 10;done >logfile
>> >
>> >>> does it grow continuously? does the oom situation reproducibly
>> >>> happen at a certain size ? what`s the client and server
>> >>> platform? how many files? (->
>> >>> https://rsync.samba.org/FAQ.html#5 ! )
>> >
>> >>> regards roland
>> >
>> >>> *Gesendet:* Donnerstag, 19. März 2015 um 12:24 Uhr *Von:*
>> >>> "Aron Rotteveel" <rotteveel.aron at gmail.com
>> >>> <mailto:rotteveel.aron at gmail.com>
>> >>> <mailto:rotteveel.aron at gmail.com
>> >>> <mailto:rotteveel.aron at gmail.com>>
>> >>> <mailto:rotteveel.aron at gmail.com
>> >>> <mailto:rotteveel.aron at gmail.com>
>> >> <mailto:rotteveel.aron at gmail.com
>> >> <mailto:rotteveel.aron at gmail.com>>>> *An:*
>> > devzero at web.de <mailto:devzero at web.de>
>> >> <mailto:devzero at web.de <mailto:devzero at web.de>>
>> >>> <mailto:devzero at web.de <mailto:devzero at web.de>
>> > <mailto:devzero at web.de <mailto:devzero at web.de>>> *Betreff:* Re:
>> >> rsync 3.0.9 segmentation
>> >>> fault In addition to my last message:
>> >
>> >>> * Client (sender) has 16GB's or RAM, of which only 6.5GB is
>> >>> used during peak. * I tried using --no-inc-recursive, but it
>> >>> does not solve the issue.
>> >
>> >>> What currrently is puzzling me is the question of why I am
>> >>> receiving these errors when my server seems to have plenty of
>> >>> memory to spare.
>> >
>> >>> -- Best regards / Met vriendelijke groet,
>> >
>> >>> Aron Rotteveel
>> >
>> >>> 2015-03-19 11:52 GMT+01:00 Aron Rotteveel
>> >>> <rotteveel.aron at gmail.com <mailto:rotteveel.aron at gmail.com>
>> > <mailto:rotteveel.aron at gmail.com
>> > <mailto:rotteveel.aron at gmail.com>>>:
>> >
>> >>> Hi Roland,
>> >
>> >>> I just upgrade both the client and host to 3.1.1 and seem to
>> >>> memory related issues now:
>> >
>> >>> ERROR: out of memory in make_file [sender] rsync error: error
>> >>> allocating core memory buffers (code 22) at util2.c(102)
>> >>> [sender=3.1.1] [sender] _exit_cleanup(code=22, file=util2.c,
>> >>> line=102): about to call exit(22) [Receiver]
>> >>> _exit_cleanup(code=22, file=io.c, line=1633): about to call
>> >>> exit(22)
>> >
>> >
>> >
>> >
>> > -- Please use reply-all for most replies to avoid omitting the
>> > mailing list. To unsubscribe or change options:
>> > https://lists.samba.org/mailman/listinfo/rsync Before posting,
>> > read: http://www.catb.org/~esr/faqs/smart-questions.html
>> >
>> >
>>
>> - --
>>
>> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
>>         Kevin Korb                      Phone:    (407) 252-6853
>>         Systems Administrator           Internet:
>>         FutureQuest, Inc.               Kevin at FutureQuest.net  (work)
>>         Orlando, Florida                kmk at sanitarium.net (personal)
>>         Web page:                       http://www.sanitarium.net/
>>         PGP public key available on web site.
>>
>> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG v2
>>
>> iEYEARECAAYFAlUVW/0ACgkQVKC1jlbQAQeKNQCgkZF+PmmIgXUCrFWGRIBgTNI0
>> WTIAoNfGiD91ubrz/StpXNmMxZ86gzZe
>> =RIfE
>> -----END PGP SIGNATURE-----
>> --
>> Please use reply-all for most replies to avoid omitting the mailing list.
>> To unsubscribe or change options:
>> https://lists.samba.org/mailman/listinfo/rsync
>> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.samba.org/pipermail/rsync/attachments/20150407/88088723/attachment.html>