rsync 3.0.9 segmentation fault

[Kevin Korb]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Try it without any --delete options.

On 03/27/2015 09:31 AM, Aron Rotteveel wrote:
> I am now running with --delete --numeric-ids --relative but the
> problem still persists.
> 
> -- Best regards / Met vriendelijke groet,
> 
> Aron Rotteveel
> 
> 2015-03-27 14:22 GMT+01:00 Kevin Korb <kmk at sanitarium.net 
> <mailto:kmk at sanitarium.net>>:
> 
> Try also removing --delete-excluded.  Without those two options
> there should be no reason for rsync to require gigs of RAM.  Well,
> unless the other system has rsync 2.x.
> 
> On 03/27/2015 07:29 AM, Aron Rotteveel wrote:
>> Yes, I removed "--no-inc-recursive", without success.
> 
>> -- Best regards / Met vriendelijke groet,
> 
>> Aron Rotteveel
> 
>> 2015-03-27 12:24 GMT+01:00 Kevin Korb <kmk at sanitarium.net
>> <mailto:kmk at sanitarium.net> <mailto:kmk at sanitarium.net
>> <mailto:kmk at sanitarium.net>>>:
> 
>> Have you tried removing --no-inc-recursive yet?
> 
>> On 03/27/2015 07:19 AM, Aron Rotteveel wrote:
>>> Hi Roland,
> 
>>> Thanks for the reply. Memory usage on both machines seem fine. 
>>> The server has 4GB's of RAM, of which about 3GB is used during 
>>> the file list build and about 1.5GB is used during the actual 
>>> transfer. The client has 16GB of RAM with a peak usage of
>>> 8.5GB.
> 
>>> I just tried three transfers in a row and it consistently
>>> breaks at a certain point, after which I get the "ERROR: out of
>>> memory in flist_expand [sender]" error. There is not much
>>> special to mention regarding the file on which it breaks: it's
>>> a 22KB JPEG file with no special attributes.
> 
>>> The backup server is running Debian 7.8, the client runs on 
>>> CentOS 5.11.
> 
>>> A `find . | wc -l` in the backup directory results in 7434013 
>>> files.
> 
>>> -- Best regards / Met vriendelijke groet,
> 
>>> Aron Rotteveel
> 
>>> 2015-03-19 20:10 GMT+01:00 <devzero at web.de
>>> <mailto:devzero at web.de> <mailto:devzero at web.de
>>> <mailto:devzero at web.de>>
> <mailto:devzero at web.de <mailto:devzero at web.de>
>>> <mailto:devzero at web.de <mailto:devzero at web.de>>>>:
> 
>>> Hi Aron,
> 
>>> i hope it`s ok for you if i bring this back on-list. Your
>>> issue and the way or possible fix to resolve it may be
>>> interesting for others too (that may include future searches
>>> etc)
> 
>>> so with 3.1.1 we are a step further....
> 
>>> i don`t really have a clue what`s happening here but my next 
>>> step would be taking a closer look on how the memory usage of 
>>> rsync on the client and server grows.
> 
>>> you could log it like this: while true;do ps -eo 
>>> vsz,rss,sz,rsync|grep cron;sleep 10;done >logfile
> 
>>> does it grow continuously? does the oom situation reproducibly 
>>> happen at a certain size ? what`s the client and server 
>>> platform? how many files? (->
>>> https://rsync.samba.org/FAQ.html#5 ! )
> 
>>> regards roland
> 
>>> *Gesendet:* Donnerstag, 19. März 2015 um 12:24 Uhr *Von:*
>>> "Aron Rotteveel" <rotteveel.aron at gmail.com
>>> <mailto:rotteveel.aron at gmail.com> 
>>> <mailto:rotteveel.aron at gmail.com
>>> <mailto:rotteveel.aron at gmail.com>> 
>>> <mailto:rotteveel.aron at gmail.com
>>> <mailto:rotteveel.aron at gmail.com>
>> <mailto:rotteveel.aron at gmail.com
>> <mailto:rotteveel.aron at gmail.com>>>> *An:*
> devzero at web.de <mailto:devzero at web.de>
>> <mailto:devzero at web.de <mailto:devzero at web.de>>
>>> <mailto:devzero at web.de <mailto:devzero at web.de>
> <mailto:devzero at web.de <mailto:devzero at web.de>>> *Betreff:* Re:
>> rsync 3.0.9 segmentation
>>> fault In addition to my last message:
> 
>>> * Client (sender) has 16GB's or RAM, of which only 6.5GB is
>>> used during peak. * I tried using --no-inc-recursive, but it
>>> does not solve the issue.
> 
>>> What currrently is puzzling me is the question of why I am 
>>> receiving these errors when my server seems to have plenty of 
>>> memory to spare.
> 
>>> -- Best regards / Met vriendelijke groet,
> 
>>> Aron Rotteveel
> 
>>> 2015-03-19 11:52 GMT+01:00 Aron Rotteveel 
>>> <rotteveel.aron at gmail.com <mailto:rotteveel.aron at gmail.com>
> <mailto:rotteveel.aron at gmail.com
> <mailto:rotteveel.aron at gmail.com>>>:
> 
>>> Hi Roland,
> 
>>> I just upgrade both the client and host to 3.1.1 and seem to 
>>> memory related issues now:
> 
>>> ERROR: out of memory in make_file [sender] rsync error: error 
>>> allocating core memory buffers (code 22) at util2.c(102) 
>>> [sender=3.1.1] [sender] _exit_cleanup(code=22, file=util2.c, 
>>> line=102): about to call exit(22) [Receiver] 
>>> _exit_cleanup(code=22, file=io.c, line=1633): about to call 
>>> exit(22)
> 
> 
> 
> 
> -- Please use reply-all for most replies to avoid omitting the
> mailing list. To unsubscribe or change options: 
> https://lists.samba.org/mailman/listinfo/rsync Before posting,
> read: http://www.catb.org/~esr/faqs/smart-questions.html
> 
> 

- -- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
	Orlando, Florida		kmk at sanitarium.net (personal)
	Web page:			http://www.sanitarium.net/
	PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iEYEARECAAYFAlUVW/0ACgkQVKC1jlbQAQeKNQCgkZF+PmmIgXUCrFWGRIBgTNI0
WTIAoNfGiD91ubrz/StpXNmMxZ86gzZe
=RIfE
-----END PGP SIGNATURE-----