rsync 3.0.9 segmentation fault

[Kevin Korb]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Try also removing --delete-excluded.  Without those two options there
should be no reason for rsync to require gigs of RAM.  Well, unless
the other system has rsync 2.x.

On 03/27/2015 07:29 AM, Aron Rotteveel wrote:
> Yes, I removed "--no-inc-recursive", without success.
> 
> -- Best regards / Met vriendelijke groet,
> 
> Aron Rotteveel
> 
> 2015-03-27 12:24 GMT+01:00 Kevin Korb <kmk at sanitarium.net 
> <mailto:kmk at sanitarium.net>>:
> 
> Have you tried removing --no-inc-recursive yet?
> 
> On 03/27/2015 07:19 AM, Aron Rotteveel wrote:
>> Hi Roland,
> 
>> Thanks for the reply. Memory usage on both machines seem fine.
>> The server has 4GB's of RAM, of which about 3GB is used during
>> the file list build and about 1.5GB is used during the actual
>> transfer. The client has 16GB of RAM with a peak usage of 8.5GB.
> 
>> I just tried three transfers in a row and it consistently breaks
>> at a certain point, after which I get the "ERROR: out of memory
>> in flist_expand [sender]" error. There is not much special to
>> mention regarding the file on which it breaks: it's a 22KB JPEG
>> file with no special attributes.
> 
>> The backup server is running Debian 7.8, the client runs on
>> CentOS 5.11.
> 
>> A `find . | wc -l` in the backup directory results in 7434013 
>> files.
> 
>> -- Best regards / Met vriendelijke groet,
> 
>> Aron Rotteveel
> 
>> 2015-03-19 20:10 GMT+01:00 <devzero at web.de
>> <mailto:devzero at web.de> <mailto:devzero at web.de
>> <mailto:devzero at web.de>>>:
> 
>> Hi Aron,
> 
>> i hope it`s ok for you if i bring this back on-list. Your issue 
>> and the way or possible fix to resolve it may be interesting for 
>> others too (that may include future searches etc)
> 
>> so with 3.1.1 we are a step further....
> 
>> i don`t really have a clue what`s happening here but my next
>> step would be taking a closer look on how the memory usage of
>> rsync on the client and server grows.
> 
>> you could log it like this: while true;do ps -eo 
>> vsz,rss,sz,rsync|grep cron;sleep 10;done >logfile
> 
>> does it grow continuously? does the oom situation reproducibly 
>> happen at a certain size ? what`s the client and server
>> platform? how many files? (-> https://rsync.samba.org/FAQ.html#5
>> ! )
> 
>> regards roland
> 
>> *Gesendet:* Donnerstag, 19. März 2015 um 12:24 Uhr *Von:* "Aron 
>> Rotteveel" <rotteveel.aron at gmail.com
>> <mailto:rotteveel.aron at gmail.com> 
>> <mailto:rotteveel.aron at gmail.com
> <mailto:rotteveel.aron at gmail.com>>> *An:* devzero at web.de 
> <mailto:devzero at web.de>
>> <mailto:devzero at web.de <mailto:devzero at web.de>> *Betreff:* Re:
> rsync 3.0.9 segmentation
>> fault In addition to my last message:
> 
>> * Client (sender) has 16GB's or RAM, of which only 6.5GB is used 
>> during peak. * I tried using --no-inc-recursive, but it does not 
>> solve the issue.
> 
>> What currrently is puzzling me is the question of why I am 
>> receiving these errors when my server seems to have plenty of 
>> memory to spare.
> 
>> -- Best regards / Met vriendelijke groet,
> 
>> Aron Rotteveel
> 
>> 2015-03-19 11:52 GMT+01:00 Aron Rotteveel 
>> <rotteveel.aron at gmail.com <mailto:rotteveel.aron at gmail.com>>:
> 
>> Hi Roland,
> 
>> I just upgrade both the client and host to 3.1.1 and seem to
>> memory related issues now:
> 
>> ERROR: out of memory in make_file [sender] rsync error: error 
>> allocating core memory buffers (code 22) at util2.c(102) 
>> [sender=3.1.1] [sender] _exit_cleanup(code=22, file=util2.c, 
>> line=102): about to call exit(22) [Receiver]
>> _exit_cleanup(code=22, file=io.c, line=1633): about to call
>> exit(22)
> 
> 
> 

- -- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
	Orlando, Florida		kmk at sanitarium.net (personal)
	Web page:			http://www.sanitarium.net/
	PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iEYEARECAAYFAlUVWZIACgkQVKC1jlbQAQfQ4wCeOjzVgtBt0t9LQ4Mf9X3kOhjF
tEcAoJAh158PF51O3Vnn8alkd7q0iSHQ
=pQeg
-----END PGP SIGNATURE-----