<div dir="ltr">I am now running with --delete --numeric-ids --relative but the problem still persists.<div class="gmail_extra"><br clear="all"><div><div>--<br>Best regards / Met vriendelijke groet,<br><br>Aron Rotteveel</div></div>
<br><div class="gmail_quote">2015-03-27 14:22 GMT+01:00 Kevin Korb <span dir="ltr"><<a href="mailto:kmk@sanitarium.net" target="_blank">kmk@sanitarium.net</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span>-----BEGIN PGP SIGNED MESSAGE-----<br>
Hash: SHA1<br>
<br>
</span>Try also removing --delete-excluded. Without those two options there<br>
should be no reason for rsync to require gigs of RAM. Well, unless<br>
the other system has rsync 2.x.<br>
<span><br>
On 03/27/2015 07:29 AM, Aron Rotteveel wrote:<br>
> Yes, I removed "--no-inc-recursive", without success.<br>
><br>
> -- Best regards / Met vriendelijke groet,<br>
><br>
> Aron Rotteveel<br>
><br>
> 2015-03-27 12:24 GMT+01:00 Kevin Korb <<a href="mailto:kmk@sanitarium.net" target="_blank">kmk@sanitarium.net</a><br>
</span>> <mailto:<a href="mailto:kmk@sanitarium.net" target="_blank">kmk@sanitarium.net</a>>>:<br>
<span>><br>
> Have you tried removing --no-inc-recursive yet?<br>
><br>
> On 03/27/2015 07:19 AM, Aron Rotteveel wrote:<br>
>> Hi Roland,<br>
><br>
>> Thanks for the reply. Memory usage on both machines seem fine.<br>
>> The server has 4GB's of RAM, of which about 3GB is used during<br>
>> the file list build and about 1.5GB is used during the actual<br>
>> transfer. The client has 16GB of RAM with a peak usage of 8.5GB.<br>
><br>
>> I just tried three transfers in a row and it consistently breaks<br>
>> at a certain point, after which I get the "ERROR: out of memory<br>
>> in flist_expand [sender]" error. There is not much special to<br>
>> mention regarding the file on which it breaks: it's a 22KB JPEG<br>
>> file with no special attributes.<br>
><br>
>> The backup server is running Debian 7.8, the client runs on<br>
>> CentOS 5.11.<br>
><br>
>> A `find . | wc -l` in the backup directory results in 7434013<br>
>> files.<br>
><br>
>> -- Best regards / Met vriendelijke groet,<br>
><br>
>> Aron Rotteveel<br>
><br>
>> 2015-03-19 20:10 GMT+01:00 <<a href="mailto:devzero@web.de" target="_blank">devzero@web.de</a><br>
</span>>> <mailto:<a href="mailto:devzero@web.de" target="_blank">devzero@web.de</a>> <mailto:<a href="mailto:devzero@web.de" target="_blank">devzero@web.de</a><br>
<span>>> <mailto:<a href="mailto:devzero@web.de" target="_blank">devzero@web.de</a>>>>:<br>
><br>
>> Hi Aron,<br>
><br>
>> i hope it`s ok for you if i bring this back on-list. Your issue<br>
>> and the way or possible fix to resolve it may be interesting for<br>
>> others too (that may include future searches etc)<br>
><br>
>> so with 3.1.1 we are a step further....<br>
><br>
>> i don`t really have a clue what`s happening here but my next<br>
>> step would be taking a closer look on how the memory usage of<br>
>> rsync on the client and server grows.<br>
><br>
>> you could log it like this: while true;do ps -eo<br>
>> vsz,rss,sz,rsync|grep cron;sleep 10;done >logfile<br>
><br>
>> does it grow continuously? does the oom situation reproducibly<br>
>> happen at a certain size ? what`s the client and server<br>
>> platform? how many files? (-> <a href="https://rsync.samba.org/FAQ.html#5" target="_blank">https://rsync.samba.org/FAQ.html#5</a><br>
>> ! )<br>
><br>
>> regards roland<br>
><br>
>> *Gesendet:* Donnerstag, 19. März 2015 um 12:24 Uhr *Von:* "Aron<br>
>> Rotteveel" <<a href="mailto:rotteveel.aron@gmail.com" target="_blank">rotteveel.aron@gmail.com</a><br>
>> <mailto:<a href="mailto:rotteveel.aron@gmail.com" target="_blank">rotteveel.aron@gmail.com</a>><br>
</span>>> <mailto:<a href="mailto:rotteveel.aron@gmail.com" target="_blank">rotteveel.aron@gmail.com</a><br>
<span>> <mailto:<a href="mailto:rotteveel.aron@gmail.com" target="_blank">rotteveel.aron@gmail.com</a>>>> *An:* <a href="mailto:devzero@web.de" target="_blank">devzero@web.de</a><br>
> <mailto:<a href="mailto:devzero@web.de" target="_blank">devzero@web.de</a>><br>
</span>>> <mailto:<a href="mailto:devzero@web.de" target="_blank">devzero@web.de</a> <mailto:<a href="mailto:devzero@web.de" target="_blank">devzero@web.de</a>>> *Betreff:* Re:<br>
<span>> rsync 3.0.9 segmentation<br>
>> fault In addition to my last message:<br>
><br>
>> * Client (sender) has 16GB's or RAM, of which only 6.5GB is used<br>
>> during peak. * I tried using --no-inc-recursive, but it does not<br>
>> solve the issue.<br>
><br>
>> What currrently is puzzling me is the question of why I am<br>
>> receiving these errors when my server seems to have plenty of<br>
>> memory to spare.<br>
><br>
>> -- Best regards / Met vriendelijke groet,<br>
><br>
>> Aron Rotteveel<br>
><br>
>> 2015-03-19 11:52 GMT+01:00 Aron Rotteveel<br>
</span>>> <<a href="mailto:rotteveel.aron@gmail.com" target="_blank">rotteveel.aron@gmail.com</a> <mailto:<a href="mailto:rotteveel.aron@gmail.com" target="_blank">rotteveel.aron@gmail.com</a>>>:<br>
<span>><br>
>> Hi Roland,<br>
><br>
>> I just upgrade both the client and host to 3.1.1 and seem to<br>
>> memory related issues now:<br>
><br>
>> ERROR: out of memory in make_file [sender] rsync error: error<br>
>> allocating core memory buffers (code 22) at util2.c(102)<br>
>> [sender=3.1.1] [sender] _exit_cleanup(code=22, file=util2.c,<br>
>> line=102): about to call exit(22) [Receiver]<br>
>> _exit_cleanup(code=22, file=io.c, line=1633): about to call<br>
>> exit(22)<br>
><br>
><br>
><br>
<br>
</span><span>- --<br>
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~<br>
Kevin Korb Phone: (407) 252-6853<br>
Systems Administrator Internet:<br>
FutureQuest, Inc. Kevin@FutureQuest.net (work)<br>
Orlando, Florida <a href="mailto:kmk@sanitarium.net" target="_blank">kmk@sanitarium.net</a> (personal)<br>
Web page: <a href="http://www.sanitarium.net/" target="_blank">http://www.sanitarium.net/</a><br>
PGP public key available on web site.<br>
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~<br>
-----BEGIN PGP SIGNATURE-----<br>
Version: GnuPG v2<br>
<br>
</span>iEYEARECAAYFAlUVWZIACgkQVKC1jlbQAQfQ4wCeOjzVgtBt0t9LQ4Mf9X3kOhjF<br>
tEcAoJAh158PF51O3Vnn8alkd7q0iSHQ<br>
=pQeg<br>
-----END PGP SIGNATURE-----<br>
<div><div>--<br>
Please use reply-all for most replies to avoid omitting the mailing list.<br>
To unsubscribe or change options: <a href="https://lists.samba.org/mailman/listinfo/rsync" target="_blank">https://lists.samba.org/mailman/listinfo/rsync</a><br>
Before posting, read: <a href="http://www.catb.org/~esr/faqs/smart-questions.html" target="_blank">http://www.catb.org/~esr/faqs/smart-questions.html</a></div></div></blockquote></div><br></div></div>