<div dir="ltr">Hi Kevin,<div><br></div><div>Just did: same result.</div></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature">--<br>Best regards / Met vriendelijke groet,<br><br>Aron Rotteveel</div></div>
<br><div class="gmail_quote">2015-03-27 14:32 GMT+01:00 Kevin Korb <span dir="ltr"><<a href="mailto:kmk@sanitarium.net" target="_blank">kmk@sanitarium.net</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">-----BEGIN PGP SIGNED MESSAGE-----<br>
Hash: SHA1<br>
<br>
</span>Try it without any --delete options.<br>
<span class=""><br>
On 03/27/2015 09:31 AM, Aron Rotteveel wrote:<br>
> I am now running with --delete --numeric-ids --relative but the<br>
> problem still persists.<br>
><br>
> -- Best regards / Met vriendelijke groet,<br>
><br>
> Aron Rotteveel<br>
><br>
> 2015-03-27 14:22 GMT+01:00 Kevin Korb <<a href="mailto:kmk@sanitarium.net">kmk@sanitarium.net</a><br>
</span>> <mailto:<a href="mailto:kmk@sanitarium.net">kmk@sanitarium.net</a>>>:<br>
<span class="">><br>
> Try also removing --delete-excluded. Without those two options<br>
> there should be no reason for rsync to require gigs of RAM. Well,<br>
> unless the other system has rsync 2.x.<br>
><br>
> On 03/27/2015 07:29 AM, Aron Rotteveel wrote:<br>
>> Yes, I removed "--no-inc-recursive", without success.<br>
><br>
>> -- Best regards / Met vriendelijke groet,<br>
><br>
>> Aron Rotteveel<br>
><br>
>> 2015-03-27 12:24 GMT+01:00 Kevin Korb <<a href="mailto:kmk@sanitarium.net">kmk@sanitarium.net</a><br>
</span>>> <mailto:<a href="mailto:kmk@sanitarium.net">kmk@sanitarium.net</a>> <mailto:<a href="mailto:kmk@sanitarium.net">kmk@sanitarium.net</a><br>
<span class="">>> <mailto:<a href="mailto:kmk@sanitarium.net">kmk@sanitarium.net</a>>>>:<br>
><br>
>> Have you tried removing --no-inc-recursive yet?<br>
><br>
>> On 03/27/2015 07:19 AM, Aron Rotteveel wrote:<br>
>>> Hi Roland,<br>
><br>
>>> Thanks for the reply. Memory usage on both machines seem fine.<br>
>>> The server has 4GB's of RAM, of which about 3GB is used during<br>
>>> the file list build and about 1.5GB is used during the actual<br>
>>> transfer. The client has 16GB of RAM with a peak usage of<br>
>>> 8.5GB.<br>
><br>
>>> I just tried three transfers in a row and it consistently<br>
>>> breaks at a certain point, after which I get the "ERROR: out of<br>
>>> memory in flist_expand [sender]" error. There is not much<br>
>>> special to mention regarding the file on which it breaks: it's<br>
>>> a 22KB JPEG file with no special attributes.<br>
><br>
>>> The backup server is running Debian 7.8, the client runs on<br>
>>> CentOS 5.11.<br>
><br>
>>> A `find . | wc -l` in the backup directory results in 7434013<br>
>>> files.<br>
><br>
>>> -- Best regards / Met vriendelijke groet,<br>
><br>
>>> Aron Rotteveel<br>
><br>
>>> 2015-03-19 20:10 GMT+01:00 <<a href="mailto:devzero@web.de">devzero@web.de</a><br>
>>> <mailto:<a href="mailto:devzero@web.de">devzero@web.de</a>> <mailto:<a href="mailto:devzero@web.de">devzero@web.de</a><br>
>>> <mailto:<a href="mailto:devzero@web.de">devzero@web.de</a>>><br>
> <mailto:<a href="mailto:devzero@web.de">devzero@web.de</a> <mailto:<a href="mailto:devzero@web.de">devzero@web.de</a>><br>
</span>>>> <mailto:<a href="mailto:devzero@web.de">devzero@web.de</a> <mailto:<a href="mailto:devzero@web.de">devzero@web.de</a>>>>>:<br>
<div><div class="h5">><br>
>>> Hi Aron,<br>
><br>
>>> i hope it`s ok for you if i bring this back on-list. Your<br>
>>> issue and the way or possible fix to resolve it may be<br>
>>> interesting for others too (that may include future searches<br>
>>> etc)<br>
><br>
>>> so with 3.1.1 we are a step further....<br>
><br>
>>> i don`t really have a clue what`s happening here but my next<br>
>>> step would be taking a closer look on how the memory usage of<br>
>>> rsync on the client and server grows.<br>
><br>
>>> you could log it like this: while true;do ps -eo<br>
>>> vsz,rss,sz,rsync|grep cron;sleep 10;done >logfile<br>
><br>
>>> does it grow continuously? does the oom situation reproducibly<br>
>>> happen at a certain size ? what`s the client and server<br>
>>> platform? how many files? (-><br>
>>> <a href="https://rsync.samba.org/FAQ.html#5" target="_blank">https://rsync.samba.org/FAQ.html#5</a> ! )<br>
><br>
>>> regards roland<br>
><br>
>>> *Gesendet:* Donnerstag, 19. März 2015 um 12:24 Uhr *Von:*<br>
>>> "Aron Rotteveel" <<a href="mailto:rotteveel.aron@gmail.com">rotteveel.aron@gmail.com</a><br>
>>> <mailto:<a href="mailto:rotteveel.aron@gmail.com">rotteveel.aron@gmail.com</a>><br>
>>> <mailto:<a href="mailto:rotteveel.aron@gmail.com">rotteveel.aron@gmail.com</a><br>
>>> <mailto:<a href="mailto:rotteveel.aron@gmail.com">rotteveel.aron@gmail.com</a>>><br>
>>> <mailto:<a href="mailto:rotteveel.aron@gmail.com">rotteveel.aron@gmail.com</a><br>
>>> <mailto:<a href="mailto:rotteveel.aron@gmail.com">rotteveel.aron@gmail.com</a>><br>
>> <mailto:<a href="mailto:rotteveel.aron@gmail.com">rotteveel.aron@gmail.com</a><br>
>> <mailto:<a href="mailto:rotteveel.aron@gmail.com">rotteveel.aron@gmail.com</a>>>>> *An:*<br>
> <a href="mailto:devzero@web.de">devzero@web.de</a> <mailto:<a href="mailto:devzero@web.de">devzero@web.de</a>><br>
>> <mailto:<a href="mailto:devzero@web.de">devzero@web.de</a> <mailto:<a href="mailto:devzero@web.de">devzero@web.de</a>>><br>
>>> <mailto:<a href="mailto:devzero@web.de">devzero@web.de</a> <mailto:<a href="mailto:devzero@web.de">devzero@web.de</a>><br>
> <mailto:<a href="mailto:devzero@web.de">devzero@web.de</a> <mailto:<a href="mailto:devzero@web.de">devzero@web.de</a>>>> *Betreff:* Re:<br>
>> rsync 3.0.9 segmentation<br>
>>> fault In addition to my last message:<br>
><br>
>>> * Client (sender) has 16GB's or RAM, of which only 6.5GB is<br>
>>> used during peak. * I tried using --no-inc-recursive, but it<br>
>>> does not solve the issue.<br>
><br>
>>> What currrently is puzzling me is the question of why I am<br>
>>> receiving these errors when my server seems to have plenty of<br>
>>> memory to spare.<br>
><br>
>>> -- Best regards / Met vriendelijke groet,<br>
><br>
>>> Aron Rotteveel<br>
><br>
>>> 2015-03-19 11:52 GMT+01:00 Aron Rotteveel<br>
>>> <<a href="mailto:rotteveel.aron@gmail.com">rotteveel.aron@gmail.com</a> <mailto:<a href="mailto:rotteveel.aron@gmail.com">rotteveel.aron@gmail.com</a>><br>
</div></div>> <mailto:<a href="mailto:rotteveel.aron@gmail.com">rotteveel.aron@gmail.com</a><br>
<span class="">> <mailto:<a href="mailto:rotteveel.aron@gmail.com">rotteveel.aron@gmail.com</a>>>>:<br>
><br>
>>> Hi Roland,<br>
><br>
>>> I just upgrade both the client and host to 3.1.1 and seem to<br>
>>> memory related issues now:<br>
><br>
>>> ERROR: out of memory in make_file [sender] rsync error: error<br>
>>> allocating core memory buffers (code 22) at util2.c(102)<br>
>>> [sender=3.1.1] [sender] _exit_cleanup(code=22, file=util2.c,<br>
>>> line=102): about to call exit(22) [Receiver]<br>
>>> _exit_cleanup(code=22, file=io.c, line=1633): about to call<br>
>>> exit(22)<br>
><br>
><br>
><br>
><br>
</span><span class="">> -- Please use reply-all for most replies to avoid omitting the<br>
> mailing list. To unsubscribe or change options:<br>
> <a href="https://lists.samba.org/mailman/listinfo/rsync" target="_blank">https://lists.samba.org/mailman/listinfo/rsync</a> Before posting,<br>
> read: <a href="http://www.catb.org/~esr/faqs/smart-questions.html" target="_blank">http://www.catb.org/~esr/faqs/smart-questions.html</a><br>
><br>
><br>
<br>
</span><span class="">- --<br>
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~<br>
Kevin Korb Phone: (407) 252-6853<br>
Systems Administrator Internet:<br>
FutureQuest, Inc. Kevin@FutureQuest.net (work)<br>
Orlando, Florida <a href="mailto:kmk@sanitarium.net">kmk@sanitarium.net</a> (personal)<br>
Web page: <a href="http://www.sanitarium.net/" target="_blank">http://www.sanitarium.net/</a><br>
PGP public key available on web site.<br>
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~<br>
-----BEGIN PGP SIGNATURE-----<br>
Version: GnuPG v2<br>
<br>
</span>iEYEARECAAYFAlUVW/0ACgkQVKC1jlbQAQeKNQCgkZF+PmmIgXUCrFWGRIBgTNI0<br>
WTIAoNfGiD91ubrz/StpXNmMxZ86gzZe<br>
=RIfE<br>
<div class="HOEnZb"><div class="h5">-----END PGP SIGNATURE-----<br>
--<br>
Please use reply-all for most replies to avoid omitting the mailing list.<br>
To unsubscribe or change options: <a href="https://lists.samba.org/mailman/listinfo/rsync" target="_blank">https://lists.samba.org/mailman/listinfo/rsync</a><br>
Before posting, read: <a href="http://www.catb.org/~esr/faqs/smart-questions.html" target="_blank">http://www.catb.org/~esr/faqs/smart-questions.html</a></div></div></blockquote></div><br></div>