rsync 3.0.9 segmentation fault
Aron Rotteveel
rotteveel.aron at gmail.com
Fri Mar 27 09:02:56 MDT 2015
Hi Kevin,
Just did: same result.
--
Best regards / Met vriendelijke groet,
Aron Rotteveel
2015-03-27 14:32 GMT+01:00 Kevin Korb <kmk at sanitarium.net>:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Try it without any --delete options.
>
> On 03/27/2015 09:31 AM, Aron Rotteveel wrote:
> > I am now running with --delete --numeric-ids --relative but the
> > problem still persists.
> >
> > -- Best regards / Met vriendelijke groet,
> >
> > Aron Rotteveel
> >
> > 2015-03-27 14:22 GMT+01:00 Kevin Korb <kmk at sanitarium.net
> > <mailto:kmk at sanitarium.net>>:
> >
> > Try also removing --delete-excluded. Without those two options
> > there should be no reason for rsync to require gigs of RAM. Well,
> > unless the other system has rsync 2.x.
> >
> > On 03/27/2015 07:29 AM, Aron Rotteveel wrote:
> >> Yes, I removed "--no-inc-recursive", without success.
> >
> >> -- Best regards / Met vriendelijke groet,
> >
> >> Aron Rotteveel
> >
> >> 2015-03-27 12:24 GMT+01:00 Kevin Korb <kmk at sanitarium.net
> >> <mailto:kmk at sanitarium.net> <mailto:kmk at sanitarium.net
> >> <mailto:kmk at sanitarium.net>>>:
> >
> >> Have you tried removing --no-inc-recursive yet?
> >
> >> On 03/27/2015 07:19 AM, Aron Rotteveel wrote:
> >>> Hi Roland,
> >
> >>> Thanks for the reply. Memory usage on both machines seem fine.
> >>> The server has 4GB's of RAM, of which about 3GB is used during
> >>> the file list build and about 1.5GB is used during the actual
> >>> transfer. The client has 16GB of RAM with a peak usage of
> >>> 8.5GB.
> >
> >>> I just tried three transfers in a row and it consistently
> >>> breaks at a certain point, after which I get the "ERROR: out of
> >>> memory in flist_expand [sender]" error. There is not much
> >>> special to mention regarding the file on which it breaks: it's
> >>> a 22KB JPEG file with no special attributes.
> >
> >>> The backup server is running Debian 7.8, the client runs on
> >>> CentOS 5.11.
> >
> >>> A `find . | wc -l` in the backup directory results in 7434013
> >>> files.
> >
> >>> -- Best regards / Met vriendelijke groet,
> >
> >>> Aron Rotteveel
> >
> >>> 2015-03-19 20:10 GMT+01:00 <devzero at web.de
> >>> <mailto:devzero at web.de> <mailto:devzero at web.de
> >>> <mailto:devzero at web.de>>
> > <mailto:devzero at web.de <mailto:devzero at web.de>
> >>> <mailto:devzero at web.de <mailto:devzero at web.de>>>>:
> >
> >>> Hi Aron,
> >
> >>> i hope it`s ok for you if i bring this back on-list. Your
> >>> issue and the way or possible fix to resolve it may be
> >>> interesting for others too (that may include future searches
> >>> etc)
> >
> >>> so with 3.1.1 we are a step further....
> >
> >>> i don`t really have a clue what`s happening here but my next
> >>> step would be taking a closer look on how the memory usage of
> >>> rsync on the client and server grows.
> >
> >>> you could log it like this: while true;do ps -eo
> >>> vsz,rss,sz,rsync|grep cron;sleep 10;done >logfile
> >
> >>> does it grow continuously? does the oom situation reproducibly
> >>> happen at a certain size ? what`s the client and server
> >>> platform? how many files? (->
> >>> https://rsync.samba.org/FAQ.html#5 ! )
> >
> >>> regards roland
> >
> >>> *Gesendet:* Donnerstag, 19. März 2015 um 12:24 Uhr *Von:*
> >>> "Aron Rotteveel" <rotteveel.aron at gmail.com
> >>> <mailto:rotteveel.aron at gmail.com>
> >>> <mailto:rotteveel.aron at gmail.com
> >>> <mailto:rotteveel.aron at gmail.com>>
> >>> <mailto:rotteveel.aron at gmail.com
> >>> <mailto:rotteveel.aron at gmail.com>
> >> <mailto:rotteveel.aron at gmail.com
> >> <mailto:rotteveel.aron at gmail.com>>>> *An:*
> > devzero at web.de <mailto:devzero at web.de>
> >> <mailto:devzero at web.de <mailto:devzero at web.de>>
> >>> <mailto:devzero at web.de <mailto:devzero at web.de>
> > <mailto:devzero at web.de <mailto:devzero at web.de>>> *Betreff:* Re:
> >> rsync 3.0.9 segmentation
> >>> fault In addition to my last message:
> >
> >>> * Client (sender) has 16GB's or RAM, of which only 6.5GB is
> >>> used during peak. * I tried using --no-inc-recursive, but it
> >>> does not solve the issue.
> >
> >>> What currrently is puzzling me is the question of why I am
> >>> receiving these errors when my server seems to have plenty of
> >>> memory to spare.
> >
> >>> -- Best regards / Met vriendelijke groet,
> >
> >>> Aron Rotteveel
> >
> >>> 2015-03-19 11:52 GMT+01:00 Aron Rotteveel
> >>> <rotteveel.aron at gmail.com <mailto:rotteveel.aron at gmail.com>
> > <mailto:rotteveel.aron at gmail.com
> > <mailto:rotteveel.aron at gmail.com>>>:
> >
> >>> Hi Roland,
> >
> >>> I just upgrade both the client and host to 3.1.1 and seem to
> >>> memory related issues now:
> >
> >>> ERROR: out of memory in make_file [sender] rsync error: error
> >>> allocating core memory buffers (code 22) at util2.c(102)
> >>> [sender=3.1.1] [sender] _exit_cleanup(code=22, file=util2.c,
> >>> line=102): about to call exit(22) [Receiver]
> >>> _exit_cleanup(code=22, file=io.c, line=1633): about to call
> >>> exit(22)
> >
> >
> >
> >
> > -- Please use reply-all for most replies to avoid omitting the
> > mailing list. To unsubscribe or change options:
> > https://lists.samba.org/mailman/listinfo/rsync Before posting,
> > read: http://www.catb.org/~esr/faqs/smart-questions.html
> >
> >
>
> - --
> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
> Kevin Korb Phone: (407) 252-6853
> Systems Administrator Internet:
> FutureQuest, Inc. Kevin at FutureQuest.net (work)
> Orlando, Florida kmk at sanitarium.net (personal)
> Web page: http://www.sanitarium.net/
> PGP public key available on web site.
> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2
>
> iEYEARECAAYFAlUVW/0ACgkQVKC1jlbQAQeKNQCgkZF+PmmIgXUCrFWGRIBgTNI0
> WTIAoNfGiD91ubrz/StpXNmMxZ86gzZe
> =RIfE
> -----END PGP SIGNATURE-----
> --
> Please use reply-all for most replies to avoid omitting the mailing list.
> To unsubscribe or change options:
> https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.samba.org/pipermail/rsync/attachments/20150327/2e7b906c/attachment.html>
More information about the rsync
mailing list