Which rsync version?

[Sven.Hartrumpf at FernUni-Hagen.de]

Thu, 2 Aug 2007 16:52:53 +0200, paul wrote:
> On Thu 02 Aug 2007, Sven wrote:
> >
> > I once tried to rsync around 100 GB (10 million files), but version
> > 2.6.6 needed too much RAM and was too slow.
>
> I believe 2.6.7 had a number of memory-saving options; each subsequent
> version will of course have other improvements. 2.6.9 is the current
> official release.
>
> If those files don't have hardlinks (or you're not interested
> in preserving hardlinks),

No hardlinks.

> you can greatly improve performance
> by trying the 3.0.0 cvs snapshot.

Yes, the cvs version from 20070803T08 started very modestly in terms of RAM,
but after some minutes:
> rsync -a --progress -v /c/ /data/c   (source is mounted via nfs)
...
ces/002495.ces
      105851 100%  859.88kB/s    0:00:00 (xfer#23, to-check=5762/13367)
*** glibc detected *** rsync: munmap_chunk(): invalid pointer: 0x080a8cb0 ***
======= Backtrace: =========
/lib/libc.so.6[0x2a97e6e1]
rsync[0x807c0f0]
rsync[0x804b782]
rsync[0x8056215]
rsync[0x8056825]
rsync[0x806053a]
rsync[0x8060d89]
rsync[0x8062108]
rsync[0x807556f]
rsync[0x80618c5]
/lib/libc.so.6(__libc_start_main+0xdc)[0x2a92ff9c]
rsync[0x804a6c1]
======= Memory map: ========
08048000-08092000 r-xp 00000000 00:14 389384                             /linux/rsync-cvs/bin/rsync
08092000-08094000 rw-p 0004a000 00:14 389384                             /linux/rsync-cvs/bin/rsync
08094000-0866a000 rw-p 08094000 00:00 0                                  [heap]
2a7c8000-2a7d2000 r-xp 00000000 08:02 2232639                            /lib/libgcc_s.so.1
2a7d2000-2a7d4000 rw-p 00009000 08:02 2232639                            /lib/libgcc_s.so.1
2a7f8000-2a82d000 r--s 00000000 08:02 1577380                            /var/run/nscd/passwd
2a84e000-2a890000 rw-p 2a84e000 00:00 0
2a890000-2a8c5000 r--s 00000000 08:02 1577476                            /var/run/nscd/group
2a8e6000-2a919000 r--p 00000000 08:02 526055                             /usr/lib/locale/en_US/LC_CTYPE
2a919000-2a91a000 rw-p 2a919000 00:00 0
2a91a000-2aa42000 r-xp 00000000 08:02 2232595                            /lib/libc-2.5.so
2aa42000-2aa43000 r--p 00128000 08:02 2232595                            /lib/libc-2.5.so
2aa43000-2aa45000 rw-p 00129000 08:02 2232595                            /lib/libc-2.5.so
2aa45000-2aa48000 rw-p 2aa45000 00:00 0
2aa48000-2aa4f000 r-xp 00000000 08:02 480803                             /usr/lib/libpopt.so.0.0.0
2aa4f000-2aa51000 rw-p 00006000 08:02 480803                             /usr/lib/libpopt.so.0.0.0
2aa6e000-2aa75000 r--s 00000000 08:02 542502                             /usr/lib/gconv/gconv-modules.cache
2aa75000-2aa76000 rw-p 2aa75000 00:00 0
2aa76000-2aa91000 r-xp 00000000 08:02 2232588                            /lib/ld-2.5.so
2aa91000-2aa93000 rw-p 0001a000 08:02 2232588                            /lib/ld-2.5.so
ffd49000-ffd54000 rw-p ffd49000 00:00 0                                  [stack]
ffffe000-fffff000 r-xp ffffe000 00:00 0
rsync: writefd_unbuffered failed to write 4 bytes [sender]: Broken pipe (32)
rsync: connection unexpectedly closed (2322 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(605) [sender=3.0.0cvs]

[ERROR: 12] >


Is this a known problem?

Sven
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://lists.samba.org/archive/rsync/attachments/20070803/d3097621/attachment.bin