[Bug 3116] New: large tar files: 1 gig size: retransmitted: rsync_rsh

samba-bugs at samba.org samba-bugs at samba.org
Sat Sep 24 14:08:05 GMT 2005


https://bugzilla.samba.org/show_bug.cgi?id=3116

           Summary: large tar files: 1 gig size: retransmitted: rsync_rsh
           Product: rsync
           Version: 2.6.6
          Platform: x86
        OS/Version: FreeBSD
            Status: NEW
          Severity: normal
          Priority: P3
         Component: core
        AssignedTo: wayned at samba.org
        ReportedBy: drinking.coffee at gmail.com
         QAContact: rsync-qa at samba.org


Once a week, I'm performing an offsite backup. These files are each 1 gig in
size, average speedup is 1.18.

Rsync does the initial run, and then send some of the tar files again. I
initially thought the archives were too large, so I reduced them to 512M, but
had the same symtoms. My next step, is to install an rsync daemon on the remote end.

When I do a second batch of mostly static content, I don't have any issues.

Below are my options:
#!/bin/sh
TERM="VT100"
RSYNC_RSH="ssh -c blowfish -2 -i /usr/username/.ssh/id_dsa"
export RSYNC_RSH
echo ' '
/bin/date
/bin/rsync -lrtv --stats --timeout=300 --delete-after \
        /home/backup/psa username at hostname.net:/home/username/backup

source ssh version: OpenSSH_3.9p1
destination ssh version: OpenSSH_3.6.1p2

Output of rsync log:

Sat Sep 24 03:01:01 EDT 2005
building file list ... done
psa/
psa/psa.dump.full.monthly.aa
psa/psa.dump.full.monthly.ab
psa/psa.dump.full.monthly.ac
psa/psa.dump.full.monthly.ad
psa/psa.dump.full.monthly.ae
psa/psa.dump.full.monthly.af
psa/psa.dump.full.monthly.ag
psa/psa.dump.full.monthly.ah
psa/psa.dump.full.monthly.ai
psa/psa.dump.full.monthly.aj
psa/psa.dump.full.monthly.ak
psa/psa.dump.full.monthly.al
psa/psa.dump.full.monthly.am
psa/psa.dump.full.monthly.an
psa/psa.dump.full.monthly.ao
psa/psa.dump.full.monthly.aa
psa/psa.dump.full.monthly.ad
psa/psa.dump.full.monthly.ae
psa/psa.dump.full.monthly.ag
psa/psa.dump.full.monthly.ah
psa/psa.dump.full.monthly.ai
psa/psa.dump.full.monthly.aj
psa/psa.dump.full.monthly.ak
psa/psa.dump.full.monthly.al
psa/psa.dump.full.monthly.am

Number of files: 16
Number of files transferred: 25
Total file size: 15400460674 bytes
Total transferred file size: 26137878914 bytes
Literal data: 13011870082 bytes
Matched data: 13126008832 bytes
File list size: 230
File list generation time: 0.306 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 13016666532
Total bytes received: 18628384

sent 13016666532 bytes  received 18628384 bytes  892125.72 bytes/sec
total size is 15400460674  speedup is 1.18

Source Files:
-rwx------  1 root    wheel           1G Sep 22 02:19 psa.dump.full.monthly.aa
-rwx------  1 root    wheel           1G Sep 22 02:33 psa.dump.full.monthly.ab
-rwx------  1 root    wheel           1G Sep 22 02:47 psa.dump.full.monthly.ac
-rwx------  1 root    wheel           1G Sep 22 03:00 psa.dump.full.monthly.ad
-rwx------  1 root    wheel           1G Sep 22 03:16 psa.dump.full.monthly.ae
-rwx------  1 root    wheel           1G Sep 22 03:29 psa.dump.full.monthly.af
-rwx------  1 root    wheel           1G Sep 22 03:44 psa.dump.full.monthly.ag
-rwx------  1 root    wheel           1G Sep 22 03:54 psa.dump.full.monthly.ah
-rwx------  1 root    wheel           1G Sep 22 04:04 psa.dump.full.monthly.ai
-rwx------  1 root    wheel           1G Sep 22 04:15 psa.dump.full.monthly.aj
-rwx------  1 root    wheel           1G Sep 22 04:24 psa.dump.full.monthly.ak
-rwx------  1 root    wheel           1G Sep 22 04:35 psa.dump.full.monthly.al
-rwx------  1 root    wheel           1G Sep 22 04:45 psa.dump.full.monthly.am
-rwx------  1 root    wheel           1G Sep 22 05:02 psa.dump.full.monthly.an
-rwx------  1 root    wheel         351M Sep 22 05:11 psa.dump.full.monthly.ao

Second Batch Output:
-rwx------  1 backup  backup       849K Sep 23 01:01 host1-backup-etc.zip
-rwx------  1 backup  backup       581M Sep 23 01:06 host1-backup-home-a2e.zip
-rwx------  1 backup  backup       155M Sep 23 01:07 host1-backup-home-f2j.zip
-rwx------  1 backup  backup       423M Sep 23 01:09 host1-backup-home-k2o.zip
-rwx------  1 backup  backup       341M Sep 23 01:10 host1-backup-home-p2t.zip
-rwx------  1 backup  backup       374M Sep 23 01:12 host1-backup-home-u2z.zip
-rwx------  1 backup  backup        13M Sep 23 01:01 host1-backup-mysql.zip
-rwx------  1 backup  backup       264M Sep 23 01:02 host1-backup-staff.zip
-rwx------  1 backup  backup        80M Sep 23 01:01 host1-backup-var.zip
-rwx------  1 backup  backup        29B Sep 23 01:12 host1-zip-end.txt
-rwx------  1 backup  backup        29B Sep 23 01:01 host1-zip-start.txt
-rwx------  1 backup  backup       836K Sep 23 01:01 host2-backup-etc.zip
-rwx------  1 backup  backup       513M Sep 23 01:04 host2-backup-home-a2e.zip
-rwx------  1 backup  backup       393M Sep 23 01:06 host2-backup-home-f2j.zip
-rwx------  1 backup  backup       406M Sep 23 01:08 host2-backup-home-k2o.zip
-rwx------  1 backup  backup       375M Sep 23 01:10 host2-backup-home-p2t.zip
-rwx------  1 backup  backup       130M Sep 23 01:10 host2-backup-home-u2z.zip
-rwx------  1 backup  backup        11M Sep 23 01:01 host2-backup-mysql.zip
-rwx------  1 backup  backup       133M Sep 23 01:01 host2-backup-staff.zip
-rwx------  1 backup  backup        43M Sep 23 01:01 host2-backup-var.zip
-rwx------  1 backup  backup        29B Sep 23 01:10 host2-zip-end.txt
-rwx------  1 backup  backup        29B Sep 23 01:01 host2-zip-start.txt

Sat Sep 24 07:06:26 EDT 2005
building file list ... done
archive/
archive/host1-backup-etc.zip
archive/host1-backup-home-a2e.zip
archive/host1-backup-home-f2j.zip
archive/host1-backup-home-k2o.zip
archive/host1-backup-home-p2t.zip
archive/host1-backup-home-u2z.zip
archive/host1-backup-mysql.zip
archive/host1-backup-staff.zip
archive/host1-backup-var.zip
archive/host1-zip-end.txt
archive/host1-zip-start.txt
archive/host2-backup-etc.zip
archive/host2-backup-home-a2e.zip
archive/host2-backup-home-f2j.zip
archive/host2-backup-home-k2o.zip
archive/host2-backup-home-p2t.zip
archive/host2-backup-home-u2z.zip
archive/host2-backup-mysql.zip
archive/host2-backup-staff.zip
archive/host2-backup-var.zip
archive/host2-zip-end.txt
archive/host2-zip-start.txt

Number of files: 23
Number of files transferred: 22
Total file size: 4449287013 bytes
Total transferred file size: 4449287013 bytes
Literal data: 390371025 bytes
Matched data: 4058915988 bytes
File list size: 528
File list generation time: 0.016 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 391530333
Total bytes received: 1792478

sent 391530333 bytes  received 1792478 bytes  456555.79 bytes/sec
total size is 4449287013  speedup is 11.31

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.


More information about the rsync mailing list