unpredictable behaviour
tim.conway at philips.com
tim.conway at philips.com
Sat Nov 3 03:55:14 EST 2001
I see very odd results from rsync 2.4.7pre1, the latest cvs version (sept
12, i think was the last modified file).
We have a number of network-attached storage devices. 10/100 ethernet,
nfs2 mounted (under nfs3, they buffer deletes, and recursive deletions
fail). Usually, these are kept syncronized across
a wan by a nightly cronjob,
We have a few we keep in reserve, which we syncronize locally. They're
all the same, aside from whatever is left different by rsync failures.
they are syncronized, one at a time, during the week.
They fail on different files. as you see, bildmax4 failed on only one
file. bildmax2 failed on several screensful, but i trimmed it down. i
don't see any duplication between systems.
At the bottom, i show a ls of one of the files that failed on bildmax2,
both on the master (/big1/....) and on the bildmax itself
(/bildmax/bildmax2/...). What value is too big?
Oh, the cmdline:
/cadappl/encap/bin/rsync -Wa --delete --force --bwlimit=524288 source
destination
It's about 128Gb of data in about 2.8M files.
Any idea what this randomness is? might the "Value too large for defined
data type" be thrown if the system runs out of memory? These jobs get up
to over a half a gig memory used.
It was compiled (and is running) on a 64-bit machine.
My life would be greatly simplified if i could run these syncs as single
chunks, as there are things being removed, as well as added, so
automatically breaking up the runs may leave things out.
Anybody got any ideas? I think i've heard of others running much larger
distributions.
Incidentally, from this test, it's plain that the old false timeout
problem, where one process waiting for another to finish its work would
timeout and stop the run, even though the other threads were still
working, at the 60-second select_timeout value that is set when no
io_timeout is set, is fixed in this version.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
starting /bildmax/bildmax2 at Wed Oct 31 02:42:46 PST 2001
readlink
big1/cadappl1/hpux/ictools/arm_ads/1.1/common/html/stdug/general/.22-4.htm.IFaG5g:
Value too large for defined data type
readlink
big1/cadappl1/hpux/ictools/arm_ads/1.1/common/include/.sstream.MLaG5g:
Value too large for defined data type
rsync error: partial transfer (code 23) at main.c(537)
finished /bildmax/bildmax2 at Wed Oct 31 12:46:00 PST 2001
starting /bildmax/bildmax3 at Wed Oct 31 12:46:00 PST 2001
readlink big1/cadappl1/hpux/ictools/tmpTempo: No such file or directory
symlink big/tools/DI/factory_integrator2.2/data ->
/cadappl/ictools/factory_integrator/2.2/data : File exists
symlink big/tools/synopsys/synopsys1999.05/doc ->
/cadappl/ictools/synopsys/1999.05/doc : File exists
symlink big1/tools1/DI/dis2.2.2/DI/documentation ->
/cadappl/ictools/Design_Integrator/2.2.2/documentation : File exists
symlink big1/tools1/DI/dis2.2.2/DI/system ->
/cadappl/ictools/Design_Integrator/2.2.2/system : File exists
rsync error: partial transfer (code 23) at main.c(537)
finished /bildmax/bildmax3 at Wed Oct 31 18:07:04 PST 2001
starting /bildmax/bildmax4 at Wed Oct 31 18:07:04 PST 2001
readlink
big/tools/DI/dis2.2.1/DI/system/product/vsc983/spice_models/.vsc9a_wire.inc.enc.1.1.0.fobG1u:
Value too large for defined data type
rsync error: partial transfer (code 23) at main.c(537)
finished /bildmax/bildmax4 at Thu Nov 1 05:09:48 PST 2001
starting /bildmax/bildmax5 at Thu Nov 1 05:09:48 PST 2001
readlink
big1/cadappl1/hpux/iclibs/CMOS18/PcCMOS18corelib_danger_p/2.0/lib/corelib_danger_p/dly6x3pd/auLvs/.master.tag.dNsOZO:
Value too large for defined data type
readlink
big1/cadappl1/hpux/iclibs/CMOS18/PcCMOS18corelib_p/2.0.1/lib/corelib_p/ors2pd/datasheet/.master.tag.U8zOZO:
Value too large for defined data type
rsync error: partial transfer (code 23) at main.c(537)
finished /bildmax/bildmax5 at Fri Nov 2 00:41:47 PST 2001
starting /bildmax/bildmax6 at Fri Nov 2 00:41:47 PST 2001
readlink
big1/cadappl1/hpux/iclibs/CMOS18/PcCMOS18corelib_p/2.0.1/lib/corelib_p/ao6anx4pd/abstract/.layout.cdb.hSuGZO:
Value too large for defined data type
Tools at willy
/site/local/share/ToolSync/localrep>ls -l
/big1/cadappl1/hpux/ictools/arm_ads/1.1/common/html/stdug/general/22-4.htm
-rw-r--r-- 1 Tools Tools 12025 Apr 27 1999
/big1/cadappl1/hpux/ictools/arm_ads/1.1/common/html/stdug/general/22-4.htm
Tools at willy
/site/local/share/ToolSync/localrep>ls -l
/bildmax/bildmax2/big1/cadappl1/hpux/ictools/arm_ads/1.1/common/html/stdug/general/22-4.htm
-rw-r--r-- 1 Tools Tools 12025 Apr 27 1999
/bildmax/bildmax2/big1/cadappl1/hpux/ictools/arm_ads/1.1/common/html/stdug/general/22-4.htm
Tools at willy
/site/local/share/ToolSync/localrep>
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
main.c line 537 is just exit_cleanup(status)
+++++++++++++++++++++++++++++++++++++++Memory
usage+++++++++++++++++++++++++++++++++++++++++++++++++++++
load averages: 0.25, 0.33, 0.35 08:46:15
102 processes: 100 sleeping, 1 zombie, 1 on cpu
CPU states: 94.1% idle, 0.5% user, 5.3% kernel, 0.0% iowait, 0.0% swap
Memory: 3072M real, 1709M free, 644M swap in use, 5501M swap free
PID USERNAME THR PRI NICE SIZE RES STATE TIME CPU COMMAND
10838 Tools 1 33 0 535M 176M sleep 55:19 0.00% rsync
19393 Tools 1 33 0 535M 1728K sleep 5:04 0.59% rsync
10837 Tools 1 33 0 285M 78M sleep 26:46 0.28% rsync
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Tim Conway
tim.conway at philips.com
303.682.4917
Philips Semiconductor - Longmont TC
1880 Industrial Circle, Suite D
Longmont, CO 80501
Available via SameTime Connect within Philips, n9hmg on AIM
perl -e 'print pack(nnnnnnnnnnnn,
19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970),
".\n" '
"There are some who call me.... Tim?"
More information about the rsync
mailing list