rsync 3.0.9 partial file left after CTRL-C WITHOUT using --partial
Henri Shustak
henri.shustak at gmail.com
Sat Apr 6 06:11:23 MDT 2013
Hello,
I was unable to reproduce this issue on an OS X system with rsync 3.0.9 (compiled from source) or 2.6.9 (default on 10.8.2). Output from the testing I performed is below.
> <snip>
> ╭─henri at mac /tmp/test_rsync_2.6.9_osx/src ‹› 13-04-07 - 0:41:08
> ╰─$ dd if=/dev/urandom of=source_file bs=1000000000 count=1
> 1+0 records in
> 1+0 records out
> 1000000000 bytes transferred in 105.312418 secs (9495556 bytes/sec)
> ╭─henri at mac /tmp/test_rsync_2.6.9_osx/src ‹› 13-04-07 - 0:43:17
> ╰─$ du -hs ./source_file
> 954M ./source_file
> ╭─henri at mac /tmp/test_rsync_2.6.9_osx/src ‹› 13-04-07 - 0:43:25
> ╰─$ cd ..
> ╭─henri at mac /tmp/test_rsync_2.6.9_osx ‹› 13-04-07 - 0:43:31
> ╰─$ mkdir dst
> ╭─henri at mac /tmp/test_rsync_2.6.9_osx ‹› 13-04-07 - 0:43:36
> ╰─$ rsync -a ./src/ ./dst
> ^Crsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at /SourceCache/rsync/rsync-42/rsync/rsync.c(244) [sender=2.6.9]
> ╭─henri at mac /tmp/test_rsync_2.6.9_osx ‹› 13-04-07 - 0:44:26
> ╰─$ ps -a | grep rsync 20 ↵
> 2181 ttys000 0:00.00 grep rsync
> ╭─henri at mac /tmp/test_rsync_2.6.9_osx ‹› 13-04-07 - 0:44:37
> ╰─$ ls -la ./dst
> total 0
> drwxr-xr-x 2 henri wheel 68 Apr 7 00:44 .
> drwxr-xr-x 4 henri wheel 136 Apr 7 00:43 ..
> ╭─henri at mac /tmp/test_rsync_2.6.9_osx ‹› 13-04-07 - 0:45:03
> ╰─$ rsync -a ./src/ ./dst
> ╭─henri at mac /tmp/test_rsync_2.6.9_osx ‹› 13-04-07 - 0:45:59
> ╰─$ ls -la ./dst
> total 1953128
> drwxr-xr-x 3 henri wheel 102 Apr 7 00:41 .
> drwxr-xr-x 4 henri wheel 136 Apr 7 00:43 ..
> -rw-r--r-- 1 henri wheel 1000000000 Apr 7 00:43 source_file
> ╭─henri at mac /tmp/test_rsync_2.6.9_osx ‹› 13-04-07 - 0:46:03
> ╰─$ rm ./dst/source_file
> ╭─henri at mac /tmp/test_rsync_2.6.9_osx ‹› 13-04-07 - 0:46:15
> ╰─$ rsync -a ./src/ ./dst
> ^Crsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at /SourceCache/rsync/rsync-42/rsync/rsync.c(244) [sender=2.6.9]
> ╭─henri at mac /tmp/test_rsync_2.6.9_osx ‹› 13-04-07 - 0:46:39
> ╰─$ rsync -a ./src/ ./dst 20 ↵
> ^Crsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at /SourceCache/rsync/rsync-42/rsync/rsync.c(244) [sender=2.6.9]
> ╭─henri at mac /tmp/test_rsync_2.6.9_osx ‹› 13-04-07 - 0:46:54
> ╰─$ ps -a | grep rsync 20 ↵
> 2280 ttys000 0:00.00 grep rsync
> ╭─henri at mac /tmp/test_rsync_2.6.9_osx ‹› 13-04-07 - 0:46:59
> ╰─$ ls -la ./dst
> total 0
> drwxr-xr-x 2 henri wheel 68 Apr 7 00:46 .
> drwxr-xr-x 4 henri wheel 136 Apr 7 00:43 ..
> ╭─henri at mac /tmp/test_rsync_2.6.9_osx ‹› 13-04-07 - 0:47:03
> ╰─$
> ╭─henri at mac /tmp/test_rsync_2.6.9_osx ‹› 13-04-07 - 0:47:39
> ╰─$ rsync -a --partial ./src/ ./dst
> ^Crsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at /SourceCache/rsync/rsync-42/rsync/rsync.c(244) [sender=2.6.9]
> ╭─henri at mac /tmp/test_rsync_2.6.9_osx ‹› 13-04-07 - 0:48:00
> ╰─$ ls -la ./dst 20 ↵
> total 402432
> drwxr-xr-x 3 henri wheel 102 Apr 7 00:48 .
> drwxr-xr-x 4 henri wheel 136 Apr 7 00:43 ..
> -rw-r--r-- 1 henri wheel 206045184 Apr 7 00:48 source_file
> ╭─henri at mac /tmp/test_rsync_2.6.9_osx ‹› 13-04-07 - 0:48:05
> ╰─$ rm ./dst/source_file
> ╭─henri at mac /tmp/test_rsync_2.6.9_osx ‹› 13-04-07 - 0:50:02
> ╰─$ rsync -a --partial --inplace ./src/ ./dst
> ^Crsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at /SourceCache/rsync/rsync-42/rsync/rsync.c(244) [sender=2.6.9]
> ╭─henri at mac /tmp/test_rsync_2.6.9_osx ‹› 13-04-07 - 0:50:48
> ╰─$ ls -la ./dst 20 ↵
> total 1065472
> drwxr-xr-x 3 henri wheel 102 Apr 7 00:50 .
> drwxr-xr-x 4 henri wheel 136 Apr 7 00:43 ..
> -rw------- 1 henri wheel 545521664 Apr 7 00:50 source_file
> ╭─henri at mac /tmp/test_rsync_2.6.9_osx ‹› 13-04-07 - 0:50:54
> ╰─$
>
> <snip>
>
> ╭─henri at mac /tmp/test_rsync_3.0.9_osx ‹› 13-04-07 - 0:55:56
> ╰─$ /usr/local/bin/rsync_v3.0.9 -a ./src/ ./dst
> ^Crsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(657) [sender=3.0.9]
> ╭─henri at mac /tmp/test_rsync_3.0.9_osx ‹› 13-04-07 - 0:56:46
> ╰─$ ls -la ./dst 20 ↵
> total 0
> drwxr-xr-x 2 henri wheel 68 Apr 7 00:56 .
> drwxr-xr-x 4 henri wheel 136 Apr 7 00:43 ..
> ╭─henri at mac /tmp/test_rsync_3.0.9_osx ‹› 13-04-07 - 0:56:53
> ╰─$ /usr/local/bin/rsync_v3.0.9 -a ./src/ ./dst
> ^Crsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(657) [sender=3.0.9]
> ╭─henri at mac /tmp/test_rsync_3.0.9_osx ‹› 13-04-07 - 0:57:17
> ╰─$ ls -la ./dst 20 ↵
> total 0
> drwxr-xr-x 2 henri wheel 68 Apr 7 00:57 .
> drwxr-xr-x 4 henri wheel 136 Apr 7 00:43 ..
> ╭─henri at mac /tmp/test_rsync_3.0.9_osx ‹› 13-04-07 - 0:57:20
> ╰─$ /usr/local/bin/rsync_v3.0.9 -a --partial ./src/ ./dst
> ^Crsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(657) [sender=3.0.9]
> ╭─henri at mac /tmp/test_rsync_3.0.9_osx ‹› 13-04-07 - 0:57:39
> ╰─$ ls -la ./dst 20 ↵
> total 1027072
> drwxr-xr-x 3 henri wheel 102 Apr 7 00:57 .
> drwxr-xr-x 4 henri wheel 136 Apr 7 00:43 ..
> -rw-r--r-- 1 henri wheel 525860864 Apr 7 00:57 source_file
> ╭─henri at mac /tmp/test_rsync_3.0.9_osx ‹› 13-04-07 - 0:57:41
> ╰─$ rm ./dst/source_file
> ╭─henri at mac /tmp/test_rsync_3.0.9_osx ‹› 13-04-07 - 0:57:47
> ╰─$ /usr/local/bin/rsync_v3.0.9 -a --partial --inplace ./src/ ./dst
> ^Crsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(657) [sender=3.0.9]
> ╭─henri at mac /tmp/test_rsync_3.0.9_osx ‹› 13-04-07 - 0:58:03
> ╰─$ ls -la ./dst 20 ↵
> total 1132544
> drwxr-xr-x 3 henri wheel 102 Apr 7 00:57 .
> drwxr-xr-x 4 henri wheel 136 Apr 7 00:43 ..
> -rw------- 1 henri wheel 579862528 Apr 7 00:58 source_file
> ╭─henri at mac /tmp/test_rsync_3.0.9_osx ‹› 13-04-07 - 0:58:06
> ╰─$
> <snip>
I also suggest that you confirm that the rsync process has in fact terminated. Using the 'ps' (as shown above) is another approach to using 'lsof' (which is also a good idea).
Finally, if you are using rsync for backup then you may be interested to know that when using LBackup if a failed / partial backup is resumed the destination is always a fresh directory.
Perhaps the way the trap is reported to rsync on Ubuntu GNU/LINUX is slightly different than the way it is reported to rsync by OS X? I am pretty sure they are both POSIX systems : http://manpages.ubuntu.com/manpages/intrepid/man7/signal.7.html
Disclaimer : I am involved with the development of LBackup. As such, I have been contemplating the consequences of enabling an option which would allow multiple failed backup attempts to be used as hard-link sources in an effort to reduce the time additional backup runs are performed should the link be interrupted for some reason during a backup (e.g. full disk space / system crash / network failure).
--------------------------------------------------------------------
This email is protected by LBackup, an open source backup solution
http://www.lbackup.org
On 5/04/2013, at 11:45 PM, Paul Lopez <lopiuh at googlemail.com> wrote:
> Hi folks,
>
> man page says "By default, rsync will delete any partially transferred file if the transfer is interrupted"
>
> I have (reproducible) a partial file left, if I do CTRL-C
>
> source-dir: mounted LVM XFS
> dest-dir: see source-dir
> Ubuntu 12.04.1 (LTS)
> kernel 3.2.0-39-generic
>
> command:
> rsync -a /mnt/data/\$sortin_linux/rsnapshot_stable/hourly.1/usr/include/c++/4.4/ext/pb_ds/detail/binary_heap_/ /mnt/data/delete/
> ^Crsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(549) [sender=3.0.9]
>
>
> Source-Dir:
> root at yaVDR:~# ls -lsa /mnt/data/\$sortin_linux/rsnapshot_stable/hourly.1/usr/include/c++/4.4/ext/pb_ds/detail/binary_heap_/ total 108
> 4 drwxr-xr-x 2 root root 4096 Jul 24 2011 .
> 4 drwxr-xr-x 25 root root 4096 Jul 24 2011 ..
> 12 -rw-r--r-- 1 root root 8618 Nov 15 2010 binary_heap_.hpp
> 8 -rw-r--r-- 1 root root 4373 Nov 15 2010 const_iterator.hpp
> 8 -rw-r--r-- 1 root root 4458 Nov 15 2010 const_point_iterator.hpp
> 8 -rw-r--r-- 1 root root 4555 Nov 15 2010 constructors_destructor_fn_imps.hpp
> 4 -rw-r--r-- 1 root root 2509 Nov 15 2010 debug_fn_imps.hpp
> 4 -rw-r--r-- 1 root root 2636 Nov 15 2010 entry_cmp.hpp
> 4 -rw-r--r-- 1 root root 2599 Nov 15 2010 entry_pred.hpp
> 8 -rw-r--r-- 1 root root 5837 Nov 15 2010 erase_fn_imps.hpp
> 4 -rw-r--r-- 1 root root 2614 Nov 15 2010 find_fn_imps.hpp
> 4 -rw-r--r-- 1 root root 2068 Nov 15 2010 info_fn_imps.hpp
> 8 -rw-r--r-- 1 root root 5379 Nov 15 2010 insert_fn_imps.hpp
> 4 -rw-r--r-- 1 root root 2280 Nov 15 2010 iterators_fn_imps.hpp
> 4 -rw-r--r-- 1 root root 1896 Nov 15 2010 policy_access_fn_imps.hpp
> 8 -rw-r--r-- 1 root root 6300 Nov 15 2010 resize_policy.hpp
> 8 -rw-r--r-- 1 root root 5311 Nov 15 2010 split_join_fn_imps.hpp
> 4 -rw-r--r-- 1 root root 2441 Nov 15 2010 trace_fn_imps.hpp
> root at yaVDR:~#
>
>
> Dest-Dir (last File is the partial)
> root at yaVDR:~# ls /mnt/data/delete/ -lsa
> total 44
> 4 drwxr-xr-x 2 root root 4096 Apr 5 12:05 .
> 0 drwxrwxrwx 9 root root 133 Apr 5 12:05 ..
> 12 -rw-r--r-- 1 root root 8618 Nov 15 2010 binary_heap_.hpp
> 8 -rw-r--r-- 1 root root 4373 Nov 15 2010 const_iterator.hpp
> 8 -rw-r--r-- 1 root root 4458 Nov 15 2010 const_point_iterator.hpp
> 8 -rw-r--r-- 1 root root 4555 Nov 15 2010 constructors_destructor_fn_imps.hpp
> 4 -rw-r--r-- 1 root root 2509 Nov 15 2010 debug_fn_imps.hpp
> 0 -rw------- 1 root root 0 Apr 5 12:05 .entry_cmp.hpp.543CXr
> root at yaVDR:~#
>
> It is reproducible, any ideas? (If i do the same with huge files (1 GB) the partial file GETS deleted!
>
> Thanks
>
> lopiuh
>
>
>
>
> --
> Please use reply-all for most replies to avoid omitting the mailing list.
> To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
More information about the rsync
mailing list