disconnected synchronization (mostly unidirectional)

[Konrad Karl]

Hi,

On Mon, Mar 12, 2007 at 03:26:04PM -0700, Wayne Davison wrote:
> On Mon, Mar 12, 2007 at 07:31:52PM +0100, Konrad Karl wrote:
> > it just would need the fileinfo of another_local_directory from
> > somewhere (database, whatever) in order to generate rsync batch 
> > files with --whole-files or am I missing something?
> 
> Yes, if you want to either code up a fuse filesystem that makes it
> appear that there is a hierarchy of files present, that would work
> with an unmodified rsync --whole-files using --write-batch (assuming
> that you made the fuse filesystem discard the file data and update
> the file info).  Other than that, you'd need to dig into the various
> stat(), readdir(), etc. functions that the receiving side calls and
> direct them to a DB.
> 
> ..wayne..

OK, I tried to hack something together it the quick way: 
(it needs a filesystem which supports sparse data, e.g ext[23] 
or similar to hold the "metadata")

First I modified the cpio code for -p (the pass option) 
to not copy the file contents but instead lseek
to len-1 (if len > 0) and write a zero byte at the end. 
The source file is not being read at all.

Using this modified cpio (called zcpio :-) I now can do:
find -depth -print0 |zcpio -0pmdu destdir 

This command creates a "copy" which intact metadata and very little
space needs (~5 percent in my test case) on sparse-capable
filesystems. GNU tar seems to be able to archive those but I did not
check for correctness yet.

Using --whole-files and --write-batch I am now able to create
an rscync batch file which can be applied to the original tree.

However it is not all that fast and I would really like to have
some option to rsync for generating, dumping and reloading 
filelists. (still trying to understand how exactly the two
or three processes are working together. Is there a description
somewhere?)

Greetings,
Konrad


If anyone is interested, here is the minor mod to cpio src/util.c:

void
copy_files_disk_to_disk (int in_des, int out_des, off_t num_bytes,
                         char *filename)
{
  long size;
  long k;
  off_t original_num_bytes;
  int rc;

>>  char nul=0;
>>  if (num_bytes==0) return;
>>  lseek(out_des,num_bytes-1,0); // should be SEEK_SET
>>  write (out_des,&nul,1);
>>  return;