Making --inplace and --sparse compatible

[David Taylor]

On Wed, 11 Dec 2013, Matthias Schniedermeyer wrote:

> On 10.12.2013 21:05, Perry Hutchison wrote:
> > It seems to me that it should be possible to make --inplace and
> > --sparse compatible.
> > 
> > The manpage says
> > 
> >     -S, --sparse
> >            ... Conflicts with --inplace because it's not
> >            possible to overwrite data in a sparse fashion.
> > 
> > but I think it _is_ possible to do that, provided the receiving
> > side opens the dest file read-write instead of write-only.
> > 
> > When writing within the existing length of the dest, it should be
> > possible -- when processing a src block which is all zero and thus
> > would be seeked over instead of written when writing sparsely to a
> > file being newly created -- to read that block from the dest and
> > refrain from writing it back if it is already zero in the dest file.
> > (Of course, if it is _not_ all zero in the dest, the non-zero bytes
> > would have to be overwritten with zeros.)
> > 
> > Granted this approach will not introduce sparseness into an existing
> > non-sparse dest, but it at least will preserve any sparseness which
> > already exists in the dest.
> > 
> > I'm currently using 3.0.7, but I think the same issue still exists
> > in 3.1.0 since there's no mention of such a change in the release
> > notes and the 3.1.0 manpage still documents the same limitation.
> 
> And there is hole-punching nowadays (at least for linux and it should 
> work on the major filesystems. Not sure about others). So a file can be 
> made sparse after the fact.
> 
> And nowadays (linux again) there is fiemap and the older(?) fibmap and 
> the newest addition seek_hole/seek_data to determine sparseness.
> Altough i'm unsure which work for which filesystems and also if support 
> is mainlined or still in development for the major filesytems.
> 
> So nowadays or in the near future (at least for linux with the major 
> filesystems) determining sparseness and creating sparseness "after the 
> fact" is possible.

I did start writing a patch to use SEEK_HOLE / SEEK_DATA and
fallocate FALLOC_FL_PUNCH_HOLE to "synchronise holes".

It's actually quite easy and I got a very basic proof of concept
working quickly.

Unfortunately trying to hack it into the rsync protocol and codebase
was more... challenging. 

In the end I got distracted and started working on something else.
But it's certainly not impossible.

-- 
David Taylor