TODO hardlink performance optimizations

Wayne Davison wayned at samba.org
Wed Jan 7 23:25:39 GMT 2004


On Wed, Jan 07, 2004 at 01:30:19AM -0800, jw schultz wrote:
> On Wed, Jan 07, 2004 at 02:45:46AM -0600, John Van Essen wrote:
> > The point of this exercise was to find a way to avoid unnecessary
> > transfers of already existing files
> I thought the point was to reduce the memory footprint and
> then get rid of the binary search.

They are both desireable goals, and I'd like to see one other:  a
reduction in number of bytes transmitted when sending hard-link data.
If we omit the dev/inode data for items that can't be linked together,
we should be able to save a large amount of transmission size (but
this will require a protocol bump).  Of course this does not mean that
the new optimized hard-link code would require this optimized sending
in order to work.

> - Create the union and change file_struct and the routines
>   that reference and populate it to use the union for dev
>   and inode.  This may include not allocating the union for
>   unlinkable files.

I had been considering possible ways to avoid having the extra pointer
in the flist_struct, and a suggestion John made has made me think that
we can leave it out if we allow the file_struct to be of variable
length.  We'd set a flag if it has the extra trailing data, and never
refer to this data if the flag is not set.

> - Reduce the hlink_list to just the heads and change
>   do_hard_links.

I'm not sure this is worth the cost of copying the bytes, but we'll
see.

> Each of these is a discrete step that when complete the code
> will function correctly.

Yes.  Nice plan.  If either of you have started coding the next stuff,
let me know -- I'm thinking about doing some coding.

..wayne..


More information about the rsync mailing list