Versioned files (take 2)
jw at pegasys.ws
Fri Oct 17 17:47:48 EST 2003
On Tue, Oct 14, 2003 at 01:09:03PM -0400, Jason M. Felice wrote:
> I've pondered the feedback and revised my proposal to the client. Here
> is the revised project objectives. Notably, this is the addition of 4),
> the deletion of the whole slew of items actually related to handling
> versioned files, and mention of preexisting work on 1).
> I've took a little gander at some of the backup wrappers, and it looks
> like I will probably use one of these. I'll have to look a bit closer
> to see which ones best fit my needs.
> -Jay 'Eraserhead' Felice
> Project objectives
> The backup system project will meet the following objectives:
> 1. Implement SSL connections.
> The modified client will use SSL for encryption of the protocol
> stream. Existing clients can use an external shell program such as SSH
> to provide encryption, but this is not portable and it is difficult to
> An "--ssl" option will be added to the rsync program to enable this
> feature. This option will be accepted in both client and daemon mode.
> Patches to rsync exist which do this. They will be evaluated and
> applied or modified as appropriate.
The patches that exist have issues with the three-way
connection. That is the primary reason they have not been
accepted although they have tended to have problems with
key management as well.
A better use of your developer time might be to work on
fixing the cygwin hang problem running rsync over ssh. I
suspect that the delays in fixing this problem have been a
lack of resources committed to fixing it. Using ssh is very
manageable and only a portability problem for legacy
> 2. Write a Windows backup service.
> 3. Write a configuration GUI for the Windows backup service.
> 4. Add a "--link-dest-type" option.
> Currently, rsync's "--link-dest" option will hard link files against
> an older copy in an identical directory structure when they have not
> changed in order to save space. With this option, the user would be
> able to specify the link destination type as either "mirror" or
> "hash." Mirror is the default, and will behave like existing versions
> of rsync.
> The "hash" type will calculate a directory name based on a strong hash
> of the file and the file's size, for example
> "/f7/d6/22/d9e9a6d8b9e9e4f00/1ff". rsync will search this directory
> for a file with identical contents to the one being transferred. If it
> finds one, it will hard link the transferred file to it. If it does
> not, it will create a new file with the next available integer
> containing the new file and hard link it to the destination.
Cute idea. Will be dog slow compared to a normal rsync.
You may want to sixel the hash which is 20 bytes.
--link-by-hash=dir would be a better name.
> This will allow us to store only one copy of a file which might exist
> in multiple places in a filesystem or even on multiple clients.
I wouldn't recommend accepting this option as part of a
proposal. It is vaporware that even if created has no
assurance of becoming a part of the mainline codebase.
Unless accepted into mainline the customer would then have a
patched rsync that needs to be repatched every time there is
an important (read security) update to mainline and no
support from the community.
> 5. Write a restore GUI for Windows.
> 6. Create Windows installer.
J.W. Schultz Pegasystems Technologies
email address: jw at pegasys.ws
Remember Cernan and Schmitt
More information about the rsync