superlifter design notes (was Re: Latest rZync release: 0.06)

[Bennett Todd]

2002-07-21-04:12:55 jw schultz:
> On Thu, Jul 11, 2002 at 07:06:29PM +1000, Martin Pool wrote:
> >    6. No arbitrary limits: this is related to scalability.
> >       Filesizes and times should be 64-bit; names should be
> >       arbitrarily long. 
> 
> File sizes, yes.  Times, no.  unsigned 32 bit integers will
> last us for another 90 years.  I suspect that by the time we
> need 64 bit timestamps the units will be milliseconds.
> I just don't see the need to waste an extra 4 bytes per
> timestamp per file.

If bandwidth is of any interest at all, compress; any compression
algorithm will have no trouble making hay with bulky, redundant
timestamp formats. Rather than trying to optimize the protocol for
bandwidth without compression, wouldn't it be better to try to
optimize to future-proof in the face of changing time
representations across systems?

If I were designing a protocol at this level, I'd be using TAI;
there's 64-bit time with 1 second resolution covering pretty much
all time (more or less, depending on the whimsies of
cosmologists:-); there are also longer variations with finer
resolution. TAI, with appropriately fine resolution, should be able
to represent any time that any other representation can, closer than
anyone could care.

TAI can be converted to other formats with more or less pain,
depending on how demented the other formats are; djb's libtai is a
reasonable starting point.

<URL:http://cr.yp.to/time.html> has links to some pages discussing
time formats.

In short, though, "Time since the epoch" has a complication:
leap-seconds. Either you end up having to move the epoch every time
you bump into a leap-second, thereby redefining all times before
that; or else you have duplicate times, where two different seconds
have the same representation in seconds-since-the-epoch. Well,
there's a third possibility, you could also let the current time
drift further and further from what everybody else is using, but
nobody seems to go for that one.

-Bennett
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 232 bytes
Desc: not available
Url : http://lists.samba.org/archive/rsync/attachments/20020725/f67bb604/attachment.bin