Smoother bandwidth limiting
jw at pegasys.ws
Wed Jun 18 21:38:12 EST 2003
On Wed, Jun 18, 2003 at 10:02:37PM +1000, Martin Pool wrote:
> On 15 May 2003, Paul Slootman <paul+rsync at wurtel.net> wrote:
> > I can't really see that doing smaller writes will lead to packets being
> > padded, unless you're doing really small writes (ref. the ATM 48-byte
> > packets); the TCP and IP headers will always be added, which means that
> > the extra overhead of those will have a larger impact than any
> > padding.
> > So, I'd suggest that 1024 isn't that bad a number for all cases; it'll
> > fit comfortably into most MTU sizes, and for dialup PPP it'll be split
> > into two packets without that much overhead. If not concerned with the
> > dialup PPP case, I'd go for something like 1400.
> Of course a write() does not necessarily correspond to a TCP frame,
> which does not necessarily correspond to an IP packet.
> But nevertheless I would suggest avoiding writes that are this short.
> In addition to the headers that Paul mentioned, there are other
> per-packet costs such as Ethernet leadin and trailer times, and the
> hardware, interrupt and OS overhead for processing packets.
> Consider also that some people use rsync on fast networks, and they
> won't appreciate small packets *or* getting more system calls to
> process a given amount of data.
> Needlessly causing each packet hold 30% less data than it normally
> would is very wasteful. The point of bwlimit is after all to help
> users have more bandwidth for other applications.
> Checking for bwlimit after every say 4k I can imagine but below that
> is dubious. I'm happy to be proved wrong though.
Exactly. It seemed to me that the smaller the writes the
more likely internal (not network) fragmentation would
become a problem.
I almost like the fact that bwlimit is crude. Getting it to
work with both fast and slow networks in an optimal way may
J.W. Schultz Pegasystems Technologies
email address: jw at pegasys.ws
Remember Cernan and Schmitt
More information about the rsync