David.Collier-Brown at Sun.COM
Wed Jun 8 14:33:05 GMT 2005
Volker Lendecke wrote:
> Hi, David!
> When you look at the pattern XPSP2 gives you, this does not really work that
> way, at least I don't know how to keep sanity. The following table shows the
> read pattern I see coming from XPSP2 when the Win32 app reads 256k blocks, the
> tables shows offset/length:
> This looks like two threads reading in parallel, blocks starting at 256k and
> 512k. This happens for example with cygnus dd bs=256k which most probably does
> not issue multi-threaded requests, as well as with the XDD tool from
> ioperformance.com with explicit setting to sequentially read.
> .... And sequential again (not shown anymore). This pattern proceeds until the
> file is ended.
> Now tell me how I can sanely detect this kind of pattern with a cache... :-)
This is a bizzare enough that unless your cache is large enough (> 262K) or
your systems pre-read/post-write is large enough to catch **both** of these...
... then the cache will ping-pong and the thrashing will **slow** the system
instead of speeding it up..
> De-install SP1 or use any other Windows version and you're fine.
Looks like an undocumented feature of SP1 (;-))
Windows Media Server reads individual files sequentially, but it
starts reading (according my leaky memory)
3 * 1K, to get to a 4K boundary
2 * 4K to get to an 8K boundary
1 * 8K to get to a 16K boundary
1 * 16K to get to a 32K boundary
1 * 32K to get to a 64K boundary
then 64K reads to the end of the file.
This was utterly pessimal, and there were multiple files
being read, but a per-file buffer of 64K made it fast
no matter what the stripe size was, and a buffer of
the stripe width gave downright wonderful performance. The
numbers are in the samba-technical archive somewhere...
If the mismatch wasn't 524288 - 262144 = 262,144 bytes, then
a per-file cache would catch both read streams and smooth out
access at the system level.
A 256K per-file buffer might be reasonable, but a 512
of 1024K buffer per file sounds excessive at first glance (:-))
If I were building a dedicated server for storing streaming
data, or if my stripe width was 1024K, that might well change.
David Collier-Brown, | Always do right. This will gratify
Sun Microsystems, Toronto | some people and astonish the rest
davecb at canada.sun.com | -- Mark Twain
(416) 263-5733 (x65733) |
More information about the samba-technical