Modifying Samba to skip file system reads for IOZONE READ
capps at iozone.org
Sat Oct 29 05:55:27 GMT 2005
----- Original Message -----
From: "Jeremy Allison" <jra at samba.org>
To: "Iozone" <capps at iozone.org>
Cc: "Andrew Bartlett" <abartlet at samba.org>; "Richard Sharpe"
<rsharpe at richardsharpe.com>; <samba-technical at lists.samba.org>; "Jeremy
Allison" <jra at samba.org>
Sent: Saturday, October 29, 2005 12:10 AM
Subject: Re: Modifying Samba to skip file system reads for IOZONE READ
> On Fri, Oct 28, 2005 at 09:06:28PM -0500, Iozone wrote:
>> I forgot to mention the side-effect of the release of the hack.
>> With the hack anyone could create a system that has a bogus
>> performance advantage.
>> So.... I had to change Iozone to make sure that this hack
>> will not work. The change was to alter the pattern used
>> with each revision of Iozone. Cool... problem solved.
>> BUT, it has an unfortunate side-effect on Iozone's functionality !
>> Before this hack became available, Iozone users could
>> create data sets, store them for as long as they wished,
>> upgrade to new versions of Iozone and then continue
>> to use their old data sets. Well, that's gone now. Now
>> the user will be forced to create new data sets, anytime
>> they upgrade their version of Iozone. A very unfortunate
>> side effect of release the hack, but alas, now mandatory.
>> (That 2 pedabyte file, that you were using for test, well, you
>> get to re-create it now) Arrrrgh....
>> I prefer synergy, not tossing out hand grenades and watching
>> the fireworks. In this case, the users just lost a feature :-(
> Well I think that's a shame. Look, there are always silly hacks that
> people can do to 'fix' performance for a particular benchmark.
> Throwing away a useful feature because someone posted some test
> code is (IMHO) an over-reaction and the way you describe it here
> "unfortunate...alas, now mandatory" is the sign of someone not
> thinking clearly about this.
> By all means add this as an option (even the default if you wish)
> but at least allow people to select the old behaviour. If you
> caused me to have to regenerate a 2 petabyte file each time I
> did a test on server code I *know* is clean I'd be quite cross
> with you.
Now that the hack is public knowledge, my hands are
tied. I'm forced to close the hold to protect the integrity
of the benchmark. It is also unfortunate that the hack
didn't simply use the -V option of Iozone, (permits one
to set the pattern to a specific value) instead of invisibly
grabbing the default pattern. The whole mess could have
If what you are asking for is a new option to permit
the 0xA5 pattern, I think that this is similar to
the -V option. I'll take a look in the code and validate
that the -V option provides this capability.
Right now users have the several options for data
1. Default. Simple pattern automatically generated.
Was 0xA5, and is now based on Iozone version.
2. -V # User specifies pattern to validate.
3. -+d Very complex random patterns are generated
and validated. Each byte, in the buffer, is as random
as possible, and each child process uses a different
seed, so each file is also as unique as possible.
One possibility would be to add an option for compatibility
with old data sets. This would let the user keep the previous
data sets, and also inform the user that the option has
been enabled, and is not default behavior. Its use can
be documented and users will see the option and know
what's going on. It's possible use, by short circuit optimizations
in filesystems can be fully explained and the users
warned of possible intercepts.
Another possibility is to add an option specifically for
filesystem testing that locks the pattern to something that
filesystem code can expect to always contain the pattern
that it needs for short circuit testing. This has several nice
A. Everyone can see the option has been enabled, and
know exactly what it does, and why.
B. Samba, and other filesystem types, can depend on
this pattern remaining a constant. Enabling a long
term testability feature for the filesystem developers.
Or perhaps both of the above ?
Again, I don't believe that going back an option, but going forward
together, this time, would be logical.
More information about the samba-technical