[clug] Ext2 vs Ext3: weird benchmark result

[Nathan O'Sullivan]

I've been doing some basic benchmarking with Unixbench 5.1.2 of 
different Xen kernels and noticed that the reported disk performance was 
quite variable, but with a distinct pattern - some kernels had roughly 
double the disk speed of others.

I dont use an initrd with our Xen seutp; after a lot of hunting I 
narrowed it down to how the root disk was being mounted. I have a single 
ext3 partition on /dev/xvda2 and our kernel is normally started with:

root=/dev/xvda2 ro

I found that some of my kernels had ext3 compiled in (CONFIG_EXT3_FS=y), 
while others had it as a module (CONFIG_EXT3_FS=m) - all have 
CONFIG_EXT2_FS=y. Checking in dmesg, the kernels that have ext3 as a 
module would print:

  EXT2-fs warning (device xvda2): ext2_fill_super: mounting ext3 
filesystem as ext2
  VFS: Mounted root (ext2 filesystem) readonly on device 202:2.

This makes sense, since without an initrd the ext3 module isnt available 
so ext2 is the only option to mount with during kernel boot. In the 
kernels with ext3 compiled in (CONFIG_EXT3_FS=y) the device just mounts 
normally as ext3 readonly.


What doesn't make sense is that UnixBench reports roughly double the 
speed when CONFIG_EXT3_FS=m was used - that is, when the initial 
rootdisk mounted as ext2.

Through editing /etc/fstab and rebooting (And checking via 'mount'), I 
determined that whether ext2 or ext3 was used when the filesystem was 
mounted read-write made no difference. Whether UnixBench would be 'fast' 
or 'slow' depended solely on how the root disk was initially mounted 
readonly - ext2 or ext3.

I was able to confirm this by using my CONFIG_EXT3_FS=y kernel with the 
command line:

root=/dev/xvda2 rootfstype=ext2 ro

With this in place, dmesg shows the same warning message as the 
CONFIG_EXT3_FS=m kernel and it initially mounts as ext2 while readonly. 
This on its own with no other change, pushed the UnixBench back up to 
full (double) speed.

Its worth repeating, editing /etc/fstab had no effect - if the initial 
readonly mount was ext3, the readwrite mount could be ext2 or ext3 - 
both were slow. And if the initial readonly mount was ext2, the 
readwrite mount could be ext2 or ext3 - both were fast.

I'm having a hard time understanding how this might even be possible - 
that mounting as ext2 or 3 during the initial root startup could somehow 
affect speed later. And even assuming UnixBench is exaggerating the 
difference somewhat, I find it unlikely this kind of difference would 
not have been observed as a bug and corrected.

Does anyone have any advice on what might explain this difference, or 
other related things I could investigate?

Regards
Nathan