failing LOCK1 smbtorture test

[MCCALL,DON (HP-USA,ex1)]

Thanks Jeremy,
I thought that might be the case, as I looked thru the code, I couldn't 
find anywhere that would ensure consistency between smbds.  One thing that I
found odd was
that I was able to consistently make LOCK1-4 tests pass IF I added the 
MAP_FIXED flag to the mmap call in tdb_mmap. I still don't understand 
why that is making a difference, other than it doesnt allow mmap to 
'choose' where to put the memory, like a value of 0 does when you don't
have MAP_FIXED specified.

On the topic of syncing up everyone to revert to read/write when ANY
tdb_mmap call fails, that's a tough one - I wouldn't suppose we would
want to go back and reinstitute a shared memory segment at this point.
perhaps we could use a hidden global and institute an lp_ method
to set that to False in tdb_mmap if it fails, and then have tdb_read
and tdb_write check that global in addition to the (map->ptr)test.
We could also check it in tdb_mmap right before the call to mmap,
and just skip the mmap if anyone has set it to False.
If we didn't hide it (actually added it to loadparm.c, etc) we could
even use it to force non-use of mmap on systems that DO have mmap
capability for debugging purposes...

I suppose we would have to notify the smbd parent somehow so that 
he could notify the current smbd processes to reread the smb.conf file...

it would be a one time perf hit; we would only set the global the first
time mmap failed, since we check the global before calling mmap;  new
smbd's would know it's false at startup; current smbds would take a onetime
hit to reread the smb.conf file.

But I'm sure there's a much better way to do this....
Please let me know what you decide, and thanks so much for clarifying this 
for me.

Don

-----Original Message-----
From: Jeremy Allison [mailto:jeremy at valinux.com]
Sent: Wednesday, July 25, 2001 2:48 PM
To: MCCALL,DON (HP-USA,ex1)
Cc: 'samba-technical at lists.samba.org'
Subject: Re: failing LOCK1 smbtorture test


"MCCALL,DON (HP-USA,ex1)" wrote:
> 
> Hi Jeremy,
> I have been having a hard time getting samba 2.2.latestcvs to pass LOCK1
> smbtorture test,
> and I think i MAY have discovered why...
> I added some debug statements to the code to print out the tdb->map_ptr
> value after
> opening/expanding/ect  a tdb file when calling tdb_mmap, and discovered
that
> 
> HP-UX mmap can return a tdb_mmap value of '0'.  The man page for mmap SAYS
> that this
> should not happen, but hey...
> 
> I think what is happening in my case is that because we are falling thru
to
> filesystem
> reads and writes if mmap isn't working, and HP-UX doesn't have a unified
> buffer cache, that
> when we DO use a tdb_mmap of 0, we start using reads and writes (see code
> snippet from tdb.c
> below)
> /* read a lump of data at a specified offset, maybe convert */
> static int tdb_read(TDB_CONTEXT *tdb,tdb_off off,void *buf,tdb_len len,int
> cv)
> {
>         if (tdb_oob(tdb, off + len, 0) != 0) return -1;
> 
>         if (tdb->map_ptr) memcpy(buf, off + (char *)tdb->map_ptr, len);
>         else if (lseek(tdb->fd, off, SEEK_SET) != off
>                  || read(tdb->fd, buf, len) != (ssize_t)len) {
> ...
> 
> but not all of the other tdb calls (like tdb_fetch, for example)  test
> against tdb->map_ptr
> being 0...

All tdb fetch/set calls eventually go through tdb_read()
and tdb_write(). These calls check for tdb->map_ptr being
null and then use lseek/read/write instead.

The problem is that when one tdb using process changes to
read/write, under a system with a non-coherent mmap/read/write
cache all processes must change to doing so. Currently this doesn't
happen, as when one process tries to re-mmap and it fails,
other processes that happily have it mmapped will continue
to write to areas within the mmapped range using memcpy,
only switching to read/write when they need to write to
an area outside their current mmapped range.

I'm looking at a way to cause tdb to switch to read/write
for all processes once a mmap has failed (or returned zero,
which is the same thing).

It does look like HPUX does have a broken mmap implementation
however.

Jeremy.


-- 
--------------------------------------------------------
Buying an operating system without source is like buying
a self-assembly Space Shuttle with no instructions.
--------------------------------------------------------