TDB on OpenBSD problem.

MCCALL,DON (HP-USA,ex1) don_mccall at hp.com
Tue May 8 20:47:33 GMT 2001


Hi Tridge,
As someone else on the list responded, this problem is basically because on
HP_UX
we don't have a unified buffer cache; the mmap stuff uses page cache and the
fileio
uses buffer cache - we're looking at implementing a unified buffer cache in
our next
major HP-UX release, but that doesn't help here.  As far as limits, if we
are using
mmap shared, then basically we are not limited - a 32bit implementation has
about 
1.9GIG of shared space across all processes running on HP-UX, and individual
processes
are not limited on the size of shared mmap'ings, as this goes into the
shared space
instead of the data segment space, which WOULD be limited by the maxdsize
kernel
parameter (tunealble...).  So I think Jeremy's fix should be applicable on
HP-UX as 
well; unmapping before any lseeks, etc, and then remapping (assuming that
the fileio
buffers have been flushed to the file before the remapping) should ensure
syncronization.


Hope this helps,
Don
 "Reason, not volume, is the primary
differentiator between a discussion, and an
argument."

-----Original Message-----
From: tridge at samba.org [mailto:tridge at samba.org]
Sent: Saturday, May 05, 2001 10:02 PM
To: don_mccall at hp.com
Cc: jeremy at valinux.com; vmn at bom.gov.AU; samba-technical at lists.samba.org
Subject: Re: TDB on OpenBSD problem.


>  The application must ensure correct synchronization when using mmap()
>       in conjunction with any other file access method, such as read() and
>       write(), standard input/output, and shmat().

But does it say *how* that synchronisation can be done by the app??
This sounds like a cop-out to me. "we can't work out how to do it -
its the apps responsibility".

> So I added a call to msync() right before the call to test() to the test
> program, and with this I resolve the inconsistency on the 11.0 box I was
> seeing the error on.

an msync() is potentially very expensive. It might even be faster to
drop mmap support on HPUX and use read/write.

The patch jeremy committed seems to work for simple cases, but I'm
concerned that it leaves us with a potential problem if the database
grows beyond the size allowed for mmap by the rlimits for the
process. In that case some of the smbds will drop back to read/write
IO and thus will become inconsistent with the ones that are still
using mmap.

Is this a serious enough problem to worry about? I'm not sure. Can
someone tell me what limits apply to mmap size on HPUX? I tested on
OpenBSD 2.5 and it appears that none of the limits from "ulimit -a"
affect mmap size, so the fix should be OK there.

Cheers, Tridge




More information about the samba-technical mailing list