[Samba] Missing Files/Missing Folders from an NFS Share

[Conor Armstrong]

Ok, starting to make sense now.  In order to fix it....

I note that the lower level calls are wrapped in dir.c - eg
dptr_SeekDir(...) wraps SeekDir(...)

this might allow for some code to use array indexes instead of NFS cookies
as noted by Chris Chilvers here for the 3.10 kernel:
https://lore.kernel.org/all/CAAmbk-e-YQAPo6QyNB0aJyc9qzUShmEC+x5eTR7wqp1ABWADsg@mail.gmail.com/T/

> On the older 3.10 kernel, this was not an issue as the 3.10 kernel uses
> the array index position for the offset value instead of the NFS cookie.

I am not yet 100% familiar with all the code but perhaps we could store an
array of all the directory entries when we open the directory first time
and then utilise the array index instead of the cookie.  Problems I see
with this though are the increase in memory consumption.  If there was a
way to check the underlying filesystem type and if it is NFS use this
approach as a fallback???



On Thu, 16 Feb 2023 at 21:24, Jeremy Allison <jra at samba.org> wrote:

> On Thu, Feb 16, 2023 at 12:20:27PM -0800, Jeremy Allison via samba wrote:
> >
> >doesn't return us to the same position in the directory
> >listing, then we are broken and will not return the complete
> >list of files.
> >
> >This is what Ralph is saying when he says that NFS is
> >broken w.r.t listing directories. What NFS server are
> >you using ?
>
> Oh I just remembered, you're using the Amazon NFS
> server.
>
> >This looks relevent:
> >
> >
> https://lore.kernel.org/all/CAAmbk-e-YQAPo6QyNB0aJyc9qzUShmEC+x5eTR7wqp1ABWADsg@mail.gmail.com/T/
>
> Indeed. Note the comment from Trond (NFS client
> maintainer in the Linux kernel).
>
> ---------------------------------------------------------------
> On Thu, 2022-09-08 at 10:45 +0100, Chris Chilvers wrote:
> > I should have flagged this as a bug on the first email.
> >
> > I've run into an interesting issue using VAST data, their NFS
> > implementation
> > makes use of the full 64 bit unsigned range allowed by NFS cookie for
> > READDIR
> > and READDIRPLUS.
> >
> > The issue is that this 64 unsigned value is used for the directory's
> > file
> > position (d_off), which is a 64 bit signed value. This can cause
> > readdir and
> > telldir to return negative values.
>
> Known issue that we had to deal with 25 years ago for 32 bit systems
> when glibc first decided to make lseek() return signed values (struct
> old_linux_direct still has it as an unsigned value).
>
> So VAST have had a long time to learn not to do this...
>
> --
> Trond Myklebust
> Linux NFS client maintainer, Hammerspace
> trond.myklebust at hammerspace.com
> ---------------------------------------------------------------
>
> Looks like Amazon may have the same bug.
>