[Samba] Need help with file corruption issue

[Volker Lendecke]

On Mon, Jun 03, 2013 at 06:41:33PM -0400, David Coppit wrote:
> > So you are creating files on the server side, access it from
> > the client side, remove it on the server side again and
> > create a new file server side under the same name?
> 
> No, This is much more serious. Please see the strace.txt log. Let me
> step you through the last bit:
> 
> 1) Here, I create a file SdLajo6RXt on the share. I read it from the
> raw disk location and also read it from the mounted location, and it
> matches.
> 
> Same!
> /grid/samba_stress_test/SdLajo6RXt     :
> 0.5406506065286610.5406506065286610.5406506065286610.5406506065286610.540650606528661
> /root/grid/samba_stress_test/SdLajo6RXt:
> 0.5406506065286610.5406506065286610.5406506065286610.5406506065286610.540650606528661
> 
> 2) Next I delete it
> 
> unlink("/grid/samba_stress_test/SdLajo6RXt") = 0
> 
> 3) Next I create a new file **with a different name**, write to it
> directly on disk, and read it from the samba mount:
> 
> Different!
> /grid/samba_stress_test/85fsYXTNhJ     :
> 0.9504576548397450.9504576548397450.9504576548397450.9504576548397450.950457654839745
> /root/grid/samba_stress_test/85fsYXTNhJ:
> 0.5406506065286610.5406506065286610.5406506065286610.5406506065286610.540650606528661
> 
> **Note that the NEW file has incorrect content. It matches the OLD,
> DELETED file.** I double-checked the trace, and the filenames in the
> trace are all unique.

Could it be that the inode numbers are the same for the
deleted and the newly created file? Possibly the caching on
the Linux client depends on them.

> I mounted the share using "forcedirectio" and couldn't get it to repro.
> 
> I would think that the file name is a part of the key used for
> caching! Is there some way to get visibility into the caching, so see
> why it's apparently returning invalid data for a brand new file that
> it should have *no* data for?

If it's the same inode number and caching depends on that,
this could be possible I guess. One factor could also be
that under Unix due to hard links the filename:file
relationship is not 1:1, so it is entirely possible for
files with different names to have the same content.

You can check inode numbers with ls -li.

> > Does the same also happen if you do the file
> > creation/deletion via Samba as well?
> 
> It does not.
> 
> For fun, I self-mapped the share twice and wrote to one mapped share
> while reading from the other, to simulate 1 client writing and another
> reading. I was able to repro the issue.

Same problem possibly. If your file system gives you the
same inode number, this might fool the linux client.

> I also went ahead and implemented a test where I used winexe to fetch
> the file from a Windows machine that had the samba share mounted. I
> was *not* able to repro it. So it's possible that there's something
> wrong in the Linux cifs module, or it's a race condition and the
> latencies of doing the remote command to "type
> C:\path\to\mount\samba_stress_test\random_file" mean I can't repro it.
> (It's possible that the corrupt files we saw on Windows before were
> due to something else.)

Good.

Volker

-- 
SerNet GmbH, Bahnhofsallee 1b, 37081 Göttingen
phone: +49-551-370000-0, fax: +49-551-370000-9
AG Göttingen, HRB 2816, GF: Dr. Johannes Loxen
http://www.sernet.de, mailto:kontakt at sernet.de