[Samba] Need help with file corruption issue

[David Coppit]

> So you are creating files on the server side, access it from
> the client side, remove it on the server side again and
> create a new file server side under the same name?

No, This is much more serious. Please see the strace.txt log. Let me
step you through the last bit:

1) Here, I create a file SdLajo6RXt on the share. I read it from the
raw disk location and also read it from the mounted location, and it
matches.

Same!
/grid/samba_stress_test/SdLajo6RXt     :
0.5406506065286610.5406506065286610.5406506065286610.5406506065286610.540650606528661
/root/grid/samba_stress_test/SdLajo6RXt:
0.5406506065286610.5406506065286610.5406506065286610.5406506065286610.540650606528661

2) Next I delete it

unlink("/grid/samba_stress_test/SdLajo6RXt") = 0

3) Next I create a new file **with a different name**, write to it
directly on disk, and read it from the samba mount:

Different!
/grid/samba_stress_test/85fsYXTNhJ     :
0.9504576548397450.9504576548397450.9504576548397450.9504576548397450.950457654839745
/root/grid/samba_stress_test/85fsYXTNhJ:
0.5406506065286610.5406506065286610.5406506065286610.5406506065286610.540650606528661

**Note that the NEW file has incorrect content. It matches the OLD,
DELETED file.** I double-checked the trace, and the filenames in the
trace are all unique.

I mounted the share using "forcedirectio" and couldn't get it to repro.

I would think that the file name is a part of the key used for
caching! Is there some way to get visibility into the caching, so see
why it's apparently returning invalid data for a brand new file that
it should have *no* data for?

> Does the same also happen if you do the file
> creation/deletion via Samba as well?

It does not.

For fun, I self-mapped the share twice and wrote to one mapped share
while reading from the other, to simulate 1 client writing and another
reading. I was able to repro the issue.

I also went ahead and implemented a test where I used winexe to fetch
the file from a Windows machine that had the samba share mounted. I
was *not* able to repro it. So it's possible that there's something
wrong in the Linux cifs module, or it's a race condition and the
latencies of doing the remote command to "type
C:\path\to\mount\samba_stress_test\random_file" mean I can't repro it.
(It's possible that the corrupt files we saw on Windows before were
due to something else.)


On Mon, Jun 3, 2013 at 7:56 AM, Volker Lendecke
<Volker.Lendecke at sernet.de> wrote:
> On Fri, May 31, 2013 at 12:51:40PM -0400, David Coppit wrote:
>> Hey Volker, thanks for the reply.
>>
>> > Can you explain for really stupid people what this does and where the problem is?
>>
>> Here's what the perl code is doing:
>>
>> 1) In a loop...
>> 1.1) Write a file to the local disk, using a random filename and 5
>> random floats followed by a newline as the content.
>> 1.2) chown the file so that the samba mount user can read it
>> 1.3) Read that file from a cifs mount of that very same local disk
>> location, hosted by samba
>> 1.4) Compare the written content versus the read content, exiting if
>> they are different.
>> 1.5) Delete the temp file
>
> So you are creating files on the server side, access it from
> the client side, remove it on the server side again and
> create a new file server side under the same name? I would
> really think this is a caching issue, the client does not
> notice the file changed. The wireshark trace you sent does
> not contain any file related operations, so this time the
> client did not even ask the server to close and open the
> file again.
>
> Does the same also happen if you do the file
> creation/deletion via Samba as well?
>
> Volker
>
> --
> SerNet GmbH, Bahnhofsallee 1b, 37081 Göttingen
> phone: +49-551-370000-0, fax: +49-551-370000-9
> AG Göttingen, HRB 2816, GF: Dr. Johannes Loxen
> http://www.sernet.de, mailto:kontakt at sernet.de