[Samba] Samba large file support negotiation when accessed via UNC path

[Andrew Boyce]

I am experiencing a strange problem that I hope someone out there has
experience withf.

We have a large fileserver running samba 3.6 backed by a 12T ext4
filesystem. We keep lots of VMware VMs on the filestore and I am trying to
make some improvements to how we manage/launch/use the VMs. To that end,
I've been trying to launch the VMs via the VMware VIX API by UNC path (not
mapped drive).

The trouble is that when accessed via UNC path, it seems that VIX can't
properly determine that the fileserver supports large files and refuses to
open the VM.

VIX Error:

DISKLIB-SPARSE:
"\\VMWAREFS.org.net\images\windows_7_ultimate_x64_en\Windows_7_x64-000003.vmdk"
: failed to open (36872): Underlying file system does not support files that
large.
DISKLIB-LINK  :
"\\VMWAREFS.org.net\images\\windows_7_ultimate_x64_en\Windows_7_x64-000003.vmdk"
: failed to open (The file is too large).
DISKLIB-CHAIN :"\\VMWAREFS.org.net\images\\windows_7_ultimate_x64_en\Windows_7_x64-000006.vmdk":
Failed to open parent
"\\VMWAREFS.org.net\images\windows_7_ultimate_x64_en\Windows_7_x64-000003.vmdk":
The file is too large.
DISKLIB-CHAIN :
"\\VMWAREFS.org.net\images\windows_7_ultimate_x64_en\Windows_7_x64-000003.vmdk"
: failed to open (The parent of this virtual disk could not be opened).
DISKLIB-LIB   : Failed to open
'\\VMWAREFS.org.net\images\windows_7_ultimate_x64_en\Windows_7_x64-000006.vmdk'
with flags 0x1e (The parent of this virtual disk could not be opened).

At face value this seems like a VIX error however when accessed from samba
via a mapped drive, it works as expected. Also, when the files are hosted on
a Windows server VIX can launch the VM via UNC path or mapped drive without
any problem.

This is a known problem in the VMware KB:
http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&externalId=1008633(KB
is about converter, but the error is the same with workstation and
VIX).


Unfortunately the VMware community seems to have no idea what the cause is,
only a workaround (and a workaround that I can't use at that .

So to you alll, I ask: What is different about the way that samba serves
files or the way that windows negotiates a UNC path that might cause this
problem?

I have not enabled SMB2, so that isn't part of the problem. Also, I've used
wireshark to analyze the protocol negotiation and can confirm that the
capabilities are identical between a windows and samba server (most
importantly, the Large Files bit is set on both responses). The only
interesting difference that I have found in the negotiation is that Samba
doesn't ever seem to set the "Canonicalized Pathnames" flag even though I
read that it should "always be set on modern CIFS implementations".

Thanks in advance,
  -Andrew