[Samba] Can't copy large files to Windows with SMB2/3 on 10G network
MI
mi.lists at alma.ch
Sun Sep 16 07:53:59 UTC 2018
We have a 10G network with a mix of Windows 10 and MacOS 10.12 machines,
and a CentOS 7 machine.
Copying large files "fast" from the CentOS machine to Windows, using
SMB2 or SMB3 fails almost every time, at the end of the copy.
- With SMB1, it seems to work, but is much slower.
- Copying to a smb share on a Mac seems to work, but is also much slower.
- Using a normal Gigabit network card instead of 10G, it always works.
By "large files" I mean 50 to 100 GB.
By "fast" I mean over 300 MBytes per second (usually 500 to 700 MB/s.)
as reported by various monitors on the Linux client or the windows servers.
By "slower" I mean under 150 MB/s.
I tried both smbclient and mount.cifs:
- smbclient [no -m option] is OK and slow (uses default SMB1/NT1?)
- mount.cifs -o vers=1.0 is OK and slow
- smbclient -m SMB2 (or -m SMB3) fails
- mount.cifs -o vers=3.0 (or vers=2.1 or vers=2.0) fails.
I also tried 2 different 10G cards on the Linux client (Intel and Asus).
The few times when the fast copy worked, it was to a Windows share on a
very fast RAID (faster than 10G). But even copying to that specific fast
disk didn't work every time.
Example errors when it fails:
cp: failed to close ‘/mnt/dcp/temp/file.mxf’: Input/output error
or
rsync: close failed on "/mnt/dcp2/temp/.file.mxf.41mPmS": Input/output
error (5)
rsync error: error in file IO (code 11) at receiver.c(859) [receiver=3.1.2]
When I turned on debugging, I got an avalanche of "Status code returned
0xc000009a STATUS_INSUFFICIENT_RESOURCES" in the log, and tons of
"callbacks suppressed". Example:
...
[Fri Aug 31 17:21:53 2018] Status code returned 0xc000009a
STATUS_INSUFFICIENT_RESOURCES
[Fri Aug 31 17:21:53 2018] Status code returned 0xc000009a
STATUS_INSUFFICIENT_RESOURCES
[Fri Aug 31 17:22:21 2018] cifs_demultiplex_thread: 927 callbacks suppressed
[Fri Aug 31 17:22:21 2018] fs/cifs/connect.c: RFC1002 header 0x50
[Fri Aug 31 17:22:21 2018] smb2_check_message: 929 callbacks suppressed
[Fri Aug 31 17:22:21 2018] fs/cifs/smb2misc.c: smb2_check_message
length: 0x54, smb_buf_length: 0x50
[Fri Aug 31 17:22:21 2018] smb2_calc_size: 929 callbacks suppressed
[Fri Aug 31 17:22:21 2018] fs/cifs/smb2misc.c: SMB2 len 85
[Fri Aug 31 17:22:21 2018] smb2_check_message: 3 callbacks suppressed
[Fri Aug 31 17:22:21 2018] fs/cifs/smb2misc.c: Calculated size 85 length
84 mismatch mid 3265029
[Fri Aug 31 17:22:31 2018] fs/cifs/smb2pdu.c: In echo request
[Fri Aug 31 17:22:31 2018] __smb_send_rqst: 916 callbacks suppressed
...
I tried this all with (on the same machine) :
- Samba 4.7.1 on CentOS 7.5 with kernel 3.10.0-862.11.6.el7.x86_64
- Samba 4.3.11 on Ubuntu 16.04 with kernel 4.4.0-134-generic
Many Google searches didn't bring up anything which seems relevant, so I
hope someone here has ideas...
Thanks for any help or pointers,
MI
More information about the samba
mailing list