[linux-cifs-client] Re: Transfering large files with CIFS over an openvpn tunnel

[Stanislav Maslovski]

Hello,

Finally, I got some time to experiment with the options:

On Fri, May 11, 2007 at 09:40:46AM -0500, Steve French (smfltc) wrote:
> linux-cifs-client-request at lists.samba.org wrote:
> 
> >I got this problem when switched from smbfs to cifs.
> > 
> >
> smbfs does much smaller writes than cifs
> 
> >I mounted a share over an openvpn tunnel, like this:
> >sudo mount -t cifs '\\172.16.1.2\share' /mnt 
> >-ousername=xxx,password=yyy,uid=1000,gid=1000,file_mode=0644,dir_mode=0755,iocharset=utf8
> >
> >On the other side of the tunnel there was Windows XP Professional SP2.
> >The share was on an NTFS volume.
> >
> >
> Do you know if smb signing is set to required on the windows side (it 
> usually would not be)


No, it is not enabled and is not set to required.


> >Then I tried to copy a large file (about 1.3 Gb) to the new mount. The
> >coping started with a speed of about 20-30 Mbytes/sec (the network
> >connection speed is much less), and after it "copied" (to where?) about 
> >300 Mbytes
> >of data the transfer stopped and the following error messages
> >began to appear on the console:
> >
> >CIFS VFS: server not responding
> >CIFS VFS: No response to cmd 47 mid 16722
> >CIFS VFS: No response to cmd 47 mid 16721
> >CIFS VFS: Write2 ret -11, written=0
> >CIFS VFS: Write2 ret -112, written=0
> >CIFS VFS: Write2 ret -112, written=0
> >CIFS VFS: Write2 ret -112, written=0
> >............................
> >
> >It was impossible to kill the copying process, and in a few tens of secods
> >the system became practically not responsible: I could not login as root 
> >on the other
> >console, Ctl-Alt-Del was not effective, etc, so I had to hard reset.
> >
> >After some experimentation I found that adding 'directio' to the mount 
> >options
> >solved this problem.
> >
> >Before I switched to cifs, smbfs worked flawlessly with the same set-up.
> >
> >I am not an expert at all, but for me it seems that either cifs in kernel 
> >or
> >something related to openvpn eats lot of RAM (in kernel space?) till the 
> >system
> >effectively dies.
> > 
> >
> You can get a feeling for how much memory cifs is using by doing:
>    cat /proc/slabinfo | grep "cifs"


I did this and I did not notice any significant increase in the numbers
during the time of copying (until it freezes and the system becomes
hardly responding).


> that may also show how much memory your vpn is using


Unfortunately, I could not identify there anything related to vpn.


> Typically cifs woud use less than 256K of memory but that can be higher 
> if the number of inodes (metadata cache)
> in the vfs large (which is somewhat out of cifs's control).   The mm can 
> cache a lot of inode data though
> look at:
>    cat /proc/meminfo
> to see how much memory is being used for caching by the Linux memory 
> manager.


>From here I see that the amount of cached data increases rapidly from about
200 M before the copying up to 400+ M during the copying. Also this parameter
rapidly increases:

Inactive:       107412 kB  <- before coping
Inactive:       338504 kB  <- during the coping

The total amount of RAM in the computer is 512 M.


> >Versions of the related software:
> >OS: Debian Etch 4.0
> >Kernel: 2.6.21.1 (also happens with the default 2.6.18 from Debian Etch)
> >OpenVPN: 2.0.9 (on both sides of the tunnel)
> >
> > 
> >
> -11 means "EAGAIN" (sometimes due to network stack not being able to 
> send data - e.g. no memory
> available)
> 
> -112 means "EHOSTDOWN" (tcp/ip can not connect to the server)
> 
> Since it worked with directio (which disables the mm layer caching on 
> the client) that may be an indication that
> the vpn is running short of memory.   writing to the server is where 
> memory allocation problems under
> cifs could come into play (since cifs may be freeing memory when the 
> underlying network stack in your
> case may be allocating memory).   One thought ... maybe the network 
> stack can not handle
> writing more than 1 page efficiently (cifs sends a list of 14 pages 
> typically on write over the network - unless signing is enabled in
> which case it is smaller, 16K).   Have you tried changing the wsize 
> (e.g. setting it to 4096) and see if that
> eliminates the problem.

I tried setting wsize to 4096. It did not eliminate the problem.

-- 
Stanislav