[Samba] smbd reads entire directory when creating files?

Tue Dec 7 20:08:56 GMT 2004

Apologies if this is a FAQ.

We are saddled with some directories full of production image files that 
number in the hundreds of thousands to the millions.  (Yes, I know this 
is bad.)  They reside on a couple of Win 2003 boxes that are flaky and 
unreliable.  I was hoping to copy them over to a Linux box running samba 
but it is taking forever.

A strace on the smbd process receiving the files from the windows box 
(it is mapped as a share on the 2k3 server) reveals that smbd is looking 
up the entire directory (with getdents64) every time it writes a file. 
Several times, in fact.  So as the number of files grows, it churns more 
and more.  I'm sure most of it is in cache but the data still has to be 
moved around in memory.

I know this is not necessary for writing/copying files with unix 
semantics, but I wonder if the case-fiddling or any other Windows 
imitative behavior is making this getdents64 orgy necessary.  Is there 
any way to disable it?

I was going to post a bit of the strace output but it is extremely long 
and (to my eyes, anyway) not very interesting.  I can supply it to 
anyone who is interested in seeing it.

Any suggestions appreciated.  I also note that while using smbclient 
instead of mapping the drive from the Windows box might help with this 
copy, it will not help if we were to try to use this setup to replace 
the 2k3 servers (the eventual goal).  So any pointers in that direction, 
while perhaps interesting for comparison, are not especially relevant.

greenville:~# smbd -V
Version 3.0.7-Debian

smb.conf is attached.  Not much interesting in there, it is pretty much 
stock.  "vfs_ownerwrite" is a small VFS module I hacked up to do an 
extra chown on a file that is written to the share (used in a common 
PDC/Unix environment on some shared directories.)  I am using the 
"deadwood_ftp" share in this case, so the module is not involved.

-m
-------------- next part --------------
# Samba configuration file for media servers.
# -----
#
# $Id: smb.conf,v 1.2.4.6 2002/03/13 18:56:16 peloy Exp $
#
# This is the main Samba configuration file. You should read the
# smb.conf(5) manual page in order to understand the options listed
# here. Samba has a huge number of configurable options most of which 
# are not shown in this example
#
# Any line which starts with a ; (semi-colon) or a # (hash) 
# is a comment and is ignored. In this example we will use a #
# for commentary and a ; for parts of the config file that you
# may wish to enable
#
# NOTE: Whenever you modify this file you should run the command
# "testparm" to check that you have not many any basic syntactic 
# errors. 
#

#======================= Global Settings =======================

[global]

# can this be turned back on?
use sendfile = no 

# Do something sensible when Samba crashes: mail the admin a backtrace
   panic action = /usr/share/samba/panic-action %d

# Change this for the workgroup/NT-domain name your Samba server will part of
   workgroup = VP

# server string is the equivalent of the NT Description field
   server string = %h server (Samba %v)

# If you want to automatically load your printer list rather
# than setting them up individually then you'll need this
;   load printers = yes

# You may wish to override the location of the printcap file
;   printcap name = /etc/printcap

# 'printing = cups' works nicely
;   printing = bsd

;   guest account = nobody
   invalid users = root

# This tells Samba to use a separate log file for each machine
# that connects
   log file = /var/log/samba/log.%m

# Put a capping on the size of the log files (in Kb).
   max log size = 1000

# If you want Samba to log though syslog only then set the following
# parameter to 'yes'. Please note that logging through syslog in
# Samba is still experimental.
;   syslog only = no

# We want Samba to log a minimum amount of information to syslog. Everything
# should go to /var/log/samba/log.{smb,nmb} instead. If you want to log
# through syslog you should set the following parameter to something higher.
   syslog = 0

# "security = user" is always a good idea. This will require a Unix account
# in this server for every user accessing the server. See
# security_level.txt for details.
   security = domain

# You may wish to use password encryption. Please read ENCRYPTION.txt,
# Win95.txt and WinNT.txt in the Samba documentation. Do not enable this
# option unless you have read those documents
   encrypt passwords = true
   passdb backend = tdbsam guest

# Using the following line enables you to customise your configuration
# on a per machine basis. The %m gets replaced with the netbios name
# of the machine that is connecting
;   include = /home/samba/etc/smb.conf.%m

# Most people will find that this option gives better performance.
# See speed.txt and the manual pages for details
# You may want to add the following on a Linux system:
#         SO_RCVBUF=8192 SO_SNDBUF=8192
   socket options = TCP_NODELAY

# --- Browser Control Options ---

# Please _read_ BROWSING.txt and set the next four parameters according
# to your network setup. The defaults are specified below (commented
# out.) It's important that you read BROWSING.txt so you don't break
# browsing in your network!

# set local master to no if you don't want Samba to become a master
# browser on your network. Otherwise the normal election rules apply
;   local master = yes

# OS Level determines the precedence of this server in master browser
# elections. The default value should be reasonable
;   os level = 20

# Domain Master specifies Samba to be the Domain Master Browser. This
# allows Samba to collate browse lists between subnets. Don't use this
# if you already have a Windows NT domain controller doing this job
;   domain master = auto

# Preferred Master causes Samba to force a local browser election on startup
# and gives it a slightly higher chance of winning the election
;   preferred master = auto

# --- End of Browser Control Options ---

# Windows Internet Name Serving Support Section:
# WINS Support - Tells the NMBD component of Samba to enable it's WINS Server
;   wins support = no

# WINS Server - Tells the NMBD components of Samba to be a WINS Client
# Note: Samba can be either a WINS Server, or a WINS Client, but NOT both
   wins server = 10.132.4.21

# This will prevent nmbd to search for NetBIOS names through DNS.
   dns proxy = no

# What naming service and in what order should we use to resolve host names
# to IP addresses
;   name resolve order = lmhosts host wins bcast

# Name mangling options
;   preserve case = yes
;   short preserve case = yes

# This boolean parameter controlls whether Samba attempts to sync. the Unix
# password with the SMB password when the encrypted SMB password in the
# /etc/samba/smbpasswd file is changed.
;   unix password sync = false

# For Unix password sync. to work on a Debian GNU/Linux system, the following
# parameters must be set (thanks to Augustin Luton <aluton at hybrigenics.fr> for
# sending the correct chat script for the passwd program in Debian Potato).
   passwd program = /usr/bin/passwd %u
   passwd chat = *Enter\snew\sUNIX\spassword:* %n\n *Retype\snew\sUNIX\spassword:* %n\n .

# This boolean controls whether PAM will be used for password changes
# when requested by an SMB client instead of the program listed in
# 'passwd program'. The default is 'no'.
;   pam password change = no

# The following parameter is useful only if you have the linpopup package
# installed. The samba maintainer and the linpopup maintainer are
# working to ease installation and configuration of linpopup and samba.
;   message command = /bin/sh -c '/usr/bin/linpopup "%f" "%m" %s; rm %s' &

   obey pam restrictions = yes

# Some defaults for winbind (make sure you're not using the ranges
# for something else.)
;   winbind uid = 10000-20000
;   winbind gid = 10000-20000
;   template shell = /bin/bash

#======================= Share Definitions =======================

# The next two parameters show how to auto-mount a CD-ROM when the
#	cdrom share is accesed. For this to work /etc/fstab must contain
#	an entry like this:
#
#       /dev/scd0   /cdrom  iso9660 defaults,noauto,ro,user   0 0
#
# The CD-ROM gets unmounted automatically after the connection to the
#
# If you don't want to use auto-mounting/unmounting make sure the CD
#	is mounted on /cdrom
#
;   preexec = /bin/mount /cdrom
;   postexec = /bin/umount /cdrom

[teaneck_m]
	comment = TEANECK media root
	path = /mnt/ha/teaneck_media
	public = yes
	writable = yes
	vfs objects = vfs_ownerwrite
	create mode = 0664
	directory mode = 2775

[teaneck_ftp]
	comment = TEANECK FTP root
	path = /mnt/ha/teaneck_media/ftp
	public = yes
	writable = yes
	vfs objects = vfs_ownerwrite
	create mode = 0664
	directory mode = 2775

[deadwood_e]
	comment = Old DEADWOOD media root
	path = /mnt/ha/teaneck_media
	public = yes
	writable = yes
	#vfs objects = vfs_ownerwrite
	create mode = 0664
	directory mode = 2775

[deadwood_ftp]
	comment = Old DEADWOOD FTP root
	path = /mnt/ha/teaneck_media/ftp
	public = yes
	writable = yes
	#vfs objects = vfs_ownerwrite
	create mode = 0664
	directory mode = 2775