Implementing SAMBA w/ Automounter (SAMBA digest 1429)

[Keith Farrar]

Well, creating one share for automount may cause your Samba server to
thrash through NFS mounts and unmounts every time a client PC file manager
or explorer does a directory listing update.

Here are the rough sizes of some automount directories (and one large
directory) I needed to share:
	auto.tilde	= 2200 entries	(home directories)
	auto.project	= 800 entries	(projects and misc mount points)
	/import		= 580 subdirs	(shared Unix binaries in AFS)

We once created a single directory with symlinks to 1000+ home directory
mount points (spread across ~ 40 servers).

There also was a share in use named 'root', which mounted '/'. This caused
the servers to beat themselves into the ground each time a PC user navigated
their way into a shared binaries tree in AFS. 

The Samba servers would run amok (90%+ CPU, starting one client connection)
when retrieving stats and name mangling any of these shares.

With Windows for Workgroups clients, a PC windows session would appear to
lockup while the File Manager was generating file listings.
An NT machine would (only) wedge the File Manager or Explorer process.
Either client would return to normal function between refreshing the display.

Translation: mounts and logins took several painful minutes to complete.
And things would get interesting when someone wandered into /net or /afs.

So, I hacked together a perl script and process to work around this problem.

The script builds a tree which contains symlinks to each directory in a
particular map.

It builds a similar tree from the directory entries of a repository for our
shared Unix binaries (named /import).

The trees are built (in AFS) under the path /afs/parc/forest/<mountpoint>/.
Each update throws away the old trees, builds new trees, then releases AFS
read-only replicas with the new trees.

I used a read-only AFS volume so the trees would cache the trees on local
disk and share one AFS call-back in the client cache for update notification.

To cut down on the number of remote filesystems queries forwarded for each
SMB mount, the links are stored in a directory hash.

Each tree holds 36 subdirectories for the hash. Each subdirectory name is
the first character of each symlink name, converted to lower-case. I needed
to use case-insensitive mapping, or Workgroups clients would have been
unable to use this hack.

On an administrative workstation, a cron job rebuilds the trees each hour
by running the perl script. Note that this requires cron to hold an AFS admin
token, or for some mechanism for delegation of admin authority for the
"vos release" command.

DETAILS...

The smb.conf for each Samba server shares these trees with the following
share definitions (using an include directive).

######################################################################
; NB:   Top-level mount-point shares. These are used
; NB:   to navigate using the automounter.
 
[root]
        comment = Top-level view of the world
        path = /afs/parc/forest
        public = no
        create mask = 0775
        writable = yes
        printable = no
        locking = yes
[project]
        comment = Project Directories
        path = /afs/parc/forest/project
        public = no
        create mask = 0775
        writable = yes
        printable = no
        locking = yes
[org]
        comment = PARC Organizational Directories
        path = /afs/parc/forest/org
        public = no
        create mask = 0775
        writable = yes
        printable = no
        locking = yes
[tilde]
        comment = PARC Home Directories
        path = /afs/parc/forest/tilde
        public = no
        create mask = 0775
        writable = yes
        printable = no
        locking = yes
[import]
        comment = PARC AFS packages
        path = /afs/parc/forest/import
        public = no
        create mask = 0775
        writable = no
        printable = no
        locking = yes
[import_rw]
        comment =Writable PARC AFS package volumess
        path = /afs/parc/forest/importw
        public = no
        writable = yes
        printable = no
        create mask = 0775
        locking = yes

######################################################################


Cron entry:
######################################################################
## Job to help in Samba Server global link updates..
##
17 * * * * /import/samba/bin/mk-linksdirs.pl
######################################################################



Script mk-linksdirs.pl is below:

######################################################################

#!/usr/bin/perl
# 
# @(#) /import/samba/bin/mk-linksdirs.pl
# @(#) A perl script to build a tree of symbolic links to automounted
# @(#) directory hierarchies at PARC. 
# @(#)	Keith Farrar <farrar at parc.xerox.com>
# 
#	This is an egregious hack, but it works... 8-)
#	Now prunes empty link dirs (only called for auto.org map)

$debug=0; # set to non-zero for voluminous output...

#$linksroot="/tmp/projlinks";
$linksroot="/afs/.parc.xerox.com/forest";

$orglinks="$linksroot/org";
$projlinks="$linksroot/project";
$implinks="$linksroot/import";
$iwlinks="$linksroot/importw";
$tildelinks="$linksroot/tilde";

umask 002;

foreach $tree ( $orglinks, $projlinks, $implinks, $iwlinks, $tildelinks ) {
	&mktop( $tree );
	chdir $tree || die "chdir to linksroot failed";;
}
foreach $tree ( $orglinks, $projlinks, $implinks, $iwlinks, $tildelinks ) {
	&wipeoldtree( $tree );
}
foreach $tree ( $orglinks, $projlinks, $implinks, $iwlinks, $tildelinks ) {
	&mknewtree( $tree );
}
foreach $tree ( $orglinks, $projlinks, $implinks, $iwlinks, $tildelinks ) {
	&mknewlinks( $tree );
}
#foreach $tree ( $orglinks, $projlinks, $implinks, $iwlinks, $tildelinks ) {
foreach $tree ( $orglinks ) {
	&pruneempties( $tree );
}

#system("/afs/parc/import/afs/@sys/etc/vos release -f -id parc.forest -verbose");
system("/import/import-support-1.0/bin/releasevol parc.forest");
exit 0;

sub mktop {
	local($thistree);
	$thistree = shift(@_);
	if ( ! -d "$thistree" ) {
		system("/bin/mkdir -p $thistree");
	} else {
		system("/bin/rm -f $thistree/*");
	}
	if ( ! -d "$thistree" ) {
		die "Unable to create $thistree";
	}
}

sub mknewlinks {
	local($thistree);
	$thistree = shift(@_);
	if ( $thistree eq $orglinks ) {
		open(DIRSLIST, "ypcat -k auto.org|");
		@dirslist = <DIRSLIST>;
		close DIRSLIST;
		foreach $_ ( @dirslist ) {
			($key, $junk) = split;
			($subdir = substr($key,0,1) ) =~ tr/A-Z/a-z/;
			$newlink = "$thistree/$subdir/$key";
			$debug && print " Link to make == <$newlink> \n";
			symlink("/org/$key", $newlink);
		}
	} elsif ( $thistree eq $projlinks ) {
		open(DIRSLIST, "ypcat -k auto.project|");
		@dirslist = <DIRSLIST>;
		close DIRSLIST;
		foreach $_ ( @dirslist ) {
			($key, $junk) = split;
			($subdir = substr($key,0,1) ) =~ tr/A-Z/a-z/;
			$newlink = "$thistree/$subdir/$key";
			$debug && print " Link to make == <$newlink> \n";
			symlink("/project/$key", $newlink);
		}
	} elsif ( $thistree eq $tildelinks ) {
		open(DIRSLIST, "ypcat -k auto.tilde|");
		@dirslist = <DIRSLIST>;
		close DIRSLIST;
		foreach $_ ( @dirslist ) {
			($key, $junk) = split;
			($subdir = substr($key,0,1) ) =~ tr/A-Z/a-z/;
			$newlink = "$thistree/$subdir/$key";
			$debug && print " Link to make == <$newlink> \n";
			symlink("/tilde/$key", $newlink);
		}
	} elsif ( $thistree eq $implinks ) {
		opendir(DIRSLIST, "/afs/parc.xerox.com/import");
		readdir DIRSLIST;
		readdir DIRSLIST;
		@dirslist = readdir DIRSLIST;
		close DIRSLIST;
		foreach $_ ( @dirslist ) {
			($key, $junk) = split;
			($subdir = substr($key,0,1) ) =~ tr/A-Z/a-z/;
			$newlink = "$thistree/$subdir/$key";
			$debug && print " Link to make == <$newlink> \n";
			symlink("/afs/parc.xerox.com/import/$key", $newlink);
		}
	} elsif ( $thistree eq $iwlinks ) {
		opendir(DIRSLIST, "/afs/.parc.xerox.com/import");
		readdir DIRSLIST;
		readdir DIRSLIST;
		@dirslist = readdir DIRSLIST;
		close DIRSLIST;
		foreach $_ ( @dirslist ) {
			($key, $junk) = split;
			($subdir = substr($key,0,1) ) =~ tr/A-Z/a-z/;
			$newlink = "$thistree/$subdir/$key";
			$debug && print " Link to make == <$newlink> \n";
			symlink("/afs/.parc.xerox.com/import/$key", $newlink);
		}
	} else {
		return 0;
	}
}

sub wipeoldtree {
	local($thistree);
	#$thistree = shift(@_);
	$thistree = pop(@_);
	$debug && print STDERR "Removing tree <$thistree>\n";
	for $ltr ( "0".."9", "A".."Z", "a".."z" ) {
	    if ( -d "$thistree/$ltr" ) {
		$debug && print "Removing old links in subdir $thistree/$ltr\n";
		system("/bin/rm -rf $thistree/$ltr");
	    }
	}
}

sub mknewtree {
	local($thistree);
	$thistree = shift(@_);
	$debug && print STDERR "Building tree <$thistree>\n";
	for $ltr ( "0".."9", "a".."z" ) {
	    if ( ! -d "$thistree/$ltr" ) {
		$debug && print "Building new links subdir $thistree/$ltr\n";
		mkdir("$thistree/$ltr",0755);
	    }
	}
}


# remove the top level dirs if there are no subdirs
sub pruneempties{
	local($thistree, @dircontents, $linkcount);
	$thistree = shift(@_);
	$debug && print STDERR "Pruning tree <$thistree>\n";
	for $ltr ( "0".."9", "a".."z" ) {
	    $debug && print "Pruning new links subdir $thistree/$ltr\n";
	    if ( -d "$thistree/$ltr" ) {
		opendir(SUBDIR, "$thistree/$ltr");
		@dircontents = readdir(SUBDIR);
		closedir(SUBDIR);
		$linkcount = @dircontents;
		$debug && print "Found $linkcount links in $thistree/$ltr\n";
		$debug && print "Content list:\n at dircontents\n";
		$linkcount < 3 && rmdir("$thistree/$ltr");
	    }
	}
}

######################################################################


--
| Keith Farrar | Xerox PARC CSNS  | Palo Alto, CA | (650) 812-4292	|
| DOMAIN: farrar at parc.xerox.com   | "That which does not kill you gives	|
|				  |  you interesting scars."  - Kyle B.	|