[Samba] memory consumption with treesize pro and cifs shares
samba at tlinx.org
Mon Jul 29 18:39:30 MDT 2013
(some more followup---sorry if I ask too much / too many Q's,
if so, just don't respond! I won't be offended)..
You might look for a file system loop and check for options in treesize pro
to detect such.
Another program to try is "WinDirStat's home is
The reason I mention the loop stuff is that windir stat has options to
remote mounts and remote symbolic links and to follow them or not.
That can cause it to go in loops but not exhaust memory on th server.
Running it now in normal mode, it runs more aggressively against the
I see the smbd process at 90% cpu usage (the fact that the protocol is
makes it difficult to parallelize cpu usage, so 90% is how much of 1 core
it is using, vs. system wide it would be about 6.3%).
Note -- my instance of 3.6.12 is running with millions of files as well
(a bit over
9 million at last count) and is running on linux-3.9.8. I'm not sure,
but I think
linux's multi-tasking ability is considered more efficient than BSD's,
though BSD is
has had some record of better security -- though to both those figures
the relative user
bases need to be considered (fewer users, fewer bugs found, more users,
for different types of HW and efficient algorithms to handle high loads
platforms, as well as the ability to keep source closed under BSD
(security through obscurity)).
Of note -- on the server, I see two instances of smbd running -- the
other is from
another machine where I have a logon instance in windows that I left
(disconnected from a remote session, so not really suspended -- looks
like explorer doing
some sort of indexing (which MS refuses to allow their indexer to use to
update a computer's
Anyway, the two of them are running right around 129-137MB Virtual size
with 24MB resident and 20M Shared...
Server's cpu's are Xeon X5660 @2.8Gh currently running at 1600MHz
scheduler), so they aren't the fastest or the slowest. Main disk
subsystem is a RAID50
of reasonable speed. I would think it unlikely, but perhaps a slow disk
a backlog of requests... but I don't see that as likely.
Evidence points to the BSD-samba combination you are using. :(.
You didn't mention --
* what processor/how many cores the NAS is using/has available?
* What type of disk are in use (Sata/SAS 4000RPM - 15K RPM);
* Is the system using RAID? Type?
* What file system is it using (options?)
Personally, in my limited exploration of home NAS units, I didn't
find any that were "well powered"; not as even as much as a
low-end workstation based server. With
48G, your's already sounds better than most, but that's only 1 measure.
Also note, my version of treesizepro isn't the latest, it's officially
so next to the version 18.104.22.1687)....;-). Newer versions may be more
or have different options.
Cy Mike wrote:
> More info on this:
> The NAS running FreeBSD has 48GB RAM, same as the test NAS we are
> duplicating the error on. Both machines see this error with 3.6.9
> Samba. The initial try at duplicating the error didn't produce it. It
> wasn't until we increased the amount of files in the CIFS share that
> we were able to duplicate it. Number of files is in the millions.
> Drive freespace is large on the test machine and the error still
> occurs. According to LindaW here, the test hasn't been reproduce yet
> using Samba 3.6.16, so we're looking into another test on our box
> using the updated version.
> Has anyone else encountered an issue like with using TreeSize Pro?
> Does anyone need more information to help sort this out? We'll be
> running additional tests today looking for a solution and I will post
> back more on this later.�
> On Tue, Jul 23, 2013 at 6:59 PM, Cy Mike <cymike at gmail.com
> <mailto:cymike at gmail.com>> wrote:
> Hi everyone. I'm looking to solve an issue with Samba on a NAS
> being accessed with TreeSize Pro. Using that program to scan
> through millions of files is eating up memory on swap and
> eventually crashing the system. It's scanning mounted CIFS shares
> on the NAS running TrueNAS with samba version 3.6.9
> We have a test case and have been able to replicate the issue on
> another machine.
> The "solution" right now is to simply not run TreeSize Pro. Not
> the best of plans.
> In the meantime, I'm going to continue to check the usual
> manuals/google sources to see if I can find anything. I haven't as
> yet and am short on time with this. Basically looking to see if
> this is an actual bug that might require a patch/upgrade, or
> something I can fix with some tuneables.�
> "that's not a bald head, that's a solar panel for a dumbass machine" -
> jon stewart 5/9/12
More information about the samba