[Samba] memory consumption with treesize pro and cifs shares

[Linda W]

(some more followup---sorry if I ask too much / too many Q's,
if so, just don't respond! I won't be offended)..

You might look for a file system loop and check for options in treesize pro
to detect such.

Another program to try is "WinDirStat's home is 
http://windirstat.sourceforge.net/.

The reason I mention the loop stuff is that windir stat has options to 
detect
remote mounts and remote symbolic links and to follow them or not.

That can cause it to go in loops but not exhaust memory on th server.

Running it now in normal mode, it runs more aggressively against the 
server and
I see the smbd process at 90% cpu usage  (the fact that the protocol is 
single-server/client
makes it difficult to parallelize cpu usage, so 90% is how much of 1 core
it is using, vs. system wide it would be about 6.3%).

Note -- my instance of 3.6.12 is running with millions of files as well 
(a bit over
9 million at last count) and is running on linux-3.9.8.  I'm not sure, 
but I think
linux's multi-tasking ability is considered more efficient than BSD's, 
though BSD is
has had some record of better security -- though to both those figures 
the relative user
bases need to be considered (fewer users, fewer bugs found, more users, 
more need
for different types of HW and efficient algorithms to handle high loads 
across diverse
platforms, as well as the ability to keep source closed under BSD 
(security through obscurity)).


Of note -- on the server, I see two instances of smbd running -- the 
other is from
another machine where I have a logon instance in windows that I left 
"suspended"
(disconnected from a remote session, so not really suspended -- looks 
like explorer doing
some sort of indexing (which MS refuses to allow their indexer to use to 
update a computer's
local index)).

Anyway, the two of them are running right around 129-137MB Virtual size 
for each
with 24MB resident and 20M Shared...
Server's cpu's are Xeon X5660 @2.8Gh currently running at 1600MHz 
(demand based
scheduler), so they aren't the fastest or the slowest.  Main disk 
subsystem is a RAID50
of reasonable speed.  I would think it unlikely, but perhaps a slow disk 
might cause
a backlog of requests... but I don't see that as likely.

Evidence points to the BSD-samba combination you are using.  :(.
You didn't mention --
* what processor/how many cores the NAS is using/has available?
* What type of disk are in use (Sata/SAS 4000RPM - 15K RPM);
* Is the system using RAID? Type?
* What file system is it using (options?)

Personally, in my limited exploration of home NAS units, I didn't
find any that were "well powered"; not as even as much as a
low-end workstation based server.  With
48G, your's already sounds better than most, but that's only 1 measure.

Also note, my version of treesizepro isn't the latest, it's officially 
'OUTDATED' (says
so next to the version 5.4.4.707)....;-).  Newer versions may be more 
aggressive
or have different options.



Cy Mike wrote:
> More info on this:
>
> The NAS running FreeBSD has 48GB RAM, same as the test NAS we are 
> duplicating the error on. Both machines see this error with 3.6.9 
> Samba. The initial try at duplicating the error didn't produce it. It 
> wasn't until we increased the amount of files in the CIFS share that 
> we were able to duplicate it. Number of files is in the millions. 
> Drive freespace is large on the test machine and the error still 
> occurs. According to LindaW here, the test hasn't been reproduce yet 
> using Samba 3.6.16, so we're looking into another test on our box 
> using the updated version.
>
> Has anyone else encountered an issue like with using TreeSize Pro? 
> Does anyone need more information to help sort this out? We'll be 
> running additional tests today looking for a solution and I will post 
> back more on this later.�
>
> Thanks,
> Mike�
>
>
> On Tue, Jul 23, 2013 at 6:59 PM, Cy Mike <cymike at gmail.com 
> <mailto:cymike at gmail.com>> wrote:
>
>     Hi everyone. I'm looking to solve an issue with Samba on a NAS
>     being accessed with TreeSize Pro. Using that program to scan
>     through millions of files is eating up memory on swap and
>     eventually crashing the system. It's scanning mounted CIFS shares
>     on the NAS running TrueNAS with samba version 3.6.9
>
>     We have a test case and have been able to replicate the issue on
>     another machine.
>
>     The "solution" right now is to simply not run TreeSize Pro. Not
>     the best of plans.
>
>     In the meantime, I'm going to continue to check the usual
>     manuals/google sources to see if I can find anything. I haven't as
>     yet and am short on time with this. Basically looking to see if
>     this is an actual bug that might require a patch/upgrade, or
>     something I can fix with some tuneables.�
>
>     Thanks,
>     Mike
>
>
>
>
> -- 
> "that's not a bald head, that's a solar panel for a dumbass machine" - 
> jon stewart 5/9/12