[Samba] Can't restore from GPFS snapshots, disk_free error

[Eric Chris Garrison]

Thank you for the suggestions and advice, Jonathan.  We will try out the
snapshot options you mention.

How onerous are snapshots on a prod system? I think we're going to do just
once a day, in the wee hours when not much is going on. What kind of
problems do you see? How badly does it hang, for how long? What sorts of
things aggravate that?

Thanks again,

Chris


On 10/17/13 11:46 AM, "Jonathan Buzzard" <jonathan at buzzard.me.uk> wrote:

>On Thu, 2013-10-17 at 09:16 -0400, Eric Chris Garrison wrote:
>> Hello,
>> 
>> We're trying to set up a GPFS system with Samba running on top with CTDB
>> managing it.
>> 
>> I have snapshots set up to be accessible in every directory as the
>>invisible
>> directory .snap
>> 
>> The snapshots are in the following format:
>> 
>> /usr/lpp/mmfs/bin/mmcrsnapshot 1MB `TZ=GMT date + at GMT-%Y.%m.%d-%H.%M.%S`
>> 
>> ?and look like this from the UNIX level:
>> 
>> ~ecgarris/RSFS/.snap/@GMT-2013.10.16-20.00.01/thing1
>> 
>> I've set up shares like this:
>> 
>> [homes] 
>> path = %H/RSFS
>>         comment = RSFS Home Directories
>>         browseable = No
>>         shadow:snapdir = .snap
>> #        shadow:basedir = %H/RSFS
>>         shadow:fixinodes = yes
>>         shadow:snapdirseverywhere=yes
>> 
>
>[SNIP]
>
>> I'd appreciate any help. It feels like we're really close. Thanks.
>> 
>
>I am assuming that you are loading the shadow_copy2 and gpfs VFS
>modules. Assuming that you don't have independent filesets as well then
>scrap the snapshot directories all over the place and in the general
>configuration do
>
># enable shadow copies
>	shadow : snapdir = /gpfs/.snapshots
>	shadow : basedir = /gpfs
>	shadow : fixinodes = yes
>
>Works for certain with GPFS 3.4.x and Samba 3.5.x and 3.6.x
>
>Now the bad news, give it up anyway as snapshots are unworkable on a
>GPFS files system that is in production. The following commands in my
>experience should only be run under very light load (aka maintenance
>window) on a production file system mmcrsnapshot, mmdelsnapshot and
>mmunlinkfileset.
>
>I have a wonderful Perl script that can be called from all nodes in the
>file system able to run admin commands; will create snapshots as
>required, will remove old snapshots as required, complete with full
>locking (the script is called from crontab on multiple nodes so a node
>down for maintenance does not cause you to loose snapshots) and it all
>works beautifully in test. Runs for months without a problem.
>
>Put it on a production system, and within days if not hours you will get
>a total freeze on the file system when it blocks trying to get a
>quiescent lock to either take or remove a snapshot.
>
>If you wish I can send you a copy of my Perl script, my advice is forget
>it as ahead only pain and anguish :-(
>
>
>JAB.
>
>-- 
>Jonathan A. Buzzard                 Email: jonathan (at) buzzard.me.uk
>Fife, United Kingdom.
>