[Samba] shadow_copy and glusterfs not working

Wed Jan 4 06:47:51 UTC 2017

On Tue, 2017-01-03 at 15:16 +0100, Stefan Kania via samba wrote:
> Hello,
> 
> we are trying to configure a CTDB-Cluster with Glusterfs. We are using
> Samba 4.5 together with gluster 3.9. We set up a lvm2 thin-provisioned
> volume to use gluster-snapshots.
> Then we configured the first share without using shadow_copy2 and
> everything was working fine.
> 
> Then we added the shadow_copy2 parameters, when we did a "smbclient" we
> got the following message:
> 
> root at ng-data1:~# smbclient -m SMB3 //ctdb/gluster -k
> smb: \> ls
> NT_STATUS_NOT_FOUND listing \*
> 
> On a windows-system we got the message "Element not found"
> 
> Here is our smb.conf:
> -------------------
> [global]
>         workgroup = uni
>         netbios name = ctdb
>         security = ads
>         realm = UNI.DE
>         idmap config *: range = 10000-19999
>         idmap config uni: range = 1000000-1999999
>         idmap config uni: backend = rid
>         winbind use default domain = yes
>         winbind refresh tickets = yes
>         store dos attributes = yes
>         map acl inherit = yes
>         vfs objects = acl_xattr
>         server min protocol = SMB2
> 
> 
> [gluster]
>         comment = Daten im Cluster
>         guest ok = no
>         read only = no
>         vfs objects = acl_xattr glusterfs shadow_copy2

Assuming you have configured CTDB-GlusterFS cluster in the right way, one correction to your
smb.conf here is that glusterfs module should be always the last one in vfs objects list. See the
following snippet from man page for vfs_glusterfs(8):

. . .
This module can be combined with other modules, but it should be the last module in the vfs objects
list. Modules added to this list to the right of the glusterfs entry may not have any effect at all.
. . .

Even then we do have an issue when vfs_shadow_copy2 module is used along with vfs_glusterfs as
explained in the following thread:

https://lists.samba.org/archive/samba-technical/2016-October/116834.html

If you encounter this issue you can workaround it by setting mountpoint parameter for shadow_copy2
module to / as mentioned in the above link.

>         glusterfs:volume = gv1
>         glusterfs:logfile = /var/log/samba/gluster-gv1.log
>         glusterfs:loglevel = 10
>         gluster:volfile_server = localhost
>         kernel share modes = no
>         path = /win-share
>         shadow:snapdir = /win-share/.snaps
>         shadow:basedir = /win-share
>         shadow:sort = desc
>         shadow:snapprefix = ^S[A-Za-z0-9]*p$
>         shadow:format = _GMT-%Y.%m.%d-%H.%M.%S
> -------------
> 
> when we try to excess the share we get the following errormessage insite
> the logfile:
> --------------------
> [2017-01-03 13:55:14.757049] T [rpc-clnt.c:1659:rpc_clnt_submit]
> 0-rpc-clnt: submitted request (XID: 0x4a Program: GlusterFS 3.3,
> ProgVers: 330, Proc: 18) to rpc-transport (gv1-client-1
> )
> [2017-01-03 13:55:14.757291] T [rpc-clnt.c:687:rpc_clnt_reply_init]
> 0-gv1-client-2: received rpc message (RPC XID: 0x4b Program: GlusterFS
> 3.3, ProgVers: 330, Proc: 18) from rpc-transpo
> rt (gv1-client-2)
> [2017-01-03 13:55:14.757320] T [rpc-clnt.c:687:rpc_clnt_reply_init]
> 0-gv1-client-1: received rpc message (RPC XID: 0x4a Program: GlusterFS
> 3.3, ProgVers: 330, Proc: 18) from rpc-transpo
> rt (gv1-client-1)
> [2017-01-03 13:55:14.757347] D [MSGID: 0]
> [client-rpc-fops.c:1095:client3_3_getxattr_cbk] 0-gv1-client-2: remote
> operation failed: No data available. Path: /win-share/. (0dafa3a2-5928-4
> b05-8cb2-bd90ac84d7c4). Key: security.NTACL
> [2017-01-03 13:55:14.757355] D [MSGID: 0]
> [client-rpc-fops.c:1095:client3_3_getxattr_cbk] 0-gv1-client-1: remote
> operation failed: No data available. Path: /win-share/. (0dafa3a2-5928-4
> b05-8cb2-bd90ac84d7c4). Key: security.NTACL
> [2017-01-03 13:55:14.757378] D [MSGID: 0]
> [client-rpc-fops.c:1106:client3_3_getxattr_cbk] 0-stack-trace:
> stack-address: 0x7fd99d682310, gv1-client-1 returned -1 error: No data
> available
>  [No data available]
> [2017-01-03 13:55:14.757378] D [MSGID: 0]
> [client-rpc-fops.c:1106:client3_3_getxattr_cbk] 0-stack-trace:
> stack-address: 0x7fd99d682310, gv1-client-2 returned -1 error: No data
> available
>  [No data available]
> [2017-01-03 13:55:14.757400] D [MSGID: 0]
> [afr-inode-read.c:505:afr_getxattr_cbk] 0-stack-trace: stack-address:
> 0x7fd99d682310, gv1-replicate-0 returned -1 error: No data available [No
> data available]
> [2017-01-03 13:55:14.757409] D [MSGID: 0]
> [afr-inode-read.c:505:afr_getxattr_cbk] 0-stack-trace: stack-address:
> 0x7fd99d682310, gv1-replicate-1 returned -1 error: No data available [No
> data available]
> [2017-01-03 13:55:14.757429] D [MSGID: 0]
> [dht-common.c:3032:dht_getxattr_cbk] 0-stack-trace: stack-address:
> 0x7fd99d682310, gv1-dht returned -1 error: No data available [No data availa
> ble]
> [2017-01-03 13:55:14.757445] D [MSGID: 0]
> [defaults.c:1346:default_getxattr_cbk] 0-stack-trace: stack-address:
> 0x7fd99d682310, gv1-io-threads returned -1 error: No data available [No da
> ta available]
> [2017-01-03 13:55:14.757461] D [MSGID: 0]
> [io-stats.c:2349:io_stats_getxattr_cbk] 0-stack-trace: stack-address:
> 0x7fd99d682310, gv1 returned -1 error: No data available [No data availab
> le]
> --------------------
> When we do a "gluster snapshot create Snap1 gv1" we can create and use
> the snapshot on the commandline, so the snapshot it self is working.
> Problem seams to be the settings for shadow_copy2.
> 
> Any help will help ;-)
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 299 bytes
Desc: This is a digitally signed message part
URL: <http://lists.samba.org/pipermail/samba/attachments/20170104/358e3365/signature.sig>