[Samba] Samba in Containers/Kubernetes Status Update 1

Thu Nov 4 14:41:38 UTC 2021

On 11/4/21 07:17, Robert Marcano via samba wrote:
> On 10/18/21 2:57 PM, John Mulligan via samba wrote:
>> Samba in Containers & Kubernetes Status Update vol. 1
>> ======================================================
>>
>> As some of you may remember, I've been working on an effort to include 
>> SMB,
>> via Samba, in the container ecosystem [1] and Kubernetes. It's been a 
>> while
>> since I wrote anything about these projects. I want to give an update 
>> on our
>> work to keep others informed, and possibly get others interested in what
>> we've been doing. I hope to make this a regular thing (unless you get
>> annoyed and tell me to stop ;-) ).
>>
>> We gave a presentation on these projects at sambaXP [2] and also have 
>> had a
>> few posts on the samba mailing list about them. Since then we've been 
>> doing
>> a lot of varied work and below is a small slice of it.
> 
> Thanks for working on this, as someone that deploy Samba AD exclusively 
> in containers I appreciate it. I will take a look at the presentation.
> 
> Up until know I have needed to run the containers as privileged ones and 
> the work about ACL Xattr is a step forward. Another thing I need to do 
> is to run the container using host networking, not ideal, but it has 
> been needed because for some reason Samba registered the internal 
> container IP address on DNS

I mean, it's just registering the IP address it knows about, right? I 
previously built some LXD containers using the default LXD bridge, and, 
while secure, I find that to be an endless pain to deal with. My go to 
strategy now is to bind one system NIC to a virtual bridge with an IP 
address and then attach the containers to the bridge for networking, 
each with their own network IP address. This way the containers are 
indistinguishable from bare metal servers; I can easily ssh into them, etc..

Regarding running in unprivileged containers, the Samba DC is a perfect 
candidate for this since you're not mounting filesystems, so don't need 
to deal with id mappings, shiftfs, or anything like this.  I'd love to 
know the technical reason why this doesn't work

, causing problems with clients that
> obviously can´t reach that IP. I will test with teh latest releases 
> again to see if that isn't happening anymore, maybe something to look at.
> 
>>
>> CTDB
>> -----
>>
>> I took on the task of trying to "containerize" CTDB-enabled Samba over 
>> this
>> summer. With assistance of the samba-technical mailing list we 
>> successfully
>> started running CTDB in a containerized environment. I proceeded to 
>> automate
>> parts of the CTDB configuration in our sambacc library [3]. Finally, we
>> created new example YAML files in samba-containers [4] that demonstrate a
>> clustered instance of Samba running under Kubernetes orchestration.
>>
>> There were a lot of challenges getting CTDB working automatically in a
>> container and there's still a lot to do. For example, the code I wrote 
>> to help
>> manage the CTDB nodes file isn't as robust as I'd like it to be and is 
>> still
>> generally immature. We'd like to discuss ways we might update CTDB to 
>> make the
>> nodes easier to manage through automation (in our environment) and 
>> hope such
>> discussions could improve things for other use cases as well.
>>
>> At the moment, we've been using a Kubernetes based mechanism [5] for 
>> getting
>> clients running outside of the cluster access to the shares. Without 
>> plunging
>> deep into the particulars of Kubernetes networking; the method works well
>> enough but does not use the CTDB public address mechanism. As such, we 
>> feel the
>> failover behavior we have right now may not be as nice at getting 
>> clients to
>> quickly fail-over when compared to CTDB's tickle-ack. This is an area 
>> we're
>> going to be actively investigating and would like to hear additional 
>> feedback
>> on this topic.
>>
>> We're also adding support for the CTDB enabled Samba instances to the
>> Kubernetes operator code. The plan is to continue to use the single SMB
>> instance by default but add an experimental mode for running a 
>> StatefulSet of
>> Pods that include CTDB for clustering across multiple nodes.
>>
>> We are also discussing the need for using the fileid vfs module on top 
>> the
>> clustered file system we intend to use (cephfs). However, we don't 
>> want to
>> require a PVC to be any particular file system and so we have been 
>> discussing
>> ways to enable the fileid module, and how to configure it, only in 
>> cases that
>> we need it in an easy-to-use manner.
>>
>> ACL Xattr
>> ----------
>>
>> We are very interested in being able to use NTACLs on the shares that 
>> we host
>> in containerized Samba servers. Today, the acl_xattr module stores the 
>> NTACL
>> metadata in an xattr named "security.NTACL". The "security" namespaces 
>> on Linux
>> requires the use of the CAP_SYS_ADMIN capability - basically the "root 
>> user" of
>> Linux capabilities. This would require running the containers with 
>> escalated
>> privileges; something we would prefer not to do. So, Günther Deschner, 
>> with
>> feedback from Ralph Böhme and Michael Adam and others, has been 
>> working on the
>> patches in MR#1908 [6].
>>
>> With these changes in place, we'd be able to configure Samba with a 
>> custom
>> xattr for our containerized Samba instances and thus avoid needing to 
>> run a
>> privileged container while still supporting NTACLs.
>>
>> Nightly Builds
>> ---------------
>>
>> We've been looking in an easy way to try out the latest code in Samba 
>> and are
>> planning on adding support for building container images based on nightly
>> package builds. Anoop C S has already added support to the sambacc 
>> layer in
>> order to support the newer CLI options present in Samba 4.15. Next we 
>> plan to
>> add new build rules to the samba-containers project that will create 
>> new build
>> tags in our container registry repo(s). Soon we should be able to more 
>> easily
>> test and experiment with the latest versions Samba has to offer!
>>
>> Metrics
>> --------
>>
>> In the slightly longer term, we would like to add metrics support to our
>> pods running in Kubernetes. Our plan is to follow the de-facto standard
>> in this space and export Prometheus metrics. In order to do this we need
>> to get the data out of our Samba stack and rather than try and directly
>> scrape the output of a tool like smbstatus we are very interested in the
>> effort to add JSON output support to samba tools. In our case, we want 
>> JSON
>> output from smbstatus. As such we're very interested in MR#1269 [7].
>> We'll probably be getting more involved here in the near future.
>>
>> Offline Domain Join
>> --------------------
>>
>> One new feature that landed in Samba 4.15 is the new Offline Domain 
>> Join (ODJ)
>> support. Currently, to connect our containers with Active Directory we're
>> asking users to store a username/password in a Kubernetes secret [8]. 
>> We are
>> aware that storing a password (even within a secret) isn't the best thing
>> to do, so we're looking forward to trying to integrate the ODJ feature
>> into our projects so that users never have to store a password.
>> Watch this space! :-)
>>
>> Wrap Up
>> --------
>>
>> If you have any questions or comments feel free to ask! I'm hoping this
>> update both serves to inform you of what we're up to as well as 
>> serving as
>> a prompt for you to give us feedback.
>>
>> Thanks for reading!
>>
>> [1] - Specifically OCI/Docker style containers.
>>
>> [2] - 
>> https://sambaxp.org/fileadmin/user_upload/sambaxp2021-slides/Mulligan_Adam_samba_operator.pdf 
>>
>>
>> [3] - https://github.com/samba-in-kubernetes/sambacc/
>>
>> [4] - https://github.com/samba-in-kubernetes/samba-container/
>>
>> [5] - 
>> https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/ 
>>
>>
>> [6] - https://gitlab.com/samba-team/samba/-/merge_requests/1908
>>
>> [7] - https://gitlab.com/samba-team/samba/-/merge_requests/1269
>>
>> [8] - Not strictly true, as you can leave out the secret and then run
>>   a command in the container to complete the join. However, this
>>   runs counter to the level of automation we'd like to provide, but
>>   is an option.
>>
>> -- John M.
>>
>>
>>
>>
>>
> 
> 