[Samba] Samba in Containers/Kubernetes Status Update 1

Thu Nov 4 14:12:20 UTC 2021

On Thursday, November 4, 2021 8:17:55 AM EDT Robert Marcano via samba wrote:
> On 10/18/21 2:57 PM, John Mulligan via samba wrote:
> > Samba in Containers & Kubernetes Status Update vol. 1
> > ======================================================
> > 
> > As some of you may remember, I've been working on an effort to include
> > SMB,
> > via Samba, in the container ecosystem [1] and Kubernetes. It's been a
> > while
> > since I wrote anything about these projects. I want to give an update on
> > our work to keep others informed, and possibly get others interested in
> > what we've been doing. I hope to make this a regular thing (unless you
> > get annoyed and tell me to stop ;-) ).
> > 
> > We gave a presentation on these projects at sambaXP [2] and also have had
> > a
> > few posts on the samba mailing list about them. Since then we've been
> > doing
> > a lot of varied work and below is a small slice of it.
> 
> Thanks for working on this, as someone that deploy Samba AD exclusively
> in containers I appreciate it. I will take a look at the presentation.
> 
> Up until know I have needed to run the containers as privileged ones and
> the work about ACL Xattr is a step forward. Another thing I need to do
> is to run the container using host networking, not ideal, but it has
> been needed because for some reason Samba registered the internal
> container IP address on DNS, causing problems with clients that
> obviously can´t reach that IP. I will test with teh latest releases
> again to see if that isn't happening anymore, maybe something to look at.
> 

To be very honest, the AD DC side of things for our project is very primitive. 
So far we've only needed it as a component for testing and thus it has a "hard 
coded" configuration and I've put no effort to try to make it multi-node either.
So far the priority for us has been the file server components. That said, I'd 
love to get it into a state where the container image can be used for general 
use!

The acl xattr issue is being worked on at https://gitlab.com/samba-team/
samba/-/merge_requests/1908/  - as I mentioned later in my previous mail. 
AFAIK it continues to make progress.

The IP thing is interesting, I suspect that if the join command for AD DCs is 
similar to that of a domain member server you can try an approach I took for 
our k8s stack: disable dns auto-registration at join time (using --no-dns-
updates), then use another tool to look up the external IP of the container 
[1], then pass that as a command line argument to `net ads dns register` (my 
code for this is in our sambacc tool [2] for reference). There may be other 
similar approaches for this that could avoid the need for host networking too.

Please feel free to file issues in our repo [3] with regard to your AD use 
cases... even if we are not able to get to it right away we'd like to keep 
track of the desire to have containerized AD DCs too!  Or if you're feeling 
adventurous - patches. :-D

However you prefer to do it, it would be great to collaborate some on this 
topic.

[1] - https://github.com/samba-in-kubernetes/svcwatch
[2] - https://github.com/samba-in-kubernetes/sambacc/blob/
f6480c5861a56ef9d1ebb965aebb14732ec2690c/sambacc/commands/dns.py#L48
[3] - https://github.com/samba-in-kubernetes/samba-container

(Apologies if I sent a duplicate message - in my haste to reply I think I sent 
this from the wrong address that wasn't registered with the mailing list.)

> > CTDB
> > -----
> > 
> > I took on the task of trying to "containerize" CTDB-enabled Samba over
> > this
> > summer. With assistance of the samba-technical mailing list we
> > successfully
> > started running CTDB in a containerized environment. I proceeded to
> > automate parts of the CTDB configuration in our sambacc library [3].
> > Finally, we created new example YAML files in samba-containers [4] that
> > demonstrate a clustered instance of Samba running under Kubernetes
> > orchestration.
> > 
> > There were a lot of challenges getting CTDB working automatically in a
> > container and there's still a lot to do. For example, the code I wrote to
> > help manage the CTDB nodes file isn't as robust as I'd like it to be and
> > is still generally immature. We'd like to discuss ways we might update
> > CTDB to make the nodes easier to manage through automation (in our
> > environment) and hope such discussions could improve things for other use
> > cases as well.
> > 
> > At the moment, we've been using a Kubernetes based mechanism [5] for
> > getting clients running outside of the cluster access to the shares.
> > Without plunging deep into the particulars of Kubernetes networking; the
> > method works well enough but does not use the CTDB public address
> > mechanism. As such, we feel the failover behavior we have right now may
> > not be as nice at getting clients to quickly fail-over when compared to
> > CTDB's tickle-ack. This is an area we're going to be actively
> > investigating and would like to hear additional feedback on this topic.
> > 
> > We're also adding support for the CTDB enabled Samba instances to the
> > Kubernetes operator code. The plan is to continue to use the single SMB
> > instance by default but add an experimental mode for running a StatefulSet
> > of Pods that include CTDB for clustering across multiple nodes.
> > 
> > We are also discussing the need for using the fileid vfs module on top the
> > clustered file system we intend to use (cephfs). However, we don't want to
> > require a PVC to be any particular file system and so we have been
> > discussing ways to enable the fileid module, and how to configure it,
> > only in cases that we need it in an easy-to-use manner.
> > 
> > ACL Xattr
> > ----------
> > 
> > We are very interested in being able to use NTACLs on the shares that we
> > host in containerized Samba servers. Today, the acl_xattr module stores
> > the NTACL metadata in an xattr named "security.NTACL". The "security"
> > namespaces on Linux requires the use of the CAP_SYS_ADMIN capability -
> > basically the "root user" of Linux capabilities. This would require
> > running the containers with escalated privileges; something we would
> > prefer not to do. So, Günther Deschner, with feedback from Ralph Böhme
> > and Michael Adam and others, has been working on the patches in MR#1908
> > [6].
> > 
> > With these changes in place, we'd be able to configure Samba with a custom
> > xattr for our containerized Samba instances and thus avoid needing to run
> > a
> > privileged container while still supporting NTACLs.
> > 
> > Nightly Builds
> > ---------------
> > 
> > We've been looking in an easy way to try out the latest code in Samba and
> > are planning on adding support for building container images based on
> > nightly package builds. Anoop C S has already added support to the
> > sambacc layer in order to support the newer CLI options present in Samba
> > 4.15. Next we plan to add new build rules to the samba-containers project
> > that will create new build tags in our container registry repo(s). Soon
> > we should be able to more easily test and experiment with the latest
> > versions Samba has to offer!
> > 
> > Metrics
> > --------
> > 
> > In the slightly longer term, we would like to add metrics support to our
> > pods running in Kubernetes. Our plan is to follow the de-facto standard
> > in this space and export Prometheus metrics. In order to do this we need
> > to get the data out of our Samba stack and rather than try and directly
> > scrape the output of a tool like smbstatus we are very interested in the
> > effort to add JSON output support to samba tools. In our case, we want
> > JSON
> > output from smbstatus. As such we're very interested in MR#1269 [7].
> > We'll probably be getting more involved here in the near future.
> > 
> > Offline Domain Join
> > --------------------
> > 
> > One new feature that landed in Samba 4.15 is the new Offline Domain Join
> > (ODJ) support. Currently, to connect our containers with Active Directory
> > we're asking users to store a username/password in a Kubernetes secret
> > [8]. We are aware that storing a password (even within a secret) isn't
> > the best thing to do, so we're looking forward to trying to integrate the
> > ODJ feature into our projects so that users never have to store a
> > password.
> > Watch this space! :-)
> > 
> > Wrap Up
> > --------
> > 
> > If you have any questions or comments feel free to ask! I'm hoping this
> > update both serves to inform you of what we're up to as well as serving as
> > a prompt for you to give us feedback.
> > 
> > Thanks for reading!
> > 
> > [1] - Specifically OCI/Docker style containers.
> > 
> > [2] -
> > https://sambaxp.org/fileadmin/user_upload/sambaxp2021-slides/Mulligan_Ada
> > m_samba_operator.pdf
> > 
> > [3] - https://github.com/samba-in-kubernetes/sambacc/
> > 
> > [4] - https://github.com/samba-in-kubernetes/samba-container/
> > 
> > [5] -
> > https://kubernetes.io/docs/tasks/access-application-cluster/create-extern
> > al-load-balancer/
> > 
> > [6] - https://gitlab.com/samba-team/samba/-/merge_requests/1908
> > 
> > [7] - https://gitlab.com/samba-team/samba/-/merge_requests/1269
> > 
> > [8] - Not strictly true, as you can leave out the secret and then run
> > 
> >   a command in the container to complete the join. However, this
> >   runs counter to the level of automation we'd like to provide, but
> >   is an option.
> > 
> > -- John M.