[PATCH] Make AD DC build require Jannsson JSON libs, fix fileserver without it

Fri Jun 22 22:05:00 UTC 2018

On Fri, 2018-06-22 at 13:57 -0700, Jeremy Allison wrote:
> On Fri, Jun 22, 2018 at 10:43:08PM +0200, Ralph Böhme wrote:
> > On Fri, Jun 22, 2018 at 10:49:00AM -0700, Jeremy Allison wrote:
> > > On Fri, Jun 22, 2018 at 07:35:45PM +0200, Ralph Böhme wrote:
> > > > hm, I disagree. I strongly feel that it doesn't add too much upstream
> > > > maintanance having a --without-json-audit switch that allows building on older
> > > > systems that are lacking libjansson.
> > > 
> > > Yeah but it's json today, it'll be something else tomorrow.
> > 
> > Nah, today is today and tomorrow is tomorrow. Today, RHEL 6 is still a supported
> > release and will be til 2020. Or am I missing something?

Why is 'Supported by Red Hat' the same as 'supported by the Samba
Team'?

> > > IMHO I think it's OK for the AD-DC to require more packages/resources
> > > than a fileserver build.
> > 
> > sorry, but I don't get it. Why is json audit logging a *requirement*? It might
> > be a required features for some downstream consumers, it certainly it not a
> > requirement for all, is it?
> 
> Well the auditing is a big deal for customers using an AD-DC.
> Many laws around compliance require proper auditing. We have
> it in the fileserver already (vfs_audit etc.), the AD-DC needs
> it too.
> 
> Happens that the auditing code in the AD-DC requires json. I
> didn't write it that way but IMHO it's a reasonable technical
> decision.

Thanks.  Our clients are really keen to have this easily imported into
the big log aggregation engines. 

> > > It's a big complex beast that does a lot of stuff, and it's
> > > still rapidly under development with more features being added.
> > > 
> > > If we try and make it compatible with only the packages available
> > > on 8+ year old distro's we're going to make development more complex
> > > than it already is.
> > 
> > Hm, how much work can it be maintaining a build without libjansson? 
> 
> Well IMHO this isn't just around json. We can probably maintain
> such a build. It's more around can we freely develop an AD-DC
> requiring technologies that aren't packed by default in a distro
> from 2010 ?
> 
> I think that's too old for requirements. What do others think
> about the minimum/maximum distro age we should support out of
> the box ?

We should be able to reasonably freely depend on any package in:

For the AD DC: Current (RHEL + EPEL), Current Ubuntu LTS.

For the fileserver: Current -1 (RHEL + EPEL), Current -1 Ubuntu LTS.

This certainly isn't about JSON, it is in EPEL6, the same place we get
the python we require to build:
https://centos.pkgs.org/6/epel-i386/jansson-2.11-1.el6.i686.rpm.html

But as an implementer, it is really expensive to develop around
technologies from 2010:  

* We have two whole copies of complex cryptographic code for BackupKey
because we might not have a new enough GnuTLS, but only test one (I
don't know which) in autobuild!  

* We have two, and developed three different crypto implementations for
a simple encryption of secret values in the AD DC[1].  

* We still carry a copy of zlib in third_party 'just in case' someone
needs it. 

It is incredibly cheap to ask for a build 'without this dependency'. 
The #ifdef magic seems so quick!  

I spent around three hours yesterday building the patch attached, which
I could have spent on other things if we had just accepted it as a
build requirement overall. 
 - setting up autobuild targets without that feature
 - wrangling WAF to still link without this feature
 - fixing the fallback code used in the fileserver
 - ensuring it still passes tests

Let alone now arguing that this was enough, and that we shouldn't go
further.  What is asked for, making it truly optional then puts it out
of what is tested, in #ifdef paths that are not compiled and so never
run in autobuild. 

It all takes time, and while it is 'only' developer time, it continues
for feature after feature.  It constrains the options we even consider,
given that use of third_party is also considered a really big ask.  

We need to be able to use current external libraries without fear. 
Samba is already too strong with the Not-Invented-Here.  When we
switch, we need to be able to rely on a current Python3, not the one
from a decade ago.  We need to assume current Kerberos libraries if we
are to ever move past Heimdal for the fileserver (the DC is harder of
course)!  We need to be able to assume current GnuTLS if we are to ever
remove Heimdal or in-tree crypto. 

Now yes, this makes it harder for packagers and those manually
installing.  SerNet in particular may have extra costs developing the
subscription Samba+ packages.  Those are costs that should be bone by
those who suggest that the very latest AD DC should run on the very
oldest systems, not by the Samba Team.

[1] This was a bit of a stuff-up, Gary spent ages working on the GnuTLS
one to find the algorithm wasn't actually in current RHEL, so then did
the libnetttle one.  Then that wasn't good enough so Metze re-did it
using the internal crypto.  Weeks and weeks of developer time were
spent. 

Thanks,

Andrew Bartlett

-- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba