[PATCH] Auto-generate param_functions.c at build time

Andrew Bartlett abartlet at samba.org
Tue Jan 14 15:38:12 MST 2014

On Tue, 2014-01-14 at 16:23 +0100, Michael Adam wrote:
> Hi Andrew and Garming,
> it is good to see that you are making progress on this!
> Next step would be to generate the param_table right?
> And then to not have mkparamdefs generate the headers
> from the generated param_functions but have the script
> generate_param.py generate them in one go. Correct?

The next step would be to generate all the things that are based on
param_functions.c, not only because it won't be very hard to do, but
because generating code based on generated code is just one too many
layers.  We certainly don't want this to be more convoluted than the
current situation. 

param_table is a bit harder, but I've seen Garming's first prototype
here.  Preceding that will need to be another series of patches to fix
or regularise more of the incorrect metadata.  In particular, we need to
decide what information we still want to keep in a post-SWAT world. 

> I will try hard to find the time to look very closely
> at the patches and the resulting code in the next days.


I hope that when you do, you will see how much of this is about cleaning
up inconsistency.  The fundamental changes that worry you are the small
and in many ways easier part!

> It is good to move towards a central place of definition of
> conf parameters. But I do second Volker's request to not rush
> the patchset into master, and let it ripe a bit in a private
> tree, because it is very important that a few people understand
> it properly... :-)

It is important that other folks understand this.  I certainly find it
much easier to follow Garming's generate_param.py than the perl scripts
they will replace.  I'm also happy to commit to making the steps to add
a new parameter clear, and including pointers at the places where we
once used to add parameters. 

This is about much more than auto-generation, it is a once in 20 years
spring-clean of the loadparm code!

The auto-generation step is primarily to ensure that such
inconsistencies don't come back, and to capitalise on the work done by
reducing the number of places a new parameter must be added.  

I would however ask that we don't delay for delays sake.  We are in a
unique situation right now where I've been allowed the time to
intensively work on this with Garming.  Putting it off - particularly as
far as SambaXP - would simply make this much harder to merge.  What it
needs is review of the actual changes, each of which I still maintain
stands on it's own. 

> The last step, generating the headers from the FN macros
> etc, had the positive effect to remove one (or so) place to
> touch when adding a parameter, but for me at least the changes
> done at that time had the negative effect that they made it
> much more difficult to understand the loadparm code.

Hopefully the generation of these headers from the docs will be easier
to follow.  If we can rid the code of the last of the manual FN_ macros,
we may even be able to generate the CPP output directly, which would
certainly be clearer. 

> This is probably mainly related to the splitting between
> s4- and s3-loadparm, and I hence think it is very important
> to refactor the code to be able to share loadparm.
> I would like to be assured that this goal (which I would have
> loved to see reached before generating the code) is not made
> much more difficult to achieve by the new autogeneration-changes.

None of these tasks make this any more difficult, instead this work
makes that easier.  It does that because it puts a price on the
exceptions (and there is a shocking number of them).

There are multiple layers to loadparm:
 - the file parsing layer
 - the parameter table layer
 - the internal API to Samba (lp_ and lpcfg_)
 - the testparm/output layer

While certainly these are interactions, they stand surprisingly
independently.  With this work, we are merging loadparm, but slowly: not
all of it at once, and focusing on the parameter table at this point. 

We need to remove these exceptions and other special cases, so that, one
step at a time, the two systems can be merged.  We have come across some
small functions that can indeed be merged at the testparm/output layer,
and I'll post patches for this when I get a chance.

We have all spent a little time in loadparm, and both you and I have
tried to improve it at points.  We both know quite how difficult a beast
it is to manage.  These patches do make a big improvement to the
loadparm situation, and fix release-critical bugs by finally addressing
the divergent defaults and incorrect documentation.  


Andrew Bartlett

Andrew Bartlett
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba

More information about the samba-technical mailing list