Generating loadparm from our XML documentation

[Andrew Bartlett]

On Thu, 2014-01-09 at 14:51 +0100, Volker Lendecke wrote:
> On Fri, Jan 10, 2014 at 12:02:57AM +1300, Andrew Bartlett wrote:
> > The new docs.py and any future generator is written in pure python,
> > using the builtin ElementTree, and by design requires no more of the
> > build host than the rest of our build system already does. 
> 
> What minimum python version does this require?

http://docs.python.org/2/library/xml.etree.elementtree.html claims 2.5,
which is what we require anyway (see the very short discussion earlier
this week). 

> > As to if the XML documentation is the best source, I think it is.  While
> > XML is a highly frustrating language, it is structured, and we have
> > demonstrated the use of the python ElementTree API to read it and
> > extract the relevant elements, without disturbing the generation of the
> > manpages.   
> 
> Ok, you got a point here. I just hope that the internship
> does not end half-way and we end up with just yet another
> half-done situation that nobody has the energy to really
> push through to the end. Every now and then we throw in
> sprinkles of stuff without any real direction that anybody
> can follow with years of time in between. If Garming has to
> stop working before everything is done, we will drop it
> again for years. If there is a slight chance that Garming
> moves on before he is done with it, I would rather not
> change the current situation at all, bad as it might be.

All the stages here so far have been independent, while they help the
next stage, they stand on their own as reasonable changes, passing make
test etc.  Even so, I can allay your concerns because Garming will be
with Catalyst as an intern until the end of February.  

I'm amazed by the progress we are making.  I'm expecting we will have
auto-generation of param_functions.c by the end of today, and a
replacement for the dodgy mkproto.pl derived perl scripts by the end of
next week.  Even if we finished at that point, it would have been a
worthwhile project, because those scripts really suck.

Once we have that working, then we can move on to generating the
param_table.c file, or the defaults in lib/param/loadparm.c, and
confirming that still works.  Again, that would be a step that could
stand on it's own.  

The other task we noticed is simply to merge more of the code.  A number
of functions are still essentially duplicated between source3/param and
lib/param, and can be merged.  This will also assist the creation of a
merged subsystem. 

We are taking this step by step because a grand 'boil the ocean' scheme
just isn't reasonable here.  With over 20 years of history in these
systems, we need to take this step by step, and we need to prove each
step.  Every time we take one of these steps, we have found oddities and
inconsistencies, because we have tests.  Even if this effort was
rejected outright, we have already made massive progress on incorrect
defaults and often outright wrong documentation. 

Finally, once we have proof that we can correctly generate the current
loadparm systems, with correct types and defaults (by diff of the old
and new output), then we can look to generate better things.  

Thanks,

Andrew Bartlett

-- 
Andrew Bartlett
http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba