Generating loadparm from our XML documentation

Thu Jan 9 14:27:41 MST 2014

On Thu, 2014-01-09 at 18:20 +0100, Michael Adam wrote:
> Hi Andrew,
> 
> On 2014-01-10 at 00:02 +1300, Andrew Bartlett wrote:
> > On Thu, 2014-01-09 at 08:28 +0100, Volker Lendecke wrote:
> > > On Thu, Jan 09, 2014 at 12:24:14PM +1300, Andrew Bartlett wrote:
> > > > On Wed, 2014-01-08 at 11:13 +0100, Volker Lendecke wrote:
> > > > > On Tue, Jan 07, 2014 at 04:34:41PM +1300, Garming Sam wrote:
> > > > > > I've just put all the changes into a repository as the patch was too
> > > > > > big for the list.
> > > > > > 
> > > > > > git://git.catalyst.net.nz/samba.git rename-param
> > > > > 
> > > > > I agree that the names do not follow Samba's variable naming
> > > > > convention, but just doing a sweeping change is questionable
> > > > > at best: It will cause pain for people who have to port back
> > > > > and forth patches for no good functional reason, they don't
> > > > > even clean up convoluted code.
> > > > 
> > > > > Can you please explain why you need to rename all the
> > > > > parameters?
> > > > 
> > > > The future step of auto-generation requires this, as otherwise each of
> > > > these variable names would need to be encoded into our XML
> > > > documentation, which would increase the risk of errors being introduced
> > > > at that step.  While not perfectly consistent, the current _-separated
> > > > names generally match the parameter string. 
> > > 
> > > It does. However -- can we discuss how you envision the
> > > parameters to be handled in the future? The current
> > > situation is a mess at best, and before we go too far into
> > > one direction, I would love to see consensus how to do this
> > > in the future. You seem to indicate to generate code from
> > > XML? Wouldn't it be better to generate the XML from code?
> > > This would make XML handling optional for people who *just*
> > > want the code and don't care about manpages.
> > 
> > I'm very happy to discuss this, which is why I've tried to foreshadow
> > where I hope Garming's work might go since the first patches sent on
> > Christmas Eve. 
> > 
> > First, the fundamental idea (of generating this all from the docs) isn't
> > mine, or particularly new.  I'm pretty sure Michael Adam was the one to
> > suggest it to me during one of the previous re-factoring stages.  We
> > discussed it as a long-term goal in 
> > https://lists.samba.org/archive/samba-technical/2011-July/078525.html
> > and I also mentioned it in October when we fleshed out lib/param/README
> > with aca475b6bcbe364d09b83d17bf0504741ed3a84a.
> 
> We certainly discussed this more than once, and I definitely
> said I would love to introduce once central place to store
> all information about a parameter, and generate all and doc
> from that. I would even be surprised if anyone would
> question this.
> 
> But I definitely never suggested to generate code from the xml
> docs. Instead, what I suggested at the time is to have one
> central python program to generate all code and the xml doc
> sources for the parameters from python data structures.
> 
> In principle the effect is the same of course.
> And I consent that using an existing standard parser
> for extracting the information from xml and then generating
> the code from there might have advantages over self-written(?)
> code that generates xml doc from python data.
> Still, I think I'd like the latter better.
> (But that is just a detail... :-)
> 
> My main critique now (just like 2+ years ago) is that I think
> that we should first refactor the two separate loadparm instances
> to the point that they are the same and can be moved to the base
> dir completely. And then start generating this. We should not
> start to add generating code for the convoluted state we
> currently have.

I understand your critique, but as nobody has done that work
substantially, I suggest we should at least take the progress we have. 

While merging parameters and fixing the docs has been an overwhelming
task for us all so far, it boils down to a pile of small, verifiable
steps.  Even auto-generation isn't hard when you have the expected
output and regular (eg XML) input.  Re-factoring loadparm's internals
however needs a level of both experience and patience that we don't have
a great supply of right now.

> That being said, regarding the currently prosed patchset of
> renaming lots of variables: Is this really necessary right now?

Yes.  The alternatives would require the same amount of review and embed
Hungarian notation into our XML documentation, probably forever.  

> Can't it be held off until we really do the autogenerating bit,
> even more if the patches are more or less automatically
> generated.

I expect to see patches for some auto-generation to be posted today, and
more next week, so there is no value in putting this off. 

> There are also quite some changes of defaults.
> These have to be reviewed and tested very, very carefully, imho.

One of the important steps in this process has been to extend docs.py to
check defaults.  This has been key to ensuring this process is safe.
I'll post the 4 patches we have concerns about to the list for separate
comment and advice shortly. 

> I have to look more deeply into the patches and the last mails,
> but wanted to give a pong to indicated that I am also interested
> in this and thinking about it, before leaving office... :-)

Thanks,

Andrew Bartlett

-- 
Andrew Bartlett
http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: This is a digitally signed message part
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20140110/aa34d33b/attachment.pgp>