The Wrapper Project

[Andrew Bartlett]

On Wed, 2013-07-10 at 23:56 +0200, Andreas Schneider wrote:
> On Wednesday, July 10, 2013 14:05:02 Jelmer Vernooij wrote:
> > Hi Andreas,
> 
> Hi Jelmer,
> 
> > Great, thanks very much for working on that! It'd be great to have
> > these available elsewhere.
> 
> thanks. 
>  
> > > On the way to get 'make test' in Samba working I found serveral bugs
> > > which I already sent patches for. The remaining patches are my git
> > > repository [4].  I've changed selftest to write a hosts file for
> > > name resolution and changed waf to look for the preloadable system
> > > version of the wrappers.  

Once we get this in, we should remove the ugly hack I currently have
doing DNS in a hosts-like file at the libcli/resolv layer. 

> I have removed nss_wrapper, socket_wrapper
> > > and uid_wrapper from the Samba source code for now. I want to
> > > maintain and improve them outside of the Samba tree.
> >
> > > Currently the git trees live on my own infrastructure but I'm also
> > > happy to create git repos on git.samba.org if you prefer that.
> > 
> > What committers/commit policy would you suggest for these?
> 
> The same as Samba once they are in a state I would consider them useable. 
> Currently I need to be able to do changes very quickly. I try to write unit 
> tests for all the changes. All the new hosts code in nss_wrapper has unit 
> tests and I also wrote a testsuite for uid_wrapper.
> 
> > 
> > > Yes, you have to install them first into the system before running
> > > them. If you can't do that you can still set the LD_LIBRARY_PATH to
> > > point to a directory where you have installed them. If you vote
> > > against the removal of them from the Samba tree, then we could just
> > > pull the code from time to time from the other repsositories like we
> > > do it with heimdal or subunit.  I'm against this but we have
> > > developers which would like to have everthing in the source tree.
> > > Alternatively, we could use git submodules for that.
> > > 
> > > My work on the wrappers is not finished yet. They still need to be
> > > checked on different platforms and we need more tests to easier
> > > verify that they are working correctly. socket_wrapper is already
> > > tested on several platforms [5] and works on MacOSX too.
> > 
> > The argument for keeping copies of the other software in-tree has
> > always been that it should be possible to hack on samba without having
> > to install a bunch of dependencies, as that may not be easy on all
> > platforms. I guess you can still build Samba without the wrappers and that
> > is probably sufficient - you only need to install these if you want to run
> > the testsuite.
> 
> Well and if you run the testsuite it is normally efficient to install them 
> once. The libc code doesn't change very often and I don't expect a lot of 
> changes to happen. Maybe that someone adds new wrappers for functions but it 
> is working very well on Linux right now.
> 
> > We need a strategy for the build farm hosts. It is annoying to
> > install something as root on all build farm hosts - and to install
> > newer versions when we come across regressions.
> 
> You don't have to install them into the system. You either set LD_LIBRARY_PATH 
> or set LD_PRELOAD to point to them directly like
> 
> LD_PRELOAD=/path/to/libsocket_wrapper.so:/path/to/libnss_wrapper.so
> 
> > We should probably do the same (bundle, submodule, require
> > installation) for testtools/subunit as we do for the
> > wrappers.
> 
> Yes, this sounds reasonable!

I agree.  

> > Personally, I would prefer using submodules and just having "make
> > dist" include copies of the wrappers and testtools/subunit in the
> > tarball and the tree that is used by the buildfarm hosts.
> 
> I don't expect that they need to be updated very often once they are in a 
> working state. So they need only be updated when we add new features we will 
> use in Samba.

The biggest reason I have for keeping a copy in the Samba tree (rather
than require installation) is that when I've found bugs in
socket_wrapper, it tends to be of the incredibly subtle kind - the kind
that could take another developer just as many hours to debug.  It might
only need a proper pkg-config dependency, but whatever we do needs to
ensure we always use the same version that Samba was autobuilt with. 

I guess the same applies just as much to all the other users however :-)

It certainly is incredibly encouraging to see this little project grow
wings and gain a life of it's own, as it has been the keystone to the
level of automated testing we have, and it would be great to have other
projects use it too. 

I'm still sceptical about loosing support for handling this without
LD_PRELOAD (it seems an unfortunate feature to loose), but your testing
across BSD, Solaris and Linux variants seems to rebut my concerns.

My biggest concern is ensuring we don't make it any harder than we need
to build it, to test Samba on an unusual platform (where we are seeing
other issues, and want to run make test), and in particular for the
build farm.  

My preference would be for it to build with wafsamba like our other
subprojects, and for it to stay in samba.git like tdb/talloc/etc.

That said, provided you patch the build farm scripts with whatever magic
is required to build this before the test run, I'll consider my concerns
here addressed.   (This is not unprecedented:  We once had to build
samba4 to get smbtorture4 for the samba3 build to test with). 

Finally, congratulations on taking a Samba-specific hack and exposing it
to the world!

Andrew Bartlett

-- 
Andrew Bartlett                                http://samba.org/~abartlet/
Authentication Developer, Samba Team           http://samba.org