Investigating CI options for Samba test branches

[Michael Adam]

Hi Andrew,

On 2015-05-04 at 15:28 +1200, Andrew Bartlett wrote:
> Towards my proposal for faster release cycles, I've been looking into
> how we can ensure developers use their own time better.  In particular,
> I'm really keen to see a situation where folks push 'try' branches to a
> review server, and even if that isn't how the patch is finally merged,
> that an autobuild 'just happens' against that tree.
> 
> That way, it is much less likely that the reviewer will have to re-push
> things due to an autobuild failure, as if they look at the patch over 2
> hours after it was written, the patch would have been automatically
> tested.

While I don't see how this would spare the reviewer to re-push
things, I generally agree with the idea of forcing some
build and smoke-test (if not a complete make test, etc even)
before any reviewer starts to look at a patchset.

There is a lot of potential for avoiding wasted work
by catching things with such pre-review automatisms!

Speaking of this, I have recently worked a little with
such a system that does this from the original review
request up to the final push, namely the gluster development
process. This is build around self-maintainted instances
of gerrit for review and jenkins for CI.
This is set up in such a way that a patch(set) that is submitted
to gerrit first undergoes some tests that are performed by
jenkins. And only if the tests pass, a reviewer starts looking
at the change request. A patch needs verified ticks (for the
tests) and enough review ticks before it can be merged into
the upstream tree. This final merge/push does not happen
automatically but has to be triggered by the reviewer.

While we have tried gerrit once and it was not so well received,
and I know that generally some people have objections against
gerrit and jenkins because they are written in java, my point
is not so much about the actual components but that it gets
really nice and useful if something like this is carried
through from start to end! The experience of the gluster
workflow is very positive for me, personally!


Regarding the options that you have listed below,
I am not quite certain what to do. There are so many
influencing factors...
I think for CI jenkins is the de-facto open-source
standard at the moment, downside (for some): written in java.
trevis-ci is hosted and mostly tied to github and not
completely free. One can probably set it up oneself, too.

For the sake of flexibility and control, it would be good
to have a system that is self-maintained. But that needs
resources for administration. And we need machines (HW/VM)
to schedule the compile/test jobs on.

The main point still: Such a thing will really start to
rock when it is integrated with a review tool and with
the authoritative upstream repository.

My unfinished thoughts on these matters...

Cheers - Michael

> So far, I've looked at the practical details of three systems.
>  - my cloud-autobuild scripts (on the Catalyst Cloud)
> 
> I've got a patch to support the latter two solutions attached and here
> https://git.samba.org/?p=abartlet/samba.git/.git;a=shortlog;h=refs/heads/cloud-build
> 
> The cloud-autobuild scripts are at:
> http://git.catalyst.net.nz/gw?p=samba-cloud-autobuild.git;a=summary
> git://git.catalyst.net.nz/samba-cloud-autobuild.git master
> 
> However, this solution requires an active step, pushing the build, and
> of course an account on a cloud.
> 
>  - drone.io
> 
> For drone.io, the build doesn't even start properly, perhaps there is a
> disk space limit on their free plans.  This is however an interesting
> option, perhaps if we host it on our own (virtual) systems, as it meets
> our requirement of being an open source project.  I've asked for
> increased limits on the hosted system, and am intrigued that if we
> hosted it, it could apparently link back to a gitlab server. 
> 
>  - travis-ci.org
> 
> For travis-ci.org, the build starts, but they have a hard limit after 50
> mins, so the build also fails.  It is also not open source.  
> 
> Are there any other solutions we should be looking at, beyond just
> running autobuld on sn-devel?
> 
> Thanks,
> 
> Andrew Bartlett
> -- 
> Andrew Bartlett
> http://samba.org/~abartlet/
> Authentication Developer, Samba Team  http://samba.org
> Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20150505/f65b4122/attachment.pgp>