[RFC] Suggested "combined tree" for easier development and Franky integration.

[Kai Blin]

Hi folks,

at the last Franky meeting, we realized that our current attempt at creating a 
combined Samba3/Samba4 build using git submodules was clumsy and fragile.

So we came up with a combined tree solution to address this issue. 
Incidentally, this combined tree might be useful before we get the integrated 
build sorted out already.

There is a small one-time effort involved migrating personal branches over, 
more about that below. Let me first explain how the merged tree looks like.

As you all know, the Samba3 tree looks like this:

kai at blackjack:~/samba/samba3$ ls -1p
COPYING
docs-xml/
examples/
MAINTAINERS
Manifest
packaging/
pcp/
PFIF.txt
Read-Manifest-Now
README
README.Coding
release-scripts/
Roadmap
source/
swat/
testsuite/
WHATSNEW.txt

The Samba4 tree looks like this:
kai at blackjack:~/samba/samba4$ ls -1p
BUGS.txt
COPYING
howto.txt
NEWS
packaging/
PFIF.txt
prog_guide.txt
source/
swat/
swat2.txt
testdata/
testprogs/
TODO
WHATSNEW.txt

Now, it's obvious that a plain merge of these directories isn't possible as 
there is a couple of clashes. Some of those are easy to fix (COPYING and 
PFIF.txt), some are hard (source/).

As trying to combine the Samba3 and Samba4 source/ folder into a single folder 
would be madness, we decided to merge the Samba3 source/ folder as source3/ 
and merge the Samba4 source/ folder as source4/. For my proof of concept 
merge, I dropped the other conflicting files that weren't identical and as a 
result, the combined tree looks like this:
kai at blackjack:~/samba/samba-devel$ ls -1p
BUGS.txt
COPYING
docs-xml/
examples/
howto.txt
MAINTAINERS
Manifest
merged_branches.txt
pcp/
PFIF.txt
prog_guide.txt
Read-Manifest-Now
README
README.Coding
release-scripts/
Roadmap
source3/
source4/
swat/
testdata/
testprogs/
testsuite/
TODO

The nice part about this is that nothing really changed for building Samba3 or 
Samba4 standalone, just that to e.g. build Samba4, you now do "cd 
source4; ./configure; make" instead of "cd source; ./configure; make".

So far so good, our first attempt looked pretty much like this. However, git's 
tracking of file renames is very basic and breaks down if a lot of files are 
renamed. For both Sambas, source is big enough to break. So while history 
wasn't lost completely (e.g. git blame path/to/file still worked), git log 
path/to/file didn't work anymore. The Git folks think this is a minor 
annoyance, Volker hated it, and I tend to agree with Volker. There's a couple 
of workarounds, but they were all pretty clumsy.

In the end, we found a solution that preserved the history across the merge 
for a small one-time migration effort. Using some fancy git magic (git 
filter-branch, in case you're interested), Volker created a Samba3 branch 
that had all commits since the beginning use "source3/" instead of "source/", 
and then repeated the same for a Samba4 branch. I then merged those two 
branches using some more git magic Jelmer and I had discovered on Wednesday 
(I'll describe the whole process in another email for people interested in 
this) and uploaded the combined branches to git.samba.org [1].

Now, because we changed the commit objects doing this, the sha1-sums of the 
commit objects don't work anymore, meaning git rebase or similar will blow up 
attempting to rebase on the new, unknown objects. However, it's easy to work 
around this with little effort, described on the Franky wiki page [2]

So while I don't propose to go live with this setup right now, I'm hoping for 
some feedback on the general approach. The incredible advantage of this is 
that (Franky considerations aside) code that is shared between Samba3 and 
Samba4 can be pulled out of the source3/4 dirs and needs to be updated only 
once instead of twice. So even though this looks like "yet one more branch" 
right now, it will mean less branches to check things in in the long and even 
in the medium term. Also, instead of having to get a separate Samba4 branch 
to run smbtorture tests with Samba3, you just need to build source4 once, and 
not change the repo.

We need to discuss how to handle the remaining duplicates from the old Samba3 
and Samba4 trees. I'm looking forward to suggestions on how to improve the 
migration guide as well.

Cheers,
Kai

[1] Check out the combined tree at git://git.samba.org/kai/samba/franky.git
[2] The migration guide is at 
http://wiki.samba.org/index.php/Franky#Updating_your_branches_to_the_combined_build

-- 
Kai Blin
WorldForge developer  http://www.worldforge.org/
Wine developer        http://wiki.winehq.org/KaiBlin
Samba team member     http://www.samba.org/samba/team/
--
Will code for cotton.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part.
Url : http://lists.samba.org/archive/samba-technical/attachments/20080905/753033c7/attachment.bin