[development] CVS branch work best practices?

[Derek Wright]

sorry for the delayed reply to this thread -- i just got back from  
vacation and have started reading through the backlog...  i'll  
respond directly to a few key points here, and start new threads for  
a few others.


A) the answers to Ber's original questions can be found here:

http://drupal.org/node/17570#HEAD

i laid out the 2 basic scenarios for how to use HEAD, when you'd  
create branches, and where you'd apply patches.  that's been there  
since the night the release system went live.  it's worth re-reading  
in light of this thread.  no, you don't *have* to make a DRUPAL-5--2  
branch to add new features to your 5.x-* version of your module.   
read what i wrote, and follow the first scenario i outline [1].

these 2 scenarios don't quite match the 2 laid out by Dries here:
http://lists.drupal.org/archives/development/2007-02/msg00082.html

some of the other debates in this thread came from misunderstandings  
coming from the difference between what Dries wrote in that email,  
and what's in the handbook.  that handbook page is *really* important  
for developers to read and understand, so any comments on how to make  
it more clear would be greatly welcomed.


B) why this is so complicated...

- Drupal contrib development is nearly worst-case for revision  
control complexity:
-- core's API changes (usually radically) at least twice per year.
-- people actually *using* drupal try to rely on various older  
(stable) versions of core for at least a whole year, sometimes longer.
-- nearly every site depends on at least a few (if not dozens of)  
contrib modules.
-- real sites often want new functionality in contrib for older  
(stable) versions of core.
-- contrib is full of bugs and security holes (like every other piece  
of software on earth), and those are actively being uncovered and  
fixed by a very clueful, responsive, and dedicated security team  
(unlike most pieces of software).
-- many modules depend on other modules.

(i didn't invent any of this when i designed the new release system.   
those are just the facts i found myself confronting when trying to do  
contrib development for a real site that needed new functionality not  
found in the existing modules.  the release system doesn't create  
this complexity, but it partly reflects it.  yes, complexity can be a  
disease, but nature is complicated.  again, einstein: "everything  
should be as simple as possible, but not simpler".)

- many (most?) Drupal developers have little or no experience with  
revision control, release management, etc.  some (many?) don't even  
understand basic revision control concepts like branching and  
tagging, let alone advanced operations of the specific tool we're using.

- we make a virtue out of flexibility and distributed decision  
making.  every maintainer is basically free to do whatever they want  
regarding their own revision control activities.  this makes perfect  
sense given a huge, distributed, volunteer development community, but  
it adds another layer of complexity to the overall picture.  we can  
document best practices all we want (and i've done lots of that), but  
in the end, it's all a gamble as to how responsible and consistent  
the maintainers are being for the modules you actually care about...   
there's a limit to how much the system can force the maintainers to do.

- we had a long period where none of these problems were confronted,  
so there are a lot of accumulated practices, expectations, and  
(frankly) superstitions that have been built up over the years.  in  
order to get the new release system accepted and installed at all, i  
had to maintain certain links to the past that, if we were starting  
clean, i would have removed [2].  these compromises for "backwards  
compatibility" (a little ironic, given the usual "the drop is always  
moving" mentality), make a few things about the release system (in  
particular, the fate of the contrib branches that have the same names  
as core branches, like DRUPAL-5, etc) a little more confusing than  
they need to be.

- every contrib module has a different audience/community of users,  
and how much effort you need to put into the release/branch  
management depends on the maturity of the module and the size of the  
user base.  new modules with few or no users don't need to be as  
thoughtful as well-established modules that 1000s of sites depend  
on.  in general, i document best practices for the safe, mature,  
cautious approach, but for lots (perhaps most?) of the stuff in  
contrib, it's not *really* necessary.  i'm very torn about this  
point.  i'd like to encourage consistency (and therefore, the safe,  
good approach), but i don't want to saddle people with burdens and  
concerns that they don't necessarily have to worry about (yet).   
advice on this dilemma would be most appreciated.


C) idiot-proofing the system

Earl Miles wrote:
>> Khalid Baheyeldin wrote:
>> 1. The new release system does not enforce a branch for the major  
>> number. I know
>> because I used tags only for a 1.1 and 2.0 and the system just  
>> accepted it.
>
> This may qualify as a bug and deserve correction.

there are already a bunch of protective measures in place to help  
prevent people from screwing up the branching and tagging. [3]  right  
now, for example, you can not add "DRUPAL-5--2" as a tag... that has  
to be a branch.  contrib tags have to include the full version  
information, like DRUPAL-5--2-0 (for the 5.x-2.0 release).  but, it  
does let you check out a DRUPAL-5 branch workspace (or even HEAD  
workspace), and add a DRUPAL-5--2-0 tag directly to that workspace,  
without ever creating a DRUPAL-5--2 branch.  this behavior (at least  
regarding HEAD) is a feature, not a bug, and is the basis for the  
"recommended" scenario for HEAD i put in the handbook.

the larger problem is that certainly in CVS, and, i fear, in SVN,  
too, there's no notion of what branch a specific tag belongs to, so  
it's impossible to enforce things like this [4].

thanks,
-derek


[1] ... and consider sponsoring work on http://drupal.org/node/89699 ;)

[2] for example, see http://drupal.org/node/92452

[3] there are still some on the todo-list:
http://drupal.org/node/90968
(http://groups.drupal.org/node/1830 for the full list)

[4] details for the interested reader:

a tag is just a symbolic name for a set of revisions of files (in  
CVS, a whole set of file/revision pairs, in SVN, a unique revision #  
across the entire repository for all files).  a branch tracks changes  
to a set of files, but not all files on the branch change.   
therefore, a given version of a given file might be the same revision  
on many branches.  so, when you mark that file as part of a tag  
across multiple files, there's no way to know "you're adding your  
DRUPAL-5--2-0 tag to the XXXX branch".  i can elaborate if anyone is  
really interested in the details, but basically, there's no real  
notion (as far as i know, in *any* of the revision control systems  
out there) that a tag belongs to a specific branch.  given that,  
these sorts of "idiot-proofing" requests are a) feature requests, not  
bugs and b) impossible to fulfill until the revision control  
technology catches up with our desires and notions.  if you were  
doing a "cvs rtag -r DRUPAL-5--2 DRUPAL-5--2-0", that'd be more of  
what you have in mind but a) cvs fails to provide this info in the  
right hooks we need b) rtag is scary and error prone if you don't  
know what you're doing and c) i'm not sure SVN solves this particular  
failing of CVS, either (see my other post).  granted, it's been a  
while since i looked really closely at SVN, so things might have  
changed, but from all i've ever known about it, an SVN tag is  
fundamentally just like a CVS tag: a symbolic name for a repository  
revision, and there's no inherent binding of tags to branches.  if  
SVN has adding more tag meta data in recent versions, and someone  
wants to point me to the relevant docs, i'd be happy to learn more.