[development] Very concerned over Drupal's core development
larry at garfieldtech.com
Tue Apr 21 06:05:19 UTC 2009
On Monday 20 April 2009 4:03:08 pm Dries Buytaert wrote:
> I think catch hits a lot of good points about how our reward and trust
> system could be improved. I'm all for better profile pages, giving
> credit where credit is due, and providing better visibility into who
> is working on what. It might be useful to provide a "Reviewer rating"
> or "Reviewer score" -- something which could be an incentive for more
> people to review patches. Maybe this can be accomplished through some
> project module extensions? I'm not sure what the best approach would
> be, but it is worthy of a discussion.
Dear god this is a monster thread... :-) I'm going to jump in here for lack
of anywhere better to do so.
As others have noted, the problem isn't the speed of commits, it's the speed
of reviews and the ability of sufficient people to grok them to review them
usefully. I would add a related problem: The speed of decision making.
Some have suggested giving subsystem maintainers commit access. I am not sure
that's a good idea, given Drupal's current architecture. It's still too
intertwined, with poor separation between systems. Hooks are well and good
but they don't separate subsystems from each other. (I freely admit that one
of my ulterior motives with the handlers project is to encourage further
separation between components, making it easier to grok them separately, test
them separately, and develop them separately, even in contrib. I hope to get
back to that soon, as I've been heavily distracted by personal matters of
For subsystem committers to work, we would need to have a way for there to be
layered levels of commit. That is, as others have mentioned before, the Linux
model. Linus almost never reviews or even sees individual patches. He gets
periodic rollups from subsystem maintainers, reviews them in general but not
at a line-by-line level, and adds in a great deal of trust that said subsystem
maintainer is doing his job and that what he's getting has already been vetted
by someone who understands, say, the power management system far better than
he does. It's the power management subsystem maintainer's job to go line by
line, not Linus'.
We could adopt that sort of model without giving subsystem maintainers core
commit access, if and only if 1) We switched to a distributed VCS, because CVS
really can't handle that. Really, I see greater autonomy for submaintainers
and a DVCS for core as fundamentally linked; doing one without the other is
pointless. 2) We need to trust that subsystem maintainers have and are
implementing long-term planning that makes sense, and then let them sweat the
As webchick noted earlier, and as I discussed with her in IRC, we need more
"maintainer teams" like we have for the DB system. The DB system is sort of
an oddball case, though, since it breaks into subcomponents so easily (each
driver is its own subcomponent).
Of course, if we want to get more people to be domain experts, whether
formally recognized or not, there needs to be an incentive. "Pleeeeeese???"
is our current incentive model, and I don't think it works all that well. :-)
Informal karma only goes so far. What is the incentive for people to become
and remain domain experts? Now that I have finally managed to get query
builders into core so that I never have to deal with the frickin' INSERT query
syntax again, what's the incentive for me to remain active and on top of the
DB system? (Not that I'm planning to wander off and ignore it, mind you; I'm
just using that as an example.)
I first became the DB maintainer right before Szeged. In Szeged, I asked Dries
and Angie what they wanted me to do, exactly. "OK, so what did I just sign up
for?" Their answer was "just keep doing what you're doing". That didn't
really make much sense to me, as what I'd been doing was developing a several
hundred KB patch over the course of a year and a half and not sleeping for two
weeks in order to get it committed and I really didn't want to keep doing
things that way. :-) I've not gotten a particularly different answer since.
Sometimes I see "I'd like to get Crell's thoughts on this issue" type
comments, and sometimes I get emails with an issue that a core committer wants
me to have a look at, but usually I know about them already because I have the
DB issue queue bookmarked.
I don't mean that to sound like a whiny complaint. The point is that if I, as
a current submaintainer / domain expert, don't really know what the benefit is
for being one besides feeding my own ego, then what incentive is there for
anyone else to become a domain expert on FAPI, or to give forum.module the
extensive love it desperately needs?
Which brings me to something else that Dries said earlier about the micro vs.
macro level. Looking at the macro level and planning things out to ensure
consistency and so forth is very important. It also has a name, which is even
dirtier to mention around here than "backward compatibility" but I shall do so
anyway: A roadmap.
Roadmaps help ensure that small tasks fit together into a cohesive whole, and
that everyone knows what that whole is. They also give people a clear task
list of things that need to be done and WHY. That makes life easier for
developers and for reviewers, because they can see why we're making this
change to a given piece of code; it's a small change that is part of a larger
effort, and the larger effort makes sense. OK, call them community initiatives
or whatever; that's still what they are. We're seeing more of them appear
haphazardly in the issue queue with multi-part issues that reference each
other. That's a very good thing. Drupal is, really, past the size at which
"scratch an itch somewhere" can be the primary organizational mechanism,
especially if we actually want clean and consistent APIs.
OK, so we don't necessarily need one huge roadmap for all of core. Given our
development model I doubt that would work anyway. But, if we tone down the
anti-planning rhetoric and instead allow for smaller, subsystem roadmaps or
particular campaign roadmaps (like Fields-ifying core), that gives people more
context in which to review, as well as more documentation on why something is
being done, which again makes it easier to review. Then, allow submaintainers
to direct roadmaps.
If someone wants to become the big fish in the little pond for blog.module and
do amazing things with it, and it's someone that has already shown that they
are reliable in general, OK. Let's see what your plan (roadmap) is. Discuss
and approve it in general. Let them work on it and also be the go-to person
for anyone wanting to work on it to make sure that that particular piece of
core is consistent and clean and robust. That's their responsibility... and
also their authority. When something comes down to a subjective opinion
question, the submaintainer's opinion needs to carry more than usual weight.
(Objective measures like performance, sure, benchmarks win. I'm referring to
judgment calls.) Then the subsystem maintainers, together with the
maintainers (Dries and Angie for D7), coordinate to ensure consistency and
cleanliness and elegance across core, so that the API decisions for
node.module and the API decisions for comment.module and the decisions for
FAPI all mirror each other where appropriate, which in turn leads to more
consistent and therefore more learnable APIs, which lowers the barrier to
entry for future reviewers, etc.
Positive feedback loops are a good thing.
larry at garfieldtech.com
More information about the development