[development] Very concerned over Drupal's core development

[Robert Douglass]

Nedjo has given us the first response to chx's origninal problem  
statement that addresses the real intent of the concern, and proposes  
solutions.

Of the three solutions Nedjo proposed, I favor c) giving targeted  
commit rights to people who have historically been the maintainers for  
certain subsystems. We would need to develop guidelines around where  
their turf ends and they are no longer free to commit without  
escalating to either the branch maintainer or a permanent core  
maintainer (Dries), but this is a feasible solution that could speed  
the development of certain subsystems (actions, triggers, openID, i18n).

The other solution which chx and I have discussed, and which is a long  
time goal of mine, would be to reduce the size of core. We carry  
around a lot of modules that don't absolutely need to be part of core:

- aggregator
- blog
- blogapi
- forum
- help
- poll
- profile
- search
- statistics
- taxonomy
- tracker

I believe that all of these modules could have brilliant lives outside  
of core. A slimmer core means more focused development for the core  
team. It would also make us focus our attention on the APIs that  
support these very important modules. For example, the search API is  
in great need of being decoupled from the node module. If our goal  
were to remove search from core we'd need to really make sure the API  
were in place for letting core be searched.

My argument squarely reflects opinion that "core" Drupal should be an  
application framework that doesn't offer any specialized  
functionality. Forums, polls, profiles, blogs and even taxonomy are  
specialized web application functions that don't need the 10,000  
layers of red tape that core has. I firmly believe that taxonomy, for  
example, is hampered by its presence in core. There has been amazing  
work done on turning taxonomy into something far beyond what we have  
today, but the very presence of taxonomy IN CORE means that nobody  
uses it. Same with search. When I first wrote the ApacheSolr module it  
utilized as much of the search "API" as possible. Now we're moving to  
make it completely independent of core search because there is no way  
core can keep up with our development pace.

So, in my opinion, if you want to take Drupal to the moon, you do two  
things: 1) give more people the keys to the car, and 2) shed a bunch  
of the pieces that we currently call "core".

Robert Douglass

The RobsHouse.net Newsletter: http://robshouse.net/newsletter/robshousenet-newsletter
Follow me on Twitter: http://twitter.com/robertDouglass

On Apr 20, 2009, at 5:35 PM, Nedjo Rogers wrote:

> Karoly Negyesi wrote:
>> On Mon, Apr 20, 2009 at 7:06 AM, Bryan Ollendyke <bto108 at psu.edu>  
>> wrote:
>>
>> I had concerns and I shared that with the community. I had no
>> solutions. I stil do not have. I am not sure postponing Drupal 7 even
>> further is the solution.
> I agree that bottlenecks in the reviewing and applying of patches is a
> huge problem majorly holding up Drupal development. I suspect we need
> some significant structural changes to address it.
>
> Partly I think we need to look at changes in the drivers of Drupal
> development.
>
> There was a time when it was mainly hobbyists, who weren't  
> particularly
> tied to external timelines. No longer. The biggest amount of focused
> development time now comes from medium to large companies needing
> specific functionality within set timelines. The typical development
> cycle for a large project (e.g., a major site or multisite  
> installation)
> is a matter of months. Development can't wait the many months or even
> years before a new version is ready and can't deal with the  
> uncertainty
> of fluxuating timelines. So the significant work they sponsor is  
> almost
> never for core, which is years ahead of what can be used. It's instead
> for the current usable release version.
>
> Meanwhile, with fewer resources, core goes its different direction,
> choosing its own solutions to many of the same problems. And so the
> contrib-core gap increases.
>
> Even when a corporate-sponsored project is specifically designed to
> contribute to core, the results can be mixed at best. Take the
> internationalization work CivicActions did, sponsored by Sony Music,  
> see
> http://drupal.org/node/383954. We worked just as hard or harder on
> numerous core fixes and enhancements as we did on work in contrib. But
> at the end of the six or seven months, when pretty much every
> significant patch had been accepted and applied in dozens of contrib
> modules (Views, CCK, Flag, Voting API, Internationalization, Date,
> etc.), practically none of our core patches had got in.
>
> I conclude:
>
> 1. More than tweaks and further exhortations to just do more  
> reviewing,
> we probably need to consider some major changes.
> 2. Any changes should increase the ease and incentive of contributing
> major improvements to core as part of the regular development work.
>
> Here are some concrete ideas:
>
> 1. Expand our team of core committers.
>
> Since 4.7 we have had a single maintainer per major version in  
> addition
> to the permanent core committers (now reduced to one). We could  
> consider
> changing this. Possible changes:
>
> a. Each core committer either becomes a permanent core committer or is
> maintained for two or three versions instead of one. E.g., while Angie
> is the primary maintainer of 7.x, some or all of Gerhard, Neil and  
> Gabor
> retain the ability to accept and apply patches for HEAD.
>
> b. Two or three core committers are appointed per major release  
> instead
> of just one.
>
> c. The maintainers listed at
> http://cvs.drupal.org/viewvc.py/drupal/drupal/MAINTAINERS.txt?view=co
> are given permanent commit access to be used only in the areas they  
> are
> responsible for.
>
> 2. Shorten our release cycle. Instead of trying to fit many major API
> changes and improvements into each major release, we instead limit
> ourselves to a focused few changes and a much shorter timeline. This
> quickens the pace of change, reducing the contrib-core gap.
>
> 3. Rather than always limiting new functionality to the new version,  
> we
> accept specific, break-nothing backports in core of new functionality
> and APIs. E.g., we have two branches of each major version, one a
> feature-frozen branch (what we have now) and the other one (yes,
> somewhat less stable) that accepts selected new APIs and features that
> have been added already to HEAD. This allows contrib modules to use  
> the
> new APIs in the current version.
>
> I know that all of these potential changes - and others that I hope
> others will come up with - would have potential pitfalls and  
> disasters.
> But I think these are the sorts of discussions we should be having.
> We've recognized a long-standing issue. We need some creative  
> thinking,
> drawing on our strengths, to consider afresh how to keep core
> development accessible, rewarding, and effective.
>
> Nedjo