[development] Very concerned over Drupal's core development

[Matt Farina]

A few idea (I'm flinging IT at the wall to see what sticks):

1) Building on what Angie wrote about teams. We could put some time  
into tools to help people build teams. g.d.o helps with the tools. We  
could see what's missing and bring the tools there to help form the  
teams. Maybe come up with common solutions to making successful teams  
and sharing them. Maybe a guide to team building. It may seem basic  
but a guide can help people who don't know how.

2) The complex systems (like filters, menus, etc.) can be scary to  
people. Even really smart and capable developers. We could create  
training and teaching to explain how these work and why these work the  
way they do. The caching systems interaction with filters has caused  
more than a few questions, arguments, and violent beatings on drupal  
code. Tools to lower the barrier to entry to understand the more  
complex systems can invite people to learn them and learn how to code  
better raising the bar overall.

3) dww started to work on a distribution system for drupal some time  
ago. A package with an install profile, drupal core, modules, themes,  
etc. Imagine drupal.org creating distributions. There could be one for  
blogging, a video site, a knowledge management system, etc. We could  
be less worried about what core puts out for people to use and let it  
be a tool that things are built upon.

The concern I've had is more along the lines of what chx noted. Go  
back and read it. It's not the committing of patches. It's the people  
with expertise in different areas working on and reviewing patches.  
For all the hundreds of people working on core there are areas we need  
more people qualified to work on the tough stuff, more people  
reviewing patches, and more people crafting them to be core worthy.  
More committers doesn't help with this part of the process (though  
that may eventually become a bottleneck).

Additionally, there is the time aspect. Core development isn't a high  
priority for many people. How can we change that? If it's a priority  
people will find to be involved.

Matt

On Apr 20, 2009, at 1:25 PM, Angela Byron wrote:

>
> On 20-Apr-09, at 11:35 AM, Nedjo Rogers wrote:
>
>> <snip>
>> c. The maintainers listed at
>> http://cvs.drupal.org/viewvc.py/drupal/drupal/MAINTAINERS.txt?view=co
>> are given permanent commit access to be used only in the areas they  
>> are
>> responsible for.
>
> Let's examine the consequences of this, since it's always a popular  
> option when it comes up. I'm going to choose the menu system as a  
> random example, but you can easily substitute  'menu system' as  
> $subsystem and 'chx and pwolanin' as $subsystem_maintainers in the  
> following narrative.
>
> The menu system is a bit tweaky and its innards not easily  
> understood by the "common developer" in the Drupal community. As a  
> result, patches that improve the menu system tend to languish in the  
> queue in "code needs review" for weeks or months. When a review does  
> happen, it's normally on more minor things like coding style or  
> similar; not about the fundamental architecture of the menu system.  
> This is extremely frustrating to chx and pwolanin, the menu system  
> maintainers, since they want to keep going with their crazy  
> development ideas and are blocked.
>
> So, to resolve the issue of menu system patches languishing in the  
> queue, chx and pwolanin are granted commit access to the menu  
> system. Suddenly, all of our problems with languishing patches  
> magically disappear! After a quick discussion in #drupal-dev, patch  
> after patch that improves the menu system is committed. The menu  
> system has never evolved so quickly in such a short period of time!  
> And better still, the committers for the field API, the database  
> system, actions/triggers, etc. are all empowered to do so as well,  
> so Drupal evolves at lightning speed! WIN!
>
> Of course, without any kind of peer oversight, while these patches  
> that get committed make perfect sense to chx and pwolanin, they  
> completely mystify the "common developer." When Drupal 7 is  
> released, it becomes painfully obvious that we've developed  
> extremely well-engineered subsystems without ever exposing a "real"  
> human developer to having to work with them. And since these patches  
> only needed to make sense to two subsystem maintainers with intimate  
> knowledge of the underlying subsystem already, documentation is  
> extremely sparse. It's now way harder than ever before to jump in  
> and get involved in core development.
>
> Worse, because all of these subsystems were rapidly evolving within  
> their own sphere, 20, 30, perhaps 50 patches committed per day, it's  
> become impossible for anyone to oversee all of them; at best you  
> have people inspecting one-off patches here and there. That means  
> things like coding standards compliance, performance benchmarking,  
> algorithm optimization, and the collective DX (not to mention UX)  
> experience of Drupal 7 plummets. Drupal 7 is usable by chx,  
> pwolanin, DamZ, catch, and about 20 other people, and everyone else  
> switches to Django. ;)
>
> We have this kind of decentralized development model, where one or  
> two people are solely responsible for code with basically zero peer  
> review. It's called contrib. And it's notoriously filled with sub- 
> standard, shoddily documented code that needs to be closely  
> inspected by individual site maintainers before being deployed on  
> any serious production sites.
>
> The entire strength of Drupal core *is* that it is peer reviewed  
> (sometimes mercilessly so :)), it has oversight from others who are  
> *not* deeply immersed in APIs who can give sanity checks, and has  
> people who are bringing a very *diverse* range of experience levels,  
> specializations, and use cases. You remove that, you remove what I  
> would argue is the very heart and soul of Drupal.
>
> "But webchick!" I hear you say. "Obviously, we wouldn't let these  
> subsystem maintainers work /completely/ bereft of peer review. We'd  
> surely have someone else check over their work." And I say, "Great!  
> Then *why aren't these mythical someones doing this right now*?" and  
> *then* you get at what I believe is the /actual/ root of the  
> problem. ;)
>
> *We need to grow the base of people who have understanding in these  
> subsystems.*
>
> People who consider themselves subsystem maintainers should be  
> actively working to build teams around their subsystem of choice.  
> They should be documenting how their underlying APIs work so that  
> someone without familiarity could jump in and get oriented quickly.  
> They should be recruiting from contrib authors who are doing  
> interesting/innovative work in their areas. They should be  
> maintaining a "patch spotlight" page under http://drupal.org/community-initiatives/drupal-core 
>  so that people who want to jump in and don't know where to start  
> have a direction. They should be guiding and mentoring "younger"  
> contributors on how to do patch reviews, and actively advocating for  
> others to do so as well.
>
> If subsystem maintainers do this, the collective IQ of the Drupal  
> development community goes up, we grow the larger pool of  
> contributors, and no patch gets left as code needs review. Then you  
> can come back to me when the RTBC queue is 6 pages long (as opposed  
> to cleared down to zero about once every couple of months as it has  
> been) and say "webchick, we need subsystem maintainers to have  
> commit rights, because these patches have all been thoroughly peer  
> reviewed and are languishing at RTBC." and I will probably back you  
> up 110%.
>
> -Angie
>