Very concerned over Drupal's core development
Hi, I know there are many patches are written, reviewed and committed still. But, without intending to critique of anyone in particular, I see a very dangerous cycle: As Drupal 5 and 6 are very good, rock solid releases, people are satisfied and very busy there, much less is the need, the drive and the availability for core development. This means less patches are written. Also core got bigger. Very, very few people touch actions/triggers, fields, filter system, openid or for that matter, menu just to name a few. Once a patch is written , it needs a review. However, the testing bot, curiously, made patch reviews even harder -- no longer can a novice just install a patch, click around and report back "works" ( the testing bot, of course added a lot of quality to Drupal, I am not saying, down with the bot). This has very severe consequences: we always had too few reviewers and now the entry barrier is even harder as you need to do meaningful code reviews all the time. Low reviewer activity means patches do not get 'bumped' and they linger. Lingering is exacerbated by the loss of Steven and that the single guy who can call the big decisions now has a kid (soon two) and two companies to run. Of course, this leads to frustration on behalf of the patch writers and even less patches get written or reviewed and people draw back into their own little contrib empire where they call the shots... Regards Karoly Negyesi
Hi chx, I completely agree. However, how do you suggest we make this better? Dmitri On Sun, Apr 19, 2009 at 1:38 PM, Karoly Negyesi <karoly@negyesi.net> wrote:
Hi,
I know there are many patches are written, reviewed and committed still. But, without intending to critique of anyone in particular, I see a very dangerous cycle:
As Drupal 5 and 6 are very good, rock solid releases, people are satisfied and very busy there, much less is the need, the drive and the availability for core development. This means less patches are written. Also core got bigger. Very, very few people touch actions/triggers, fields, filter system, openid or for that matter, menu just to name a few. Once a patch is written , it needs a review. However, the testing bot, curiously, made patch reviews even harder -- no longer can a novice just install a patch, click around and report back "works" ( the testing bot, of course added a lot of quality to Drupal, I am not saying, down with the bot). This has very severe consequences: we always had too few reviewers and now the entry barrier is even harder as you need to do meaningful code reviews all the time. Low reviewer activity means patches do not get 'bumped' and they linger. Lingering is exacerbated by the loss of Steven and that the single guy who can call the big decisions now has a kid (soon two) and two companies to run. Of course, this leads to frustration on behalf of the patch writers and even less patches get written or reviewed and people draw back into their own little contrib empire where they call the shots...
Regards
Karoly Negyesi
<karoly@negyesi.net> wrote:
As Drupal 5 and 6 are very good, rock solid releases, people are satisfied and very busy there, much less is the need, the drive and the availability for core development. This means less patches are written. Also core got bigger. Very, very few people touch actions/triggers, fields, filter system, openid or for that matter, menu just to name a few. Once a patch is written , it needs a review.
I completely agree. However, how do you suggest we make this better?
Dmitri
I completely agree as well. Core is way ahead of contrib and far ahead of actual implementations. Only a minority of contrib modules has really adopted all new features of D6 thus far. There is an obvious gap between possible implementations and actual implementations of core features in modules. This also explains the fact that only a few contrib developers have the energy to work on improvements to Drupal core that would actually improve our APIs for contrib. The answer to solve this is simple, but somehow disliked by some: Let it mature. i.e. defer the code freeze for D7. I know that I'm not the only one who would prefer that (but capitulated in disagreement). sun
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Daniel F. Kudwien schrieb:
I completely agree as well. Core is way ahead of contrib and far ahead of actual implementations. Only a minority of contrib modules has really adopted all new features of D6 thus far. There is an obvious gap between possible implementations and actual implementations of core features in modules.
This also explains the fact that only a few contrib developers have the energy to work on improvements to Drupal core that would actually improve our APIs for contrib.
The answer to solve this is simple, but somehow disliked by some:
Let it mature.
i.e. defer the code freeze for D7. I know that I'm not the only one who would prefer that (but capitulated in disagreement).
I don't think you are making much sense here. You claim that module developers haven't found time to add the new D6 features in their code and at the same time want to have a D7 which is more mature (and presumably thus contains even more features). In the end your proposal would only serve to disconnect core and contrib even more. Cheers, Ger»release early, release often«hard -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAknsexQACgkQfg6TFvELooTtRQCfZ6yHwd4VH0fvWXnso5r7XOVS pN0An1T7yjRF7+bybYCXFxkDSS/WWQx4 =Mguz -----END PGP SIGNATURE-----
I don't think you are making much sense here. You claim that module developers haven't found time to add the new D6 features in their code and at the same time want to have a D7 which is more mature (and presumably thus contains even more features).
In the end your proposal would only serve to disconnect core and contrib even more.
Cheers, Ger»release early, release often«hard
In case this really was ambigious: Let it [D6] mature. => i.e. defer the code freeze for D7. Obviously, I mean the exact opposite of what you understood. Contrib needs time to catch up with core. sun
lol yeah, I believe the intent of the original author was to hold off on D7 until D6 matured more from a crontrib standpoint. I'll +1 that idea. D5 was and still is a very mature platform that you can develop and build off of, seemingly without "need" to upgrade if you have a site up and running stablly; this isn't always the case with 6 with some of the more major projects just getting to full releases in the last 6 months. I think once D6 stablizes and you start to feel that clamoring for D7 is the time to move to it. I know there's a push to keep the community / versioning alive and active since it's a very deliver or die out in the open source world and we want to stay on top of over big projects in our space but having a full functional D6 that people are adopting the APIs / features of and THEN working towards 7 is probably a better idea. I've started to notice a bunch of dev / alpha D7 module ports too; maybe having versions of D7 released and keeping it in the alpha phase longer (once it reaches that point) will encourage developers to play around with it and start porting things over in the interum. No timeline, no "this is the date we move", a more organic window to maturation much like the nature of open source in the first place... Or get it out tomorrow; I'll end up using it either way ;) On Mon, Apr 20, 2009 at 9:58 AM, Daniel F. Kudwien <news@unleashedmind.com>wrote:
I don't think you are making much sense here. You claim that module developers haven't found time to add the new D6 features in their code and at the same time want to have a D7 which is more mature (and presumably thus contains even more features).
In the end your proposal would only serve to disconnect core and contrib even more.
Cheers, Ger»release early, release often«hard
In case this really was ambigious:
Let it [D6] mature.
=> i.e. defer the code freeze for D7.
Obviously, I mean the exact opposite of what you understood. Contrib needs time to catch up with core.
sun
On Mon, Apr 20, 2009 at 7:06 AM, Bryan Ollendyke <bto108@psu.edu> wrote:
lol yeah, I believe the intent of the original author was to hold off on D7 until D6 matured more from a crontrib standpoint. I'll +1 that idea.
I had concerns and I shared that with the community. I had no solutions. I stil do not have. I am not sure postponing Drupal 7 even further is the solution. Because then, as Gerhard says, what will you do when Drupal 7 gets out, send core development on vacation for two years?
From: Karoly Negyesi
I had concerns and I shared that with the community. I had no solutions. I stil do not have. I am not sure postponing Drupal 7 even further is the solution. Because then, as Gerhard says, what will you do when Drupal 7 gets out, send core development on vacation for two years?
Well, I asked myself the very same questions like you:
This means less patches are written. [why?] Very, very few people touch actions/triggers, fields, filter system, openid or for that matter, menu just to name a few. [why?]
The most obvious that crosses my mind is: Most developers did not really work with the current feature-set and APIs, they did not experience (and suffer from) current limitations, and have no clear idea or vision of how Drupal *should* really work. Once this prerequisite is there, we will see much more interest and traction in core development. Releasing yet another new API, developers have to catch up with, means that it will take them even longer to get into core development business. sun
Karoly Negyesi wrote:
what will you do when Drupal 7 gets out, send core development on vacation for two years? Hmm, I wonder what would happen if we all realized that every member of DO is a "core developer?" There may be a few people who are more or less directly responsible for consolidating proposed changes, but it is the community at large that is responsible for making those proposals - and patches. Even though we sometimes refer to those privileged (burdened) few as "Drupal-gods," they are not perfect; they cannot predict every way someone will use the code, nor can they test everything all together the way we put them together. Those people have real lives too (I think), and have to do the same thing we do: fit in time for Drupal amidst all our other demands. I don't always agree with everything they do or say - I dare say chx is one of them who knows that already. But Drupal is a great system, and it will stay that way only if we all think of ourselves as core developers.
No virus found in this outgoing message. Checked by AVG - http://www.avg.com Version: 8.0.176 / Virus Database: 270.12.0/2068 - Release Date: 4/19/2009 8:04 PM
Karoly Negyesi wrote:
On Mon, Apr 20, 2009 at 7:06 AM, Bryan Ollendyke <bto108@psu.edu> wrote:
I had concerns and I shared that with the community. I had no solutions. I stil do not have. I am not sure postponing Drupal 7 even further is the solution. I agree that bottlenecks in the reviewing and applying of patches is a huge problem majorly holding up Drupal development. I suspect we need some significant structural changes to address it.
Partly I think we need to look at changes in the drivers of Drupal development. There was a time when it was mainly hobbyists, who weren't particularly tied to external timelines. No longer. The biggest amount of focused development time now comes from medium to large companies needing specific functionality within set timelines. The typical development cycle for a large project (e.g., a major site or multisite installation) is a matter of months. Development can't wait the many months or even years before a new version is ready and can't deal with the uncertainty of fluxuating timelines. So the significant work they sponsor is almost never for core, which is years ahead of what can be used. It's instead for the current usable release version. Meanwhile, with fewer resources, core goes its different direction, choosing its own solutions to many of the same problems. And so the contrib-core gap increases. Even when a corporate-sponsored project is specifically designed to contribute to core, the results can be mixed at best. Take the internationalization work CivicActions did, sponsored by Sony Music, see http://drupal.org/node/383954. We worked just as hard or harder on numerous core fixes and enhancements as we did on work in contrib. But at the end of the six or seven months, when pretty much every significant patch had been accepted and applied in dozens of contrib modules (Views, CCK, Flag, Voting API, Internationalization, Date, etc.), practically none of our core patches had got in. I conclude: 1. More than tweaks and further exhortations to just do more reviewing, we probably need to consider some major changes. 2. Any changes should increase the ease and incentive of contributing major improvements to core as part of the regular development work. Here are some concrete ideas: 1. Expand our team of core committers. Since 4.7 we have had a single maintainer per major version in addition to the permanent core committers (now reduced to one). We could consider changing this. Possible changes: a. Each core committer either becomes a permanent core committer or is maintained for two or three versions instead of one. E.g., while Angie is the primary maintainer of 7.x, some or all of Gerhard, Neil and Gabor retain the ability to accept and apply patches for HEAD. b. Two or three core committers are appointed per major release instead of just one. c. The maintainers listed at http://cvs.drupal.org/viewvc.py/drupal/drupal/MAINTAINERS.txt?view=co are given permanent commit access to be used only in the areas they are responsible for. 2. Shorten our release cycle. Instead of trying to fit many major API changes and improvements into each major release, we instead limit ourselves to a focused few changes and a much shorter timeline. This quickens the pace of change, reducing the contrib-core gap. 3. Rather than always limiting new functionality to the new version, we accept specific, break-nothing backports in core of new functionality and APIs. E.g., we have two branches of each major version, one a feature-frozen branch (what we have now) and the other one (yes, somewhat less stable) that accepts selected new APIs and features that have been added already to HEAD. This allows contrib modules to use the new APIs in the current version. I know that all of these potential changes - and others that I hope others will come up with - would have potential pitfalls and disasters. But I think these are the sorts of discussions we should be having. We've recognized a long-standing issue. We need some creative thinking, drawing on our strengths, to consider afresh how to keep core development accessible, rewarding, and effective. Nedjo
Nedjo has given us the first response to chx's origninal problem statement that addresses the real intent of the concern, and proposes solutions. Of the three solutions Nedjo proposed, I favor c) giving targeted commit rights to people who have historically been the maintainers for certain subsystems. We would need to develop guidelines around where their turf ends and they are no longer free to commit without escalating to either the branch maintainer or a permanent core maintainer (Dries), but this is a feasible solution that could speed the development of certain subsystems (actions, triggers, openID, i18n). The other solution which chx and I have discussed, and which is a long time goal of mine, would be to reduce the size of core. We carry around a lot of modules that don't absolutely need to be part of core: - aggregator - blog - blogapi - forum - help - poll - profile - search - statistics - taxonomy - tracker I believe that all of these modules could have brilliant lives outside of core. A slimmer core means more focused development for the core team. It would also make us focus our attention on the APIs that support these very important modules. For example, the search API is in great need of being decoupled from the node module. If our goal were to remove search from core we'd need to really make sure the API were in place for letting core be searched. My argument squarely reflects opinion that "core" Drupal should be an application framework that doesn't offer any specialized functionality. Forums, polls, profiles, blogs and even taxonomy are specialized web application functions that don't need the 10,000 layers of red tape that core has. I firmly believe that taxonomy, for example, is hampered by its presence in core. There has been amazing work done on turning taxonomy into something far beyond what we have today, but the very presence of taxonomy IN CORE means that nobody uses it. Same with search. When I first wrote the ApacheSolr module it utilized as much of the search "API" as possible. Now we're moving to make it completely independent of core search because there is no way core can keep up with our development pace. So, in my opinion, if you want to take Drupal to the moon, you do two things: 1) give more people the keys to the car, and 2) shed a bunch of the pieces that we currently call "core". Robert Douglass The RobsHouse.net Newsletter: http://robshouse.net/newsletter/robshousenet-newsletter Follow me on Twitter: http://twitter.com/robertDouglass On Apr 20, 2009, at 5:35 PM, Nedjo Rogers wrote:
Karoly Negyesi wrote:
On Mon, Apr 20, 2009 at 7:06 AM, Bryan Ollendyke <bto108@psu.edu> wrote:
I had concerns and I shared that with the community. I had no solutions. I stil do not have. I am not sure postponing Drupal 7 even further is the solution. I agree that bottlenecks in the reviewing and applying of patches is a huge problem majorly holding up Drupal development. I suspect we need some significant structural changes to address it.
Partly I think we need to look at changes in the drivers of Drupal development.
There was a time when it was mainly hobbyists, who weren't particularly tied to external timelines. No longer. The biggest amount of focused development time now comes from medium to large companies needing specific functionality within set timelines. The typical development cycle for a large project (e.g., a major site or multisite installation) is a matter of months. Development can't wait the many months or even years before a new version is ready and can't deal with the uncertainty of fluxuating timelines. So the significant work they sponsor is almost never for core, which is years ahead of what can be used. It's instead for the current usable release version.
Meanwhile, with fewer resources, core goes its different direction, choosing its own solutions to many of the same problems. And so the contrib-core gap increases.
Even when a corporate-sponsored project is specifically designed to contribute to core, the results can be mixed at best. Take the internationalization work CivicActions did, sponsored by Sony Music, see http://drupal.org/node/383954. We worked just as hard or harder on numerous core fixes and enhancements as we did on work in contrib. But at the end of the six or seven months, when pretty much every significant patch had been accepted and applied in dozens of contrib modules (Views, CCK, Flag, Voting API, Internationalization, Date, etc.), practically none of our core patches had got in.
I conclude:
1. More than tweaks and further exhortations to just do more reviewing, we probably need to consider some major changes. 2. Any changes should increase the ease and incentive of contributing major improvements to core as part of the regular development work.
Here are some concrete ideas:
1. Expand our team of core committers.
Since 4.7 we have had a single maintainer per major version in addition to the permanent core committers (now reduced to one). We could consider changing this. Possible changes:
a. Each core committer either becomes a permanent core committer or is maintained for two or three versions instead of one. E.g., while Angie is the primary maintainer of 7.x, some or all of Gerhard, Neil and Gabor retain the ability to accept and apply patches for HEAD.
b. Two or three core committers are appointed per major release instead of just one.
c. The maintainers listed at http://cvs.drupal.org/viewvc.py/drupal/drupal/MAINTAINERS.txt?view=co are given permanent commit access to be used only in the areas they are responsible for.
2. Shorten our release cycle. Instead of trying to fit many major API changes and improvements into each major release, we instead limit ourselves to a focused few changes and a much shorter timeline. This quickens the pace of change, reducing the contrib-core gap.
3. Rather than always limiting new functionality to the new version, we accept specific, break-nothing backports in core of new functionality and APIs. E.g., we have two branches of each major version, one a feature-frozen branch (what we have now) and the other one (yes, somewhat less stable) that accepts selected new APIs and features that have been added already to HEAD. This allows contrib modules to use the new APIs in the current version.
I know that all of these potential changes - and others that I hope others will come up with - would have potential pitfalls and disasters. But I think these are the sorts of discussions we should be having. We've recognized a long-standing issue. We need some creative thinking, drawing on our strengths, to consider afresh how to keep core development accessible, rewarding, and effective.
Nedjo
On Mon, Apr 20, 2009 at 6:03 PM, Robert Douglass <rob@robshouse.net> wrote:
So, in my opinion, if you want to take Drupal to the moon, you do two things: 1) give more people the keys to the car, and 2) shed a bunch of the pieces that we currently call "core".
Would that solve our other problem: that many contrib modules got updates later, so it turns out after a stable release that certain APIs are not complete or suitable for important contrib stuff? If we move more of the modules which use these APIs outside of core, we go to pure API design without validation of those APIs with real life modules, no? While I agree some modules would be best to go, maybe moving all is not the solution. Poll was joked to be one of the best API validation modules (for drag and drop, AHAH, etc). While that was a joke to some degree, it also has roots in reality. Gábor
On Apr 20, 2009, at 6:13 PM, Gábor Hojtsy wrote:
On Mon, Apr 20, 2009 at 6:03 PM, Robert Douglass <rob@robshouse.net> wrote:
So, in my opinion, if you want to take Drupal to the moon, you do two things: 1) give more people the keys to the car, and 2) shed a bunch of the pieces that we currently call "core".
Would that solve our other problem: that many contrib modules got updates later, so it turns out after a stable release that certain APIs are not complete or suitable for important contrib stuff? If we move more of the modules which use these APIs outside of core, we go to pure API design without validation of those APIs with real life modules, no? While I agree some modules would be best to go, maybe moving all is not the solution. Poll was joked to be one of the best API validation modules (for drag and drop, AHAH, etc). While that was a joke to some degree, it also has roots in reality.
Gábor
I agree that your concerns are valid, but I think that solving those concerns would be easier than solving the larger core concerns raised by chx if we continue to follow the exact strategy that we have now. API validation is vitally important, and should be driven by contrib. Contrib should be instructing core "hey, I need this in the API so that I can do what I want". Contrib module maintainers currently submit patches to core to tend to the important APIs that affect their modules. Witness Earl Mile's recent patch that has major ramifications on future Views development. http://drupal.org/node/322344 Contrib modules are real modules. They do a fine job of finding the limitations and strengths of core's APIs. The feedback loop from contrib to core is there, and it is strong. In one direction. Most of that feedback currently sits idle in the core issue queue, however, and contrib module authors often choose to just work around core instead of enhancing it. Poll, despite being "core's regression test", is the best example. The only time anyone pays any attention to poll module development is when other people scream "get poll out of core!" loudly enough :P These days, if we need regression tests, we should put them in the Simpletest framework. Another solution would be to split core into two parts. "Framework", and "CMS". The framework would be the things like user, node, block, system, actions, fields, etc., and the CMS would be things like poll, blog, forum and who knows what. The smart thing would be to make "Framework" in this case as small as possible, and push as much as we can into "Framework". Then "Framework" could adopt a nice, long release cycle and focus on big API changes, whereas "CMS" could focus on shorter release cycles upon any given "Framework" release. "CMS" would get a new set of "core" committers.
On Mon, Apr 20, 2009 at 10:40 AM, Robert Douglass <rob@robshouse.net> wrote:
API validation is vitally important, and should be driven by contrib. Contrib should be instructing core "hey, I need this in the API so that I can do what I want". Contrib module maintainers currently submit patches to core to tend to the important APIs that affect their modules. Witness Earl Mile's recent patch that has major ramifications on future Views development. http://drupal.org/node/322344
Contrib modules are real modules. They do a fine job of finding the limitations and strengths of core's APIs. The feedback loop from contrib to core is there, and it is strong. In one direction. Most of that feedback currently sits idle in the core issue queue, however, and contrib module authors often choose to just work around core instead of enhancing it.
I think contrib authors submitting back core patches is really the exception. I know it too me two full versions of working around a crappy file APIs before I finally decided to wade in an experience the utter frustration that is core development. And even then it was two versions before things really started to improve. Someone at DCDC described core development as not a meritocracy but a who's-got-more-free-time-ocracy and as Nedjo identified most corporate sponsored development doesn't come with that free time leading to the idle feedback you identified.
Another solution would be to split core into two parts. "Framework", and "CMS". The framework would be the things like user, node, block, system, actions, fields, etc., and the CMS would be things like poll, blog, forum and who knows what. The smart thing would be to make "Framework" in this case as small as possible, and push as much as we can into "Framework". Then "Framework" could adopt a nice, long release cycle and focus on big API changes, whereas "CMS" could focus on shorter release cycles upon any given "Framework" release.
I like this idea but it sort of goes into the whole "golden" module list that seems to come up ever now and again. andrew
Another solution would be to split core into two parts. "Framework", and "CMS". The framework would be the things like user, node, block, system, actions, fields, etc., and the CMS would be things like poll, blog, forum and who knows what. The smart thing would be to make "Framework" in this case as small as possible, and push as much as we can into "Framework". Then "Framework" could adopt a nice, long release cycle and focus on big API changes, whereas "CMS" could focus on shorter release cycles upon any given "Framework" release.
I like this idea but it sort of goes into the whole "golden" module list that seems to come up ever now and again.
andrew
I think the problem is less the number of modules we have in core (mainly CMS-ish ones), and much more that the ones we have in core are only there for arbitrary / historical reasons. A patch to add forum, blog, help, poll, tracker, taxonomy to core now would be almost immediately won't fixed or stay at CNW for a very long time requiring years of work to get right - either because there are better contrib alternatives, or because the code is crappy, or both. However taking them out requires an upgrade path and a responsible contrib maintainer, not to mention consensus, and getting all three of those at the same time is hard too. On the other hand, getting eternally useful modules like token, CCK, Views etc. into core is a very long and drawn out process too - for good reason - if only in part because we know that once they're in core it's very hard to take them out, and any wrong design implications have a much longer lifespan and are harder to fix (hence the main query in tracker module which was bringing Drupal.org down regularly on Drupal 5 still hasn't been fixed in core for three releases). Packaging and better install profiles might help, getting those major APIs into core will hopefully help us push the old-style modules out of core eventually as well - i.e. replace tracker.module with a couple of default views. I don't think this is the root of the various recurring frustrations with core development though at all - either those of us very active in the core issue queue or those who avoid it like the plague. In my opinion it's a combination of factors, most very hard to fix - hence threads like this and discussions in #drupal on a regular basis. Nat
I would strongly agree with chx and others that in some way we need additional committers. I think that it does make sense for Dries to be providing overall guidance, but I think it would be helpful if in some/many cases Dries would lay out criteria for a proposed change to be accepted and then delegate review and commit implementation details to others. A few of the modules Robert mention (maybe blog, tracker, and help) should either go or get a serious revamping. I general, however, many of the module in core serve as reference implementations of certain APIs and allow you to build a reasonable little site with just core - there can and should be other or more feature-rich implementations in contrib. -Peter On Mon, Apr 20, 2009 at 12:13 PM, Gábor Hojtsy <gabor@hojtsy.hu> wrote:
On Mon, Apr 20, 2009 at 6:03 PM, Robert Douglass <rob@robshouse.net> wrote:
So, in my opinion, if you want to take Drupal to the moon, you do two things: 1) give more people the keys to the car, and 2) shed a bunch of the pieces that we currently call "core".
Would that solve our other problem: that many contrib modules got updates later, so it turns out after a stable release that certain APIs are not complete or suitable for important contrib stuff? If we move more of the modules which use these APIs outside of core, we go to pure API design without validation of those APIs with real life modules, no? While I agree some modules would be best to go, maybe moving all is not the solution. Poll was joked to be one of the best API validation modules (for drag and drop, AHAH, etc). While that was a joke to some degree, it also has roots in reality.
Gábor
On Mon, Apr 20, 2009 at 10:06 AM, Peter Wolanin <pwolanin@gmail.com> wrote:
I would strongly agree with chx and others that in some way we need additional committers.
You can't agree with me because I have not said so. Check the thread again :) I have not suggested anything and will not (for some time at least). I deliberately avoid taking sides and rather play devil's advocate. Do not presume I agree or disagree with any idea presented in here. The rest of this mail will be reflections on many of the things were raised in here. It really must be noted that the Drupal 6 cycle was unusual because major contribs decided they do not want to maintain old crap and did a complete refactoring and started that only once D6 was out the door. This, hopefully, won't repeat so in the future contribs will be ready somewhat sooner. This does not help the contrib-core gap, of course. One of the reasons I left contrib is that my perfectionist half could not stand the crappy code my programmer half produced. I can't write pretty code without peer review. If you think you can, my best advice is to think again. Really, peer review is almost mandatory. Simply allowing people to commit patches will not help our code quality. Really, check most contrib vs core. I started with a post which showed how a shortage of patch writers, reviewers and committers create a self strenghening cycle. Making a half-assed fix at committers is not enough to fix the whole cycle. Slimming down core is an idea but as was noted we do we need them to validate. Also, if you slim down core and thus speed up development, how again will that help the contrib-core gap? Now, Drupal shops. Sure, you can't sell to a client directly core development but what about this, say, you would ask for 100/hr on a project which takes 400 hrs (less hree man-months, not that much). If you ask 110/hr instead then that client sponsored 40 hrs of core work, too. On the other hand if you can get away with 110/hr then why not just bill that and go to the beach?
On Mon, Apr 20, 2009 at 10:03 AM, Robert Douglass <rob@robshouse.net> wrote:
The other solution which chx and I have discussed, and which is a long time goal of mine, would be to reduce the size of core. We carry around a lot of modules that don't absolutely need to be part of core:
While there might be specific modules that would be better off in contrib, on the whole, I think we need to have code in core that actually uses our APIs. Converting core modules to use a new API is one of the first and most informative tests the API will get. And importantly it happens while the original developers are still engaged and working with the code rather than months later when no one remembers exactly why a decision was made. I've heard poll.module referred to as the canary of the FormsAPI because it helps spot subtle flaws in the API that might have gone unnoticed for months after core ships. Back in the D5 beta days I tried to keep the audio module updated for HEAD to help test it but HEAD was insanely unstable. These days, due to the unittests, HEAD is the most stable I've seen since I stared with 4.6 but most of my modules are have enough dependencies on other contrib modules that it would be impossible to keep them updated. The one success I've had in this regard was back porting the fixes that quicksketch did moving the ImageAPI module into core (http://drupal.org/node/373613) back to the contrib module. There's been one core bug fixed and another identified. I'm hoping something similar comes from the push to get ImageCache into core (http://drupal.org/node/371374). So while I'm all for reviewing the list of modules in core and dropping some I think we'd be well served to have several higher level modules that exist solely to help smoke test our core APIs. andrew
Notwithstanding the fact that modules shed from core tend to die, I think some of the modules you mention could indeed be made contrib, because their are "application"-type (like aggregator, forum, or poll), whereas one (taxonomy) is what really defines Drupal to many people outside the coder group, and others (search, statistics, ....profile(!) ) are part of anyone's basic expectations in a CMS. ----- Original Message ----- From: "Robert Douglass" <rob@robshouse.net> To: <development@drupal.org> Sent: Monday, April 20, 2009 6:03 PM Subject: Re: [development] Very concerned over Drupal's core development [...] The other solution which chx and I have discussed, and which is a long time goal of mine, would be to reduce the size of core. We carry around a lot of modules that don't absolutely need to be part of core: - aggregator - blog - blogapi - forum - help - poll - profile - search - statistics - taxonomy - tracker I believe that all of these modules could have brilliant lives outside of core. A slimmer core means more focused development for the core team. It would also make us focus our attention on the APIs that support these very important modules. For example, the search API is in great need of being decoupled from the node module. If our goal were to remove search from core we'd need to really make sure the API were in place for letting core be searched. [...]
On 20-Apr-09, at 11:35 AM, Nedjo Rogers wrote:
<snip> c. The maintainers listed at http://cvs.drupal.org/viewvc.py/drupal/drupal/MAINTAINERS.txt?view=co are given permanent commit access to be used only in the areas they are responsible for.
Let's examine the consequences of this, since it's always a popular option when it comes up. I'm going to choose the menu system as a random example, but you can easily substitute 'menu system' as $subsystem and 'chx and pwolanin' as $subsystem_maintainers in the following narrative. The menu system is a bit tweaky and its innards not easily understood by the "common developer" in the Drupal community. As a result, patches that improve the menu system tend to languish in the queue in "code needs review" for weeks or months. When a review does happen, it's normally on more minor things like coding style or similar; not about the fundamental architecture of the menu system. This is extremely frustrating to chx and pwolanin, the menu system maintainers, since they want to keep going with their crazy development ideas and are blocked. So, to resolve the issue of menu system patches languishing in the queue, chx and pwolanin are granted commit access to the menu system. Suddenly, all of our problems with languishing patches magically disappear! After a quick discussion in #drupal-dev, patch after patch that improves the menu system is committed. The menu system has never evolved so quickly in such a short period of time! And better still, the committers for the field API, the database system, actions/ triggers, etc. are all empowered to do so as well, so Drupal evolves at lightning speed! WIN! Of course, without any kind of peer oversight, while these patches that get committed make perfect sense to chx and pwolanin, they completely mystify the "common developer." When Drupal 7 is released, it becomes painfully obvious that we've developed extremely well- engineered subsystems without ever exposing a "real" human developer to having to work with them. And since these patches only needed to make sense to two subsystem maintainers with intimate knowledge of the underlying subsystem already, documentation is extremely sparse. It's now way harder than ever before to jump in and get involved in core development. Worse, because all of these subsystems were rapidly evolving within their own sphere, 20, 30, perhaps 50 patches committed per day, it's become impossible for anyone to oversee all of them; at best you have people inspecting one-off patches here and there. That means things like coding standards compliance, performance benchmarking, algorithm optimization, and the collective DX (not to mention UX) experience of Drupal 7 plummets. Drupal 7 is usable by chx, pwolanin, DamZ, catch, and about 20 other people, and everyone else switches to Django. ;) We have this kind of decentralized development model, where one or two people are solely responsible for code with basically zero peer review. It's called contrib. And it's notoriously filled with sub- standard, shoddily documented code that needs to be closely inspected by individual site maintainers before being deployed on any serious production sites. The entire strength of Drupal core *is* that it is peer reviewed (sometimes mercilessly so :)), it has oversight from others who are *not* deeply immersed in APIs who can give sanity checks, and has people who are bringing a very *diverse* range of experience levels, specializations, and use cases. You remove that, you remove what I would argue is the very heart and soul of Drupal. "But webchick!" I hear you say. "Obviously, we wouldn't let these subsystem maintainers work /completely/ bereft of peer review. We'd surely have someone else check over their work." And I say, "Great! Then *why aren't these mythical someones doing this right now*?" and *then* you get at what I believe is the /actual/ root of the problem. ;) *We need to grow the base of people who have understanding in these subsystems.* People who consider themselves subsystem maintainers should be actively working to build teams around their subsystem of choice. They should be documenting how their underlying APIs work so that someone without familiarity could jump in and get oriented quickly. They should be recruiting from contrib authors who are doing interesting/ innovative work in their areas. They should be maintaining a "patch spotlight" page under http://drupal.org/community-initiatives/drupal-core so that people who want to jump in and don't know where to start have a direction. They should be guiding and mentoring "younger" contributors on how to do patch reviews, and actively advocating for others to do so as well. If subsystem maintainers do this, the collective IQ of the Drupal development community goes up, we grow the larger pool of contributors, and no patch gets left as code needs review. Then you can come back to me when the RTBC queue is 6 pages long (as opposed to cleared down to zero about once every couple of months as it has been) and say "webchick, we need subsystem maintainers to have commit rights, because these patches have all been thoroughly peer reviewed and are languishing at RTBC." and I will probably back you up 110%. -Angie
A few idea (I'm flinging IT at the wall to see what sticks): 1) Building on what Angie wrote about teams. We could put some time into tools to help people build teams. g.d.o helps with the tools. We could see what's missing and bring the tools there to help form the teams. Maybe come up with common solutions to making successful teams and sharing them. Maybe a guide to team building. It may seem basic but a guide can help people who don't know how. 2) The complex systems (like filters, menus, etc.) can be scary to people. Even really smart and capable developers. We could create training and teaching to explain how these work and why these work the way they do. The caching systems interaction with filters has caused more than a few questions, arguments, and violent beatings on drupal code. Tools to lower the barrier to entry to understand the more complex systems can invite people to learn them and learn how to code better raising the bar overall. 3) dww started to work on a distribution system for drupal some time ago. A package with an install profile, drupal core, modules, themes, etc. Imagine drupal.org creating distributions. There could be one for blogging, a video site, a knowledge management system, etc. We could be less worried about what core puts out for people to use and let it be a tool that things are built upon. The concern I've had is more along the lines of what chx noted. Go back and read it. It's not the committing of patches. It's the people with expertise in different areas working on and reviewing patches. For all the hundreds of people working on core there are areas we need more people qualified to work on the tough stuff, more people reviewing patches, and more people crafting them to be core worthy. More committers doesn't help with this part of the process (though that may eventually become a bottleneck). Additionally, there is the time aspect. Core development isn't a high priority for many people. How can we change that? If it's a priority people will find to be involved. Matt On Apr 20, 2009, at 1:25 PM, Angela Byron wrote:
On 20-Apr-09, at 11:35 AM, Nedjo Rogers wrote:
<snip> c. The maintainers listed at http://cvs.drupal.org/viewvc.py/drupal/drupal/MAINTAINERS.txt?view=co are given permanent commit access to be used only in the areas they are responsible for.
Let's examine the consequences of this, since it's always a popular option when it comes up. I'm going to choose the menu system as a random example, but you can easily substitute 'menu system' as $subsystem and 'chx and pwolanin' as $subsystem_maintainers in the following narrative.
The menu system is a bit tweaky and its innards not easily understood by the "common developer" in the Drupal community. As a result, patches that improve the menu system tend to languish in the queue in "code needs review" for weeks or months. When a review does happen, it's normally on more minor things like coding style or similar; not about the fundamental architecture of the menu system. This is extremely frustrating to chx and pwolanin, the menu system maintainers, since they want to keep going with their crazy development ideas and are blocked.
So, to resolve the issue of menu system patches languishing in the queue, chx and pwolanin are granted commit access to the menu system. Suddenly, all of our problems with languishing patches magically disappear! After a quick discussion in #drupal-dev, patch after patch that improves the menu system is committed. The menu system has never evolved so quickly in such a short period of time! And better still, the committers for the field API, the database system, actions/triggers, etc. are all empowered to do so as well, so Drupal evolves at lightning speed! WIN!
Of course, without any kind of peer oversight, while these patches that get committed make perfect sense to chx and pwolanin, they completely mystify the "common developer." When Drupal 7 is released, it becomes painfully obvious that we've developed extremely well-engineered subsystems without ever exposing a "real" human developer to having to work with them. And since these patches only needed to make sense to two subsystem maintainers with intimate knowledge of the underlying subsystem already, documentation is extremely sparse. It's now way harder than ever before to jump in and get involved in core development.
Worse, because all of these subsystems were rapidly evolving within their own sphere, 20, 30, perhaps 50 patches committed per day, it's become impossible for anyone to oversee all of them; at best you have people inspecting one-off patches here and there. That means things like coding standards compliance, performance benchmarking, algorithm optimization, and the collective DX (not to mention UX) experience of Drupal 7 plummets. Drupal 7 is usable by chx, pwolanin, DamZ, catch, and about 20 other people, and everyone else switches to Django. ;)
We have this kind of decentralized development model, where one or two people are solely responsible for code with basically zero peer review. It's called contrib. And it's notoriously filled with sub- standard, shoddily documented code that needs to be closely inspected by individual site maintainers before being deployed on any serious production sites.
The entire strength of Drupal core *is* that it is peer reviewed (sometimes mercilessly so :)), it has oversight from others who are *not* deeply immersed in APIs who can give sanity checks, and has people who are bringing a very *diverse* range of experience levels, specializations, and use cases. You remove that, you remove what I would argue is the very heart and soul of Drupal.
"But webchick!" I hear you say. "Obviously, we wouldn't let these subsystem maintainers work /completely/ bereft of peer review. We'd surely have someone else check over their work." And I say, "Great! Then *why aren't these mythical someones doing this right now*?" and *then* you get at what I believe is the /actual/ root of the problem. ;)
*We need to grow the base of people who have understanding in these subsystems.*
People who consider themselves subsystem maintainers should be actively working to build teams around their subsystem of choice. They should be documenting how their underlying APIs work so that someone without familiarity could jump in and get oriented quickly. They should be recruiting from contrib authors who are doing interesting/innovative work in their areas. They should be maintaining a "patch spotlight" page under http://drupal.org/community-initiatives/drupal-core so that people who want to jump in and don't know where to start have a direction. They should be guiding and mentoring "younger" contributors on how to do patch reviews, and actively advocating for others to do so as well.
If subsystem maintainers do this, the collective IQ of the Drupal development community goes up, we grow the larger pool of contributors, and no patch gets left as code needs review. Then you can come back to me when the RTBC queue is 6 pages long (as opposed to cleared down to zero about once every couple of months as it has been) and say "webchick, we need subsystem maintainers to have commit rights, because these patches have all been thoroughly peer reviewed and are languishing at RTBC." and I will probably back you up 110%.
-Angie
On Mon, Apr 20, 2009 at 8:02 PM, Matt Farina <matt@mattfarina.com> wrote:
The concern I've had is more along the lines of what chx noted. Go back and read it. It's not the committing of patches. It's the people with expertise in different areas working on and reviewing patches. For all the hundreds of people working on core there are areas we need more people qualified to work on the tough stuff, more people reviewing patches, and more people crafting them to be core worthy. More committers doesn't help with this part of the process (though that may eventually become a bottleneck).
This is spot on. It's important that we look at Drupal core development at the macro-level -- "Drupal core is a very healthy project that is making steady progress, but it risks going too fast and becoming too complex for (contrib) developers". Often, contributors, including myself, are thinking at the micro-level -- "my patch is really important, but it doesn't bubble to the top, and it doesn't get committed so let's get grumpy because things are too slow". Giving more people commit access doesn't solve the macro-problem. It seemingly solves the micro-level problem, but at the expense of making the macro-level problem worse. That is key to understand -- and explained by webchick. Matt is spot on because he succeeds at looking at the macro-level. It is what I call a 'community responsibility' to help solve this problem. From doing a better job educating people to fighting complexity. The fact that we have poll module is not a source of complexity because it can be safely ignored 98% of the time. The fact that we have difficult to understand subsystems and increasingly more Drupal-isms is a problem. -- Dries Buytaert :: http://buytaert.net/
From my point of view, this discussion has some real value, but targets other areas that could use improvement.
Braindump: It's not about the speed of patches being committed. It's about core patches lacking review, and core evolving in ways and areas that leave contrib behind. It's about contrib development not being able to catch up with core. Core development is driven by some core developers. Only a few of them being contrib maintainers. But the real applications live in contrib. Contrib developers need to catch up with new core APIs and features before they can think about and work on improving core. But next to catching up with that, they also have to maintain, solve issues, and develop new stuff for their projects/applications. By doing so, they identify areas in core that need improvement. At the time they realize what needs improvement, new core APIs are released, so they start from scratch. Contrib should influence and drive core development. Currently, I experience the opposite. I have to decide whether I want to improve my XYZ module or whether I want to work on core without taking into account what XYZ module would really need in X months when new core is released. If I do both, neither core nor my module is mature. Gotta go. sun
Angela Byron wrote:
We have this kind of decentralized development model, where one or two people are solely responsible for code with basically zero peer review. It's called contrib. And it's notoriously filled with sub-standard, shoddily documented code that needs to be closely inspected by individual site maintainers before being deployed on any serious production sites.
I stop following your argument here. The only way this argument holds up is that if *all* of contrib is substandard crap. It is not. There is a level of contrib that is above that, and there is a reason those particular pieces of contrib are above that. Chew on that.
On 20-Apr-09, at 2:04 PM, Earl Miles wrote:
Angela Byron wrote:
We have this kind of decentralized development model, where one or two people are solely responsible for code with basically zero peer review. It's called contrib. And it's notoriously filled with sub- standard, shoddily documented code that needs to be closely inspected by individual site maintainers before being deployed on any serious production sites.
I stop following your argument here. The only way this argument holds up is that if *all* of contrib is substandard crap. It is not. There is a level of contrib that is above that, and there is a reason those particular pieces of contrib are above that. Chew on that.
And this level contrib is the kind that has enough users so that peer reviews occur naturally, or at the very least peer QA after the fact. As we've noted, peer review and peer QA is *not* happening with these core subsystems, which makes it a lot closer to the "darker, scarier" contrib. So I'm not sure what there is to chew on. :) -Angie
Angela Byron wrote:
On 20-Apr-09, at 2:04 PM, Earl Miles wrote:
Angela Byron wrote:
We have this kind of decentralized development model, where one or two people are solely responsible for code with basically zero peer review. It's called contrib. And it's notoriously filled with sub-standard, shoddily documented code that needs to be closely inspected by individual site maintainers before being deployed on any serious production sites.
I stop following your argument here. The only way this argument holds up is that if *all* of contrib is substandard crap. It is not. There is a level of contrib that is above that, and there is a reason those particular pieces of contrib are above that. Chew on that.
And this level contrib is the kind that has enough users so that peer reviews occur naturally, or at the very least peer QA after the fact. As we've noted, peer review and peer QA is *not* happening with these core subsystems, which makes it a lot closer to the "darker, scarier" contrib. So I'm not sure what there is to chew on. :)
Perhaps you attribute more peer review and QA to some of these modules than is truly there. I don't think there is very much. There is peer review pretty much after the fact; that's the review that said "it's ok to use this module." But the responsibility of the committer is what made that happen in the first place. I personally believe that anyone who is given the right to commit directly to Drupal core is going to think several times longer before committing something than he or she would to contrib. Obviously this varies based on personality, but the suggestion that adding more committers to core will turn core into contrib does not work for me.
On 20-Apr-09, at 2:20 PM, Earl Miles wrote:
Angela Byron wrote:
On 20-Apr-09, at 2:04 PM, Earl Miles wrote:
Angela Byron wrote:
We have this kind of decentralized development model, where one or two people are solely responsible for code with basically zero peer review. It's called contrib. And it's notoriously filled with sub-standard, shoddily documented code that needs to be closely inspected by individual site maintainers before being deployed on any serious production sites.
I stop following your argument here. The only way this argument holds up is that if *all* of contrib is substandard crap. It is not. There is a level of contrib that is above that, and there is a reason those particular pieces of contrib are above that. Chew on that. And this level contrib is the kind that has enough users so that peer reviews occur naturally, or at the very least peer QA after the fact. As we've noted, peer review and peer QA is *not* happening with these core subsystems, which makes it a lot closer to the "darker, scarier" contrib. So I'm not sure what there is to chew on. :)
Perhaps you attribute more peer review and QA to some of these modules than is truly there. I don't think there is very much. There is peer review pretty much after the fact; that's the review that said "it's ok to use this module."
But the responsibility of the committer is what made that happen in the first place. I personally believe that anyone who is given the right to commit directly to Drupal core is going to think several times longer before committing something than he or she would to contrib. Obviously this varies based on personality, but the suggestion that adding more committers to core will turn core into contrib does not work for me.
I guess I view it as the following simple pseudo-query, assuming an active maintainer: $relative_project_quality = db_result(db_query('SELECT (number of active participants in issues, filtering out '+1 subscribe') FROM {the_issue_queue} WHERE project = %d', $project->nid)). On a project such as Views, with a very high number, $relative_project_quality will be very high. On a project such as any of mine ;) with a very low number, code quality will be much lower. This is not because I'm a horrible coder or that I'm not conscientious about my work or that I don't care about quality. It's because there's no body of diverse people chiming in to say "actually that will break my use case and here's why," or "I actually find that a bit hard to grasp; what if we did it this way instead?" There's no one looking over my shoulder for obvious brain-os that I miss in the height of my caffeine-powered coding rush. And furthermore, even if I *wanted* to bounce ideas off of more experienced developers or peers (which I do!), I quite simply can't; they don't have expertise in this space. So I do what most contributed module authors do, which is muddle through and try my best. Now, if you do the above query with project = 3060, you will get an extremely healthy project. And by having only a small number of committers, through whom *all* changes are run, this number holds true. However, If we move to a subsystem maintainer committing model, we need to change that query for the Drupal project so that it's WHERE project_component = %d. And if you do /that/, you do not get a rosy Views-esque picture of things like the menu system. You get a very worst-case scenario picture of webchick-maintained contributed modules. So my argument is quite simply that without building this number much higher for each of the underlying subsystems, I don't see what choice core has *but* to turn into the dark, scary kind of contrib by adding more committers to the mix. -Angie
Angela Byron wrote:
On 20-Apr-09, at 2:20 PM, Earl Miles wrote:
Angela Byron wrote:
On 20-Apr-09, at 2:04 PM, Earl Miles wrote:
Angela Byron wrote:
We have this kind of decentralized development model, where one or two people are solely responsible for code with basically zero peer review. It's called contrib. And it's notoriously filled with sub-standard, shoddily documented code that needs to be closely inspected by individual site maintainers before being deployed on any serious production sites.
I stop following your argument here. The only way this argument holds up is that if *all* of contrib is substandard crap. It is not. There is a level of contrib that is above that, and there is a reason those particular pieces of contrib are above that. Chew on that. And this level contrib is the kind that has enough users so that peer reviews occur naturally, or at the very least peer QA after the fact. As we've noted, peer review and peer QA is *not* happening with these core subsystems, which makes it a lot closer to the "darker, scarier" contrib. So I'm not sure what there is to chew on. :)
Perhaps you attribute more peer review and QA to some of these modules than is truly there. I don't think there is very much. There is peer review pretty much after the fact; that's the review that said "it's ok to use this module."
But the responsibility of the committer is what made that happen in the first place. I personally believe that anyone who is given the right to commit directly to Drupal core is going to think several times longer before committing something than he or she would to contrib. Obviously this varies based on personality, but the suggestion that adding more committers to core will turn core into contrib does not work for me.
I guess I view it as the following simple pseudo-query, assuming an active maintainer:
$relative_project_quality = db_result(db_query('SELECT (number of active participants in issues, filtering out '+1 subscribe') FROM {the_issue_queue} WHERE project = %d', $project->nid)).
What if a project is small and so mature that there simply aren't issues? Does that make it poor quality? This metric does. Also, flexinode. That also would hit your metric, but the code quality of flexinode was not high. It had a huge number of issue participants because it was a cool idea, not good code. In fact, what your metric gets you are "Things people really want but may not work right," not "quality contrib modules."
On Mon, Apr 20, 2009 at 2:48 PM, Angela Byron <drupal-devel@webchick.net> wrote:
So my argument is quite simply that without building this number much higher for each of the underlying subsystems, I don't see what choice core has *but* to turn into the dark, scary kind of contrib by adding more committers to the mix.
-Angie
I have to take issue with this in as much as any added committers would almost certainly not (as you do not) commit their own patches. If we were to move towards the notion of subsystem committers I would imagine that they would have to be in close communication with you and Dires about the overall direction and API design, get one or more serious reviews in addition to their own review, and probably would have much less time for writing patches as opposed to reviewing and committing them. In other words, they would behave w.r.t. a subset of the code in much as we observe you (and Gabor, and Neil) behaving. -Peter
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Earl Miles schrieb:
Angela Byron wrote:
We have this kind of decentralized development model, where one or two people are solely responsible for code with basically zero peer review. It's called contrib. And it's notoriously filled with sub-standard, shoddily documented code that needs to be closely inspected by individual site maintainers before being deployed on any serious production sites.
I stop following your argument here. The only way this argument holds up is that if *all* of contrib is substandard crap. It is not. There is a level of contrib that is above that, and there is a reason those particular pieces of contrib are above that. Chew on that.
Because they follow core's development model? Cheers, Gerhard -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAknswFcACgkQfg6TFvELooSYZwCgiXjOIsgkkaXzHMh9et0yGIY3 L2MAn3VLhZwL0Th9kHDtQAIUmuZ9QNPV =3YWz -----END PGP SIGNATURE-----
Gerhard Killesreiter wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Earl Miles schrieb:
Angela Byron wrote:
We have this kind of decentralized development model, where one or two people are solely responsible for code with basically zero peer review. It's called contrib. And it's notoriously filled with sub-standard, shoddily documented code that needs to be closely inspected by individual site maintainers before being deployed on any serious production sites. I stop following your argument here. The only way this argument holds up is that if *all* of contrib is substandard crap. It is not. There is a level of contrib that is above that, and there is a reason those particular pieces of contrib are above that. Chew on that.
Because they follow core's development model?
They don't, actually, or at least, mine don't. I don't post patches and have half a dozen people review them before I commit code. If someone posts a patch, chances are there's only 1 or 2 reviews on it, and I have to do the real review prior to deciding whether or not to commit it. Sometimes some patch will be both simple enough and important enough for more than a couple people to review it, but that is the exception, not the rule. Right now, it's because I try to hold myself to a high standard. And I admit that I can be sloppy and commit too fast due to the workload, but ultimately even with that I do a better job than a lot of contrib, simply because I try to see the bigger picture prior to committing. The reason I think more committers in targeted areas will work is because I think it will create more activity in those areas. The people who have the right to commit will have a more vested interest, and therefore you will get more of their time, effort and energy, plus any that they can draw to them. webchick commented earlier that one of the issues was providing an incentive to do reviews. And that's certainly a key issue, right there. There isn't much incentive to do reviews. There's the good of the project, but there is no pride of ownership in a review. The pride of ownership drives a lot of the initial development, and it drives the committers, but the reviewers? I cannot remember a time in the 4 years I've been with this project that we haven't had a longstanding complaint about lack of good patch reviews. That is one of the first complaints I heard then, and it is one of the complaints I hear now. It has not changed, and asking people to do more reviews has already proven to be ineffective in combatting the issue. Give your reviewers some ownership and you'll find a lot more people interested in reviewing.
The people who have the right to commit will have a more vested interest
There isn't much incentive to do reviews. There's the good of the project, but there is no pride of ownership in a review. The pride of ownership drives a lot of the initial development, and it drives the committers, but the reviewers?
I am giving up my neutral position (already...) and I must say: I agree. I *am* sloppy but if I am given the keys to Drupal core? I will second-guess every decision of mine for sure.
On Monday 20 April 2009 14:08:42 Karoly Negyesi wrote:
The people who have the right to commit will have a more vested interest
There isn't much incentive to do reviews. There's the good of the project, but there is no pride of ownership in a review. The pride of ownership drives a lot of the initial development, and it drives the committers, but the reviewers?
I am giving up my neutral position (already...) and I must say: I agree.
I *am* sloppy but if I am given the keys to Drupal core? I will second-guess every decision of mine for sure.
Hear hear for that. Nothing like the knowledge that if you commit a screw-up, it'll be noted by core devs within hours (if not minutes), which any self- respecting dev would find at least a smidge humiliating. When reading this little part of the thread, I got this mental image of Angie and Dries each driving enormous vans along a precarious cliffside road, trying to listen to a cacophony of conflicting navigational advice (ranging in quality from "local expert guide" to "pin the tail on the donkey") mixed in with a healthy dose of griping and "ARE WE THERE YET?!?" Not intended to paint us as whiny or unhelpful. My only point: personally, I got a lot better at navigating after I started driving.
On Mon, Apr 20, 2009 at 8:38 PM, Sam Boyer
Hear hear for that. Nothing like the knowledge that if you commit a screw-up, it'll be noted by core devs within hours (if not minutes), which any self- respecting dev would find at least a smidge humiliating.
This is interesting, because I find it really comforting that we manage to pick up screw-ups both before and after commit, especially mine, maybe I'm not self-respecting enough. I don't really like the experience of coming across an issue which just got committed and have to mark it code needs work (sometimes with rollback patch ready if it's really, really bad) - but when people do it to me I don't get offended or overly humiliated - I'm just glad it never made it into a final release. Actually I think a big part of the problem is this perception that core is hard and contrib is easy, personally I find the contrib workflow really hard. A lot of people's first attempt to contribute to Drupal is often when they apply for a CVS account to contribute back a module. A lot of companies view that as their primary way to contribute back resources as well. What we end up with is 4,000 projects of very varying quality, and the projects which /everyone/ uses - Views, CCK etc. are crying out for people to help in their issue queues - not just writing and reviewing patches at a high level, but answering support requests, closing out duplicates, stuff like that. One issue with this is the way our user profiles work in terms of highlighting contributions. I'd been contributing to core for more than a year before I got a CVS account, and that was when one of my core patches removed a slice of user.module on the condition it was maintained in contrib. So for a long time I never had any projects listed on my profile page, nor any entries in 'track code'. I've still only got commit access on a handful of projects, and don't actively maintain any of them - a situation I'm quite happy with. However if I posted a couple of crappy modules and committed loads of tiny patches to them, then to the casual observer that'd make the user profile look a lot beefier. The only way that code contributions to core get measured is when Greg Knaddison parses commit logs and puts everything into a spreadsheet once or twice a year - our commit model also completely breaks sites like ohloh. Personally I'd really like better statistics on how many people are reviewing/submitting patches, getting them committed etc. since it'd at least help to spot clear trends when having discussions like these. And I think it'd be a bit of incentive to review and post patches more if there was some way of having that data recorded more firmly than just in commit messages themselves. There was talk of having 'issue queue maintainers' without actual commit access for contrib projects as well - for people who answer lots of support requests and write documentation etc. Similar issue and it goes two ways. Identifying the people who you need to speak to about a particular area is hard unless you know who they are. Identifying what any particular individual is doing is hard unless you trawl through their tracker. There ought to be better ways to deal with this than filing patches to add people to maintainers.txt and having to write big long lists of names in commit messages (then what if you forgot someone or spell their name wrong). Nat
I think catch hits a lot of good points about how our reward and trust system could be improved. I'm all for better profile pages, giving credit where credit is due, and providing better visibility into who is working on what. It might be useful to provide a "Reviewer rating" or "Reviewer score" -- something which could be an incentive for more people to review patches. Maybe this can be accomplished through some project module extensions? I'm not sure what the best approach would be, but it is worthy of a discussion. On Mon, Apr 20, 2009 at 10:30 PM, Nathaniel Catchpole <catch56@googlemail.com> wrote:
On Mon, Apr 20, 2009 at 8:38 PM, Sam Boyer
Hear hear for that. Nothing like the knowledge that if you commit a screw-up, it'll be noted by core devs within hours (if not minutes), which any self- respecting dev would find at least a smidge humiliating.
This is interesting, because I find it really comforting that we manage to pick up screw-ups both before and after commit, especially mine, maybe I'm not self-respecting enough. I don't really like the experience of coming across an issue which just got committed and have to mark it code needs work (sometimes with rollback patch ready if it's really, really bad) - but when people do it to me I don't get offended or overly humiliated - I'm just glad it never made it into a final release.
Actually I think a big part of the problem is this perception that core is hard and contrib is easy, personally I find the contrib workflow really hard.
A lot of people's first attempt to contribute to Drupal is often when they apply for a CVS account to contribute back a module. A lot of companies view that as their primary way to contribute back resources as well. What we end up with is 4,000 projects of very varying quality, and the projects which /everyone/ uses - Views, CCK etc. are crying out for people to help in their issue queues - not just writing and reviewing patches at a high level, but answering support requests, closing out duplicates, stuff like that.
One issue with this is the way our user profiles work in terms of highlighting contributions. I'd been contributing to core for more than a year before I got a CVS account, and that was when one of my core patches removed a slice of user.module on the condition it was maintained in contrib. So for a long time I never had any projects listed on my profile page, nor any entries in 'track code'. I've still only got commit access on a handful of projects, and don't actively maintain any of them - a situation I'm quite happy with. However if I posted a couple of crappy modules and committed loads of tiny patches to them, then to the casual observer that'd make the user profile look a lot beefier.
The only way that code contributions to core get measured is when Greg Knaddison parses commit logs and puts everything into a spreadsheet once or twice a year - our commit model also completely breaks sites like ohloh. Personally I'd really like better statistics on how many people are reviewing/submitting patches, getting them committed etc. since it'd at least help to spot clear trends when having discussions like these. And I think it'd be a bit of incentive to review and post patches more if there was some way of having that data recorded more firmly than just in commit messages themselves.
There was talk of having 'issue queue maintainers' without actual commit access for contrib projects as well - for people who answer lots of support requests and write documentation etc. Similar issue and it goes two ways. Identifying the people who you need to speak to about a particular area is hard unless you know who they are. Identifying what any particular individual is doing is hard unless you trawl through their tracker. There ought to be better ways to deal with this than filing patches to add people to maintainers.txt and having to write big long lists of names in commit messages (then what if you forgot someone or spell their name wrong).
Nat
-- Dries Buytaert :: http://buytaert.net/
Something else which is worth to discuss. In private conversations, various people have expressed concerns that our release cycles are too long, and that, as a result, they are not interested in working on core. If they have a problem, and they fix it in core, they have to wait 1-2 years before they can actual use their own improvement in a production environment. In a lot of cases, people don't bother with it because they can't wait that long. We see evidence of that around code freeze time, when people actually do start to bother and patches tend to move through the review cycle quite a bit faster -- creating it own set of frustrations. In other words, moving to shorter release cycles could help. Not only does it increase the incentive for people to work on core, it could also provide additional focus. Because we have been working on Drupal 6 so long, people might have lost some of their incentive and motivation. There is a certain engery-cycle/rhythm that we might have broken with the longer Drupal 6 development cycle. In fact, a lot of big projects, like the Linux kernel, Gnome, KDE, Ubuntu, Fedora and Wordpress all switched to 6 month release cycles. Mark Shuttleworth posted some good insights about that in a recent blog post: http://www.markshuttleworth.com/archives/288. I'm not suggesting that we should switch to 6 month release cycles, I'm merely bringing it up to get feedback on the idea. On Mon, Apr 20, 2009 at 11:03 PM, Dries Buytaert <dries.buytaert@gmail.com> wrote:
I think catch hits a lot of good points about how our reward and trust system could be improved. I'm all for better profile pages, giving credit where credit is due, and providing better visibility into who is working on what. It might be useful to provide a "Reviewer rating" or "Reviewer score" -- something which could be an incentive for more people to review patches. Maybe this can be accomplished through some project module extensions? I'm not sure what the best approach would be, but it is worthy of a discussion.
-- Dries Buytaert :: http://buytaert.net/
On Monday 20 April 2009 23:19:03 Dries Buytaert wrote:
In fact, a lot of big projects, like the Linux kernel, Gnome, KDE, Ubuntu, Fedora and Wordpress all switched to 6 month release cycles. Mark Shuttleworth posted some good insights about that in a recent blog post: http://www.markshuttleworth.com/archives/288. I'm not suggesting that we should switch to 6 month release cycles, I'm merely bringing it up to get feedback on the idea. I don't think any of those projects have 6 month release cycles that break backwards compatibility, which I think major drupal releases do...
Marijn
Marijn Kruisselbrink wrote:
On Monday 20 April 2009 23:19:03 Dries Buytaert wrote:
In fact, a lot of big projects, like the Linux kernel, Gnome, KDE, Ubuntu, Fedora and Wordpress all switched to 6 month release cycles. Mark Shuttleworth posted some good insights about that in a recent blog post: http://www.markshuttleworth.com/archives/288. I'm not suggesting that we should switch to 6 month release cycles, I'm merely bringing it up to get feedback on the idea. I don't think any of those projects have 6 month release cycles that break backwards compatibility, which I think major drupal releases do...
Marijn
Looking at that list, one thing sticks out - there is only one blogging/CMS platform listed. Even though I know that list isn't a comprehensive collection of project release cycles, can anyone find any other blogging or CMS system pushing releases out every six months? I'm not saying its a bad thing, but I believe serious consideration needs to be given to backwards compatibility and support length of releases. Perhaps one good route for Drupal would be to mimic the Ubuntu system of offering a "long term support"release after X releases. That would benefit sites with a large custom code base, or even the smaller guy, who is less tech-savvy and dreads the upgrade for fear of breaking their site.
On Monday 20 April 2009, Marijn Kruisselbrink wrote:
On Monday 20 April 2009 23:19:03 Dries Buytaert wrote:
In fact, a lot of big projects, like the Linux kernel, Gnome, KDE, Ubuntu, Fedora and Wordpress all switched to 6 month release cycles. Mark Shuttleworth posted some good insights about that in a recent blog post: http://www.markshuttleworth.com/archives/288. I'm not suggesting that we should switch to 6 month release cycles, I'm merely bringing it up to get feedback on the idea.
I don't think any of those projects have 6 month release cycles that break backwards compatibility, which I think major drupal releases do...
I think that's just another argument in favor of Robert's suggested move of core modules to contrib. The Form/Fields/File/whatever APIs deserve longer release cycles so that contrib has at least some chance to stick with feature work for a single core version only. However, for the modules built on top of the APIs (Forum, Aggregator, etc.) nobody really cares about API compatibility, those are the ones that would actually benefit from significantly shorter release cycles. While I'm not yet convinced that stripping down Drupal to a bare framework is the right approach, I do think that the two aspects of core should get different treatment. -- Jakob
To my knowledge, every module removed from Drupal core has died. I use several of the modules on Roberts list (blog, aggregator, blogapi) and if they were removed, then I would be out of luck as one of the few remaining 'hobbyist' users. And Stefan, I too miss the old front page where it wasn't all site implementations and Drupalcons but things people found interesting, but that's just what happened. -sepeck On Mon, Apr 20, 2009 at 10:50 PM, Jakob Petsovits <jpetso@gmx.at> wrote:
On Monday 20 April 2009, Marijn Kruisselbrink wrote:
On Monday 20 April 2009 23:19:03 Dries Buytaert wrote:
In fact, a lot of big projects, like the Linux kernel, Gnome, KDE, Ubuntu, Fedora and Wordpress all switched to 6 month release cycles. Mark Shuttleworth posted some good insights about that in a recent blog post: http://www.markshuttleworth.com/archives/288. I'm not suggesting that we should switch to 6 month release cycles, I'm merely bringing it up to get feedback on the idea.
I don't think any of those projects have 6 month release cycles that break backwards compatibility, which I think major drupal releases do...
I think that's just another argument in favor of Robert's suggested move of core modules to contrib. The Form/Fields/File/whatever APIs deserve longer release cycles so that contrib has at least some chance to stick with feature work for a single core version only.
However, for the modules built on top of the APIs (Forum, Aggregator, etc.) nobody really cares about API compatibility, those are the ones that would actually benefit from significantly shorter release cycles.
While I'm not yet convinced that stripping down Drupal to a bare framework is the right approach, I do think that the two aspects of core should get different treatment.
-- Jakob
Steven Peck wrote:
To my knowledge, every module removed from Drupal core has died. I use several of the modules on Roberts list (blog, aggregator, blogapi) and if they were removed, then I would be out of luck as one of the few remaining 'hobbyist' users.
And Stefan, I too miss the old front page where it wasn't all site implementations and Drupalcons but things people found interesting, but that's just what happened.
I don't believe this would be true if we were still packaging them together, but treating them differently. Think of the Drupal Framework as the Linux Kernel, and then the Drupal distribution as Linux. The framework includes only the APIs -- node, user, base object stuff. The foundation. Drupal then provides applications built upon that foundation. It starts with just one: Something resembling Drupal as we know it today. We split core, and we devote effort to the most important projects. This might allow us to better focus efforts, anyway. One of the issues I see is that core itself is too big. Neither of the two committers really understand everything they've been committing, lately. (If you doubt me...field API =)
Earl Miles wrote:
Think of the Drupal Framework as the Linux Kernel, and then the Drupal distribution as Linux. The framework includes only the APIs -- node, user, base object stuff. The foundation. Drupal then provides applications built upon that foundation. It starts with just one: Something resembling Drupal as we know it today. We split core, and we devote effort to the most important projects. This might allow us to better focus efforts, anyway. One of the issues I see is that core itself is too big. Neither of the two committers really understand everything they've been committing, lately. (If you doubt me...field API =) I think Drupal growing in size is natural. Drupal doesn't have to fit onto a floppy disk. Code bloat, and overall code size are two different things.
I'm really surprised and scared at how quickly Drupal is progressing. At this rate, contrib will never be able to keep up all the features in core, with only handfuls of people really understanding it, and so it will be a complete waste of time coding the features in the first place. Each radical change between major core versions means more time investment for contrib developers to upgrade their modules. So, instead of trying to clump lots of really great cool features into the next major release and blowing everyone away, how about picking a few major features, and make them the best they can be, and then adding other new features at a later date? This would give contrib developers time to develop for them, and the docs team time to document.
Or we can have one Drupal Legacy Module that we can suport the modules of previous drupal release. Pros: * Lots of previous module will work. * The "contrib developer" have more time to decide to upgrade the module. (instead 1 years will have 2 years or more) * More users will update their Drupal version. * I think with that, instead contrib modules became more not up to date, will became more up to date. because with this we can use the modules with the new version of drupal, so the process of migrating modules will became more faster.. Why? because some developers wait until all others modules (that they use) became migrated, before start.. Cons: * Modules that use this module will run more slow * Some Modules will not run and others have some break's. -- ------------------------------------- Marco Sousa
I forget, we can start suporting Drupal 7 Contrib Modules to work with Drupal 8. Because of the realy change in BD and POO... On Tue, Apr 21, 2009 at 1:38 PM, Marco Sousa <marcomsousa@gmail.com> wrote:
Or we can have one Drupal Legacy Module that we can suport the modules of previous drupal release.
Pros: * Lots of previous module will work. * The "contrib developer" have more time to decide to upgrade the module. (instead 1 years will have 2 years or more) * More users will update their Drupal version. * I think with that, instead contrib modules became more not up to date, will became more up to date. because with this we can use the modules with the new version of drupal, so the process of migrating modules will became more faster.. Why? because some developers wait until all others modules (that they use) became migrated, before start..
Cons: * Modules that use this module will run more slow * Some Modules will not run and others have some break's.
-- ------------------------------------- Marco Sousa
-- ------------------------------------- Marco Sousa
See the joomla way: Legacy Mode: *http://tinyurl.com/cxr45d* -- ------------------------------------- Marco Sousa
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Marco Sousa schrieb:
See the joomla way:
Legacy Mode: *http://tinyurl.com/cxr45d*
Feel free to provide such a module in contrib and package it as part of an install profile. Or just use Joomla. Cheers, Gerhard -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAknttcUACgkQfg6TFvELooRMCgCaAxzzc+Pc/JH1/f/I0pGkr3Zo 1fgAoIRZKF5kqJ2YrSOJHGWrFQtVLuJN =Vw90 -----END PGP SIGNATURE-----
On Tue, Apr 21, 2009 at 2:02 PM, Gerhard Killesreiter < gerhard@killesreiter.de> wrote:
Feel free to provide such a module in contrib and package it as part of an install profile. Or just use Joomla.
Gehard, this comment is useful? Please stop that. I just want to give more alternatives (used by one CMS), to we have all variables to choose the better one.. I'm not inflame nothing... -- ------------------------------------- Marco Sousa
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Marco Sousa schrieb:
On Tue, Apr 21, 2009 at 2:02 PM, Gerhard Killesreiter < gerhard@killesreiter.de> wrote:
Feel free to provide such a module in contrib and package it as part of an install profile. Or just use Joomla.
Gehard, this comment is useful? Please stop that.
I just want to give more alternatives (used by one CMS), to we have all variables to choose the better one..
I'm not inflame nothing...
You are repeating a discussion we are having every couple of months or years. Last time was February, IIRC. This is not helpful and has to be stopped. The others gave you more detailed answers why your proposal is not only annoying but also not working. Cheers, Gerhard -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAkntyo4ACgkQfg6TFvELooSxngCePiG5knjL9t9XhnND4us9yVEq 2eAAn2GWA/i98wWaY4LpB8QH9CwLAnL8 =jkQ1 -----END PGP SIGNATURE-----
Or we can have one Drupal Legacy Module that we can suport the modules of previous drupal release.
No we can't. The API changes in Drupal 7 include things like changing the way modules are defined in .info files to make them compatible with the registry, and changing the syntax of every single database query. It's impossible to make a module which would allow you to run a Drupal 6 module on Drupal 7 with no changes, and that's a very, very good thing. It's already been said once in this thread, but the reason it took so long for contrib to catch up to Drupal 6 was a massive refactoring of CCK, Views, wysiwyg API and other modules all the same time - and the fact that contrib has a very complex web of dependencies (baz module extends foo module which extends bar module, so can't do an update until both bar and foo are upgraded). A Views 2 which was backwards compatible with Views 1 wouldn't be half as much fun - and if you just want to use the same D5 modules you were using already without any of the advantages, why upgrade at all? We don't make API changes for fun, we make them because we have to - either to correct previous mistakes or open up new possibilities. If an API doesn't need to change then it usually doesn't - for example hook_menu() is pretty much exactly the same between 6 and 7. All the effort talking about backwards compatibility should instead be focused on http://drupal.org/project/deadwood and http://drupal.org/project/coder - and also on the Field API and Views related patches to core which will be necessary for field, views, and views-dependent modules to upgrade to D7. Coder already has an initial port to Drupal 7 and does a great deal of the work required to update a module. The better it gets, the less work module maintainers (or upgrade patch contributors) need to do to get a basic port of their module going. Combined with unstable releases, it's quite possible to start upgrading modules to Drupal 7 now - in fact there's 7 pages of modules which have some kind of port in the works, months before code freeze - something we've never, ever had before. http://drupal.org/project/modules?filters=drupal_core:103&solrsort=sort_titl... Nat
On Tuesday 21 April 2009, Marco Sousa wrote:
Or we can have one Drupal Legacy Module that we can suport the modules of previous drupal release.
This is proposed by someone for every Drupal version, it has been tried several times, and it has never worked for reasons that have been discussed at length in previous threads on this list. Please search the archives to read up those arguments. If we really go with module-supported "automatical" compatibility, the forward-facing way (deadwood) is much more promising than the legacy one.
2009/4/21 Jakob Petsovits <jpetso@gmx.at>:
Or we can have one Drupal Legacy Module that we can suport the modules of previous drupal release.
This is proposed by someone for every Drupal version, it has been tried several times, and it has never worked for reasons that have been discussed at length in previous threads on this list. Please search the archives to read up those arguments.
Last night, just before I went to sleep, I was thinking if it would be good to have CHANGES.txt accompanied by DIDNOTCHANGE.txt to aid porting of contrib modules. In some cases people might even be able to have xyz.module and xyz.d6.inc as well as xyz.d7.inc. xyz.module would contain everthing mentioned in DIDNOTCHANGE.txt. But maybe this is silly... Eric
Quoting Jakob Petsovits <jpetso@gmx.at>:
On Tuesday 21 April 2009, Marco Sousa wrote:
Or we can have one Drupal Legacy Module that we can suport the modules of previous drupal release.
This is proposed by someone for every Drupal version, it has been tried several times, and it has never worked for reasons that have been discussed at length in previous threads on this list. Please search the archives to read up those arguments.
If we really go with module-supported "automatical" compatibility, the forward-facing way (deadwood) is much more promising than the legacy one.
I started to suggest the legacy issue yesterday but stopped because I realize that supporting legacy becomes an unending task that just postpones forever the upgrade to the newer API. I agree the deadwood module is a great benefit. One desire I would have with it is to report in foo.deadwood things of caution instead of inserting comments into foo.module. Yea, yea, I know a feature request in the deadwood issue queue. -- Earnie -- http://r-feed.com/ -- http://for-my-kids.com/ -- http://www.4offer.biz/ -- http://give-me-an-offer.com/
Quoting Marco Sousa <marcomsousa@gmail.com>:
Cons: * Modules that use this module will run more slow * Some Modules will not run and others have some break's.
* There will be less interest in upgrading a module prolonging the inevitable. * Most modules will suddenly break when moving from D7 to Dx. * Legacy remains legacy forever and the maintenance of such a module becomes unbearable as more core versions are released. As has been mentioned already deadwood and coder modules are the way to go when upgrading. Identifying to the module maintainer or interested party where the code needs to be upgraded is jump in the correct direction. Deadwood even changes what it can. Improving that module is where we need to focusing the legacy issue. -- Earnie -- http://r-feed.com/ -- http://for-my-kids.com/ -- http://www.4offer.biz/ -- http://give-me-an-offer.com/
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Steven Peck schrieb:
To my knowledge, every module removed from Drupal core has died. I use several of the modules on Roberts list (blog, aggregator, blogapi) and if they were removed, then I would be out of luck as one of the few remaining 'hobbyist' users.
There are more hobbyist users than you think. They just are not on this list. I don't think that removing current core modules to the contrib repo is a good idea. It would screw up the upgrade path for thousands of existing sites. Things like the blog module simply do not need much development, they are feature complete. Adding bells and whistles would only make them harder to use.
And Stefan, I too miss the old front page where it wasn't all site implementations and Drupalcons but things people found interesting, but that's just what happened.
Yeah, I've read enough site implementations for the rest of my life I think. Cheers, Gerhard -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAkntcgUACgkQfg6TFvELooRw5QCdFu/6omXbEZnJbEi5vVblgPpR m9sAoMaV49l1DrFNfDZqaGkWPsccTXfI =YVlr -----END PGP SIGNATURE-----
Op 21 apr 2009, om 08:20 heeft Steven Peck het volgende geschreven:
<snip></snip>
And Stefan, I too miss the old front page where it wasn't all site implementations and Drupalcons but things people found interesting, but that's just what happened.
-sepeck
Happy to hear that Sepeck, sometimes I wonder if it is just me who is experiencing this. Drupal is not for users anymore, drupal is for making money and profit. One example is the "Support the Drupal redesign! Donate Now"- button, which is currently cycling over the frontpage. Why can't we - as a community - build the new drupal.org design? It gives us a good reality check why drupal is (too) hard for true designers. That said, you could wonder that if some module is moved from core to contrib and dies silently, if it wasn't in core too long already. If nobody feels the urge to keep a certain module up to date, then you could argue about its usefullnes. I would rather see modules like views(1|2|3).module in core, which are very well designed and maintained and deserves to be in core, because of it's quality and filling the gap which there currently is in core. Building views/page-layouts is something drupal needs. Although, I do not think that the current implementation is right, the functionality surely is. Getting rid of modules like the help.module is not that smart imo. Dries started a monster project to make drupal more accessible and easier to use by attracting Mark Boulton and Leisa, which I think is a good thing. Although, I do not doubt that they could make things easier to use, but I'm pretty convinced they could not make things as easy that the help texts are becoming redundant. Kind regards, Stefan
On Tue, Apr 21, 2009 at 12:31 AM, Stefan Nagtegaal <development@standoutdesign.nl> wrote:
Op 21 apr 2009, om 08:20 heeft Steven Peck het volgende geschreven:
<snip></snip>
And Stefan, I too miss the old front page where it wasn't all site implementations and Drupalcons but things people found interesting, but that's just what happened.
-sepeck
Happy to hear that Sepeck, sometimes I wonder if it is just me who is experiencing this. Drupal is not for users anymore, drupal is for making money and profit. One example is the "Support the Drupal redesign! Donate Now"-button, which is currently cycling over the frontpage. Why can't we - as a community - build the new drupal.org design? It gives us a good reality check why drupal is (too) hard for true designers.
Now now, the redesign is complex and time consuming. People are busy, so outside perspective is useful.
That said, you could wonder that if some module is moved from core to contrib and dies silently, if it wasn't in core too long already. If nobody feels the urge to keep a certain module up to date, then you could argue about its usefullnes.
I still use them. I know a number of people I support do as well. If it's been working fine then it hasn't needed much attention, but we are still have a large non-professional, minimal coding group of people that use Drupal so I would rather see a gradual enhancement/generalization of core to absorb these remaining specialized modules.
I would rather see modules like views(1|2|3).module in core, which are very well designed and maintained and deserves to be in core, because of it's quality and filling the gap which there currently is in core. Building views/page-layouts is something drupal needs. Although, I do not think that the current implementation is right, the functionality surely is.
Different discussion. But on par with how CCK migrated into core as it and the community matured around it.
Getting rid of modules like the help.module is not that smart imo. Dries started a monster project to make drupal more accessible and easier to use by attracting Mark Boulton and Leisa, which I think is a good thing. Although, I do not doubt that they could make things easier to use, but I'm pretty convinced they could not make things as easy that the help texts are becoming redundant.
I liked the old built in help system, but we've gone 3 different ways since then.
Kind regards,
Stefan
Steven
On Tuesday 21 April 2009, Stefan Nagtegaal wrote:
Happy to hear that Sepeck, sometimes I wonder if it is just me who is experiencing this. Drupal is not for users anymore, drupal is for making money and profit. One example is the "Support the Drupal redesign! Donate Now"- button, which is currently cycling over the frontpage. Why can't we - as a community - build the new drupal.org design? It gives us a good reality check why drupal is (too) hard for true designers.
Actually we do build it, I think the steady flow of redesign sprints with a large number of community members is a testament to that. Also, the community provided plenty of input for the actual design process, it's just that *someone* has to oversee the process and has the final word, because design by committee doesn't really work. On the other hand, Drupal has actually grown very commercial, not least because the community is made up of the people who use it. (And if those people include lots of companies with monetary resources, their influence and contributions will not go unnoticed.) I don't think that's a bad idea per se, the challenge is just how to cope with this situation. Deciding how to deal with this is mainly an issue of vision and focus, which to a great extent is a responsibility of the project leader. * Do we want to focus on innovation and pushing the web forward, or is it more important to us to preserve existing functionality in an easily supportable way? * Do we want to focus on "free as in speech" political issues or are we satisfied with plain "open source" collaboration principles? * Do we give see ourselves as companies collaborating with each other, or are we still a team of individuals where certain people happen to work for certain employers? * Are sponsors (including Acquia) given special treatment, power and/or responsibility, or is code (and arguments) still gold when it comes to decisions that affect commercial reusability? * Do we emphasize the "product" aspect of Drupal or rather the "community" one? * Do we focus on excellence of code and inviting skilled contributors, or do we take an inclusive approach by tailoring to the newbie masses? Each project has to answer those questions for itself. So far, I think Dries has done a pretty good job of not letting money take over the development process of Drupal core, and has given clear answers to most of these questions. Maybe the promotional side has gotten a bit out of hand, because the number of "community" related stuff is not increasing as much as the number of commercial sites that are built and submitted as case studies. On the whole though, I think the Drupal community has always been this way, and the current state of "commercialism" is just the logical extension of how we are as a community. The thing that I worry more about than companies occupying d.o is that Drupal lacks the drive to search for solutions how the user can be guaranteed control over her data, as opposed to handing it to service providers. But that was never a stated goal or common vision, and I can live with it. Drupal has not been subverted, it just has evolved in its natural way. I hope that's not too off-topic to still fit into this thread... cheers, Jakob
Two thoughts occurred to me on the community. First, we are no longer what the community was 3 or 5 years ago. The front page of drupal has changed with a changing focus and need. Instead of a few thousand users there are hundreds of thousands of users. The front page of d.o has a different place in the ecosystem than it used to. But, we still need a place for some of those things that used to get posted. Is g.d.o the place for that? Second, we seem to have a bit of an identity crisis. Is drupal a framework or a CMS? If it's both should they be cleanly separated? How does this and should this work? Moving the CMS like features into contrib, as some have suggested, seems to be centered around this. If we have a drupal framework and a drupal cms do we have maintainers for each piece? How does this interaction work? Ah, growing pains. Now we just have to figure out how to bribe the kids to clean their rooms (i.e. review patches and clean the queue). On Apr 21, 2009, at 3:31 AM, Stefan Nagtegaal wrote:
Op 21 apr 2009, om 08:20 heeft Steven Peck het volgende geschreven:
<snip></snip>
And Stefan, I too miss the old front page where it wasn't all site implementations and Drupalcons but things people found interesting, but that's just what happened.
-sepeck
Happy to hear that Sepeck, sometimes I wonder if it is just me who is experiencing this. Drupal is not for users anymore, drupal is for making money and profit. One example is the "Support the Drupal redesign! Donate Now"- button, which is currently cycling over the frontpage. Why can't we - as a community - build the new drupal.org design? It gives us a good reality check why drupal is (too) hard for true designers.
That said, you could wonder that if some module is moved from core to contrib and dies silently, if it wasn't in core too long already. If nobody feels the urge to keep a certain module up to date, then you could argue about its usefullnes.
I would rather see modules like views(1|2|3).module in core, which are very well designed and maintained and deserves to be in core, because of it's quality and filling the gap which there currently is in core. Building views/page-layouts is something drupal needs. Although, I do not think that the current implementation is right, the functionality surely is.
Getting rid of modules like the help.module is not that smart imo. Dries started a monster project to make drupal more accessible and easier to use by attracting Mark Boulton and Leisa, which I think is a good thing. Although, I do not doubt that they could make things easier to use, but I'm pretty convinced they could not make things as easy that the help texts are becoming redundant.
Kind regards,
Stefan
On Tuesday 21 April 2009 8:04:21 am Matt Farina wrote:
Second, we seem to have a bit of an identity crisis. Is drupal a framework or a CMS? If it's both should they be cleanly separated? How does this and should this work? Moving the CMS like features into contrib, as some have suggested, seems to be centered around this. If we have a drupal framework and a drupal cms do we have maintainers for each piece? How does this interaction work?
Last week, I gave a workshop at the Museums and the Web conference in Indianapolis on remote data handling in Drupal. One of the main things I drove home during the session was that Drupal has a split personality of CMS and framework... and that is a very very good thing. That is what makes Drupal powerful and well-suited to, in that case, remote data handling within a CMS. That multiple personality disorder is great, because you get to pick which Drupal personality you speak to at any given time. "Both" is the answer that has gotten Drupal so far. Let's not drop half of our power. Balance is the answer. -- Larry Garfield larry@garfieldtech.com
Quoting Matt Farina <matt@mattfarina.com>:
Is drupal a framework or a CMS? If it's both should they be cleanly separated?
Drupal is both a framework and a CMS. The framework itself is geared toward managing dynamic content. The parts of Drupal that make it a traditional CMS must remain with the core of Drupal because that gives Drupal better definition.
How does this and should this work? Moving the CMS like features into contrib, as some have suggested, seems to be centered around this. If we have a drupal framework and a drupal cms do we have maintainers for each piece? How does this interaction work?
Since the framework is geared toward a CMS how can we separate all of those pieces. There may be some modules dwelling in core that may be moved but certainly not those that pertain to a minimal CMS. -- Earnie -- http://r-feed.com/ -- http://for-my-kids.com/ -- http://www.4offer.biz/ -- http://give-me-an-offer.com/
On Tue, Apr 21, 2009 at 6:04 AM, Matt Farina <matt@mattfarina.com> wrote:
Two thoughts occurred to me on the community.
First, we are no longer what the community was 3 or 5 years ago. The front page of drupal has changed with a changing focus and need. Instead of a few thousand users there are hundreds of thousands of users. The front page of d.o has a different place in the ecosystem than it used to. But, we still need a place for some of those things that used to get posted. Is g.d.o the place for that?
It is and I find it disturbing that it changed the way it did but it happened. I haven't decided what, if anything, I want to do about it.
Second, we seem to have a bit of an identity crisis. Is drupal a framework or a CMS? If it's both should they be cleanly separated? How does this and should this work? Moving the CMS like features into contrib, as some have suggested, seems to be centered around this. If we have a drupal framework and a drupal cms do we have maintainers for each piece? How does this interaction work?
I disagree with the identity crisis. Drupal has always been in balance. This balance is extremely beneficial and because it is 'close enough' for many it gets used, but it is also 'not quite right' for others and leads them into contributing or learning development. Some persist, some give up, some get angry and some do notable things. Drupal has been like this every release I have seen. If it hadn't been usable as a CMS (4.3), I would never have used it past learning how to install it. As long as the edge cases (toss everything out of core now vs just make it build my site telepathically) don't get their way we've probably kept a nice middle ground. As I am one of the few people on the dev list in the 'want some CMS features' camp, I feel it is important for me to speak up when the 'toss everything out' camp puts up their list. It's just the way discussions work is all.
Ah, growing pains. Now we just have to figure out how to bribe the kids to clean their rooms (i.e. review patches and clean the queue).
See historical emails for those discussions :) Steven
On Apr 21, 2009, at 3:31 AM, Stefan Nagtegaal wrote:
Op 21 apr 2009, om 08:20 heeft Steven Peck het volgende geschreven:
<snip></snip>
And Stefan, I too miss the old front page where it wasn't all site implementations and Drupalcons but things people found interesting, but that's just what happened.
-sepeck
Happy to hear that Sepeck, sometimes I wonder if it is just me who is experiencing this. Drupal is not for users anymore, drupal is for making money and profit. One example is the "Support the Drupal redesign! Donate Now"-button, which is currently cycling over the frontpage. Why can't we - as a community - build the new drupal.org design? It gives us a good reality check why drupal is (too) hard for true designers.
That said, you could wonder that if some module is moved from core to contrib and dies silently, if it wasn't in core too long already. If nobody feels the urge to keep a certain module up to date, then you could argue about its usefullnes.
I would rather see modules like views(1|2|3).module in core, which are very well designed and maintained and deserves to be in core, because of it's quality and filling the gap which there currently is in core. Building views/page-layouts is something drupal needs. Although, I do not think that the current implementation is right, the functionality surely is.
Getting rid of modules like the help.module is not that smart imo. Dries started a monster project to make drupal more accessible and easier to use by attracting Mark Boulton and Leisa, which I think is a good thing. Although, I do not doubt that they could make things easier to use, but I'm pretty convinced they could not make things as easy that the help texts are becoming redundant.
Kind regards,
Stefan
On Mon, Apr 20, 2009 at 10:19 PM, Dries Buytaert wrote:
Something else which is worth to discuss.
In private conversations, various people have expressed concerns that our release cycles are too long, and that, as a result, they are not interested in working on core. If they have a problem, and they fix it in core, they have to wait 1-2 years before they can actual use their own improvement in a production environment. In a lot of cases, people don't bother with it because they can't wait that long.
In other words, moving to shorter release cycles could help. Not only does it increase the incentive for people to work on core, it could also provide additional focus.
With testing, dbtng and Field API I think it would have been impossible to get them all in during the six month release cycle, and hard to do any of them properly in that time either (unless they were committed the first week) - actually writing tests and converting modules has taken a long time, and that's been a good thing as all three APIs are maturing during the process. However, having made all those big changes in one release, it'd be really interesting to consider a shorter cycle for Drupal 8. Some things which are looking less likely for Drupal 7 (views in core, a non-crappy comment and forum module) would be great to have in a shorter timespan than another two year release cycle. Nat
In fact, a lot of big projects, like the Linux kernel, Gnome, KDE, Ubuntu, Fedora and Wordpress all switched to 6 month release cycles. Mark Shuttleworth posted some good insights about that in a recent blog post: http://www.markshuttleworth.com/archives/288. I'm not suggesting that we should switch to 6 month release cycles, I'm merely bringing it up to get feedback on the idea.
On Mon, Apr 20, 2009 at 11:03 PM, Dries Buytaert <dries.buytaert@gmail.com> wrote:
I think catch hits a lot of good points about how our reward and trust system could be improved. I'm all for better profile pages, giving credit where credit is due, and providing better visibility into who is working on what. It might be useful to provide a "Reviewer rating" or "Reviewer score" -- something which could be an incentive for more people to review patches. Maybe this can be accomplished through some project module extensions? I'm not sure what the best approach would be, but it is worthy of a discussion.
-- Dries Buytaert :: http://buytaert.net/
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Dries Buytaert schrieb:
blog post: http://www.markshuttleworth.com/archives/288. I'm not suggesting that we should switch to 6 month release cycles, I'm merely bringing it up to get feedback on the idea.
I'd be very happy if we could at least break the trend towards longer and longer release cycles. 4.5.0 October 18, 2004 4.6.0 April 15, 2005 4.7.0 May 1, 2006 5.0 January 15, 2007 6.0 February 13, 2008 7.0 Codefreeze: Sept 2009 6, 12, 8, 13, 18+, ...? Cheers, Gerhard P.S.: Whoever thinks about complaining about the perceived lack of commercial viability of such a development model: Just can it. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAkntDd8ACgkQfg6TFvELooSpnQCgp29vFEfNBK4vx2nu4J1+T1Ge Jk8An18wUCezp/I+lcQG/xfbbJI7rZ6t =ZAq5 -----END PGP SIGNATURE-----
Quoting Gerhard Killesreiter <gerhard@killesreiter.de>:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Dries Buytaert schrieb:
blog post: http://www.markshuttleworth.com/archives/288. I'm not suggesting that we should switch to 6 month release cycles, I'm merely bringing it up to get feedback on the idea.
I'd be very happy if we could at least break the trend towards longer and longer release cycles.
I can imagine that this could happen with fewer features added in one cycle. Things like dbTNG that requires a whole rewrite of queries will take a longer cycle and should be the only feature change for that release. Other changes like fields in core could be released earlier. This though requires a different paradigm in version control. -- Earnie -- http://r-feed.com/ -- http://for-my-kids.com/ -- http://www.4offer.biz/ -- http://give-me-an-offer.com/
IMHO, we'd have to have a different stance on backward compatibility if we were planning on shortening release cycles much. The linux Kernel does not require significant refactoring of code built on top of it every 6 months. I'm still not sure if I have a migration path identified for my universities drupal sites with D6, even though I've already ported all of my modules to D6. It is possible that much of the interesting work about extending drupal really is properly in contrib and not in Core. The fact that most of what I hear being touted as new features for drupal 6/7 are the movement of contrib support into core is not necessarily a bad thing. It''s pretty proven code after all. I think finding a pathway to move old stuff out of core is not necessarily a bad idea either. Strikes me as the pathway for evolution of drupal. I think think the right answer about drupal stagnating was really about having Views, Panels, and WYSIWG api all going through major refactoring on D6, all lagging significantly the D6 release. I know it's why I'm not using my own D6 ported modules in any production sites yet. Dave On Apr 20, 2009, at 2:19 PM, Dries Buytaert wrote:
Something else which is worth to discuss.
In private conversations, various people have expressed concerns that our release cycles are too long, and that, as a result, they are not interested in working on core. If they have a problem, and they fix it in core, they have to wait 1-2 years before they can actual use their own improvement in a production environment. In a lot of cases, people don't bother with it because they can't wait that long.
We see evidence of that around code freeze time, when people actually do start to bother and patches tend to move through the review cycle quite a bit faster -- creating it own set of frustrations.
In other words, moving to shorter release cycles could help. Not only does it increase the incentive for people to work on core, it could also provide additional focus. Because we have been working on Drupal 6 so long, people might have lost some of their incentive and motivation. There is a certain engery-cycle/rhythm that we might have broken with the longer Drupal 6 development cycle.
In fact, a lot of big projects, like the Linux kernel, Gnome, KDE, Ubuntu, Fedora and Wordpress all switched to 6 month release cycles. Mark Shuttleworth posted some good insights about that in a recent blog post: http://www.markshuttleworth.com/archives/288. I'm not suggesting that we should switch to 6 month release cycles, I'm merely bringing it up to get feedback on the idea.
On Mon, Apr 20, 2009 at 11:03 PM, Dries Buytaert <dries.buytaert@gmail.com> wrote:
I think catch hits a lot of good points about how our reward and trust system could be improved. I'm all for better profile pages, giving credit where credit is due, and providing better visibility into who is working on what. It might be useful to provide a "Reviewer rating" or "Reviewer score" -- something which could be an incentive for more people to review patches. Maybe this can be accomplished through some project module extensions? I'm not sure what the best approach would be, but it is worthy of a discussion.
-- Dries Buytaert :: http://buytaert.net/
Quoting David Metzler <metzlerd@metzlerd.com>:
I think think the right answer about drupal stagnating was really about having Views, Panels, and WYSIWG api all going through major refactoring on D6, all lagging significantly the D6 release. I know it's why I'm not using my own D6 ported modules in any production sites yet.
And to avoid this the module developer needs to agree to port the existing functionality of a module from one version to the next while the new features of the module are being developed. The deadwood and coder modules can help with that and should be one of the first modules upgraded out of the door. New functionality in a module is great but once a Drupal version is out and people are knocking on the modules door for "when is Dx going to be supported" carrying over the old functionality to the new Dx should have a priority over a more powerful, better looking model. D6 still lags because modules are improving their API and developers aren't willing to put out a module that is carries the sins of the older D5 version. A worthy goal but so is getting a module that is working just like the previous. This is really a sore sticking point making using Drupal sour in some eyes. I agree that Drupal has a nice feature rich framework for building many different applications (modules). I agree that the CMS work is what has drawn many to Drupal. However, a focus on CMS often drowns the other usefulness of the framework and you have a situation similar to HORDE/IMP. Does anyone think framework when they hear HORDE? Maybe as a second thought but usually you think an email client. Unfortunately when people here Drupal they think CMS and not framework. Most users of the CMS don't care that the framework of Drupal or the framework of their favorite module is now more feature rich at first. They care that their sites will work exactly the same as it does before upgrading. Then they might look at the newer features. There are more users of Drupal than there are hackers providing code, peer reviews and module functionality. Yes, these are important but not to the non-technical user of the product. People are tired of waiting on module maintainers and probably will go elsewhere or worse create their own version of the module. This is why it is important for contrib modules to be readily available within a month of a new Dx release. -- Earnie -- http://r-feed.com/ -- http://for-my-kids.com/ -- http://www.4offer.biz/ -- http://give-me-an-offer.com/
I agree with this post, but also understand that there are times when this is not possible. I read enough of the panels thread to understand that there were some changes in the menu system that made porting Panels 2 in a feature complete way to D6 significantly difficult, and I want to acknowledge that this is a reality that maintainers must live with when frameworks change (which I continue to believe is a good thing). Dave On Apr 21, 2009, at 7:20 AM, Earnie Boyd wrote:
Quoting David Metzler <metzlerd@metzlerd.com>:
I think think the right answer about drupal stagnating was really about having Views, Panels, and WYSIWG api all going through major refactoring on D6, all lagging significantly the D6 release. I know it's why I'm not using my own D6 ported modules in any production sites yet.
And to avoid this the module developer needs to agree to port the existing functionality of a module from one version to the next while the new features of the module are being developed. The deadwood and coder modules can help with that and should be one of the first modules upgraded out of the door. New functionality in a module is great but once a Drupal version is out and people are knocking on the modules door for "when is Dx going to be supported" carrying over the old functionality to the new Dx should have a priority over a more powerful, better looking model. D6 still lags because modules are improving their API and developers aren't willing to put out a module that is carries the sins of the older D5 version. A worthy goal but so is getting a module that is working just like the previous. This is really a sore sticking point making using Drupal sour in some eyes.
I agree that Drupal has a nice feature rich framework for building many different applications (modules). I agree that the CMS work is what has drawn many to Drupal. However, a focus on CMS often drowns the other usefulness of the framework and you have a situation similar to HORDE/IMP. Does anyone think framework when they hear HORDE? Maybe as a second thought but usually you think an email client. Unfortunately when people here Drupal they think CMS and not framework. Most users of the CMS don't care that the framework of Drupal or the framework of their favorite module is now more feature rich at first. They care that their sites will work exactly the same as it does before upgrading. Then they might look at the newer features.
There are more users of Drupal than there are hackers providing code, peer reviews and module functionality. Yes, these are important but not to the non-technical user of the product. People are tired of waiting on module maintainers and probably will go elsewhere or worse create their own version of the module. This is why it is important for contrib modules to be readily available within a month of a new Dx release.
-- Earnie -- http://r-feed.com/ -- http://for-my-kids.com/ -- http://www.4offer.biz/ -- http://give-me-an-offer.com/
On Monday 20 April 2009 4:03:08 pm Dries Buytaert wrote:
I think catch hits a lot of good points about how our reward and trust system could be improved. I'm all for better profile pages, giving credit where credit is due, and providing better visibility into who is working on what. It might be useful to provide a "Reviewer rating" or "Reviewer score" -- something which could be an incentive for more people to review patches. Maybe this can be accomplished through some project module extensions? I'm not sure what the best approach would be, but it is worthy of a discussion.
Dear god this is a monster thread... :-) I'm going to jump in here for lack of anywhere better to do so. As others have noted, the problem isn't the speed of commits, it's the speed of reviews and the ability of sufficient people to grok them to review them usefully. I would add a related problem: The speed of decision making. Some have suggested giving subsystem maintainers commit access. I am not sure that's a good idea, given Drupal's current architecture. It's still too intertwined, with poor separation between systems. Hooks are well and good but they don't separate subsystems from each other. (I freely admit that one of my ulterior motives with the handlers project is to encourage further separation between components, making it easier to grok them separately, test them separately, and develop them separately, even in contrib. I hope to get back to that soon, as I've been heavily distracted by personal matters of late.) For subsystem committers to work, we would need to have a way for there to be layered levels of commit. That is, as others have mentioned before, the Linux model. Linus almost never reviews or even sees individual patches. He gets periodic rollups from subsystem maintainers, reviews them in general but not at a line-by-line level, and adds in a great deal of trust that said subsystem maintainer is doing his job and that what he's getting has already been vetted by someone who understands, say, the power management system far better than he does. It's the power management subsystem maintainer's job to go line by line, not Linus'. We could adopt that sort of model without giving subsystem maintainers core commit access, if and only if 1) We switched to a distributed VCS, because CVS really can't handle that. Really, I see greater autonomy for submaintainers and a DVCS for core as fundamentally linked; doing one without the other is pointless. 2) We need to trust that subsystem maintainers have and are implementing long-term planning that makes sense, and then let them sweat the line-by-line details. As webchick noted earlier, and as I discussed with her in IRC, we need more "maintainer teams" like we have for the DB system. The DB system is sort of an oddball case, though, since it breaks into subcomponents so easily (each driver is its own subcomponent). Of course, if we want to get more people to be domain experts, whether formally recognized or not, there needs to be an incentive. "Pleeeeeese???" is our current incentive model, and I don't think it works all that well. :-) Informal karma only goes so far. What is the incentive for people to become and remain domain experts? Now that I have finally managed to get query builders into core so that I never have to deal with the frickin' INSERT query syntax again, what's the incentive for me to remain active and on top of the DB system? (Not that I'm planning to wander off and ignore it, mind you; I'm just using that as an example.) I first became the DB maintainer right before Szeged. In Szeged, I asked Dries and Angie what they wanted me to do, exactly. "OK, so what did I just sign up for?" Their answer was "just keep doing what you're doing". That didn't really make much sense to me, as what I'd been doing was developing a several hundred KB patch over the course of a year and a half and not sleeping for two weeks in order to get it committed and I really didn't want to keep doing things that way. :-) I've not gotten a particularly different answer since. Sometimes I see "I'd like to get Crell's thoughts on this issue" type comments, and sometimes I get emails with an issue that a core committer wants me to have a look at, but usually I know about them already because I have the DB issue queue bookmarked. I don't mean that to sound like a whiny complaint. The point is that if I, as a current submaintainer / domain expert, don't really know what the benefit is for being one besides feeding my own ego, then what incentive is there for anyone else to become a domain expert on FAPI, or to give forum.module the extensive love it desperately needs? Which brings me to something else that Dries said earlier about the micro vs. macro level. Looking at the macro level and planning things out to ensure consistency and so forth is very important. It also has a name, which is even dirtier to mention around here than "backward compatibility" but I shall do so anyway: A roadmap. Roadmaps help ensure that small tasks fit together into a cohesive whole, and that everyone knows what that whole is. They also give people a clear task list of things that need to be done and WHY. That makes life easier for developers and for reviewers, because they can see why we're making this change to a given piece of code; it's a small change that is part of a larger effort, and the larger effort makes sense. OK, call them community initiatives or whatever; that's still what they are. We're seeing more of them appear haphazardly in the issue queue with multi-part issues that reference each other. That's a very good thing. Drupal is, really, past the size at which "scratch an itch somewhere" can be the primary organizational mechanism, especially if we actually want clean and consistent APIs. OK, so we don't necessarily need one huge roadmap for all of core. Given our development model I doubt that would work anyway. But, if we tone down the anti-planning rhetoric and instead allow for smaller, subsystem roadmaps or particular campaign roadmaps (like Fields-ifying core), that gives people more context in which to review, as well as more documentation on why something is being done, which again makes it easier to review. Then, allow submaintainers to direct roadmaps. If someone wants to become the big fish in the little pond for blog.module and do amazing things with it, and it's someone that has already shown that they are reliable in general, OK. Let's see what your plan (roadmap) is. Discuss and approve it in general. Let them work on it and also be the go-to person for anyone wanting to work on it to make sure that that particular piece of core is consistent and clean and robust. That's their responsibility... and also their authority. When something comes down to a subjective opinion question, the submaintainer's opinion needs to carry more than usual weight. (Objective measures like performance, sure, benchmarks win. I'm referring to judgment calls.) Then the subsystem maintainers, together with the maintainers (Dries and Angie for D7), coordinate to ensure consistency and cleanliness and elegance across core, so that the API decisions for node.module and the API decisions for comment.module and the decisions for FAPI all mirror each other where appropriate, which in turn leads to more consistent and therefore more learnable APIs, which lowers the barrier to entry for future reviewers, etc. Positive feedback loops are a good thing. -- Larry Garfield larry@garfieldtech.com
Some have suggested giving subsystem maintainers commit access. I am not sure that's a good idea, given Drupal's current architecture. ... Looking at the macro level and planning things out to ensure consistency and so forth is very important. It also has a name, which is even dirtier to mention around here than "backward compatibility" but I shall do so anyway: A roadmap.
FWIW, it mostly agree with Larry's points. But anyway. You all lost me here. I'm already considering whether the issue I face is really my own, personal issue only. I consider myself a contrib developer, reviewing and providing "upstream" patches for core occasionally. I have to stop doing that, because I still have to rewrite contrib modules for new API features in D6 and to improve them in general. These are the applications I (and the users of my modules) build Drupal sites with. My time is limited (I have a life as well), so I have to stop caring about D7/HEAD. Regarding better WYSIWYG support in Drupal, someone else has to take over the left side of the roadmap: http://groups.drupal.org/node/6492/summary Hopefully, the stuff that ends up in core will be compatible and useful for Wysiwyg API in D7 in the end. The same applies to some other modules. Especially regarding UX and Drupal's administrative interface, I've lost track and hope anyway already. I can only hope that one will be able to disable any oh-so-nifty-UX-widgets that may be added in D7. Thanks to Karoly for bringing up this discussion. sun
On Monday 20 April 2009, Nathaniel Catchpole wrote:
The only way that code contributions to core get measured is when Greg Knaddison parses commit logs and puts everything into a spreadsheet once or twice a year - our commit model also completely breaks sites like ohloh.
It's not the commit model that breaks Ohloh but the fact that we're using a distributed version control workflow with a centralized one (CVS) as tool. That's a mismatch, because most projects using centralized VCSs don't work that way. In fact, iirc then Linus Torvalds dropped CVS in favor of Bitkeeper (later Git) because the former did not work with his development model. Now they're working as a bunch of subsystem teams with each subsystem maintainer keeping track of his version of the Linux kernel, while the final word is still spoken by Linus who has a lot of say in terms of direction and the "large picture" (or should I say "macro-problems"). Sound familiar? Of course, even current DVCSs treat reviewed-by and approved-by as part of the textual message instead of core concept. As long as it's standardized though, Ohloh should be able to deal with it :)
Hi, this or similar topics come up again and again. The fact of repetition of complains and topics like this comming up from time to time (see eg. october / november 2008: http://lists.drupal.org/pipermail/development/2008-October/031263.html). So putting in the mailinglist more arguments for whatever will probably not help, since these discussions and all related arguments come up every few months. What can we learn form that fact? 1.) The topic was not solved in November 2008 - at least not for the people discussing in this thread now. 2.) The process to solve the issue '[development] How to post bug reports and patches' failed! (hope the following is clear enough for chx ;-) ) What can we do? 1.) Getting a better insight of what are the complains / bugs of the actual problem / issue. 2.) Define with all related persons (users, developers ...) a aim / goal. 3.) Define measures for the aim / goal to measure progress and completion (or for ongoing task monitoring criteria and corridors where the measured values should stay in) 4.) Identify the influencing factors for the measurement criteria. 5.) Integreate all criteria and influencing factor to a model of the problem. 6.) Identify all people who rule over the influencing factors and who are affected of the problem, identify the needs of all the involved persons. 7.) Decide with all involved persons what influencing fatctors should be changed. 8.) Create a project plan for the change. 9.) Perform the planed changes with the involved persons. 10.) Check your results on the defined measurements and criterias. 11.) Decide with all involved persons to iterate or to stop. So please let me know, if you understand these eleven steps, bevor we go to discuss how we walk the talk. Best Thomas Zahreddin (I pointed this out to Dries even in Barcelona, but we did not find the time to speak about in detail - so unsolved problems keep coming back ;-) )
On Monday 20 April 2009 15:30:03 Nathaniel Catchpole wrote:
One issue with this is the way our user profiles work in terms of highlighting contributions. I'd been contributing to core for more than a year before I got a CVS account, and that was when one of my core patches removed a slice of user.module on the condition it was maintained in contrib. So for a long time I never had any projects listed on my profile page, nor any entries in 'track code'. I've still only got commit access on a handful of projects, and don't actively maintain any of them - a situation I'm quite happy with. However if I posted a couple of crappy modules and committed loads of tiny patches to them, then to the casual observer that'd make the user profile look a lot beefier.
The only way that code contributions to core get measured is when Greg Knaddison parses commit logs and puts everything into a spreadsheet once or twice a year - our commit model also completely breaks sites like ohloh. Personally I'd really like better statistics on how many people are reviewing/submitting patches, getting them committed etc. since it'd at least help to spot clear trends when having discussions like these. And I think it'd be a bit of incentive to review and post patches more if there was some way of having that data recorded more firmly than just in commit messages themselves.
Heh. So I had a project that I started on last fall which would do exactly this kind of thing - collect richer statistics on drupal participation - and I'm gonna have to finish if I want to get my degree. Which reminds me that I've been meaning to ask if someone couldn't point me towards any colossal discussion(s) we've had about this kind of thing of thing in the past, so that I can digest what's already been thrown around. cheers Sam
On Mon, Apr 20, 2009 at 9:38 PM, Sam Boyer <drupal@samboyer.org> wrote:
When reading this little part of the thread, I got this mental image of Angie and Dries each driving enormous vans along a precarious cliffside road, trying to listen to a cacophony of conflicting navigational advice (ranging in quality from "local expert guide" to "pin the tail on the donkey") mixed in with a healthy dose of griping and "ARE WE THERE YET?!?"
Roflmao! -- Dries Buytaert :: http://buytaert.net/
On 20-Apr-09, at 2:44 PM, Earl Miles wrote:
The reason I think more committers in targeted areas will work is because I think it will create more activity in those areas. The people who have the right to commit will have a more vested interest, and therefore you will get more of their time, effort and energy, plus any that they can draw to them. webchick commented earlier that one of the issues was providing an incentive to do reviews. And that's certainly a key issue, right there. There isn't much incentive to do reviews. There's the good of the project, but there is no pride of ownership in a review. The pride of ownership drives a lot of the initial development, and it drives the committers, but the reviewers?
I cannot remember a time in the 4 years I've been with this project that we haven't had a longstanding complaint about lack of good patch reviews. That is one of the first complaints I heard then, and it is one of the complaints I hear now. It has not changed, and asking people to do more reviews has already proven to be ineffective in combatting the issue.
Give your reviewers some ownership and you'll find a lot more people interested in reviewing.
Now this "how to get more reviewers" is a line of discussion I'm *very* interested in having since, regardless if we have subsystem committers or not, we need to resolve it. Here are some ideas around that: 1. A community-led mentorship/journeyman program for people who want to get involved in core development and don't know where to start. Basically, taking what we do at code sprints at Drupalcons twice a year that always lead to a big increase in new core contributors, and making an e-version of it, or something that could be done in person at camps and local meetups. 2. Developing some sort of system that can formalize a bit more "patch review swaps." The biggest incentive people currently have for reviewing a patch is that they want one in core and need to get other reviewers to look at it in order to do so. Making it easier to identify people who are willing to engage in this would help everyone out. 3. Providing tools on g.d.o, d.o, or elsewhere to make it easier for core subsystem maintainers to nurture their pocket of Drupal core. Not really sure what those would be yet. Perhaps one aspect could be liaising with the documentation team to help get their subsystem documentation cleaned up and some clear step 1, step 2, step 3 stuff. 4. Monthly contests for "review queue smashing," like other projects have "bug fix days" or similar. Winners get their name in lights on the d.o front page, or we could pony up for some cash incentives or similar. 5. Calling specific attention out to people who did thorough reviews in commit messages, either by including them in the list of contributors or something like "#xxxx by dev1, dev1, and reviewed by rev1: wah wah wah." 6. Add more people to MAINTAINERS.txt, and have it reflect the current committer "trust" model. If that document can be trusted, this makes it easier for reviewers to know who to talk to if they're interested in helping out with X system. We could also add subsystem maintainer names to the pages on http://drupal.org/community-initiatives/drupal-core perhaps. Curious about others people might have as well. -Angie
Am Montag, den 20.04.2009, 15:39 -0400 schrieb Angela Byron: ...
Monthly contests for "review queue smashing," like other projects have "bug fix days" or similar. Winners get their name in lights on the d.o front page, or we could pony up for some cash incentives or similar.
I don't want many loosers and a few winners - I want cooperation! Is this so hard to understand? Best Thomas Zahreddin
On Apr 20, 2009, at 3:39 PM, Angela Byron wrote:
On 20-Apr-09, at 2:44 PM, Earl Miles wrote:
The reason I think more committers in targeted areas will work is because I think it will create more activity in those areas. The people who have the right to commit will have a more vested interest, and therefore you will get more of their time, effort and energy, plus any that they can draw to them. webchick commented earlier that one of the issues was providing an incentive to do reviews. And that's certainly a key issue, right there. There isn't much incentive to do reviews. There's the good of the project, but there is no pride of ownership in a review. The pride of ownership drives a lot of the initial development, and it drives the committers, but the reviewers?
I cannot remember a time in the 4 years I've been with this project that we haven't had a longstanding complaint about lack of good patch reviews. That is one of the first complaints I heard then, and it is one of the complaints I hear now. It has not changed, and asking people to do more reviews has already proven to be ineffective in combatting the issue.
Give your reviewers some ownership and you'll find a lot more people interested in reviewing.
Now this "how to get more reviewers" is a line of discussion I'm *very* interested in having since, regardless if we have subsystem committers or not, we need to resolve it.
A huge disincentive for getting involved in core is that it doesn't yield immediately deployable results. This keeps shops and individuals from putting more labor towards core work. Robert Douglass suggested on this thread to remove modules that don't absolutely need to be in core (aggregator, blog, blogapi, forum). I absolutely concur. This would not just make core easier to overlook and remove a huge tax from the drupal core infrastructure, but it would most importantly allow us to release and deploy these modules quicker. Like mentioned already by others on this thread, these modules could create a new tier of Drupal modules on d.o. that would be packaged with installer profiles and maintained for core compatibility for every release of the current Drupal release branch and Drupal HEAD. In my mind, such a third tier of modules would fill the gap between core and contrib: it would create a peer reviewed space between Drupal core and Drupal contrib while not suffering from long release cycles. In turn we would get more reviewers for the new tier of modules but I also for Drupal core as there is going to be more activity on modules that will be available in the current release branch *and* in HEAD. A third tier of Drupal modules does not solve all issues of course, but I think it makes our problem *a lot* smaller.
Here are some ideas around that:
1. A community-led mentorship/journeyman program for people who want to get involved in core development and don't know where to start. Basically, taking what we do at code sprints at Drupalcons twice a year that always lead to a big increase in new core contributors, and making an e-version of it, or something that could be done in person at camps and local meetups.
2. Developing some sort of system that can formalize a bit more "patch review swaps." The biggest incentive people currently have for reviewing a patch is that they want one in core and need to get other reviewers to look at it in order to do so. Making it easier to identify people who are willing to engage in this would help everyone out.
3. Providing tools on g.d.o, d.o, or elsewhere to make it easier for core subsystem maintainers to nurture their pocket of Drupal core. Not really sure what those would be yet. Perhaps one aspect could be liaising with the documentation team to help get their subsystem documentation cleaned up and some clear step 1, step 2, step 3 stuff.
4. Monthly contests for "review queue smashing," like other projects have "bug fix days" or similar. Winners get their name in lights on the d.o front page, or we could pony up for some cash incentives or similar.
5. Calling specific attention out to people who did thorough reviews in commit messages, either by including them in the list of contributors or something like "#xxxx by dev1, dev1, and reviewed by rev1: wah wah wah."
6. Add more people to MAINTAINERS.txt, and have it reflect the current committer "trust" model. If that document can be trusted, this makes it easier for reviewers to know who to talk to if they're interested in helping out with X system. We could also add subsystem maintainer names to the pages on http://drupal.org/community-initiatives/drupal-core perhaps.
Curious about others people might have as well.
-Angie
Alex Barth http://www.developmentseed.org/blog tel (202) 250-3633
I am in Drupal up to my eyebrows every day but still I am not finding enough time for my few contrib modules, the less for the core patches. I am not giving up though - the opposite, understanding more and more every day, gearing up. Even without being a current core or active contributor I too clearly see a problem with the differing speeds of development of core and contrib, and I very much agree with most of the refreshing proposals of Nedjo Rogers and Robert Douglass above. In general, I think that so powerfully creative and committed (!) people should be given more rights and space for driving development in their areas of expertise. It is sad to hear people who have made major contributions to Drupal feel shackled. Let me give two comments to Angie's otherwise good and useful reasoning: Worse, because all of these subsystems were rapidly evolving within their
own sphere, 20, 30, perhaps 50 patches committed per day, it's become impossible for anyone to oversee all of them; at best you have people inspecting one-off patches here and there. That means things like coding standards compliance, performance benchmarking, algorithm optimization, and the collective DX (not to mention UX) experience of Drupal 7 plummets. Drupal 7 is usable by chx, pwolanin, DamZ, catch, and about 20 other people, and everyone else switches to Django. ;)
I could read what you say as: "no commit rights because the group might not be able to oversee the changes", hence "better slow and frustrating development than fast and exciting one". That might finally stifle Drupal. I firmly believe more coders would be attracted to providing and testing patches if development took up speed in this way. Far too often we see modules sprouting with the apologetic explanation that 'it would take too long in core, see the patch xy that's laying there for months'. Probably in this way the teams of interested coders-turning-experts you talk about would naturally emerge from the exciting development activity in such areas! Further, if the maintainers of subsystems had commit rights they would probably be more motivated to manage and work with such teams, etc. Or turn it around - how can we expect the most able developers be motivated to do fantastic things in their areas of experience if they cannot commit them? "But webchick!" I hear you say. "Obviously, we wouldn't let these subsystem
maintainers work /completely/ bereft of peer review. We'd surely have someone else check over their work." And I say, "Great! Then *why aren't these mythical someones doing this right now*?" and *then* you get at what I believe is the /actual/ root of the problem. ;)
I think the development would speed up, and so those who want to follow would have to keep the pace. Its a tax you pay for the exciting new functionality and stability. It works in major and vital contrib modules, so why not in such specialized areas of core? Problem of overseeing fast changes is there at any speed of development. That can be mitigated by increasing pressure for providing documentation and decision paths along with the code. (Perhaps a condition like: you can commit in your area but only with full documentation of not only the code but also your clearly stated and referenced reasoning and immediate roadmap.) As I said, I am determined to contribute much more than I have ever done, to contrib, and to core. But I believe adding more rights to the biggest Drupal coders as proposed above would greatly help me and everybody, and Drupal as a project. Anyways; back to coding! See you around -- Tomáš -- Tomáš Fülöpp (vacilando) http://vacilando.net
Quoting Bryan Ollendyke <bto108@psu.edu>:
lol yeah, I believe the intent of the original author was to hold off on D7 until D6 matured more from a crontrib standpoint. I'll +1 that idea. D5 was and still is a very mature platform that you can develop and build off of, seemingly without "need" to upgrade if you have a site up and running stablly; this isn't always the case with 6 with some of the more major projects just getting to full releases in the last 6 months.
Especially when you have issues with core modules causing issues with cron actually executing. See http://drupal.org/node/436028 for what I'm referring to. -- Earnie -- http://r-feed.com/ -- http://for-my-kids.com/ -- http://www.4offer.biz/ -- http://give-me-an-offer.com/
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Daniel F. Kudwien schrieb:
I don't think you are making much sense here. You claim that module developers haven't found time to add the new D6 features in their code and at the same time want to have a D7 which is more mature (and presumably thus contains even more features).
In the end your proposal would only serve to disconnect core and contrib even more.
Cheers, Ger»release early, release often«hard
In case this really was ambigious:
Let it [D6] mature.
=> i.e. defer the code freeze for D7.
Obviously, I mean the exact opposite of what you understood. Contrib needs time to catch up with core.
Ah, that makes much more sense, although I still disagree. If a module author hasn't found the time to include D6 features over one year after D6 was released, he'll not find time regardless of when D7 is released. If, however, D7 was released in June, he'd look at his code again at least in July and maybe kick out some D5 leftovers. Cheers, Gerhard -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAknshAwACgkQfg6TFvELooQUXwCgwKKCLDzJJAW/GapYvVMDMRrr XnAAoJBa2oCwisnKX0sNSJFjVSCBRHMw =G4Z8 -----END PGP SIGNATURE-----
Quoting Karoly Negyesi <karoly@negyesi.net>:
Hi,
I know there are many patches are written, reviewed and committed still. But, without intending to critique of anyone in particular, I see a very dangerous cycle:
As Drupal 5 and 6 are very good, rock solid releases, people are satisfied and very busy there, much less is the need, the drive and the availability for core development. This means less patches are written. Also core got bigger. Very, very few people touch actions/triggers, fields, filter system, openid or for that matter, menu just to name a few. Once a patch is written , it needs a review. However, the testing bot, curiously, made patch reviews even harder -- no longer can a novice just install a patch, click around and report back "works" ( the testing bot, of course added a lot of quality to Drupal, I am not saying, down with the bot). This has very severe consequences: we always had too few reviewers and now the entry barrier is even harder as you need to do meaningful code reviews all the time. Low reviewer activity means patches do not get 'bumped' and they linger. Lingering is exacerbated by the loss of Steven and that the single guy who can call the big decisions now has a kid (soon two) and two companies to run. Of course, this leads to frustration on behalf of the patch writers and even less patches get written or reviewed and people draw back into their own little contrib empire where they call the shots...
While I review a few drupal core patches once in a while, there are still many prime modules needing a stable port to D6; xmlsitemap for example. I'm putting most of my time into trying to get things like this module working so I can port my sites to D6. Otherwise D7 is just a waste of time since upgrading to it requires upgrading to D6 first. -- Earnie -- http://r-feed.com/ -- http://for-my-kids.com/ -- http://www.4offer.biz/ -- http://give-me-an-offer.com/
On 19-Apr-09, at 4:38 PM, Karoly Negyesi wrote:
we always had too few reviewers and now the entry barrier is even harder as you need to do meaningful code reviews all the time
A significant issue I've encountered in participating with core patches is the time required to synthesize hundreds of comments into a cohesive overview of where the issue is at and what the next steps are. Of course, most of the comments will be valid points of discussion, but the barrier to entry can be quite high due to the initial time involved to get familiar with the issue. One idea to help simply patch reviews would be to add single checklist of review tasks for an issue. CNR is too vague of a status to be meaningful in many cases. When a patch is set to CNR for the first time, review items (such as those at http://drupal.org/patch/review) could be added to the issue automatically. One item would be "passing the test bot". Each checkbox could have associated comment ID / user name fields, so it would be easy to say "this code has been benchmarked at comment #1234". Those more involved with the issue would need to be able to add additional tasks as well. Code isn't set to RTBC until all of the tasks have been marked as done. I think this would help get more individuals reviewing patches. For example, perhaps one of the tasks for UI related patches would be "Needs screenshot of changes". A potential reviewer could search for all CNR issues where that task needs to be done, and do them all at once. Developers have specific specialities, and as-is it's not incredibly easy for new reviewers to find the issues where they can contribute the most. --Andrew
Some word from the contrib side of drupal: Why am I not participating in reviews of core patches? Naked fear. How could I possibly understand whats going on in core? Although I am using and developing for drupal since version 3.somthing (??) I still have problems with the inner workings of drupal. One reason for this is the fact that core is a moving target. All kinds of APIs change between major releases. Joe Contrib Developer (thats me) does not understand why. I am sure those changes are important, but I don't understand them. I just fought a furious fight with the fapi last weekend. Something that was working fine 5.x is behaving strangely in 6.x. If I am not understanding the USE of those APIs how can I judge patches to the code behind them? Solution: Dunno. Maybe extensive documentation? Not just docs for the APIs, but docs about the functionality and docs about design decisions. Such docs would also aid ports of modules to new core releases since devs would loose fear. I am currently maintaining scheduler and I am still struggling with the difference between D5 and D6. I dare not even think about a port to D7. One more thought about the docs: Maybe everything is I would need to loose fear is documented. But where? I always get lost in the drupal documentation... My 0.02€, Eric
Am Montag, den 20.04.2009, 18:09 +0200 schrieb Eric Schaefer:
Some word from the contrib side of drupal:
Why am I not participating in reviews of core patches? Naked fear. How could I possibly understand whats going on in core? Although I am using and developing for drupal since version 3.somthing (??) I still have problems with the inner workings of drupal. One reason for this is the fact that core is a moving target. All kinds of APIs change between major releases.
Joe Contrib Developer (thats me) does not understand why.
the answers to the question 'Why ..' are often design decisions and these are maybe documented, e.g. in issues, mailinglist, IRC, groups.drupal.org - but often in a form nobody can tell from that documentation what the decision is.
I am sure those changes are important, but I don't understand them. I just fought a furious fight with the fapi last weekend. Something that was working fine 5.x is behaving strangely in 6.x. If I am not understanding the USE of those APIs how can I judge patches to the code behind them? Solution: Dunno. Maybe extensive documentation? Not just docs for the APIs, but docs about the functionality and docs about design decisions. Such docs would also aid ports of modules to new core releases since devs would loose fear. ..
The question behind is: how are design decisions made _and_ coordinated? my suggestion form e.g. Nov. 2008 original: http://lists.drupal.org/pipermail/development/2008-November/031354.html " .. If the group of persons working on a module keeps a logbook about their decisions, then everybody is able to follow the decisions regarding a module. A short description of this method of dynamic selfgovernance can be found on http://en.wikipedia.org/wiki/Sociocracy or as free pdf http://www.governancealive.com/Links/The_Creative_Forces_of_Self_Organizatio... A longer description is in Buck, John and Sharon Villines (2007). We the People: Consenting to a Deeper Democracy, A Guide to Sociocraty " So what do you think now of a _log_ for requirements and design decisions for core, APIs and modules?
One more thought about the docs: Maybe everything is I would need to loose fear is documented. But where? I always get lost in the drupal documentation...
My 0.02€, Eric
Eric Schaefer wrote:
Why am I not participating in reviews of core patches? Naked fear. How could I possibly understand whats going on in core? Fire up your debugger (i use eclipse + pdt + zend debugger) and step through the major operations of the bootstrap process to discover the big picture. Then walk through the loading of a node that is decorated with cck fields and taxonomy terms. Finish up your tour with the editing of a form (node, admin) to get a good look at fapi and the form processing involved in form.inc and you have a VERY strong foundation from which to build. Most people involved in core don't understand 100% of it. Many specialize in a particular area. This is why we have reviews and the testbot. One reason for this is the fact that core is a moving target. This is a conscious and purposeful decision on the part of the project it will not change.
If I am not understanding the USE of those APIs how can I judge patches to the code behind them? api.drupal.org lists each function and in the declaration is detailed everywhere that function is also used in core. This is a good start but your IDE can help you alot more if you use one especially if you put a breakpoint in your function and start debugging to see how various functions arrive at the location and what the variables are set to at that time.
Please take the time and try it just once. Install D7, apply a couple patches, review their changes and you'll be surprised how quickly you fall in place. We want your help. I normally find that future contributors are simply scared away by the challenge certainly not actually unable to rise to it. Pick a patch that seems interesting to you. If you ever need help find me in IRC and I'll always be happy to lend a hand. -mf
2009/4/20 Michael Favia <michael@favias.org>:
Fire up your debugger (i use eclipse + pdt + zend debugger) and step through
I will.
api.drupal.org lists each function and in the declaration is detailed everywhere that function is also used in core. This is a good start but your IDE can help you alot more if you use one especially if you put a breakpoint in your function and start debugging to see how various functions arrive at the location and what the variables are set to at that time.
Please take the time and try it just once. Install D7, apply a couple patches, review their changes and you'll be surprised how quickly you fall in place.
I am sure you are right. But still. I can understand a lot of stuff by debugging and reading code. But it is really hard to understand the bigger picture from the code view. It can see WHAT the code does but I might not be able to understand WHY or it might be very time consuming. Eric
As core matures, we might see more patches getting stale. I think the reason is (at least) two-fold: 1) More patches compete for attention, but fewer of these patches are truly important. In this case, patches getting stale is not necessarily a bad thing because history has proven that important patches will eventually bubble to the top for the right reasons. However, I understand that it might be frustrating because it takes longer, and some patches don't make it at all. As a core contributor, I think you have to trust that you patch will eventually bubble up -- it just takes a bit longer for people with the same pain point to weigh in and participate in the review process. It is also important that you convince other people as why your patch should bubble up -- this is something a lot of people can get better at. 2) Drupal is getting increasingly complex. You already hinted that the test framework increases the barrier to participation for aspiring developers. However, it is not just that -- things like the new database abstraction layer, the registry, the form API, the new menu system, the theme preprocess functions, load multiple and jQuery make Drupal a better platform, but also increase the barrier to participation. As a result, fewer people are capable of reviewing patches in great depth. In ways, complexity is a disease. We have to strive towards making Drupal easier to develop for while maintaining its functionality and flexibility. Fighting unnecessary complexity and abstractions is increasingly important. Webchick and I have been pretty good at managing the RTBC queue. It used to be several pages long, but the last couple of months we never had more than 20 Drupal 7 patches marked RTBC (as far as I remember). We only have 8 Drupal 7 patches marked as RTBC at the time of this writing. But even today, with many patches in the 'code needs review' state (not in RTBC state), core continues to be a fast moving target. I don't think we should speed things up, or we risk de-railing contrib, and many of the contributed module authors in the process. I think we progressing at a good pace, and I think many important patches have gone in. Already, Drupal 7 promises to be a great release. On Sun, Apr 19, 2009 at 10:38 PM, Karoly Negyesi <karoly@negyesi.net> wrote:
Hi,
I know there are many patches are written, reviewed and committed still. But, without intending to critique of anyone in particular, I see a very dangerous cycle:
As Drupal 5 and 6 are very good, rock solid releases, people are satisfied and very busy there, much less is the need, the drive and the availability for core development. This means less patches are written. Also core got bigger. Very, very few people touch actions/triggers, fields, filter system, openid or for that matter, menu just to name a few. Once a patch is written , it needs a review. However, the testing bot, curiously, made patch reviews even harder -- no longer can a novice just install a patch, click around and report back "works" ( the testing bot, of course added a lot of quality to Drupal, I am not saying, down with the bot). This has very severe consequences: we always had too few reviewers and now the entry barrier is even harder as you need to do meaningful code reviews all the time. Low reviewer activity means patches do not get 'bumped' and they linger. Lingering is exacerbated by the loss of Steven and that the single guy who can call the big decisions now has a kid (soon two) and two companies to run. Of course, this leads to frustration on behalf of the patch writers and even less patches get written or reviewed and people draw back into their own little contrib empire where they call the shots...
Regards
Karoly Negyesi
-- Dries Buytaert :: http://buytaert.net/
Dries Buytaert wrote:
As core matures, we might see more patches getting stale. I think the reason is (at least) two-fold:
1) More patches compete for attention, but fewer of these patches are truly important. In this case, patches getting stale is not necessarily a bad thing because history has proven that important patches will eventually bubble to the top for the right reasons.
If the 'right reasons' are because the developers got frustrated with the process and quit pushing their patch, whereas other developers are extremely stubborn and managed to push their patch through anyway, then sure. But just because some developer is bullheaded enough to wade through the crap doesn't mean that's the right reasons.
I agree with Earl here. Several of my patches, including quite big popular patches that people have bought me beers for, have only got in because I'm a stubborn git - either re-rolling them 50 times due to patch conflicts or writing reams of arguments to get them committed. Not to mention nagging people time after time on irc to get reviews and/or advice. Plenty of other people give up (either on the individual patch or core as a whole), and then those patches either sit in the queue for years or sometimes get revived by another developer, and then they can still sit in the queue for another year or two. This doesn't mean that /all/ lingering patches are lingering for the wrong reasons, but there's a lot of overhead getting patches in other than just writing the code. Having said that, none of this overhead exists for reviewing patches - and it's slow review turnover which leads to a lot of the lingering, and being someone who reviews a lot of patches is one of the easiest ways to get your own patches reviewed faster. On Mon, Apr 20, 2009 at 7:07 PM, Earl Miles wrote:
Dries Buytaert wrote:
As core matures, we might see more patches getting stale. I think the reason is (at least) two-fold:
1) More patches compete for attention, but fewer of these patches are truly important. In this case, patches getting stale is not necessarily a bad thing because history has proven that important patches will eventually bubble to the top for the right reasons.
If the 'right reasons' are because the developers got frustrated with the process and quit pushing their patch, whereas other developers are extremely stubborn and managed to push their patch through anyway, then sure. But just because some developer is bullheaded enough to wade through the crap doesn't mean that's the right reasons.
Some responsibilities, actions, and expectations seem to be unclear to me: Is getting the patches committed the problem or is the problem getting them reviewed and to the point where they can be committed the problem? Is it getting the from needs review to RTBC? What is the role of a core committer in the review process compared to your average joe developer. Obviously there are differences between your average joe developer and a core committer. I guess I'm trying to find the bottleneck. It doesn't look to be in committing the patches when they become RTBC. It seems to be in the area of reviewing patches and understanding subsystems and good design (especially in the complex parts). Am I missing something? If this is the case, from a process perspective, adding more core committers doesn't solve the problem we are facing. Or, am I missing something? On Apr 20, 2009, at 2:24 PM, Nathaniel Catchpole wrote:
I agree with Earl here. Several of my patches, including quite big popular patches that people have bought me beers for, have only got in because I'm a stubborn git - either re-rolling them 50 times due to patch conflicts or writing reams of arguments to get them committed. Not to mention nagging people time after time on irc to get reviews and/or advice.
Plenty of other people give up (either on the individual patch or core as a whole), and then those patches either sit in the queue for years or sometimes get revived by another developer, and then they can still sit in the queue for another year or two.
This doesn't mean that /all/ lingering patches are lingering for the wrong reasons, but there's a lot of overhead getting patches in other than just writing the code.
Having said that, none of this overhead exists for reviewing patches - and it's slow review turnover which leads to a lot of the lingering, and being someone who reviews a lot of patches is one of the easiest ways to get your own patches reviewed faster.
On Mon, Apr 20, 2009 at 7:07 PM, Earl Miles wrote:
Dries Buytaert wrote:
As core matures, we might see more patches getting stale. I think the reason is (at least) two-fold:
1) More patches compete for attention, but fewer of these patches are truly important. In this case, patches getting stale is not necessarily a bad thing because history has proven that important patches will eventually bubble to the top for the right reasons.
If the 'right reasons' are because the developers got frustrated with the process and quit pushing their patch, whereas other developers are extremely stubborn and managed to push their patch through anyway, then sure. But just because some developer is bullheaded enough to wade through the crap doesn't mean that's the right reasons.
Op 20 apr 2009, om 20:36 heeft Karoly Negyesi het volgende geschreven:
Plenty of other people give up (either on the individual patch or core as a whole),
Note: this is very very and a very sad important fact.
Yup, I was one of them.. I was tired of fighting my patch up the issue queue for months. Also the lack of interest of a core committer, for answering some simple questions (what is wrong with this patch? Why isn't this committed yet?) closed the door for me. Now, after 1,5 year without drupal I am trying to get up to speed again. Mainly because the file system has been improved, and because of the fact that I wan;t to participate in the image API overhaul. Seriously, almost 60-70% of the core modules, i have never used or only used to scratch my own itch. The motto "talk is silver, code is gold" has died a long time ago imo, when drupal got more commercial. Now, whenever I look at the drupal.org homepage, drupal.org is being (ab)used for asking people money for some conference, or whatever else.. Drupal isn't the drupal anymore which I *loved* so much in the past. It is grown up, and made to be a money making machine. In my opinion, that is truly a very very sad fact. (Even when it isn't a fact, it's sad people are getting the feeling that this is the case) Kind regards, Stefan
Karoly Negyesi wrote:
Plenty of other people give up (either on the individual patch or core as a whole),
Note: this is very very and a very sad important fact.
next fact: there won't be more good core reviewers joke1: your old feature patch is still not in core. Don't worry "There are at least two modules for that!" joke2: create another account on d.o and RTBC your own patch, then at least the core committer will give you a quick review on the changes. joke3: if you need a core patch review, then you should create a blog post about it first. Károly: sorry, I will not review patches created for the menu module, as I still do not know it enough, and no time to learn it. I have a huge list what should I work on, as many Drupal developers have, but core development is not on the list any more.. I don't know anyone other than Peter who would make the time and would give you a good review. I trust you, so your changes should be in the module. My real advice, if it is possible then create a contrib module which is better than the one in core. I will switch without hesitation.. My reviews will be submitted as bug reports. If something does not work you will know it sooner or later.. Advanced menu would be a good name..
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Earl Miles schrieb:
Dries Buytaert wrote:
As core matures, we might see more patches getting stale. I think the reason is (at least) two-fold:
1) More patches compete for attention, but fewer of these patches are truly important. In this case, patches getting stale is not necessarily a bad thing because history has proven that important patches will eventually bubble to the top for the right reasons.
If the 'right reasons' are because the developers got frustrated with the process and quit pushing their patch, whereas other developers are extremely stubborn and managed to push their patch through anyway, then sure. But just because some developer is bullheaded enough to wade through the crap doesn't mean that's the right reasons.
Right or wrong: this is not new. You always had to be persistent (or stubborn) in order to push through larger changes. Cheers, Gerhard -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAknsxiQACgkQfg6TFvELooQqyACgsWljMV3QDG+bHe8pDa0RCs/s M7sAn1fA7LrXKLnypzJdKupGvxYxByOy =h5fv -----END PGP SIGNATURE-----
Gerhard Killesreiter wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Earl Miles schrieb:
Dries Buytaert wrote:
As core matures, we might see more patches getting stale. I think the reason is (at least) two-fold:
1) More patches compete for attention, but fewer of these patches are truly important. In this case, patches getting stale is not necessarily a bad thing because history has proven that important patches will eventually bubble to the top for the right reasons. If the 'right reasons' are because the developers got frustrated with the process and quit pushing their patch, whereas other developers are extremely stubborn and managed to push their patch through anyway, then sure. But just because some developer is bullheaded enough to wade through the crap doesn't mean that's the right reasons.
Right or wrong: this is not new. You always had to be persistent (or stubborn) in order to push through larger changes.
I don't disagree with this. I disagree with the interpretation: Calling 'being stubborn' the right reasons is not an indication that the right patches float to the top automatically. It is a little bit like argument trickle-down economics: Giving money to the rich enriches the poor.
On Mon, Apr 20, 2009 at 1:59 PM, Gerhard Killesreiter < gerhard@killesreiter.de> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Earl Miles schrieb:
Dries Buytaert wrote:
As core matures, we might see more patches getting stale. I think the reason is (at least) two-fold:
1) More patches compete for attention, but fewer of these patches are truly important. In this case, patches getting stale is not necessarily a bad thing because history has proven that important patches will eventually bubble to the top for the right reasons.
If the 'right reasons' are because the developers got frustrated with the process and quit pushing their patch, whereas other developers are extremely stubborn and managed to push their patch through anyway, then sure. But just because some developer is bullheaded enough to wade through the crap doesn't mean that's the right reasons.
Right or wrong: this is not new. You always had to be persistent (or stubborn) in order to push through larger changes.
I noted this wasn't new in a brief chat on irc. More its amplified. It sounds like some people feel things have changed procedurally in someway that has made things worse but its more we haven't grown in the way we do things in a way that's kept up with the size of our community. When the community was smaller it was easier for you to just hope on IRC and talk out a concern with someone about an issue before derailing things. Now there are so many people discussing and so many people with a stake in core these derailing discussions become more and more common place and harder to get around. The way we seem to have delt with this for some crucial systems in the D7 is with numerous code sprints. I don't think this is going to pan out as an answer either since not everyone can pick up and travel or get time off to sit down and contribute to things they are interested in. Another option that's happened is making little Drupal forks in external repositories where they can work on these larger problems. Without any policy or direction this is its own scary can of worms. So, I definitely feel we need to take a look at how we develop core a bit. There seems to be a need to group core into systems and initiatives that people can direct their focus on. I don't think what we really want is to remove the core commiter role but we want to widen things directly below that. Based on the external repositories, maybe the more distributed lieutenant model used in the linux kernel would work for us? That might have a heavy infrastructure cost but it might address the problem. Another option might be just something clever in the d.o redesign that more closely tie issues with groups so issues are easier to find, follow and discuss. There might even be some sort of group/core component ownership that can help funnel better reviews into patches and by proxy better patches to the core committers. Either way, I think that we need something allows the community to scale a little better as we continue to grow. -- James Gilliland
On Mon, Apr 20, 2009 at 1:07 PM, Earl Miles <merlin@logrus.com> wrote:
Dries Buytaert wrote:
As core matures, we might see more patches getting stale. I think the reason is (at least) two-fold:
1) More patches compete for attention, but fewer of these patches are truly important. In this case, patches getting stale is not necessarily a bad thing because history has proven that important patches will eventually bubble to the top for the right reasons.
If the 'right reasons' are because the developers got frustrated with the process and quit pushing their patch, whereas other developers are extremely stubborn and managed to push their patch through anyway, then sure. But just because some developer is bullheaded enough to wade through the crap doesn't mean that's the right reasons.
That, and time. Core committers, as the DCDC comment noted, are those with more time to devote to the effort. It's tough to find the time to contribute to core because the entry point is high, but additionally because it's painful to spend lots of time on a core patch, only to see it languish. The same amount of effort can create and maintain several contrib modules.
Dries Buytaert wrote:
weigh in and participate in the review process. It is also important that you convince other people as why your patch should bubble up -- this is something a lot of people can get better at.
This statement right here is what's wrong with Drupal core development. At this time, you must be better at debate, arguing and general Drupal politics than you are at actual code design and implementation. At one point 'talk is silver, code is gold' was the motto. But this has changed. Now talk is gold and code is defined by those who talk the best.
If code is gold, then documentation is platinum. -----Original Message----- From: development-bounces@drupal.org [mailto:development-bounces@drupal.org] On Behalf Of Earl Miles Sent: Monday, April 20, 2009 2:15 PM To: development@drupal.org Subject: Re: [development] Very concerned over Drupal's core development Dries Buytaert wrote:
weigh in and participate in the review process. It is also important that you convince other people as why your patch should bubble up -- this is something a lot of people can get better at.
This statement right here is what's wrong with Drupal core development. At this time, you must be better at debate, arguing and general Drupal politics than you are at actual code design and implementation. At one point 'talk is silver, code is gold' was the motto. But this has changed. Now talk is gold and code is defined by those who talk the best.
On Mon, Apr 20, 2009 at 2:14 PM, Earl Miles <merlin@logrus.com> wrote:
Dries Buytaert wrote:
weigh in and participate in the review process. It is also important that you convince other people as why your patch should bubble up -- this is something a lot of people can get better at.
This statement right here is what's wrong with Drupal core development. At this time, you must be better at debate, arguing and general Drupal politics than you are at actual code design and implementation.
At one point 'talk is silver, code is gold' was the motto. But this has changed. Now talk is gold and code is defined by those who talk the best.
The barrier for me isn't the complexity by any means. It's more the politics and sheer stubborness required to implement change. I'm not a politician. I'm a programmer.
I agree with Earl and Darrel, many patches are derailed with endless debates. The question is whether that is good or bad, because the context is missing. I can't speak much about the past, but is it possible that core's development process changed drastically? We introduced an awesome testing framework that ought to increase the quality of code that is committed to core. However, we are still introducing new core bugs (take "FAIL." [1] as an example), despite being reviewed by many eyeballs and having tests. I think this is caused by the steadily increasing complexity of Drupal core - and - by a lack of reviewers that *really* understand core, use-cases in contrib, many sub-systems (not one), and all implications of a core patch. This is the real issue (I believe) Karoly wanted to raise originally. I am not sure whether sub-system maintenance teams will lead to an improvement of this situation. Completely contrary to this proposal, I think our issue is that we are lacking the opposite of domain experts: Drupal core contributors/maintainers/reviewers who understand every single bit of core and potential implications of a changed bit. For the sake of clarity, I have revised Dries' "Drupal learning curve" graph, because I think the important part - the part we are really talking about - is missing: http://www.unleashedmind.com/files/drupal-learning-curve.png Adding new people above the "Core contributor" threshold is what we are mostly doing currently. However, it is my impression that there are very, very few people above the "Trustworthy for core maintainers" treshold. Even less, if you only count people who are known to be "trustworthy" for more than one sub-system. It is possible that you counted only one person (or less?) when climbing up the further tresholds. Maybe that is the issue we are facing? Patches languish in the queue for ages, because some sub-system maintainers subsequently chime in, each one needs own treatment, other contributors chime in, trying to understand the issue, the patch is re-rolled all over again, a branch maintainer shows up who questions the entire approach, and finally the patch may or may not be committed -- still lacking review by someone who understands all changes and implications of the patch. Given all other proposals in this thread, my guess is that adding more maintainers to core might really be a step in the right direction: Handing out responsibility (that comes with a fair amount of power) to core contributors who are already known to be trustworthy (in more than one area) will most likely let them advance in other areas, climbing up the "treshold-ladder" - not only, because people will start to ask about reviews/commits for arbitrary patches. Equally, this action could be done now. [1] http://drupal.org/node/446742 Thanks, sun (who hopes to have reached macro-level with these thoughts)
I'm not going to pretend I read this whole thread, these are some general thoughts on this discussion. I don't have a lot of answers. We should understand why this situation exists before attempting to solve it. I have not contributed a lot to Drupal 7. I didn't contribute a lot to Drupal 6, but that was because I was spending more time on Drupal 5. I can't blame this on the process, I've worked with it and know how it works. The main factor is how I use my time, but the process does make a difference. We should focus on removing barriers and pain points, not adding complexity. Many minor changes are better than major changes. The current bottleneck does seem to be reviewing, not committing. Planning out big changes is hard, but that might be okay. UI makes a big difference. Issue tags changed how people structure their attention. Flag integration will remove subscribe noise. We haven't tested testing in a code freeze situation. More problems should be found and improved. I would like to see more quantitative data. Giant mailing list discussions have their place, but we need some way to test/prove/quantify/measure. Not everything can be quantified, but we should do something better than a big bag of opinions. Structural changes are okay. We have gotten where we are with the same structure since 4.7, and that was simply adding another committer per-version. Something has been working, but the size and scope of the problem has changed. I don't know what/if structural changes would be ideal. Adding more core committers without changing the structure would likely be good short-term, but we will run into the same situation later, that might be okay. Adding subsystem maintainers might be helpful, as long as we remember that writing and committing are different processes. Any additional committers need to be skilled willing volunteers with time. I am fine with the Dries chooses process. If you are seriously interested in filling a new position, you should already be doing what you want to do, then we remove a step from your process. We should not make complex CVS branch/tag/repository systems. We should keep things simple and, if there are structural changes, consider moving to a distributed version control system. API changes per year, reguardless of release cycle, should go down, which is a good thing. This is a more important measure than release cycle length. More features and APIs, not necessarily modules, should go into core, we are too reliant on contrib. Substantive things to do to help are: - Review patches and look for annoyances in the process. - Contribute to the Drupal.org redesign, http://groups.drupal.org/drupalorg-redesign-implementers. - Contribute to project module, http://drupal.org/project/project. - If you are serious about changes, go ahead and do the work. If it works, we can codify it. -Neil -- Neil Drumm http://delocalizedham.com
participants (40)
-
Alex Barth -
Andrew Berry -
andrew morton -
Angela Byron -
Bryan Ollendyke -
Chris Johnson -
Csuthy Balint -
Daniel F. Kudwien -
Darrel O'Pry -
David Metzler -
Dmitri G -
Dries Buytaert -
Earl Miles -
Earnie Boyd -
Eric Schaefer -
Frédéric G. MARAND -
George -
Gerhard Killesreiter -
Gábor Hojtsy -
Jakob Petsovits -
James Gilliland -
Jamie Holly -
Karoly Negyesi -
Larry Garfield -
Marco Sousa -
Marijn Kruisselbrink -
Matt Farina -
Michael Favia -
Nancy Wichmann -
Nathaniel Catchpole -
Nedjo Rogers -
Neil Drumm -
Peter Wolanin -
Robert Douglass -
Sam Boyer -
Stefan Nagtegaal -
Steven Peck -
Thomas Zahreddin -
Tomáš Fülöpp (vacilando.org) -
Walt Daniels