It seems we're falling into one of two sides of the old argument over design philosophies. One side says "do the right thing", the other believes "worse is better." One side builds monolithic jewels, the other side builds simple modular tools that while messy are simple to use and understand.

More here:
http://www.jwz.org/doc/worse-is-better.html

There are also parallels here to a comparison of publishing on the internet and traditional publishing in print. Publishing a magazine or newspaper is expensive, very expensive. So nothing is printed until it passes through an army of editors, fact checkers, graphic designers etc. that mold the content into something special. Publishing on the internet is, on the other hand, extraordinarily cheap. Head over to wordpress.com and 5 minutes later, you have an international publishing platform -- for free. And millions of people have done that which is why the internet is now full of junk and useless or so critics will say.

But it's still very easy to find very high-quality useful information on the internet. Why? Because there's tools which help filter out the good content by mining the collective intelligence of human of human interaction on the web. Google works its search magic by discovering which pages have the most "votes" from incoming links. Collaborative news sites such as digg, hacker news, reddit, and others work by voting. The best news bubbles up to the top. Memetrackers analyze new content and discover which topics are being discussed the most and push those conversational clusters to the top of the news page. And so on.

Both systems of publishing content are filters -- the traditional system says -- "publishing is expensive, so we'll filter content before publishing." The second system says "we can't stop people from publishing -- so let's devise better filters to help the best content still come out on top."

Many people will argue against the second system complaining about information overload -- "there's too much stuff on the internet, we need professional editors to help us know what to read." Clay Shirky talked about this awhile ago and said "there's no such thing as information overload, just filter failure." He expanded on the argument:

So, the real question is, how do we design filters that let us find our way through this particular abundance of information? And, you know, my answer to that question has been: the only group that can catalog everything is everybody. One of the reasons you see this enormous move towards social filters, as with Digg, as with del.icio.us, as with Google Reader, in a way, is simply that the scale of the problem has exceeded what professional catalogers can do. But, you know, you never hear twenty-year-olds talking about information overload because they understand the filters they’re given. You only hear, you know, forty- and fifty-year-olds taking about it, sixty-year-olds talking about because we grew up in the world of card catalogs and TV Guide. And now, all the filters we’re used to are broken and we’d like to blame it on the environment instead of admitting that we’re just, you know, we just don’t understand what’s going on.

Now I hope I'm not distorting Marcel's argument here (and if I'm wrong I'm sure he'll correct me :) when I say he's arguing that there's too many modules, too many "bad" developers, writing too much "bad" code and we'd be all better off if the module landscape was less messy and more prestine. Perhaps this is too neat but it seems his argument falls along the same lines as those who say we need professional editors to filter content before publishing. I think his idea is a bad one (for our situation -- it works great for manufacturing, magazines, skyscrapers, etc.) for many of the same reasons as others have argued. But we do need better filters to help us find the best and right modules to use (and I'm sure some duplicate modules are written simply because the author didn't *know* there was an existing solution). And I think the drupal community is already working very hard to do just that -- provide better filters to point site builders to the current "best-of-breed" modules for a given situation. Much of the drupal.org design was directed at that problem and the (still coming) Drupal distro revolution will also greatly help at standardizing the module selection and focusing developer effort.

Kyle

Research Assistant
eBusiness Center @ BYU
kyle.mathews2000.com/blog


On Sun, Mar 8, 2009 at 3:47 PM, Marcel Partap <mpartap@gmx.net> wrote:
...

Because 99.9999% of all Drupalers are not really aware of that, I
want to take the chance to publicly say THANK YOU to Andy Kirkham
(AjK), who does an awesome job on carefully reviewing a bloat of
CVS applications - mostly alone. We get several new CVS
applications *each day*. Although there are a few other community
members who are eligible to process CVS applications, he is the
one who takes this on with the required responsibility, discipline,
and motivation.
You're right, haven't noticed him although he probably also processed
my application.. KUDOS Andy for bearing with the community ;)


That's an ungrateful job, because many applicants complain about
the slow review process and in parallel, his hard and
time-consuming work is not recognized throughout the community. :(
Yeah being in the jury just never pay off.


Thank you, Andy!  You are one of the most important people for
Drupal. Without you, Drupal contrib would certainly be a pain.
Wow i didn't realize it could be far worse..
Now on to the more profane stuff.


It might be difficult and require changes, but in the long run it
will be worth it.
That's a nice vision you have there.  But it sounds like you only
want to talk about it.
Well admittedly starting the discussion is only a first small part.
For much of the work i can not be of help simply because i have other
obligations, but i volunteered for reimplementing a mailman/drupal
integration module and would contribute as much as possible.


Merging and rewriting most of those duplicated modules from scratch
(e.g. Notifications/Subscriptions, eCommerce/Ubercart, etc)
requires a lot of development time - much more than simply porting
them to yet another Drupal core version.
Well of course it does, but a) the structural sanity benefit might be
worth it and b) the license doesn't prohibit code reuse ;)


To have any effect in the end, it also requires that all module
maintainers agree on the "roadmap", which means that they will let
 their own, duplicated modules die in favor of the new, centralized
 module.
No. It does require the community at large to do the work of
discussing, drafting and finalizing the structural design of these new
frameworks - module maintainers are not required, but of course their
experience would be of tremendous use.


This has been done before.
Fine. So uhm let's do.


- Wysiwyg API module is the successor of all client-side editor
integration modules, because someone analyzed the anatomy of all
previously existing modules and thought about a better, generic way
to do it with Drupal. Almost none of the other maintainers and
developers ever contributed to this vision, which is the reason for
why the few developers of this project are completely buried with
work, and Drupal users are (even more) confused about which module
to choose/install.  Because of that, we won't have better Wysiwyg
support in Drupal 7 core.
Shame on those module maintainers! But still, all is not lost. Let us
try making it suceed and mirroring the process, without leaving
current module maintainers out of the equation.
By the way, a policy change like described would put more pressure on
those maintainers - if they want to have at least some of their code
run on D7 - they better be cooperative!


True. How does a more swarm-like approach to code committal
hinder that? That'd just channel the flow more tightly.
People most often forget that maintaining a project is primarily
about responsibility, once it is in the wild.
Ok now that's a valid point. Maybe it can be possible for people to
take responsibility without having exclusive access rights to a module?


Yet another example: If there was some bad, not carefully reviewed
 code committed to Views module, then 72,799 sites could go down.
So as that has not yet happened it seems responsible developers exist
in the drupal community.. hurray! ;)


I have seen issues where 10 or more contributors reviewed and gave
 their thumbs up, but luckily there was this responsible maintainer
 who told them that the entire approach is wrong, followed by the
proper way to do it.
That's the scenario where the veto vote i proposed comes into play ;)


That is the primary job of maintainers (gate-keepers) and the
reason for why it is both excellent and required to have separate,
 specialized knowledge and vision per project.
Well if it work better, that'd be great. However not all individuals
live up to these standards, which is why i think maybe the drupal dev
collective at large can better fulfill this role!


Your proposal really boils down to the question whether a project
maintainer is open to improvements by the Drupal community and
thereby takes the required, additional steps to ensure that the
community is able to contribute.
My proposal was to shift the responsibility from individual persons to
a community process, to minimize possible negative influence of those.
The process should promote and support taking responsibility by
anyone, and hinder the opposite.



It's about following a certain paradigm - or not.  No automated
process or AI will be able to change the paradigms of humans who do
not want to follow a certain paradigm.
True, _but_ individuals who do not want to follow the common shared
values may be excluded from using the community to publish their code.
That will teach them ;)


We can measure quality in contrib (and there are plans to do so),
but putting regulations onto contrib hinders innovation,
evolution, contributions, and lastly freedom.
Yes and no. It restricts freedom, true. But why does experimental code
need to be committed to the central repository, and how does enforcing
an open development process hinder evolution?



--
 "Obstacles are those frightful things you see when you take
 your eyes off your goal."         -- Henry Ford (1863-1947)

 Change the world! Vote: http://hfopi.org/vote-future