First off, I am all for a freeze on the planned date. We have enough features and there is not much in the pipe line worth waiting for (unless Adrian comes forward and says that his new themes engine 2 is ready, with all the reservations on the "too many files" issue there ...)

that said, i have similar concerns with a lot of what ber raised. i've got 6 modules i personally maintain and there are about 4 others i've got commit access to and help out with. a few (dba and diff) still aren't ported to 4.7. it's going to be a lot of work to port the rest to HEAD. i've got tons of functionality i want to add to these modules, but i simply don't have the time to do it all, try to keep my eye on core development and help out where i can, be the resident CVS guru, *and* port all my modules to the latest core API every 3 months...

Let me offer some contrarian evidence to Derek and Ber. Between me and Wafaa, we have 18 modules. All of them except one (topic module) have 4.7 versions. Five of these modules were in existence since 4.5. Two of them never had a 4.6 version (favorite nodes and image watermark, for different reasons, the latter being a recent addition, the former because I developed it for a client in the pre-4.7 cycle). Of the above, Wafaa owns one module (profile CSV), and has committed to 7 others (not sure if these were ports to 4.7 or something else). The majority of upgrades are either patches from other users of the modules, or requests from clients for a 4.7. I could not have done all that without users contributing patches, and clients paying for the time for others. Some modules were updated because I needed them for my own sites as well, so that is another factor. All my sites have been upgraded from 4.6 to 4.7 as well, except for some test and demo ones. I am not doing any 4.6 work anymore, unless a client really really needs it.

On Fri, 18 Aug 2006, Richard Archer wrote:

How about doing with CCK what was done with the Forms API... commit CCK to core, freeze all the other features and then make CCK work properly.

That way the feature set for 4.8 is known and people can begin porting their contrib modules, none of which will rely on CCK (although some may benefit from it in the long term).

And since CCK will take a couple of extra months to fully debug, there will be sufficient time to port modules.

It was decided that we are not going to go down on the same route again, as we have done with the FormsAPI. People need reliable release dates and solid features fast. If CCK is not going to be updated for 4.8/5.0 to get compatible with the custom content types in core, it just shows that there is no interest. In case there is voluntary or financial interest in doing a flexinode upgrade path or a CCK update and upgrade path, it will happen. I have first hand experience with some of my module creations I was reluctant to update. People came up with patches, and since I had no time to review and apply them, I handed over maintanance. Sometimes I peek into their current state (I might need them for ongoing projects), and some of them already changed to another maintainer, but as long as there is need for them, they keep getting updated. Contrib modules, translations, themes need time for sure. If we keep pushing the freeze date, most contribs, themes and translations will push their update dates further. If interface strings are not stable, why should I bother translating them? If the database structure / API is not stable, why should I bother to update my modules for that API / database structure? If it is uncertain how themes will get handled in the next Drupal version, why should I bother updating my themes? You see, putting some big change into core which makes for an uncertain freeze date and still a lot of changes onward is not helping contribs update. The fact that CCK HEAD is not yet compatible with Drupal HEAD does not mean anything for the future. If I were the CCK maintainer, I would not care updating my module either until I see the solid codebase to update to (and this is what I do with my current maintained modules and the Hungarian translation). Gabor

On 18 Aug 2006, at 09:18, Larry Garfield wrote:

Personally, I think this is an issue best handled by the oft- discussed but not-yet-implemented golden/recommended/endorsed/first class citizen contrib concept.

We're not going to have such concept. We might have ratings but nothing official. -- Dries Buytaert :: http://www.buytaert.net/

On 8/19/06, Boris Mann <boris@bryght.com> wrote:

If you do all of the above, I, personally, will ship you a certificate with the name of the module and a big gold star on it. Note: may or may not be legal in some states or jurisdictions.

Also, Derek's work on CVS / Project means we can do per-module revisions. My initial push for "golden" modules was in fact related to such a concept.

IMHO, the first project that deserves the "Boris Mann Gold Star" certificate is the project module itself. Great work on this, Derek. Of course, you'll have to code the "Gold Star" system into the project module before you can get one. ;-) Cheers, Jaza.

Boris Mann wrote:

On 8/18/06, *Dries Buytaert* <dries.buytaert@gmail.com

On 18 Aug 2006, at 09:18, Larry Garfield wrote: > Personally, I think this is an issue best handled by the oft- > discussed but > not-yet-implemented golden/recommended/endorsed/first class citizen > contrib > concept.

We're not going to have such concept. We might have ratings but nothing official.

Want your module "golden"? * address issues in the tracker in a reasonable timeframe * be ready for the release of Drupal by the RC stage * add unit testing (simpletest) and/or black box testing (selenium) * use proper Drupal APIs (theme functions, connect with other modules, etc.) * make your module kick ass

There is just one big fat glaring problem with this -- and why I don't understand Dries's flat refusal to consider it: I write a module. I address all issues very fast. It is ready before Drupal core reaches RC state. It has great unit testing and black box testing. It properly uses all APIs, etc. It is a totally kick ass module. In fact, so much so, that thousands of people use and depend upon it. But it is an extremely complicated module that takes a long time to understand, so as the author I continue to maintain it and upgrade it, and do so with great care and speed. More and more people use it. Then Drupal comes along, breaks half a dozen APIs as it is wont to do -- and I'm spending a year sabbatical living in the Andes and not doing any work on the module at all. You release a new verison of Drupal but without the one module most people want, because I am not around to update it. What do you do? Multiply this times the dozen or so most popular modules; lather, rinse, repeat. Now what do you do? This is the road we are currently on, unless we come up with a good solution. A golden repository is one such solution. ..chrisxj

Jeff Eaton wrote:

...

Then Drupal comes along, breaks half a dozen APIs as it is wont to do -- and I'm spending a year sabbatical living in the Andes and not doing any work on the module at all.

If that is the case, then NO ONE should be shocked that there are problems. Get another developer to pick up the project, announce that it won't be supported anymore, give someone else commit rights... After a year away, I think the module can legitimately be considered 'abandoned.'

I only said a year sabbatical to make it clear that nothing was going to happen soon enough. I am not really talking about abandoned modules, just those that might not get updated "soon enough" where "soon enough" is open for interpretation.

...

Multiply this times the dozen or so most popular modules; lather, rinse, repeat.

I can think of maybe a handful of modules that are THAT complex. Views? Yes. Project? Yes. eCommerce and its suite of related modules? Absolutely. Those are all important, but declaring that we will never ship Drupal if those modules aren't released at the same time is silly.

I didn't suggest not shipping Drupal if those modules are not released at the same time. That's a straw horse. All I said is that the current course of action is not addressing the problem.

On Fri, 18 Aug 2006, Chris Johnson wrote:

Now what do you do? This is the road we are currently on, unless we come up with a good solution. A golden repository is one such solution.

How would any golden repository fix human resources problems? If active people are around a project they will eventually update it, regardless of the maintainer being around or not. If a module is in active use, people will find creative ways to utilize it, and wil delve into the code anyaway. They will get to know the inner workings and if they depend on the module, will do the update. Gabor

I believe the supposition for the golden repository is that core developers would always make sure those modules got updated for a release, either by doing the work themselves or ensuring someone else did.

It will be difficult to get core maintainers to care about contributed modules, I think. To me it seems that if a module is *that* widely used, it will (or should) eventually be committed to core in some form anyway. Then the core maintainers will have motivation to help maintain it, assuming it remains useful. That is better than any golden star, imo. Farsheed __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com

On 18 Aug 2006, at 22:50, Chris Johnson wrote:

In fact, so much so, that thousands of people use and depend upon it.

But it is an extremely complicated module that takes a long time to understand, so as the author I continue to maintain it and upgrade it, and do so with great care and speed. More and more people use it.

Blessing your module with a "golden star" is not going to magically cultivate the wilderness. If you are the single maintainer, and you happen to be a bottleneck (because you're having a sabbatical in the Andes or simply because you have too much work), a golden star is not going to magically upgrade your module nor is it going to send hordes of developers to your patch queue. No, that would be a crack dream. :) What you need (and what you really want) is people that know your module inside out (!), that share your vision for the module (!), and that can help you maintain it (!). If your module is that important, it is your responsibility to build such team, to get people on board and to delegate responsibilities by making other people co- maintainers. If you want to be a responsible maintainer, it's part of the job description to find, motivate, guide and empower people to take on a role within your project and to make it sustain. I know because I learned to do exactly that. -- Dries Buytaert :: http://www.buytaert.net/

On 19 Aug 2006, at 00:28, Chris Johnson wrote:

I think this is the crux of our difference in opinion, Dries. While it would be great and noble for me to build such a team out of desire to make my module successful and sustained, I might not do that for any number of reasons (e.g. lack of time, lack of diplomatic skills, lack of English language skills, people just think I'm ugly, whatever). The point then is that despite my "neglect" my module has become important and now there is nobody to maintain it.

If you created a module that thousands of people depend on, you created a successful Open Source project. By now, I think we all agree that the community has a desire for successful projects to sustain. It think we also agree on the fact that a successful project should not depend on a single person, especially if that person is a really busy person. :) Now, my opinion is that _you_, the maintainer of the project, should create a team around your successful project in order to grow and maintain it. Your opinion is that _we_, the community, should start caring about your successful project and that we call the squad team when you can't handle the work. In my eyes, your opinion, is flawed. If you "force" people (volunteers) to do something, they'll burn out or they'll go away. Open Source development is all about having fun. It is very hard for us to make people passionate about your project. It takes a lot more time and energy than blessing it with a golden star. And unless developers are passionate about it, they are not going to have fun working on it. If a successful project has problems, the maintainer can send an e- mail to the mailing list asking for help, ask for help raising funds, write about it in a forum topic, give a presentation about it, or what not. It all creates visibility, and people will start to care. Take the project.module, for example. It's an important module, but only few people cared about it. There was frustration, but no passion. We publicly blessed it with a doze gold stars but at the end of the day, not many people cared about it. So in the past, CivicSpace threw money at the project module to make it sustain. Money can fix short term problems quickly, but doesn't offer long time guarantees. What Derek is doing with the project.module nowadays, is much more rewarking and much more sustainable. He cares about the module, he improves it, and he writes about it. By doing so, he is spreading the passion and slowly creating a team around it. It takes him a lot of time and energy, but for god's sake, he converted Boris from a 'hater' into a 'believer'. ;-)

It just seemed that the door to any discussion was being closed.

No, we can discuss this. :) -- Dries Buytaert :: http://www.buytaert.net/

On 19 Aug 2006, at 14:08, Bèr Kessels wrote:

Op zaterdag 19 augustus 2006 10:42, schreef Dries Buytaert:

...

If a successful project has problems, the maintainer can send an e- mail to the mailing list asking for help, ask for help raising funds, write about it in a forum topic, give a presentation about it, or what not. It all creates visibility, and people will start to care.

I did that for flexinode a couple of times and received NULL. Is it me, or my behaviour that keeps away people? Probably. Is it my lack of LOADS of time to review all patches? even more probably. Is it the fact the module is not popular, that people don't use it? Unlikely.

Well, I don't remember those e-mails but it probably means you did something wrong. For example, just sending a couple of factual e- mails clearly isn't enough. Neither is begging for help. You have to transfer the passion, make them feel connected, and literally suck them in. Don't beg for help, but help them help you. (Yes, _you_ have to help _them_ (help you) -- and not the other way around.)

* A workflow. Drupal is aware that modules with such a status are important to Drupal as a whole. I dare to bet (for a Duvel) that there are at least five contribs that are more important for Drupal (very broad) then some core modules. Drupal ackonowledges this importance.

We all know what is important, and what is not. We don't need golden stars to figure out that flexinode, cck, views, e-commmerce and image are important. There is common sense amongst developers.

* Security. Abovementioned flexinode case makes me beleive that all people who use Drupal 4.7.x and flexinode, have not upgraded to 4.7.3.

I don't know what the problem is with flexinode or why it can't be upgraded. If this is due to a critical bug in core, please point us to the issue in the patch queue. Hopefully, it is marked critical. I haven't seen an e-mail about it on security@drupal.org either.

But if he did his homework, found that flexinode may or may not solve hiss issue, AND sees that flexinode is NOT marked as special-supported (or golden, whatever) and /then/ gets to see errors, we have still won. Because we (Drupal) have said to him: Core is what Drupal is, Golden contribs is what we advice, the pile of code out there is a scrapyard where you may find just what you need, but don't count on it. But never count on it.

I'm all for adding ratings, showing information about the project's health, etc. Hence my previous e-mail with the mock-up. I think such information should be auto-generated based on various metrics: - Number of contributors - Patch queue activity - Security history - Download statistiscs - Usage statistics - Number of commits - ... -- Dries Buytaert :: http://www.buytaert.net/

On Sat, 19 Aug 2006, [iso-8859-1] Bèr Kessels wrote:

around a project. Someone who looks "For a CMS that allows blogging and an image gallery" does not sense that, and wll probably need to go trough the whole range of "shazamgallery (not-user-safe)", "gallery2 (glues gallery2 to Drupal)", "img assist (not for image galleries)", "acidfree (for a very specific task, not very flexible)" and the rest to find out what he /realy/ needs, "image module". If we can show such a person that "image" is the one to go for right away, that would be great. But right now, getting that knowledge requires a lot of effort for people new to Drupal.

The broad range of "image gallery" type modules show that "image gallery" means different things to different people. The fact that *you* have found image.module to work for your need best does mean that it is the golden one? What if you have slightly different needs? Golden or not golden is a binary thing. If something is golden, it does not mean it fulfills my needs. Quality is just one factor, and we should certainly not treat it as binary. Gabor

On Aug 19, 2006, at 5:08 AM, Bèr Kessels wrote:

Golden stars are -indeed- not going to magically bring me coding ninjas.

here, here. my original post in this thread wasn't directed at golden contrib at all. i think these are basically orthogonal discussions (though there's a tiny-winy bit of overlap)... golden contrib is good for quality control, help sorting through the very large and highly mixed project download listings, etc. basically, most of the stuff ber is talking about. i see it as something to help site admins choose what modules to build their sites on top of, not as something to help our internal development workflow. of course, a gold star isn't the only thing that would help. browsing by download stats (http://drupal.org/node/66013), a project rating/review system (http://drupal.org/node/77976), exposing the kind of end-user-readable data about issue + code activity dries is talking about, etc, etc, can all help with this problem. one possible solution to my original concern about core running too fast for contrib to keep up was the proposal for a longer *freeze* period. i see a few pros to this approach: 1) more time for contrib to catch up. ;) 2) more time to work on hardening, testing, bug fixing for core itself 3) more time to work on documentation 4) more time to work on usability enhancements (so long as they're not major and don't change the API) the downside, of course, is that it would delay the next furious round of changes. however, that might just be a good thing. ;) 2 major releases a year instead of 3 isn't anything to be ashamed of. and, if the extra time means that the folks working on new core API changes have more time to design, get feedback, do initial implementations, etc, etc, it's probably going to speed up the rate at which the patches make it in, since they'll be better patches with more sane APIs in the first place. ;) but, whether or not we have more time to do the work, we still have the human resources problem. i think dries is right about what brings you "coding ninjas" -- people using the stuff, relying on it, and effort spent developing a team. that said, sometimes a "team" (or elements of it) do just fall out of the sky. for example, justin randell (cck in IRC) decided, simply because he wants to, that he's going to help adopt the project issue queue and work on patches. we've discussed some stuff via IRC and email, and i'm about to commit a set of patches from him that will end up resolving about 5 critical bugs. i suppose dries would say the only reason justin decided to do this is b/c i've been so active and vocal in working on project, developing a team, etc, but from my perspective, i did no work at all to get his help -- he just came out of no where as a blessing in my time of need. ;) only justin could really answer for sure, but i certainly didn't spend any effort trying to recruit him for the task... On Aug 18, 2006, at 9:55 AM, Boris Mann wrote:

On 18-Aug-06, at 8:32 AM, Jeremy Epstein wrote:

...

IMHO, the first project that deserves the "Boris Mann Gold Star" certificate is the project module itself. Great work on this, Derek.

/me leads round of applause for Derek.

/me blushes. thanks for all the love, everyone. ;) of course, i'm only able to do what i'm doing because i'm "standing on the shoulders of giants" -- dries, kjartan, killes, nedjo, and everyone else who wrote, maintained, and kept project.module afloat (not to mention drupal itself(!)), long before i "dropped out of the sky in the time of need". ;) cheers, -dww

Although I'm new to the group, I'm not new to programming (since 86), and I'm not new to open source (since 95). I'm just jumping off the drupal cliff (adopting it as my primary development environment for a college), so although I haven't earned a strong voice, I'm really vested in the outcome of the discussion. I thought a fresh perspective/recap might be helpful here. Agile deveopment/project management techniques are also a passion of mine. Here's some questions that the group seems to be trying to answer: 1. Would the success of drupal be improved by a module rating system to inform administrators of the stability level? On this everyone agrees. The golden repository or golden * has been proposed a solution for this too. I think this could dovetail well with the automated comment proposal too. This sounds like a feature request for the project module to me. I think it could be a really great tool for many open source developers. I'd suggest we might consider measuring: * average bug report to patch duration * number of bugs that convert to patches in released code. (that's a bad thing, yes?) * How quickly did the module release after core? * Maybe % of features requests converted to applied patches * number of downloads. * an ability for the project owner to override and say, "my module ain't golden". I don't agree with measuring the size of the community, as that doesn't always equate to stability, or value. It actually seems to discount elegent, simple, effective solutions. 2. Should the status of contributed modules that should hold up the release of Drupal? I haven't heard anyone actually suggest that the status of any one contrib module should hold up release. But some argue the need to have a formal mechanism to identify critical mass. I've been hearing a lot about the project module and flexinode as examples of examples of some such modules. Some are for, some are againts. I think we shouldn't do this. If such modules exist, perhaps they should be in core. It sounds like a flexinode replacement is being slowly rolled into core. It's not featured enough yet to be a complete replacement, I think everyone agreed on that. I often wonder why the project module hasn't been. It's one of the few modules that we actually as a community can't do with out, not because of how much its used, but because its used to maintain drupal. If we ever got more than two versions behind on this would we shut down the drupal.org site :). My point in bringing this up is that project isn't really a good case study for this discussion. It's too important, so the question about what would happen if we released with out it can't be taken seriously. (Oh we could do it once, but then even drupal.org couldn't upgrade) Could anyone think of modules for which there are not core replacements in progress, but that we'd want to consider holding off a release for? Sounds like Dries sent a suggested list once. 3. If so how would we decide which modules would be used as the benchmark. Golden repository has been proposed as a solution here. Some are concerned that it might slow core development or discourage module development. It could work, if we decided to do it. I have a feeling that unless we could automate the process of deciding which modules were "gold starred", that it would be work that takes away from improving drupal. So I'm not real hot on this. The less process the better. Its the "agile" way :). 4. If not how can we make sure that the contrib modules don't fall too far behind? Ideas I've heard are longer freeze periods, and better outreach to people want to contribute. I think the freeze period length increase is a strong idea to helping this speed issue. I also think there could be some improvements in how contributers get recruited.

On Saturday 19 August 2006 16:17, Bèr Kessels wrote:

...

* average bug report to patch duration * number of bugs that convert to patches in released code. (that's a bad thing, yes?) * How quickly did the module release after core? * Maybe % of features requests converted to applied patches * number of downloads. * an ability for the project owner to override and say, "my module ain't golden".

Good points!

...

I don't agree with measuring the size of the community, as that doesn't always equate to stability, or value. It actually seems to discount elegent, simple, effective solutions.

Also very true. This is my main reason to vote for goldstarred: They would be starred by real people, but based on facts other then simply its popularity.

Popularity/downloads say something about a project's ability to a) fill a gap/niche. b) market itself. It tells us nothing about quality. If this was not the case, then Windows would not have its monopoly ;)

...

I haven't heard anyone actually suggest that the status of any one contrib module should hold up release. But some argue the need to have a formal mechanism to identify critical mass. I've been hearing a lot about the project module and flexinode as examples of examples of some such modules. Some are for, some are againts.

I am sorry about making people think this. I think flexinode is far, far from a module that should hold up any freezes. I used flexinode as an example, because: a) It had a completely horrible transition to 4.7. I took over management because it went unmaintained for a period in which people tagged it 4.7 (the whole system, including files that were not touched since before 4.5), added hacked in, people added features that introduced critical bugs etc. But no-one was co-ordinating and working towards a solid 4.7 release. b) It has been between one of our most popular modules (in download stats). JonBob has told numerous times he never really expected this, and that he really did not design the module for such wide use. Its current quality is low, yet it fills a gap, and somehow marketed itself well. People want it, use it, regardless of its current state.

I really hope that I will manage a smoother transition to 4.8 for flexinode :)

...

I have a feeling that unless we could automate the process of deciding which modules were "gold starred", that it would be work that takes away from improving drupal. So I'm not real hot on this. The less process the better. Its the "agile" way :).

I think that is the main reason why Dries's proposal for more visible stats are what most people seem to like.

...

4. If not how can we make sure that the contrib modules don't fall too far behind? Ideas I've heard are longer freeze periods, and better outreach to people want to contribute.

A lot of people bring forward the idea that "a popular module will attract improvement anyway". This may, or may not be true. But I haven't seen proof for any of these.

...

I think the freeze period length increase is a strong idea to helping this speed issue. I also think there could be some improvements in how contributers get recruited.

Is there anyone with stats, or who remembers when people started to upgrade in the last freeze? I have a strong feeling most upgrades started only weeks after the actual release. A longer freeze might then only delay the moment people start upgrading.

Bèr

Right then, I'm going to jump back into this can of worms I reopened... First, to clarify (as it seemed there was a bit of confusion earlier on), I was talking about flagging modules, not developers. Although I do recall Chad and I once spending a few hours toying with the idea of a goldstar module of sorts as a gag, like a year ago. :-) I think Ber's picking up on the underlying message I'm trying to get at here. Drupal without contrib is an incomplete blogging system. Core itself has an emphasis on a WordPress++/phpBB type site, but even then is missing a few features. Any of the really interesting things that separate Drupal from those sorts of apps requires using contrib. The Drupal Ecosystem includes large swaths of contrib. I do not believe that placing, say, ecommerce in the same swath as NodeReview (an example I use because it's my own module, so I know it's not a major one nor can I offend anyone else <g>) does Drupal users, developers, or support people any favors. Dries points out that there are two issues involved here: Directing users toward modules that are worth their time and directing developers toward modules that need TLC. He's right, but I believe those lists overlap dramatically. How we determine which modules are "Special", both for users and developers, is open for debate. I don't think that either a "Dries orders by fiat" or "Whatever gets most votes/downloads/closed issues in the project module" method will be adequate. But whatever the method, we need some way to clearly break down contrib and separate the wheat from the chafe, so to speak. -- Larry Garfield AIM: LOLG42 larry@garfieldtech.com ICQ: 6817012 "If nature has made any one thing less susceptible than all others of exclusive property, it is the action of the thinking power called an idea, which an individual may exclusively possess as long as he keeps it to himself; but the moment it is divulged, it forces itself into the possession of every one, and the receiver cannot dispossess himself of it." -- Thomas Jefferson

On Aug 19, 2006, at 8:06 AM, Gerhard Killesreiter wrote:

Actually, I am angry about myself that I accepted some patches to event.module to keep it cvs compatible. This delays the back- merging of fixes into the stable branch since I need to remove these changes from the diff.

i agree. i feel the same way about project.module. :( devel.module is one thing -- i think that's so helpful for folks working on HEAD that it's a good idea to keep it updated with the latest. otherwise, i don't think any modules should start being ported until the code freeze. i've definitely learned this the hard way in this development cycle, and i'll remember it for the future. we should probably document this on the "updating your modules" handbook pages as a best practice, with a short explanation about why it's a bad idea to start porting to HEAD while it's still a moving target. -derek

Spent the last days thinking about this some more (couldn't help it) and summarized my thoughts at: http://buytaert.net/responsible-maintainers -- Dries Buytaert :: http://www.buytaert.net/

On Aug 19, 2006, at 8:06 AM, Gerhard Killesreiter wrote:

I don't see doing us more than 2 releases a year anyway. The catch- up period for 4.7 was long enough and still many module weren't updated before the release. This is just how it works and nothing will change this.

yeah, you're probably right. as i said in my *original* message, i'm not dissatisfied with the status quo, i'm just a little uneasy. core running too fast for (most of) contrib to catch up seems like a lingering problem, but it's not clear what, if anything, we can do to address it. either people are using/depending on the modules, and they either update them themselves, or pay someone else to do it, or the module isn't really that useful and it will eventually become abandoned. we should continue to cultivate more drupal developers in whatever ways we can (recruit your development-savvy friends, folks!) but that's a *completely* separate discussion from core development schedules, "golden contrib", etc. once we're able to tag real releases of contribs with the version of core they work with (which will be explicit in their version numbers, in fact) and stop the madness of calling "cvs" a version of anything, a lot of the confusion and woes will go away. this won't add any more development resources, and it won't automatically make more of contrib updated in a timely manner, but at least crap will be less confusing for everyone involved. ;) modules that people care about *will* be updated eventually, and no one has to upgrade to the latest core just because it's out. especially once contrib authors can maintain stable + dev branches of their modules against each version of the core API, it will be less painful for the entire drupal community to live with the fact that there will be a spread of core versions out there, not every site will upgrade right away (if ever), and yet new features can still be added to contribs that are compatible with older versions of core. dries will probably dislike this, since he'll argue this will just encourage folks to use older versions of core and therefore work on the latest release will slow down (at least, that's what he used to think -- i believe i've at least partially changed his perspective on this). but i think that's backwards. drupal's refusal to remain backwards compatible (which again, i think is great) is what encourages people to remain at older versions of core, not the continued availability of various contribs. people not wanting or being able to upgrade is a fact. we can either accept that and provide ways for people to cope, or not. the "Drupal Version-Module Support Matrix" (http://drupal.org/node/ 63491) will be very handy in all of this. it'll be a way to quickly visualize what modules are in what degree of preparedness for each version of drupal core. if core keeps moving fast, and contrib keeps struggling to keep up, then at least people will be able to easily see what subset of the "drupal ecosystem" exists in each version. in spite of the fact that it seems like nothing's going to change wrt the core development cycle as a result of this thread, i still think the discussion has helped clarify some things. ;) the tangential discussion about "project health" metrics and displays has certainly been useful, and is already generating concrete actions to make things better (thanks dries + webchick). i certainly feel more clear about my unease, and how to cope with it... -derek

*My* proposal is that the issue get visibility and some sort of mechanism to help make sure temporarily-orphaned but critical modules don't get left behind. There are likely other methods than the golden repo idea. It just seemed that the door to any discussion was being closed.

In an attempt to translate this discussion into some concrete action points, here is a first proposal: http://buytaert.net/temporary/project-information-at-a-glance.png The advantage is that is can be automated (no manual work) and that it promotes active projects over inactive projects, good maintainers over bad maintainers, etc. Also, it provides useful information to user -- something which is important on its own (if you look at Kieran's survey results). *winks at Derek* -- Dries Buytaert :: http://www.buytaert.net/

On Aug 19, 2006, at 4:37 AM, Gerhard Killesreiter wrote:

I sometimes get the impression that people don't know that http://drupal.org/project/issues/statistics exists...

It should be easy to add an individual stats page for each project.

i get the impression killes doesn't know about: http://drupal.org/project/issues/statistics/project http://drupal.org/project/issues/statistics/event ... or: http://drupal.org/project/issues/statistics/3238 (event's page by project nid) ;) it'd be trivial to add a link to each project's issue statistics page on the project nodes themselves. that said, i like dries's block proposal to summarize the issue + cvs activity in 1 place, too. those statistics pages are ok, but it's kind of hard to get a sense of the activity in the same way, and i think including commit info in the same summary is important. and yes, i'm definitely accepting patches. ;) thanks, -dww

On Aug 19, 2006, at 6:40 AM, Gerhard Killesreiter wrote:

I had a gut feeling that these pages existed, but they aren't linked from the project page (and from anywhere else?).

if you goto, e.g.: http://drupal.org/project/issues/event the "statistics" link at the top of the page brings you to that project's stats page. i agree this link should be directly visible on the project node. in fact, i think every link on that page should probably be visible on the project node. -dww

On Aug 19, 2006, at 8:17 AM, Gerhard Killesreiter wrote:

Ah. not the most obvious UI. :p

nope. ;) i've been busy on the critical bugs and totally broken crap, not so much the slickness of the UI (though, if i may say so, the UI is definitely better than it was). ;) someday (hopefully soon), things will settle down a little in the critically broken department, and i'll be able to devote more time/energy into the making it all more sane/clear/slick.

We should spice the project nodes a bit up. They look rather bland with their long lists of links.

true, that. i'm mostly working on the plumbing. if anyone wants to work on attaching nicer faucets to the pipes, by all means, submit your patches. i'll be thrilled to review them and put them in. thanks, -derek

On 19 Aug 2006, at 13:37, Gerhard Killesreiter wrote:

I sometimes get the impression that people don't know that http://drupal.org/project/issues/statistics exists...

It should be easy to add an individual stats page for each project.

Sure but ... the idea is to: 1. Take it a bit further by extending the amount of data we collect (eg. downloads). 2. To make it more visible by showing a summary on the main project page. 3. To make it more accessible to non-developers by avoiding technical lingo. -- Dries Buytaert :: http://www.buytaert.net/

On 19-Aug-06, at 7:22 AM, Dries Buytaert wrote:

...

*My* proposal is that the issue get visibility and some sort of mechanism to help make sure temporarily-orphaned but critical modules don't get left behind. There are likely other methods than the golden repo idea. It just seemed that the door to any discussion was being closed.

In an attempt to translate this discussion into some concrete action points, here is a first proposal:

http://buytaert.net/temporary/project-information-at-a-glance.png

I love this! Let's talk implementation details: http://drupal.org/node/79550

Dries Buytaert wrote:

There is a certain irony in the fact that the contrib CCK might be broken. Normally, the size and the complexity of the contrib CCK should shrink and shrink until, ultimately, there is little or nothing left from the contrib CCK. As a result, it should become easier and easier to maintain.

Contrib CCK is working with 4.7. The fact that it doesn't work ith HEAD is due to a table name collission. That's all, as far as I know. Cheers, Gerhard

Jeremy Epstein wrote:

On 8/18/06, Gerhard Killesreiter <gerhard@killesreiter.de> wrote:

...

Contrib CCK is working with 4.7. The fact that it doesn't work ith HEAD is due to a table name collission. That's all, as far as I know.

That's the main reason why contrib CCK doesn't work with 4.7. There are (as far as I can think of) 3 other (related) reasons:

1. The 'node_type' table in core HEAD has a somewhat different schema to the 'node_type' table in CCK contrib. 2. User-defined node types (i.e. all CCK node types) no longer need to be defined through hook_node_info(): they only need to have their entry in the 'node_type' table maintained. 3. There is now a UI for adding/editing/deleting node types in node.module.

(1) will take a reasonable amount of work to address. Contrib CCK will need to switch from using its own API functions, to using the new API functions provided by node.module, wherever possible. Where not possible, CCK may need to have some of its SQL queries (and dependent code) modified.

(2) should be easy to address. Just remove the content_node_info() function.

(3) should also be easy to address. Just remove the content type add/edit/delete UI from content.module (since core now provides it), and leave the field add/edit/delete UI.

This is just my predictions - I haven't actually investigated updating CCK myself (that's up to the CCK maintainers). Also, I should point out that I know a lot more about the new CCK features in core, than I know about contrib CCK.

Cheers, Jaza.

That was my biggest problem with the pave the way for cck patch.. fortunately the update will be simpler than I expected... ;)...

Yes. In fact, I beleive the only solution is the other way around: Release with less stuff in core. The less changes, the easier it is to keep in pace with all our contribs. And in the same time, put less pressure on upgrading. If stuff (contribs) is not ready, we should not tell people to upgrade.

We had this discussion before, and I still disagree. I believe that CCK+view is what the next generation of open source CMSes will look like. They have the potential of being the heart of Drupal. For Drupal to get to the next level, they have to be in core so that everyone can rely on it as well as build on it.

This particular field feature, however requires a big rewrite for CCK, again. If CCK would do it *properly* (see above), it would mean a big rewrite to remove of one of the core concepts in CCK: making nodes. I suspect this might mean a complete rewrite, after all.

Please elaborate. What would you rewrite, and why? What is not properly done in core that is a showstopper for full CCK support?

So this leaves us a long period in 4.8 with 1) a hacked up, patched up flexinode and 2) a heavy under development replacement for it, CCK. None of wich I would advice any (sane) person on his/her production site. Result: another release without a production-ready system for dynamically build (with fields) nodes.

Note that there are 3 systems here: (i) flexinode, (ii) contrib CCK and (iii) core CCK. I can't speak for (i) or (ii) but if they are "hacked up" then their maintainers are to blame -- at least to a certain extend. I can only speak for (iii), and I think you're overly pessimistic about it. 1. We made an almost insane amount of progress since Drupal 4.7, and we might make more progress by the time we freeze the code. (Go folks, go!) 2. The lightweight CCK that is in core is fully functional. Sure, you can't add extra fields yet but there are many valid use cases where you don't need additional fields. 3. It is probably at least another 2 months until Drupal 4.8/5.0 gets released. That means that there is plenty of time left to fix (i) and (ii), and to a lesser extend (iii). It's only 3 months since we released Drupal 4.7, and it is another 2 months until we might release Drupal 4.8/5.0. 4. The work we have done in CVS HEAD should be beneficial for (i) and (ii) so I'd like to believe that these modules become less "hacked up".

I don't see this as a reason for postponing a code freeze. But I do see this as a reason for concern in a more general way. Especially since the problem seems to become structural. with the rapid growth of Drupal one would expect the overal quality, amount and feature-richness of contribs and themes to improve. I don't see this happening ATM! discussion at drupalcon?

I've seen some pretty cool things that would not have been possible with Drupal 4.6. Drupal 4.7 certainly allows more feature-rich modules to be built. I've also seen modules that take advantage of Drupal 4.7 functionality to improve the user experience. As such, I disagree with your point about feature-richness and wrote about it earlier: http://buytaert.net/the-pain-before-the-payoff It's true that certain projects are slow catching up but that is because Drupal 4.7 was such a big change. Drupal 4.8 will be much easier to deal with, and will be less disruptive. Again, I think you're somewhat pessimistic: drupal.org/htdocs/files/projects $ ls -als *-4.6*gz | wc -l 461 drupal.org/htdocs/files/projects $ ls -als *-4.7*gz | wc -l 484 There are more 4.7 projects then there are 4.6 projects (and 4.6 has been around for a much longer time). -- Dries Buytaert :: http://www.buytaert.net/

From: Mark Fredrickson [mailto:mark.m.fredrickson@gmail.com] The big problem that I see with CCK right now is that it ties a lot of the logic into the form submission process. This is similar to the problems we've seen with programmatically creating nodes. This is a problem for the immediate need of allowing modules to define and import their own content types (a la views). It is also a problem for long term maintenence. There are at least three ways to create a field, all of them tied to a different form. If the data storage mechanism changes (again) that requires picking through three different areas of the code, trying to synchronize the changes.

Just a quick note that http://drupal.org/node/77919 is a patch that allows programmatic submission of forms without any user interaction. Get a form array, populate the form values, and process it -- the same validate, submit, and so on calls are executed. That won't solve problems that arise from a lack of clean API, but it WILL make scripting/automating processes that require form interaction much more straightforward in 4.8/5.0. --Jeff

Dries Buytaert wrote:

...

Yes. In fact, I beleive the only solution is the other way around: Release with less stuff in core. The less changes, the easier it is to keep in pace with all our contribs. And in the same time, put less pressure on upgrading. If stuff (contribs) is not ready, we should not tell people to upgrade.

We had this discussion before, and I still disagree. I believe that CCK+view is what the next generation of open source CMSes will look like. They have the potential of being the heart of Drupal. For Drupal to get to the next level, they have to be in core so that everyone can rely on it as well as build on it.

...

This particular field feature, however requires a big rewrite for CCK, again. If CCK would do it *properly* (see above), it would mean a big rewrite to remove of one of the core concepts in CCK: making nodes. I suspect this might mean a complete rewrite, after all.

Please elaborate. What would you rewrite, and why? What is not properly done in core that is a showstopper for full CCK support?

This is what an offhand look at content.module suggest needs to be done to me... 1) Everything done in content.module to date as far as node type creation needs to be stripped out. 2) the field specific menu items needs to be integrated with the new node.module menu. 3) content_node_info, content_access go the way of the dodo, superceeded in node.module 4) integrate content.modules node hooks using hook_nodeapi 5) titles on forms need to be moved into node.module, and out of content type modules.(seems like they already should be but page.module sets $form['title'] along with the other node type modules. There a no show stoppers here, except maybe removing title fields from the forms in node type modules since they are handled by hook_node_info && node.module sets the label accordingly.

...

So this leaves us a long period in 4.8 with 1) a hacked up, patched up flexinode and 2) a heavy under development replacement for it, CCK. None of wich I would advice any (sane) person on his/her production site. Result: another release without a production-ready system for dynamically build (with fields) nodes.

Note that there are 3 systems here: (i) flexinode, (ii) contrib CCK and (iii) core CCK.

I can't speak for (i) or (ii) but if they are "hacked up" then their maintainers are to blame -- at least to a certain extend. I can only speak for (iii), and I think you're overly pessimistic about it.

1. We made an almost insane amount of progress since Drupal 4.7, and we might make more progress by the time we freeze the code. (Go folks, go!)

2. The lightweight CCK that is in core is fully functional. Sure, you can't add extra fields yet but there are many valid use cases where you don't need additional fields.

the important part of cck is that you can have fields... right now you just have flexible node naming and node type creation.

3. It is probably at least another 2 months until Drupal 4.8/5.0 gets released. That means that there is plenty of time left to fix (i) and (ii), and to a lesser extend (iii). It's only 3 months since we released Drupal 4.7, and it is another 2 months until we might release Drupal 4.8/5.0. maybe fields.module will be working by then... I'll see if I can get a head ready field.module together today if JonBob hasn't already.

4. The work we have done in CVS HEAD should be beneficial for (i) and (ii) so I'd like to believe that these modules become less "hacked up". I don't think there was enough public communication between the people working on cck and cck's inclusion in core, to allay FUD. I had a lot of it until I just dug into content.module to see how the changed in core actually affected it, and how hard they would be in to intgrate, and its not as bad as I suspected.. There still remains a question of whether we modify node_load to contain content.modules node hooks, or do it through nodeapi.... integrating it all into node, would make core a bit fatter, but would get rid of a layer of inner loops. My personal opinion is let field.module live on its own using node api until field type modules mature. We can look at moving the rest into core after the next release if it's deemed a good idea.

...

I don't see this as a reason for postponing a code freeze. But I do see this as a reason for concern in a more general way. Especially since the problem seems to become structural. with the rapid growth of Drupal one would expect the overal quality, amount and feature-richness of contribs and themes to improve. I don't see this happening ATM! discussion at drupalcon? Its no reason to stop the code freeze, but some 'improvements' may appear to be hinderances. Especially since no one to date has taken the time to express to the community as a whole how simple the modifications required to content.module to create field.module really are.

.darrel.

Derek Wright wrote:

On Aug 17, 2006, at 2:02 PM, Bèr Kessels wrote:

...

The good news is that that means more people using, developing testing and investing in older versions: these versions simply stay alive longer.

yet another example of why we need separate stable and dev branches for contrib for *each* version/branch of the drupal core API. ;) http://drupal.org/node/77562

that said, i have similar concerns with a lot of what ber raised. i've got 6 modules i personally maintain and there are about 4 others i've got commit access to and help out with. a few (dba and diff) still aren't ported to 4.7. it's going to be a lot of work to port the rest to HEAD. i've got tons of functionality i want to add to these modules, but i simply don't have the time to do it all, try to keep my eye on core development and help out where i can, be the resident CVS guru, *and* port all my modules to the latest core API every 3 months...

i'm honestly torn. on the one hand, it's great to have new stable releases with useful improvements frequently, so that more sites at least have *access* to the latest functionality if they want it. however, given a) the rapid pace, b) the utter disregard for backwards compatibility (which is great!), c) the therefore very real cost in developer hours to keep contrib/themes/translations up to date with the latest core, and d) our relatively limited development resources (horsepower to write/review/test all the code involved), i wonder if aiming for releases every 3-4 months is too short. drupal core is great and all, but show me a drupal site *anywhere* that doesn't use at least *something* from crontrib. ;) therefore, i know it's painful to admit, but we really do have to be concerned about not having core run off far in advance of contrib.

either i can keep all 10 modules up to date with HEAD all the time, or i can make them better, but, given the other constraints on my time, i can't really do much of both. maybe the problem is that i'm trying to maintain too many things, but i don't see anyone else volunteering to take over any of these contribs. ;)

also, given that CCK in core is basically crippled as it stands (no fields) and yet CCK in contrib doesn't work w/ CCK in core, and flexinode isn't going to be ported to either 4.7 or 4.8 (according to ber), i have serious concerns about freezing. dynamic content types are essential for basically every real drupal site i'm dealing with. if core isn't going to provide fields, we absolutely *must* have a plan for working fields in 4.8 contrib (and further acknowledge that core basically requires contrib to be functional 95% of the time).

i don't have a concrete proposal, but, since dries is fishing for "zeitgeist", i just wanted to register my unease (not dissatisfaction).

thanks, -dww

cck will be easy... There isn't a whole lot to do with it.... as for translations/contrib and the release cycle.. maybe a longer freeze period would be nice... Get the devs chomping at the bit again for when head reopens. force some planning... give everyone time to catch up... and maybe get a more stable core at the end of it all... Might also encourage everyone to work really hard on getting stuff in now if they know they're gonna have 3 months of freeze to wait before they can submit new features....