What would be even better is that once the site page is back up is modify the drupal.module to also send a list of the modules enabled, this would allow people to identify how popular certain modules are.

I would make it so there is no direct correlation between the site and the modules that are running.

This has been implemented. As of Drupal 4.7.0 beta 3, Drupal optionally sends a list of installed themes and modules. While the client-side functionality has been implemented, we still need to implement the backend infrastructure to process the data, and to integrate it with the project module. Want to help? Let me know. -- Dries Buytaert :: http://www.buytaert.net/

...

client-side functionality has been implemented, we still need to implement the backend infrastructure to process the data, and to integrate it with the project module. Want to help? Let me know.

Yes I think I could spear some time for something like this.

Awesome. In that case, take a look at the database scheme Nedjo has implemented and try to work out a way to obtain ratings from it. We also need to figure out a way to make the system tamper-proof. The third and last action item is to show the ratings in the project module. We've been discussing some of this in the project module's issue tracker so make sure to tune in. I'd focus on the first two items because these are most pressing; if it turns out we have to make changes to the drupal.module, we want to do that before Drupal 4.7.0 is released. -- Dries Buytaert :: http://buytaert.net/

Can I get a url to the discussion.

Here's the patch that went in: http://drupal.org/node/39697 When a site chooses to enable the drupal.module and post data on its installed modules, cron will call XML-RPC to upload information on the site's name, url, etc. and all enabled modules. Optionally, sites can also send (a) total number of users, and (b) total number of nodes. These extra data are designed to provide some quantifable parameters of how large a site is, which we could use to add weight to that site's statistics. Data posted from sites are stored in the core tables 'client' (the basic information about a client site) and 'client_system' (the data on enabled modules, themes, theme engines--the same as what are found in the system.module). You can see the database definitions in database.mysql and database.pgsql. Clients are identified by a client id (cid), created the first time they connect and reused when they reconnect. In the table client_system: 'cid' -> this is the client id 'name' -> this the name of an enabled system component, e.g., 'page' 'type' -> this is the type of the enabled system component, e.g., 'module' To convert this into usable statistics we can use for browsing projects on drupal.org, we'll need to do at least two things: 1. Relate the system component to the project. Taking e-commerce as an example, there are many different modules, each of which will appear separately in the client_system table. We can use the data in the cvs module's tables to find out what project each component maps to. 2. Use a hook implemented by the Project module to implement a new browsing option. Project module uses a hook, _project_sort_methods(), to provide the information needed to implement project browsing options. See function project_project_sort_methods() in project.module. We'll need to implement a new sort method, e.g., 'popularity'. This could either be added to function project_project_sort_methods() or put in a different module, e.g., cvs. Thanks for stepping forward! I'll be happy to work with you, Gordon, or any others on implementing this. Nedjo

On 1/25/06, Jeff Eaton <jeff@viapositiva.net> wrote:

Not to trumpet my own module, but has anyone considered VotingAPI as a tool for rating module quality? It'd be pretty simple to whip up a module with several criteria (Stability, Features, Flexibility... Each rated on a 1-5 scale, for example) and produce an average per-module and per-criteria rating.

This usually turns into a popularity contest, which eventually reflects the same information as #downloads. imho, absolute numbers have little meaning: importance of a module is a factor of it's 1) usefulness 2) code quality (lack of bugs, buggy modules = bad reputation for drupal) 3) maintenance quality (support in future drupal versions?) while (1) can be gauged from #downloads / popularity, the latter 2 can (possibly) be estimated from project.module stats. Actually, I wouldn't mind having a small paragraph giving devel statistics for each module, like #open confirmed bugs #critical bugs in the past month (open / solved) #feature requests handled #commits in the last 4 months -Arnab

I disagree that rating and download numbers will result in similar values. There are large number of modules I've downloaded that I would rate poorly in several of those categories -- the fact that I've *investigated* something should not be taken as an indication of my *conclusion* about it.

i agree that voting would be useful. if it turns out not tobe useful, we can ditch it or restrict voting to a certain roles (i.e. 'editor votes'). editor picked modules are common on mozilla extensions, download.com, versiontracker, etc.

I thought it was a widely held consensus that giving people on Drupal.org the chance to vote on module quality was a very good idea. It feels to me like this is one of those "community process" issues that we should be addressing in my session in Vancouver. The idea exists, a consensus (or at least majority) is in favor. It seems urgent enough to merit action. Whose role is it to step forward right now and get it done? Consider the question purely rhetorical for now, I don't want to hijack this thread. cheers, Robert

On 26 Jan 2006, at 18:26, Robert Douglass wrote:

It feels to me like this is one of those "community process" issues that we should be addressing in my session in Vancouver. The idea exists, a consensus (or at least majority) is in favor. It seems urgent enough to merit action. Whose role is it to step forward right now and get it done?

We pretty much all agree that this would be a good thing. Most of this discussion is somewhat moot at this point. Before we'll deploy a community rating system, 1. we'll have to upgrade drupal.org to CVS HEAD (some of us are actively working on this) 2. we'll roll out an automatic rating system based on the installation base of themes and modules Whose role is it to work on a rating module? Whoever feels it is both interesting and important enough. :) -- Dries Buytaert :: http://www.buytaert.net/

On Thu, January 26, 2006 2:34 pm, Jeff Eaton said:

...

1. we'll have to upgrade drupal.org to CVS HEAD (some of us are actively working on this) 2. we'll roll out an automatic rating system based on the installation base of themes and modules

Whose role is it to work on a rating module? Whoever feels it is both interesting and important enough. :)

-- Dries Buytaert :: http://www.buytaert.net/

I'm working on two other projects at the moment (one of them also ratings-based) but I'd love to dive into this in the next couple of weeks. As a disclaimer, I would probably be basing it on VotingAPI's underlying engine. I hate reinventing the wheel, and I think the current crop of 'ratings solutions' for Drupal would make an example implementation on a flexible API beneficial.

If there's anyone out there who'd be willing to give VotingAPI a code review, to spot potential problems or issues if it were used on a high-volume site like Drupal.org, I'd really appreciate it.

--Jeff

I'm planning to use VotingAPI for a product review site very soon. What we're going to need is to have a separate comment for *each* review axis, so I figure that's a module that implements VotingAPI (unless said module already exists?). I'll keep my eyes open on VotingAPI's internals and submit patches as I find cause to. If any of that ends up helping the Project module, so much the better. Off the bat, my first question is whether you're caching totals or not. When 1000 nodes have 1000 reviews each, calculating all totals, averages, etc. on the fly is going to get rather slow. --Larry Garfield

If there's anyone out there who'd be willing to give VotingAPI a code review, to spot potential problems or issues if it were used on a high-volume site like Drupal.org, I'd really appreciate it.

I had a quick look at the VotingAPI code. It is well-documented and looks secure, but doesn't adhere to Drupal's coding conventions making it slightly annoying to audit the code. My problem with the VotingAPI module is this: it addresses an _extremely_ _trivial_ problem. It saves us 10 lines of code and 20 minutes of implementation work at the cost of an extra layer of code (performance cost), an additional module to monitor/track/audit before upgrades (extra dependencies, time overhead) and install (usability). Maybe the added value becomes clear after I spent some more time looking at it but right now, I prefer to go with a single, dedicated module. -- Dries Buytaert :: http://www.buytaert.net/

My problem with the VotingAPI module is this: it addresses an _extremely_ _trivial_ problem. It saves us 10 lines of code and 20 minutes of implementation work at the cost of an extra layer of code (performance cost), an additional module to monitor/track/audit before upgrades (extra dependencies, time overhead) and install (usability). Maybe the added value becomes clear after I spent some more time looking at it but right now, I prefer to go with a single, dedicated module.

I can definitely understand those objections. I think 10 lines/20 minutes is a bit of an understatement considering the more complex voting scenerios it's capable of handling, but the real benefit of it is the potential for cross-module consistency and sharing of voting data. Dedicated voting modules have a strong tendency to do the same thing in a dozen different ways. Some in contrib have a great UI, others calculate the statistics a given user needs, others are capable of working with the moderation queue. None work together. If I want to moderate nodes and comments, track the most popular pictures, and allow users to rate downloads on my site, I have to install four separate modules with four wildly different UIs and four completely different silos of data. The same objections could be raised about taxonomy: why use a complex, topheavy API when all we need to do is assign a 'category' to a node? I'll be the first to say that my skills are not equal to those who crafted Taxonomy.module, and the VotingAPI is a lot less mature. I do think that the need for a consistent way of storing and accessing voting data across modules will enable a broader range of tools for working with it, in the same way that we have loads of taxonomy-related helper modules. Frustration with those silos of voting data is what prompted me to dive into the project in the first place. Of course, whether drupal.org and project.module is the correct place to champion this is another question entirely. If folks feel it would be a bad use of resources or would add unecessary complexity to the site, that's that. --Jeff

Would it also be possible to distinguish between the criteria in the user rights system? On hotscripts they have a public voting and a closed voting. That would be a cool tool to have Best Gunnar

Not to trumpet my own module, but has anyone considered VotingAPI as a tool for rating module quality? It'd be pretty simple to whip up a module with several criteria (Stability, Features, Flexibility... Each rated on a 1-5 scale, for example) and produce an average per- module and per-criteria rating.

I don't know if anyone has taken a look at VotingAPI with a critical eye towards acceptability for use on drupal.org, but feature-wise it provides the infrastructure we'd need for that.

--Jeff

Gunnar Langemark gunnar@langemark.com

After all the back and forth discussions about rating modules/themes/etc, Adrian poked me with a stick until I whipped up an actual patch. *grin* http://drupal.org/node/50605 has the details for those passionately interested in project.module and its future. The changes make use of VotingAPI, so there may be philosophical objections by some, but it's code and it's there. Feedback/changes/ideas welcome! --Jeff

Hi, I was thinking more last night, and what would also be handy is to also split up the nodes so that we have a break down of how many nodes are being created by each module. So if we have 20 nodes, and we get the break down that 5 are from the page.module and 15 are from the story.module. Helps give additional weight to a module in that way that it is rated. Gordon. On Sat, 2006-01-28 at 16:48 +1100, Gordon Heydon wrote:

Hi,

On Wed, 2006-01-25 at 09:29 +0100, Dries Buytaert wrote:

...

...

...

client-side functionality has been implemented, we still need to implement the backend infrastructure to process the data, and to integrate it with the project module. Want to help? Let me know.

Yes I think I could spear some time for something like this.

Awesome. In that case, take a look at the database scheme Nedjo has implemented and try to work out a way to obtain ratings from it. We also need to figure out a way to make the system tamper-proof. The third and last action item is to show the ratings in the project module. We've been discussing some of this in the project module's issue tracker so make sure to tune in. I'd focus on the first two items because these are most pressing; if it turns out we have to make changes to the drupal.module, we want to do that before Drupal 4.7.0 is released.

I have had a look at the drupal.module to see what information is being past, and see if it could be tampered with so that it could alter the results.

All the information that is provided is quite good to create some kind of rating system, but I would add 1 more piece of information. I know that we have users, I would also have active users. This would most likely be something like number of users that have accessed the site in the last month, or number of users that have posted something in the last month. The later would bring the number down a lot, so maybe the first would be enough.

What I would like to see is a simple rating, like a number between 0.0 and 10.0 which would be made up of a number of different components which either increase or decrease the value.

I like calculated rating systems. Ones which require people to vote or a counter of the number or downloads isn't a good indication. Downloads for drupal is actually a poor indicator because I know that I never download though drupal.org and use cvs get a version of the required version. Now this isn't counted. Also voting means that the less sexy modules will not get voted on at all.

Something that would be a large part of the over all figure would be the the percentage of the number of sites that provide a module list who run the module. Other factors like maybe number of open bugs should detract from the rating.

Things like the total number of active users of sites which use your module. So if a module is only installed in 2% of sites, but were sites with larger user bases and your module had 60% of the active user base, then this would something that should increase your rating.

Using this method of building up the rating would also stop tampering as there would be figures out of people control. Also having something like number of open issues detract from a rating would make it a good incentive for people to work issues.

Having other measures like downloads would be ok but I would weight these so that they do not have as much effect on the rating, as say number of sites running it.

Adding active users to the rpc would be a good idea, because I know that drupal.org has over 40000 (I think) users but they would not be all active.

What we could do is create a api system so that all of the individual parts of the ratings are individual calls and then people can think of different methods of rating modules, and just implements this calculation. This could then be added and removed easily and also allow these to be rating to be weighted depending on the importance of the calculation. So if we had some obscure calculation like modules owned by user 959 get an additional 5 points will be weighted down.

We could also do something like the google page rank, or for Trekkie's warp factor in that each value is twice that of it predecessor, so getting a module to 10, would just about be impossible.

Also these calculations could get quite complicated and take a while to run, so what we could do is only update ratings only once a month, as a bad rating will hang around for a bit.

Basically the more components that harder to tamper with the rating.

Just a few of my ideas. Gordon.

!DSPAM:1000,43db05f8130621468016628!

And there are some excellent modules that get completely overlooked because their name and description doesn't necessarily speak to the masses. I've just discovered how useful the relativity.module is, but I doubt that it gets downloaded and used very much. Robert Richard Archer wrote:

At 4:42 PM -0800 24/1/06, ae2005 wrote:

...

one quick proxy for measuring quality of modules could be number of downloads, like the list kieran posted. presumably the most downloaded ones are relatively stable, due to research and filtering done by individuals first.

I downloaded all the image modules trying to find a good one. So they all got a hit, and if I were to rate them none would score more than 5/10.

The high number of downloads of all the image modules simply reflects a strong demand for image handling. Not the quality of the modules.

To give useful feedback as to quality we would need a manual rating system, or Gordon's suggestion of a "phone home" with a list of installed and enabled modules

...Richard.