Can I ask a question? This new module http://drupal.org/sandbox/robertcfi/1104066 says it's a sandbox, but it also says "This module is currently in production" in the text. My guess is that it will go nowhere (it is his first module also), but does anyone else think the criteria for scores are *ludicrous*? Since when is a site with more nodes and more modules and more users any "better" than a site with less? Should be obvious but a specialty site may be optimized and beautiful and in all ways built "perfectly" yet have a small user base and of course a small number of modules and nodes, yet far "better" than a big fat site slapped together with 1000 modules that serve only to SLOW it down. And also, since when does one have to know SEO to be a good developer?!?? Perhaps this is offtopic. Sorry. :( F PS: At least ranting made me feel better.
Well, yes, I certainly think the criteria for scores ludicrous and ultimately meaningless. I can't see there being much adoption of this module. FWIW, though - regarding the text that reads "This module is currently in production", I take it to mean "this module is currently being developed" (the way a movie is 'in production' when it is being made/filmed) rather than "this module is production site ready" Best, William On Thu, Mar 24, 2011 at 10:57 AM, Fred Jones <fredthejonester@gmail.com>wrote:
Can I ask a question? This new module http://drupal.org/sandbox/robertcfi/1104066 says it's a sandbox, but it also says "This module is currently in production" in the text. My guess is that it will go nowhere (it is his first module also), but does anyone else think the criteria for scores are *ludicrous*?
Since when is a site with more nodes and more modules and more users any "better" than a site with less?
Should be obvious but a specialty site may be optimized and beautiful and in all ways built "perfectly" yet have a small user base and of course a small number of modules and nodes, yet far "better" than a big fat site slapped together with 1000 modules that serve only to SLOW it down.
And also, since when does one have to know SEO to be a good developer?!??
Perhaps this is offtopic. Sorry. :(
F
PS: At least ranting made me feel better.
On Thu, Mar 24, 2011 at 8:57 AM, Fred Jones <fredthejonester@gmail.com> wrote:
Can I ask a question? This new module http://drupal.org/sandbox/robertcfi/1104066 says it's a sandbox, but it also says "This module is currently in production" in the text. My guess is that it will go nowhere (it is his first module also),
Yep.
but does anyone else think the criteria for scores are *ludicrous*?
Not entirely, but a little bit.
Since when is a site with more nodes and more modules and more users any "better" than a site with less?
Should be obvious but a specialty site may be optimized and beautiful and in all ways built "perfectly" yet have a small user base and of course a small number of modules and nodes, yet far "better" than a big fat site slapped together with 1000 modules that serve only to SLOW it down.
Sure it's true that measuring a site (or the site developers) by these metrics is not a complete way to measure the site's' value. And it would be easy to mess with the score by creating a bunch of modules with a .info file and an empty .module file... But taken in aggregate across a bunch of sites it could be a reasonably useful metric. If we want to compare developer's skills or the quality of sites in a scalable automated way then we have to base it on some metrics that may not be perfect but are reasonably proxies for real measures of skill.
And also, since when does one have to know SEO to be a good developer?!??
I've always considered that part of my job as a web developer, probably because the topic interested me and I learned Drupal and SEO at the same time. Increasingly I consider user experience to be part of my job. Naturally if I'm working with a team that has an SEO or UX expert on it I will defer to them but when I am working without that benefit or before they get a chance to review I consider it my job to follow at least some best practices from "other disciplines."
Perhaps this is offtopic. Sorry. :(
Not at all!
PS: At least ranting made me feel better.
That is good. If you want to rant more you should see the consulting list where that seems to be the main purpose. (and now *that* snark made *me* feel better). Cheers, Greg -- Greg Knaddison | 720-310-5623 | http://growingventuresolutions.com http://masteringdrupal.com - Videos and Tutorials
On Mar 24, 2011, at 9:39 AM, Greg Knaddison wrote:
But taken in aggregate across a bunch of sites it could be a reasonably useful metric. If we want to compare developer's skills or the quality of sites in a scalable automated way then we have to base it on some metrics that may not be perfect but are reasonably proxies for real measures of skill.
There are so many outside variables that can skew this, that I don't see how this could work: * Did the client have an adequate budget? Underbudgeted projects can have more problems, and that's not a measure of a developer's quality of work. * How are security issues handled? Will a client want a module like this on their site when it's advertising bugs and potential security flaws on their site? * Is the site being maintained? As new releases come out, maybe the client isn't interested in paying for those updates being deployed. Maybe the client is ignoring them altogether. Maybe the client has other priorities and doesn't give a crap anymore. * Does the developer have exclusive control? Unless the developer is providing ongoing maintenance on a completed site, and did all the development in the first place, the site build is not necessarily teh developer's -- all the more so if everything now is under someone else's control. Other developers may be brought on board, and their work reflects on the original developer's reputation. A site well-built could end up very troubled, dragging down the original developer's score. * Some modules are maintained better than others. A good module today may end up being unsupported for any of myriad reasons. Does a poorly maintained module end up reflecting on the site developer who never touched the code of that module? * What does number of nodes and members have to do with quality of work? Sheesh! * How does this reveal bad practices? Will it reveal that the developer has hard-coded blocks into the theme? Or makes direct database queries from a template? Or that the code that doesn't throw errors does not do what it's supposed to do? Or all the custom work is uncommented and unreadable? Or that core was hacked? * How is the difficulty of execution handled? Does this measure integration points with third-party systems? Does it measure migration of legacy data? Does not measure the difficulty of custom module development? * If you're going to measure SEO, you need to know the site goals. Does the site even answer the needs of the client? Or did the developer say "Drupal doesn't do that" and give the client something else that happens to ping the metrics this kind of module might monitor? Getting a lot of traffic from the wrong audience is not good SEO, and you can't measure that in a module that just looks for structural components. And you can't blame or credit good or bad SEO on the developer if the developer is not developing content as well as the site software. (Example: A random post I made about Marilyn Monroe is still one of the highest traffic posts on my personal blog. A blunt measure would say "great!" But that traffic has nothing to do with what I blog about in general, and if it were a business site that off-topic traffic from an audience I'm not trying to reach would be next to useless, unless I were only after selling ad impressions.) IMHO, it's next to impossible to automate scoring this way of what is knowledge work, especially when what you're measuring has so many unknowns. (Sorry for the rant.) Laura
On Thu, Mar 24, 2011 at 10:10 AM, Laura Scott <pinglaura@gmail.com> wrote:
On Mar 24, 2011, at 9:39 AM, Greg Knaddison wrote:
But taken in aggregate across a bunch of sites it could be a reasonably useful metric. If we want to compare developer's skills or the quality of sites in a scalable automated way then we have to base it on some metrics that may not be perfect but are reasonably proxies for real measures of skill.
There are so many outside variables that can skew this, that I don't see how this could work:
* Did the client have an adequate budget? Underbudgeted projects can have more problems, and that's not a measure of a developer's quality of work.
That's a problem on a specific site, but it wouldn't be a problem in broad aggregate which, I guess, is how this is meant to be used.
* How are security issues handled? Will a client want a module like this on their site when it's advertising bugs and potential security flaws on their site?
* Is the site being maintained? As new releases come out, maybe the client isn't interested in paying for those updates being deployed. Maybe the client is ignoring them altogether. Maybe the client has other priorities and doesn't give a crap anymore.
I'm not sure either of those are relevant. There's no code yet so we can't say how this will work but I don't see that as a feature on the project description.
* Does the developer have exclusive control? Unless the developer is providing ongoing maintenance on a completed site, and did all the development in the first place, the site build is not necessarily teh developer's -- all the more so if everything now is under someone else's control. Other developers may be brought on board, and their work reflects on the original developer's reputation. A site well-built could end up very troubled, dragging down the original developer's score.
Again, we don't know how this proposed system will work but your strawman could easily be fixed by taking the score at time of launch and crediting the initial developer with that score vs. later developers getting incremental scores.
* Some modules are maintained better than others. A good module today may end up being unsupported for any of myriad reasons. Does a poorly maintained module end up reflecting on the site developer who never touched the code of that module?
They make no mention of ranking modules used on the site as as proxy for site quality though that is a decent idea. But again, taking scores as snapshots in time could remove the problems you've pointed it.
* What does number of nodes and members have to do with quality of work? Sheesh!
It is a proxy for the relative size of the site. Visitors are another example. If the site is an e-commerce or donation focused site you could compare dollars in revenue. I think most folks agree that someone who can build a brochure site for the local ice cream shop has fewer skills than someone who builds a site meant to hold hundreds of thousands of nodes and users. The number of nodes/members gives a rough indication of which kind of site it is especially when taken in aggregate across several sites.
* How does this reveal bad practices? Will it reveal that the developer has hard-coded blocks into the theme? Or makes direct database queries from a template? Or that the code that doesn't throw errors does not do what it's supposed to do? Or all the custom work is uncommented and unreadable? Or that core was hacked?
* How is the difficulty of execution handled? Does this measure integration points with third-party systems? Does it measure migration of legacy data? Does not measure the difficulty of custom module development?
Great questions for the module to address.
* If you're going to measure SEO, you need to know the site goals. Does the site even answer the needs of the client? Or did the developer say "Drupal doesn't do that" and give the client something else that happens to ping the metrics this kind of module might monitor? Getting a lot of traffic from the wrong audience is not good SEO, and you can't measure that in a module that just looks for structural components. And you can't blame or credit good or bad SEO on the developer if the developer is not developing content as well as the site software. (Example: A random post I made about Marilyn Monroe is still one of the highest traffic posts on my personal blog. A blunt measure would say "great!" But that traffic has nothing to do with what I blog about in general, and if it were a business site that off-topic traffic from an audience I'm not trying to reach would be next to useless, unless I were only after selling ad impressions.)
There are currently 2 or three modules that measure progress in SEO that are relatively popular. So...some of the rough means of whether a site has achieved good SEO can be measured.
IMHO, it's next to impossible to automate scoring this way of what is knowledge work, especially when what you're measuring has so many unknowns.
It depends on your goal - if your goal is the perfect measure of knowledge work then of course it's impossible. If your goal is just something better than the other options then there's plenty of progress to be made. As always, we shouldn't let perfect be the enemy of progress. Greg -- Greg Knaddison | 720-310-5623 | http://growingventuresolutions.com http://masteringdrupal.com - Videos and Tutorials
On Mar 24, 2011, at 11:22 AM, Greg Knaddison wrote:
On Thu, Mar 24, 2011 at 10:10 AM, Laura Scott <pinglaura@gmail.com> wrote:
On Mar 24, 2011, at 9:39 AM, Greg Knaddison wrote:
But taken in aggregate across a bunch of sites it could be a reasonably useful metric. If we want to compare developer's skills or the quality of sites in a scalable automated way then we have to base it on some metrics that may not be perfect but are reasonably proxies for real measures of skill.
There are so many outside variables that can skew this, that I don't see how this could work:
* Did the client have an adequate budget? Underbudgeted projects can have more problems, and that's not a measure of a developer's quality of work.
That's a problem on a specific site, but it wouldn't be a problem in broad aggregate which, I guess, is how this is meant to be used.
How many sites will a developer develop over time? 5-10 a year, maybe? If they're relatively easy. 2000 hours baseline. It will take many years to build up a "broad aggregate." Of course, shops can work much faster, but then you don't have one developer being measured.
* Does the developer have exclusive control? Unless the developer is providing ongoing maintenance on a completed site, and did all the development in the first place, the site build is not necessarily teh developer's -- all the more so if everything now is under someone else's control. Other developers may be brought on board, and their work reflects on the original developer's reputation. A site well-built could end up very troubled, dragging down the original developer's score.
Again, we don't know how this proposed system will work but your strawman could easily be fixed by taking the score at time of launch and crediting the initial developer with that score vs. later developers getting incremental scores.
That doesn't address who else has fingers in the code. And if you follow the "release early and often" motto, the initial release is almost certainly going to run into issues.
* What does number of nodes and members have to do with quality of work? Sheesh!
It is a proxy for the relative size of the site. Visitors are another example. If the site is an e-commerce or donation focused site you could compare dollars in revenue.
I think most folks agree that someone who can build a brochure site for the local ice cream shop has fewer skills than someone who builds a site meant to hold hundreds of thousands of nodes and users. The number of nodes/members gives a rough indication of which kind of site it is especially when taken in aggregate across several sites.
I think it's a mistake to equate volume with quality. It really depends upon the use cases.
* If you're going to measure SEO, you need to know the site goals. Does the site even answer the needs of the client? Or did the developer say "Drupal doesn't do that" and give the client something else that happens to ping the metrics this kind of module might monitor? Getting a lot of traffic from the wrong audience is not good SEO, and you can't measure that in a module that just looks for structural components. And you can't blame or credit good or bad SEO on the developer if the developer is not developing content as well as the site software. (Example: A random post I made about Marilyn Monroe is still one of the highest traffic posts on my personal blog. A blunt measure would say "great!" But that traffic has nothing to do with what I blog about in general, and if it were a business site that off-topic traffic from an audience I'm not trying to reach would be next to useless, unless I were only after selling ad impressions.)
There are currently 2 or three modules that measure progress in SEO that are relatively popular. So...some of the rough means of whether a site has achieved good SEO can be measured.
Only bad SEO can be partially measured by noting what's missing. Good SEO can't be quantified programmatically, unless you can measure the goals and conversion rates. See my example above. Maybe SEO is the wrong term here. Maybe something like Search Engine Validity would be more appropriate. Real SEO is more than having the right technical architecture.
IMHO, it's next to impossible to automate scoring this way of what is knowledge work, especially when what you're measuring has so many unknowns.
It depends on your goal - if your goal is the perfect measure of knowledge work then of course it's impossible. If your goal is just something better than the other options then there's plenty of progress to be made.
As always, we shouldn't let perfect be the enemy of progress.
On the other hand, it's a mistake to believe you can draw solid conclusions from bad or unreliable data. If this were developed as a module for the developer, to help follow best practices, like your own security module, then it's a different thing. As that, in fact, it could be very useful. But otherwise I feel it would be just another standardized test that measures little while giving people the belief they know a lot more than they really do. (Like with our public schools.) Laura
On Thu, Mar 24, 2011 at 6:51 PM, Laura Scott <pinglaura@gmail.com> wrote:
On the other hand, it's a mistake to believe you can draw solid conclusions from bad or unreliable data.
Right! Ask developers to install a module for unreliable statistics is not easy. The success of CertifiedToRock is because of secret algorithm *AND* because it does not requires Drupal users to do any particular thing (except for participate to the community, which they already do). -- Hai-Nam Nguyen (aka jcisio) http://jcisio.com
IMHO, it's next to impossible to automate scoring this way of what is knowledge work, especially when what you're measuring has so many unknowns.
You made many good points, of which I touch on only this one. I make a good business doing Interesting Things (TM) in a conservative amount of Highly Maintainable (TM) code. The quality or novelty of this stuff can't really be measured in an automated way. (At least without the help of a language parsing bot who is also familiar with the intricacies of Drupal and the intricacies of databases, and the [...]) If I implement a full proprietary facial detection and facial recognition library for use on a site, and it is all contained within a single custom module space, that, presumably would make for a low score. Nevermind the mathematics, the libraries called in, the testing on multiple faces against multiple backgrounds, the database arbitrage. Nevermind the skill as a developer. There are few modules (one!) involved. Low score. And on the other hand, I know of extremely talented Drupal developers who write very little code but who can squeeze every little bit out of the modules used for the purposes required. Again, low score. For talent. I like to code novel things. Low score. They don't code but use existing tools properly. Low score. It would be very easy to game this system. The whole thing is silly. But, in the end, whomever developed the module probably learned a lot about module development in the process .. always a good thing. Yay sandboxes! (and I don't mean that sarcastically) Of course, if we could all go back and convince former clients that they need to install such a thing with our own userid, we would probably do very well by the metrics. Some of the sites that I've been involved with - and most here too, I'm sure - have blown right up over the years. But .. really? How ridiculous and shallow would that request sound? In the end, I don't blame the OP for being offended by the metrics used. At the same time, I figure, just leave it. It is a sandox module, and good luck to it.
participants (5)
-
Fred Jones -
Greg Knaddison -
jcisio -
Laura Scott -
William Smith