On Aug 9, 2006, at 7:23 AM, Gary Feldman wrote:
Kieran Lal wrote:
... How would you describe yourself as a Drupal administrator? (select all that apply) This, and several others, seem to be "pick the best one" questions, not "select all that apply" questions.
ok.
New user Non-technical user developing community or social change web site forced to become "accidental technologist" The nature of the web site doesn't seem relevant. How about: " Non-technical user developing a web site, and becoming adept at the technology"
Yes, it's very relevant. See interviews and response to Dries.
Inexperienced with Drupal but curious. Learning fast based on other web site building experience Should this just be "Inexperienced with Drupal but learning fast, based on ..."?
Ok, better.
Experienced user, comfortable with configuring site rapidly Perhaps "Experienced administrator, ..." would be better. I expect most participants won't have the benefit of the recent discussions on terminology for users, administrators, developers, etc. ... How frequently do you administer you your Drupal site? (select all that apply) Again, pick the best one.
Let me review this again. Many people see themselves as wearing several hats(particularly because they run more than one site), so I am hesitant to stick them into a single category.
... How long do you administer your Drupal site in a single sitting approximately? (select all that apply) Ditto
... How does Drupal help you accomplish your goals as a web site administrator? (select all that apply) Rapid deployment of features and ability to meet customer requirements saves time Strike "saves time", it's implied by "rapid".
ok.
Web based content publishing is easy and allows for end user contributions Learnable website that can be taught to users and allows website developers to leverage existing learning when building new sites This seems to be two separate items.
You are right. Drupal administrators learning curve, and Drupal site developer learning curve. I had hoped to cluster into one response. See response to Dries.
Allows for monitoring and logging of the website Module configuration, extensibility with new modules, and clean code make it easier to configure the website as you need it How about "easy to customize", just to read better?
Ok.
Dynamic and exciting community allows for rapid and fun learning Does community mean "Drupal user community"? Or the community that the person is creating?
Drupal developer community (core + contribs + consulting + groups + mailing lists + forums + themes). I'll clarify.
... How does Drupal help your users? ... Cost effective and winning over non-technical decision makers I can't parse this. Should it be "Cost effective, easy to convince non-technical decision makers"?
Agreed. I'll re-do it.
... What are some common Drupal administration tasks? (select all that apply) Should this be "select all", "select no more than 3", or something else? In this case, "select all" is meaningful, but I don't think it's the right question.
I'll use Dries suggestion.
I'd also change it to read "What are _your_ ...", so it's clear it's about what they do, not what they think Drupal administrators in general do.
...
Ok, good feedback.
What are some infrequent Drupal tasks? (select all that apply) Ditto, but see next comment. Add new features by adding modules or coding new features ... Modify site navigation Other_______________ The fact that this list differs from the preceding list suggests that you already believe these to be infrequent. But does that mean that if the respondent doesn't check something, it means that it's a frequent task, or that it's not a task at all? Or more to the point, what is this question trying to learn?
Think about the 400* options in Drupal's administration interface today. Some should easily accessible as they are routinely accessed. Think about the hundreds of professional Drupal consultants who live in the Drupal administration interface day in an day out. For example logs or spam seem to be the most frequently accessed. However, a task like clean URLs is only usually set once, but it has prominent real estate at administer >> settings. Infrequent tasks should be tucked away in favor of more common tasks, but made easy to find during site set-up. The first set of questions are about understanding the users. The second set of questions are about the situation the user is in when they are doing administration. In general I would say that the notion of administration configuration and administration settings need to give way to: Site set-up, Site development, site testing and maintenance. At least this is what the interviews reveal about how Drupal is actually being used.
Should it be something like "These are some Drupal tasks that are done less frequently.
infrequent, less frequently. Potato , Potato, Tomato, Tomato. Pick the one that is grammatically correct and let me know!
Which of these do you consider important?"
That question is asked below.
When you administer your site you find it easy to: (1 is not easy, 5 is very easy) ... When you administer your site you find it hard to? (1 is not hard, 5 is very hard)
The duality here is a bit more meaningful, but still confusing. Why not combine these into one question?
Response rate for long questions falls off significantly. I want to get the response rates consistent and high to avoid skewed results.
Regardless, having the first one be "1 is not easy" and the second one be "1 is not hard" will cause people to mess up. If you want to keep these as two separate questions, then perhaps it would work better if you asked "Do you agree or disagree that the following are easy tasks?", and "Do you agree or disagree that the following are difficult tasks?", with 1 being strongly agree and 5 being strongly disagree. I'm not positive this would be better, but it feels better to me.
Ok, I'll consult with Charlie Lowe, who taught audience research at Purdue, and get his input on your suggestion.
Another alternative would be to just move the questions away from each other, so that the respondent doesn't trip over the flipping of the scale.
Ok, one of the reasons I use SurveyMonkey is that it jumbles the order of questions to avoid leading bias. Insert your favorite joke here!
What other important tasks did not fit into the categories above for you? Which categories? There's frequent/infrequent and easy/hard. If something isn't checked, does that mean it's not important, or that it's neither easy nor hard, or what?
I don't think we can infer anything from a non-selection.
My expectation when I read "What other important tasks" is that there will be a text area for me to insert arbitrary items.
Yes. The part that says Othere________________ will be a text field.
If these have already been identified as possible tasks, why aren't they just included above?
It's a challenge. As the interviews progressed the responses got more and more varied. By the time I was clustering the answers for the important tasks there were 20+ unique important tasks for 10 interviewees!
... Setting up a testing process including creating a test role, soliciting feedback from users, configuring corrections, and working with module development process to get bugs resolved Should this be split into a testing item and a bug fix item?
Yes, it's a casualty of trying to cluster responses. I am trying to keep the number of responses short so that we keep the survey response rates high, at the same time get enough granularity in the responses so that we can take action . It's a tough balance.
I hope this is helpful. It it's not, let me know and I'll shut up.
It's absolutely helpful. The more review, the clearer the survey questions, the more accurate the responses, the clearer the analysis, and the greater capacity the developer community has to address the needs of their users. Cheers, Kieran
Gary