Drupal administration survey draft based on 10 interviews
The survey questions and responses listed below are derived from interviews conducted with Drupal administrators from novice to expert. The responses must be evaluated for clarity and conciseness. All questions are multiple selection in that the survey taker can select those that apply to help us best understand the Drupal users in aggregate. Each question in the survey will have an other selection so we can look for answers that did not surface during interviews. The survey will be done using SurveyMonkey.com. It will run for 45 days linked from the front page of Drupal.org. If you complete all the questions in this survey and would like to be recognized as a contributor to improving Drupal user experience please indicate your Drupal.org username Drupal.org username_______________ How would you describe yourself as a Drupal administrator? (select all that apply) New user Non-technical user developing community or social change web site forced to become "accidental technologist" Inexperienced with Drupal but curious. Learning fast based on other web site building experience Experienced user, comfortable with configuring site rapidly Module developer who understands what's going on behind the scenes Other_______________ How frequently do you administer you your Drupal site? (select all that apply) Monthly Once every couple of weeks Several times a week Daily for 30-60 minutes of website management Daily for several hours in site development and deployment Other_______________ How long do you administer your Drupal site in a single sitting approximately? (select all that apply) Thirty minutes to an hour Few hours Half a day All day long Other_______________ How does Drupal help you accomplish your goals as a web site administrator? (select all that apply) Rapid deployment of features and ability to meet customer requirements saves time Web based content publishing is easy and allows for end user contributions Learnable website that can be taught to users and allows website developers to leverage existing learning when building new sites Allows for monitoring and logging of the website Module configuration, extensibility with new modules, and clean code make it easier to configure the website as you need it Dynamic and exciting community allows for rapid and fun learning Other_______________ Why do you use Drupal? (select all that apply) Features, extensibility, integration of modules, clean code allow you to customize your site Speed of deployment and re-use of existing functionality Quality community is helpful, has smart people, is very active, and is working on community solutions not just technical capabilities Allows users to create content, categorize content, and add navigation to the site Other_______________ How does Drupal help your users? (select all that apply) Gives them the features they want quickly Allows users to create web based content such as forum posts, or blogs Drupal community innovates and provides community building capacity with Drupal Cost effective and winning over non-technical decision makers Other_______________ What are some common Drupal administration tasks? (select all that apply) Monitor site through reviewing logs, looking at user activity Manage spam through comments, track backs, forum, and user registration Configure modules Update modules, install modules, test patches, track fixes for modules Work on themes and add theme template to customize module output Learn about Drupal capabilities and features, understand terminology, and plan improvements Manage users accounts, change permissions Respond to user feedback during testing and make changes Create web pages through the creating content types Other_______________ What are some infrequent Drupal tasks? (select all that apply) Add new features by adding modules or coding new features Manage spam, delete content and comment, and banning users Modify existing content such publishing to the front page, making content sticky, or changing titles Layout content in positions with blocks, views, or designing a content section Change site theme or theme a content section Modify site navigation Other_______________ When you administer your site you find it easy to: (1 is not easy, 5 is very easy) 1-5 Add new features and install modules 1-5 Automate tasks using cron 1-5 Manage user information by editing user account permissions 1-5 Change themes or make changes to layout with theme templates 1-5 Post, edit, and version web pages and other content types 1-5 General settings such as changing titles, or date format When you administer your site you find it hard to? (1 is not hard, 5 is very hard) 1-5 Discover where to configure site settings, or where all the different places where a site needs to be configured are 1-5 Understanding Drupal specific concepts or understand technical language without pictures 1-5 Upgrade modules manually, manage module conflicts in output, download modules that don't include dependencies 1-5 Manage content specifically creating new content types, viewing differences in node versions, importing content, or seeing multiple previews of content on a single page 1-5 Manage large groups or users, nodes, menu items, comments, blocks 1-5 Get overview of new content and activities on site 1-5 Configure user permissions with existing granularity 1-5 Manage menus such as changing menu block weights for all menus on a page, or ordering menu items within a menu What other important tasks did not fit into the categories above for you? (select all that apply) Analyzing logs to understand the state of your site Integrating modules and resolving clashes in the output Setting up a testing process including creating a test role, soliciting feedback from users, configuring corrections, and working with module development process to get bugs resolved Work on the theme and templates to create structure and distinct looks to sections of the site Hide output from modules such as node links or browsing links Managing menus with the menu editing interface, bulk menu editing, and menu options Manage performance of the site Identify popular content Aggregate content from other sources into a site Here is table which was used to manually cluster responses from the interviews: http://drupal.org/node/77193. The clusters were used to make up the survey responses. If one of the survey options is confusing or needs to be reworded please consult the table and source interviews to help in making the responses clearer. Interview 1: http://groups.drupal.org/node/1136 Interview 2: http://groups.drupal.org/node/1135 Interview 3: http://groups.drupal.org/node/1133 Interview 4: http://groups.drupal.org/node/1137 Interview 5: http://groups.drupal.org/node/1134 Interview 6: http://groups.drupal.org/node/1139 Interview 7: http://groups.drupal.org/node/1143 Interview 8: http://groups.drupal.org/node/1144 Interview 9: http://groups.drupal.org/node/1146 Interview 10: http://groups.drupal.org/node/1148
Kieran Lal wrote:
... How would you describe yourself as a Drupal administrator? (select all that apply) This, and several others, seem to be "pick the best one" questions, not "select all that apply" questions. New user Non-technical user developing community or social change web site forced to become "accidental technologist" The nature of the web site doesn't seem relevant. How about: " Non-technical user developing a web site, and becoming adept at the technology" Inexperienced with Drupal but curious. Learning fast based on other web site building experience Should this just be "Inexperienced with Drupal but learning fast, based on ..."? Experienced user, comfortable with configuring site rapidly Perhaps "Experienced administrator, ..." would be better. I expect most participants won't have the benefit of the recent discussions on terminology for users, administrators, developers, etc. ... How frequently do you administer you your Drupal site? (select all that apply) Again, pick the best one. ... How long do you administer your Drupal site in a single sitting approximately? (select all that apply) Ditto ... How does Drupal help you accomplish your goals as a web site administrator? (select all that apply) Rapid deployment of features and ability to meet customer requirements saves time Strike "saves time", it's implied by "rapid". Web based content publishing is easy and allows for end user contributions Learnable website that can be taught to users and allows website developers to leverage existing learning when building new sites This seems to be two separate items. Allows for monitoring and logging of the website Module configuration, extensibility with new modules, and clean code make it easier to configure the website as you need it How about "easy to customize", just to read better? Dynamic and exciting community allows for rapid and fun learning Does community mean "Drupal user community"? Or the community that the person is creating? ... How does Drupal help your users? ... Cost effective and winning over non-technical decision makers I can't parse this. Should it be "Cost effective, easy to convince non-technical decision makers"? ... What are some common Drupal administration tasks? (select all that apply) Should this be "select all", "select no more than 3", or something else? In this case, "select all" is meaningful, but I don't think it's the right question.
I'd also change it to read "What are _your_ ...", so it's clear it's about what they do, not what they think Drupal administrators in general do.
... What are some infrequent Drupal tasks? (select all that apply) Ditto, but see next comment. Add new features by adding modules or coding new features ... Modify site navigation Other_______________ The fact that this list differs from the preceding list suggests that you already believe these to be infrequent. But does that mean that if the respondent doesn't check something, it means that it's a frequent task, or that it's not a task at all? Or more to the point, what is this question trying to learn? Should it be something like "These are some Drupal tasks that are done less frequently. Which of these do you consider important?"
When you administer your site you find it easy to: (1 is not easy, 5 is very easy) ... When you administer your site you find it hard to? (1 is not hard, 5 is very hard) The duality here is a bit more meaningful, but still confusing. Why not combine these into one question?
Regardless, having the first one be "1 is not easy" and the second one be "1 is not hard" will cause people to mess up. If you want to keep these as two separate questions, then perhaps it would work better if you asked "Do you agree or disagree that the following are easy tasks?", and "Do you agree or disagree that the following are difficult tasks?", with 1 being strongly agree and 5 being strongly disagree. I'm not positive this would be better, but it feels better to me. Another alternative would be to just move the questions away from each other, so that the respondent doesn't trip over the flipping of the scale.
What other important tasks did not fit into the categories above for you? Which categories? There's frequent/infrequent and easy/hard. If something isn't checked, does that mean it's not important, or that it's neither easy nor hard, or what?
My expectation when I read "What other important tasks" is that there will be a text area for me to insert arbitrary items. If these have already been identified as possible tasks, why aren't they just included above?
... Setting up a testing process including creating a test role, soliciting feedback from users, configuring corrections, and working with module development process to get bugs resolved Should this be split into a testing item and a bug fix item?
I hope this is helpful. It it's not, let me know and I'll shut up. Gary
On Aug 9, 2006, at 7:23 AM, Gary Feldman wrote:
Kieran Lal wrote:
... How would you describe yourself as a Drupal administrator? (select all that apply) This, and several others, seem to be "pick the best one" questions, not "select all that apply" questions.
ok.
New user Non-technical user developing community or social change web site forced to become "accidental technologist" The nature of the web site doesn't seem relevant. How about: " Non-technical user developing a web site, and becoming adept at the technology"
Yes, it's very relevant. See interviews and response to Dries.
Inexperienced with Drupal but curious. Learning fast based on other web site building experience Should this just be "Inexperienced with Drupal but learning fast, based on ..."?
Ok, better.
Experienced user, comfortable with configuring site rapidly Perhaps "Experienced administrator, ..." would be better. I expect most participants won't have the benefit of the recent discussions on terminology for users, administrators, developers, etc. ... How frequently do you administer you your Drupal site? (select all that apply) Again, pick the best one.
Let me review this again. Many people see themselves as wearing several hats(particularly because they run more than one site), so I am hesitant to stick them into a single category.
... How long do you administer your Drupal site in a single sitting approximately? (select all that apply) Ditto
... How does Drupal help you accomplish your goals as a web site administrator? (select all that apply) Rapid deployment of features and ability to meet customer requirements saves time Strike "saves time", it's implied by "rapid".
ok.
Web based content publishing is easy and allows for end user contributions Learnable website that can be taught to users and allows website developers to leverage existing learning when building new sites This seems to be two separate items.
You are right. Drupal administrators learning curve, and Drupal site developer learning curve. I had hoped to cluster into one response. See response to Dries.
Allows for monitoring and logging of the website Module configuration, extensibility with new modules, and clean code make it easier to configure the website as you need it How about "easy to customize", just to read better?
Ok.
Dynamic and exciting community allows for rapid and fun learning Does community mean "Drupal user community"? Or the community that the person is creating?
Drupal developer community (core + contribs + consulting + groups + mailing lists + forums + themes). I'll clarify.
... How does Drupal help your users? ... Cost effective and winning over non-technical decision makers I can't parse this. Should it be "Cost effective, easy to convince non-technical decision makers"?
Agreed. I'll re-do it.
... What are some common Drupal administration tasks? (select all that apply) Should this be "select all", "select no more than 3", or something else? In this case, "select all" is meaningful, but I don't think it's the right question.
I'll use Dries suggestion.
I'd also change it to read "What are _your_ ...", so it's clear it's about what they do, not what they think Drupal administrators in general do.
...
Ok, good feedback.
What are some infrequent Drupal tasks? (select all that apply) Ditto, but see next comment. Add new features by adding modules or coding new features ... Modify site navigation Other_______________ The fact that this list differs from the preceding list suggests that you already believe these to be infrequent. But does that mean that if the respondent doesn't check something, it means that it's a frequent task, or that it's not a task at all? Or more to the point, what is this question trying to learn?
Think about the 400* options in Drupal's administration interface today. Some should easily accessible as they are routinely accessed. Think about the hundreds of professional Drupal consultants who live in the Drupal administration interface day in an day out. For example logs or spam seem to be the most frequently accessed. However, a task like clean URLs is only usually set once, but it has prominent real estate at administer >> settings. Infrequent tasks should be tucked away in favor of more common tasks, but made easy to find during site set-up. The first set of questions are about understanding the users. The second set of questions are about the situation the user is in when they are doing administration. In general I would say that the notion of administration configuration and administration settings need to give way to: Site set-up, Site development, site testing and maintenance. At least this is what the interviews reveal about how Drupal is actually being used.
Should it be something like "These are some Drupal tasks that are done less frequently.
infrequent, less frequently. Potato , Potato, Tomato, Tomato. Pick the one that is grammatically correct and let me know!
Which of these do you consider important?"
That question is asked below.
When you administer your site you find it easy to: (1 is not easy, 5 is very easy) ... When you administer your site you find it hard to? (1 is not hard, 5 is very hard)
The duality here is a bit more meaningful, but still confusing. Why not combine these into one question?
Response rate for long questions falls off significantly. I want to get the response rates consistent and high to avoid skewed results.
Regardless, having the first one be "1 is not easy" and the second one be "1 is not hard" will cause people to mess up. If you want to keep these as two separate questions, then perhaps it would work better if you asked "Do you agree or disagree that the following are easy tasks?", and "Do you agree or disagree that the following are difficult tasks?", with 1 being strongly agree and 5 being strongly disagree. I'm not positive this would be better, but it feels better to me.
Ok, I'll consult with Charlie Lowe, who taught audience research at Purdue, and get his input on your suggestion.
Another alternative would be to just move the questions away from each other, so that the respondent doesn't trip over the flipping of the scale.
Ok, one of the reasons I use SurveyMonkey is that it jumbles the order of questions to avoid leading bias. Insert your favorite joke here!
What other important tasks did not fit into the categories above for you? Which categories? There's frequent/infrequent and easy/hard. If something isn't checked, does that mean it's not important, or that it's neither easy nor hard, or what?
I don't think we can infer anything from a non-selection.
My expectation when I read "What other important tasks" is that there will be a text area for me to insert arbitrary items.
Yes. The part that says Othere________________ will be a text field.
If these have already been identified as possible tasks, why aren't they just included above?
It's a challenge. As the interviews progressed the responses got more and more varied. By the time I was clustering the answers for the important tasks there were 20+ unique important tasks for 10 interviewees!
... Setting up a testing process including creating a test role, soliciting feedback from users, configuring corrections, and working with module development process to get bugs resolved Should this be split into a testing item and a bug fix item?
Yes, it's a casualty of trying to cluster responses. I am trying to keep the number of responses short so that we keep the survey response rates high, at the same time get enough granularity in the responses so that we can take action . It's a tough balance.
I hope this is helpful. It it's not, let me know and I'll shut up.
It's absolutely helpful. The more review, the clearer the survey questions, the more accurate the responses, the clearer the analysis, and the greater capacity the developer community has to address the needs of their users. Cheers, Kieran
Gary
Kieran Lal wrote:
On Aug 9, 2006, at 7:23 AM, Gary Feldman wrote: ...
New user Non-technical user developing community or social change web site forced to become "accidental technologist" The nature of the web site doesn't seem relevant. How about: " Non-technical user developing a web site, and becoming adept at the technology" Yes, it's very relevant. See interviews and response to Dries. It seems reasonable to me to have a separate question asking people to characterize the web sites on which they work. ... What are some infrequent Drupal tasks? (select all that apply) Ditto, but see next comment. Add new features by adding modules or coding new features ... Modify site navigation Other_______________ The fact that this list differs from the preceding list suggests that you already believe these to be infrequent. But does that mean that if the respondent doesn't check something, it means that it's a frequent task, or that it's not a task at all? Or more to the point, what is this question trying to learn? Think about the 400* options in Drupal's administration interface today. Some should easily accessible as they are routinely accessed. Think about the hundreds of professional Drupal consultants who live in the Drupal administration interface day in an day out. For example logs or spam seem to be the most frequently accessed. However, a task like clean URLs is only usually set once, but it has prominent real estate at administer >> settings. Infrequent tasks should be tucked away in favor of more common tasks, but made easy to find during site set-up.
The first set of questions are about understanding the users. The second set of questions are about the situation the user is in when they are doing administration. In general I would say that the notion of administration configuration and administration settings need to give way to: Site set-up, Site development, site testing and maintenance. At least this is what the interviews reveal about how Drupal is actually being used. I certainly agree that those four categories make sense and are better than the administration configuration and administration settings classification. This makes me wonder if there should be two tiers of questions. That is: A: While administering Drupal, what percentage of your time do you spend doing : a) setup; b)development; c) testing; d)maintenance; e) other? (I don't know whether SurveyMonkey can handle it, but my intent is that these add up to 100%.) B1: Which of these are the most common tasks during site setup? B2: Which of these are the most common tasks during site development? ... ...
Should it be something like "These are some Drupal tasks that are done less frequently. infrequent, less frequently. Potato , Potato, Tomato, Tomato. Pick the one that is grammatically correct and let me know! This wasn't grammatical nitpicking (not that I'm above that :-), but about the ambiguity in interpretation. There are three possibilities, frequently, infrequently, and never. Dries's suggestion of "Which of these are the LEAST common ..." solves the ambiguity, but not necessarily in a way that gets the answers that are needed. I think that asking "which of these infrequent tasks is most important" is one way of phrasing it. Another might be "how often do you do these", with answers such as "once a week or more", "2-3 times a month", "rarely", or "in bursts, several times per day or week, but then days or weeks will elapse before doing it again." I don't have strong feelings about which form of question to pick, as long as it's clear how things that fall off the scale at either end (either too frequently or never) are classified.
Regards, Gary
On Aug 10, 2006, at 11:09 AM, Gary Feldman wrote:
Kieran Lal wrote:
On Aug 9, 2006, at 7:23 AM, Gary Feldman wrote: ...
New user Non-technical user developing community or social change web site forced to become "accidental technologist" The nature of the web site doesn't seem relevant. How about: " Non-technical user developing a web site, and becoming adept at the technology" Yes, it's very relevant. See interviews and response to Dries. It seems reasonable to me to have a separate question asking people to characterize the web sites on which they work.
Ok. I do want to achieve a balance between getting enough people through the survey so that the Drupal community feels they have their interests represented in the survey and the developers and contributors to Drupal feel the responses are accurate enough to give some weight to where they put their effort. Do we go from 10 questions to 15? I already added three questions based on Neil's early request to learn more about Drupal users. Now we are adding even more. After the last survey I did at least one micro survey on categorization. I also just went and did a bunch of research in areas that seemed to show strong interest from the results and worked to get improvements into Drupal. For example we implemented multiple comment administration for example. I think we have three choices: 1) Review the relevant parts of the interviews I have posted and re- interpret the results of this question yourself. 2) Conduct additional interviews to gain further clarity and re- phrase the questions to see if the response are significantly different. 3) Go with what we have and if the results look suspicious then do more follow up interviews to dig deeper. I am still looking for folks to commit to actually working to get this done, although the public review is definitely helpful. Cheers, Kieran
...
What are some infrequent Drupal tasks? (select all that apply) Ditto, but see next comment. Add new features by adding modules or coding new features ... Modify site navigation Other_______________ The fact that this list differs from the preceding list suggests that you already believe these to be infrequent. But does that mean that if the respondent doesn't check something, it means that it's a frequent task, or that it's not a task at all? Or more to the point, what is this question trying to learn? Think about the 400* options in Drupal's administration interface today. Some should easily accessible as they are routinely accessed. Think about the hundreds of professional Drupal consultants who live in the Drupal administration interface day in an day out. For example logs or spam seem to be the most frequently accessed. However, a task like clean URLs is only usually set once, but it has prominent real estate at administer settings. Infrequent tasks should be tucked away in favor of more common tasks, but made easy to find during site set-up.
The first set of questions are about understanding the users. The second set of questions are about the situation the user is in when they are doing administration. In general I would say that the notion of administration configuration and administration settings need to give way to: Site set-up, Site development, site testing and maintenance. At least this is what the interviews reveal about how Drupal is actually being used. I certainly agree that those four categories make sense and are better than the administration configuration and administration settings classification. This makes me wonder if there should be two tiers of questions. That is: A: While administering Drupal, what percentage of your time do you spend doing : a) setup; b)development; c) testing; d) maintenance; e) other? (I don't know whether SurveyMonkey can handle it, but my intent is that these add up to 100%.) B1: Which of these are the most common tasks during site setup? B2: Which of these are the most common tasks during site development? ... ...
Should it be something like "These are some Drupal tasks that are done less frequently. infrequent, less frequently. Potato , Potato, Tomato, Tomato. Pick the one that is grammatically correct and let me know! This wasn't grammatical nitpicking (not that I'm above that :-), but about the ambiguity in interpretation. There are three possibilities, frequently, infrequently, and never. Dries's suggestion of "Which of these are the LEAST common ..." solves the ambiguity, but not necessarily in a way that gets the answers that are needed. I think that asking "which of these infrequent tasks is most important" is one way of phrasing it. Another might be "how often do you do these", with answers such as "once a week or more", "2-3 times a month", "rarely", or "in bursts, several times per day or week, but then days or weeks will elapse before doing it again." I don't have strong feelings about which form of question to pick, as long as it's clear how things that fall off the scale at either end (either too frequently or never) are classified.
Regards,
Gary
Kieran Lal schrieb:
[...] I think we have three choices: 1) Review the relevant parts of the interviews I have posted and re- interpret the results of this question yourself. 2) Conduct additional interviews to gain further clarity and re- phrase the questions to see if the response are significantly different. 3) Go with what we have and if the results look suspicious then do more follow up interviews to dig deeper. I would suggest to take the second choice. The more specific the question the merrier the results. Problem: to specific question are possibly to difficult for unexperienced users. So perhaps the questions should be rephrased and after the survey you could go ahead with choice 3.
Thanks for doing this work, Kieran! It's important. I do have some questions though: On 09 Aug 2006, at 04:54, Kieran Lal wrote:
Non-technical user developing community or social change web site forced to become "accidental technologist"
Does it matter what the user is developing? Maybe the following is sufficient: Non-technical user forced to become "accidental technologist"
Inexperienced with Drupal but curious. Learning fast based on other web site building experience
Does it matter whether I'm learning fast? What if I'm an inexperienced user learning slowly?
Learnable website that can be taught to users and allows website developers to leverage existing learning when building new sites
I don't understand what this means.
Module configuration, extensibility with new modules, and clean code make it easier to configure the website as you need it
Clean code does not affect Drupal's configurability. Maybe this needs to be rephrased?
Why do you use Drupal?
What is the difference with the previous question (eg. How does Drupal help you accomplish your goals as a web site administrator?)? Looks like both questions have sometimes similar answers.
How does Drupal help your users? (select all that apply) Gives them the features they want quickly Allows users to create web based content such as forum posts, or blogs Drupal community innovates and provides community building capacity with Drupal
I don't understand how this answers above question. Whether the Drupal community innovates shouldn't be of concern to users of, say, twit.tv.
Cost effective and winning over non-technical decision makers
Cost effective for the user, or for the administrator? Why do user have to think of 'costs'? Looks like some answers target users, while other answers target administrators.
What are some common Drupal administration tasks? (select all that apply) Monitor site through reviewing logs, looking at user activity Manage spam through comments, track backs, forum, and user registration Configure modules Update modules, install modules, test patches, track fixes for modules Work on themes and add theme template to customize module output Learn about Drupal capabilities and features, understand terminology, and plan improvements Manage users accounts, change permissions Respond to user feedback during testing and make changes Create web pages through the creating content types Other_______________
Some of the answers use technical language which not everyone might be familiar with (eg. trackback, patches, theme template).
What are some infrequent Drupal tasks?
I would rephrase this question so it is similar to the question above, to highlight the difference: What are the MOST common Drupal administration tasks? What are the LEAST common Drupal administration tasks? Would it make sense to merge the answers of both questions? Two questions, with the same answers.
1-5 Upgrade modules manually, manage module conflicts in output, download modules that don't include dependencies
What do you mean with "conflicts in output"? What do you mean with "modules that don't include dependencies"?
1-5 Manage content specifically creating new content types, viewing differences in node versions, importing content, or seeing multiple previews of content on a single page
What do you mean with "multiple previews"?
1-5 Configure user permissions with existing granularity
What do mean with "existing" here? Without adding a new module?
What other important tasks did not fit into the categories above for you?
Why do we make this a separate question? Why not add it to the answers to the previous question?
(select all that apply) Analyzing logs to understand the state of your site
Wasn't this part of the previous answers? If so, then it would have fitted in the categories, not?
Integrating modules and resolving clashes in the output
What do you mean with "clashes in the output"?
Work on the theme and templates to create structure and distinct looks to sections of the site
Wasn't this part of the previous answers? If so, then it would have fitted in the categories, not? -- Dries Buytaert :: http://www.buytaert.net/
On Aug 9, 2006, at 10:00 AM, Dries Buytaert wrote:
Thanks for doing this work, Kieran! It's important. I do have some questions though:
On 09 Aug 2006, at 04:54, Kieran Lal wrote:
Non-technical user developing community or social change web site forced to become "accidental technologist"
Does it matter what the user is developing? Maybe the following is sufficient:
Non-technical user forced to become "accidental technologist"
Yes it matters a lot. If you want to build an online community or social change website you are going to end up being recommended to use Drupal. Drupal's community knowledge extends beyond just CMS development but is now a expert community in online community building and social change websites. This is important to know because this growing user base has certain expectations and goals. This was clear from the interviews, maybe 4+/10 people fell into this category. We need to get a sense of how many Drupal sites are falling into this domain.
Inexperienced with Drupal but curious. Learning fast based on other web site building experience
Does it matter whether I'm learning fast?
Yes. Many people who are using Drupal are leaving alternatives and learning the Drupal way. They are website developer converts and we need to recognize the speed at which they understand Drupal.
What if I'm an inexperienced user learning slowly?
Then you would be a new user! I'll try to clariify.
Learnable website that can be taught to users and allows website developers to leverage existing learning when building new sites
I don't understand what this means.
Ok, I'll re-write it. Clustering answers is hard, and it's good to know what's not working. The feedback from professional site builders was that there customers can learn how to use Drupal quickly. Also, professionals will just keep building in Drupal no matter what rather than switch from WordPress, to Joomla, to home grown CMS. Drupal's learning curve has a high rate of return for Drupal consultants!
Module configuration, extensibility with new modules, and clean code make it easier to configure the website as you need it
Clean code does not affect Drupal's configurability. Maybe this needs to be rephrased?
Yes, clean code greatly impact configurability! To administrators the ability to patch clean core code is an important option, when they can't accomplish this in the theme layer or through the administration interface. You might see this differently, but to customers and administrators it's configuration, even if it's code. Note: they couldn't/wouldn't do this do other CMS code bases because the code is unclean!
Why do you use Drupal?
What is the difference with the previous question (eg. How does Drupal help you accomplish your goals as a web site administrator?)? Looks like both questions have sometimes similar answers.
True the answer do overlap significantly. But the second question speaks more to personal choice (I like the community) rather than effectiveness in accomplishing goals. I'll revisit the summarize answers to see if it's worthy to maintain this distinction.
How does Drupal help your users? (select all that apply) Gives them the features they want quickly Allows users to create web based content such as forum posts, or blogs Drupal community innovates and provides community building capacity with Drupal
I don't understand how this answers above question. Whether the Drupal community innovates shouldn't be of concern to users of, say, twit.tv.
Many communities are limited by the commercial constraints of their community tools. For example, yahoo groups feature set hasn't changed much in 5 years. Users know that with Drupal they get more features faster to better meet their needs. For commercial communities like Twit this doesn't make much of a difference. But for more grassroots communities we are seeing a tipping point. "Go with Drupal they have the best and newest community stuff! "
Cost effective and winning over non-technical decision makers
Cost effective for the user, or for the administrator?
Free tends towards cost effective for both :-)
Why do user have to think of 'costs'?
If you are currently paying $20/month for commercial service like meet-up.com then your users and administrators know they have to raise money to pay for the service every month.
Looks like some answers target users, while other answers target administrators.
Agreed, this is confusing. I'll clarify.
What are some common Drupal administration tasks? (select all that apply) Monitor site through reviewing logs, looking at user activity Manage spam through comments, track backs, forum, and user registration Configure modules Update modules, install modules, test patches, track fixes for modules Work on themes and add theme template to customize module output Learn about Drupal capabilities and features, understand terminology, and plan improvements Manage users accounts, change permissions Respond to user feedback during testing and make changes Create web pages through the creating content types Other_______________
Some of the answers use technical language which not everyone might be familiar with (eg. trackback, patches, theme template).
Ok, I try to preserve the language that the administrators use in the interviews. I'll see what I can do.
What are some infrequent Drupal tasks?
I would rephrase this question so it is similar to the question above, to highlight the difference:
What are the MOST common Drupal administration tasks? What are the LEAST common Drupal administration tasks?
Ok.
Would it make sense to merge the answers of both questions? Two questions, with the same answers.
I merged the questions for Easiest and Hardest in the last survey and used a likert scale. The response rate fell off significantly. I am recommending we go with two shorter questions and hope the response rates stay high.
1-5 Upgrade modules manually, manage module conflicts in output, download modules that don't include dependencies
What do you mean with "conflicts in output"?
For example(not a real example): categories module and freetagging module might overlap in the tags/terms they output. The result is the administrator must change, usually the output of the contributed module. Core tends to be clean, contributed modules tend to conflict.
What do you mean with "modules that don't include dependencies"?
Example: If you want sign-up module: you download a tarball. But the tarball doesn't contain the dependent event module. If you build sites professionally, the need to get each dependency is expensive.
1-5 Manage content specifically creating new content types, viewing differences in node versions, importing content, or seeing multiple previews of content on a single page
What do you mean with "multiple previews"?
Steven Peck just corrected this on Drupal.org. You are forced to preview. That preview has two actual previews. The first preview is the first part of the node. The second preview is the whole node. If you make it through that, then you get to the submit button.
1-5 Configure user permissions with existing granularity
What do mean with "existing" here? Without adding a new module?
this is wrong. It should read increasing granularity. People like Drupal's permissions, but want more.
What other important tasks did not fit into the categories above for you?
Why do we make this a separate question? Why not add it to the answers to the previous question?
Most common, Least common, Easy, Hard. That's a good cross section. But important gives the respondent an opportunity to talk about issues we didn't address. For example in the ten interviews no one talked about internationalization or about cross browser support. Just because the responses were listed above doesn't mean that they user would select to put any of those responses in those 4 categories. This is a challenge in this type of survey that what hard for some is easy for others. I'll take advice.
(select all that apply) Analyzing logs to understand the state of your site
Wasn't this part of the previous answers? If so, then it would have fitted in the categories, not?
Maybe. For some yes, for other no.
Integrating modules and resolving clashes in the output
What do you mean with "clashes in the output"?
See the categories example above.
Work on the theme and templates to create structure and distinct looks to sections of the site
Wasn't this part of the previous answers? If so, then it would have fitted in the categories, not?
It didn't come out in the interviews this way. I'll make adjustments to the available responses and do a couple more interviews to see if the duplicate responses are causing confusion. It's worth noting that I anticipate it will take 20-40 hours to analyze the results and get something meaningful to be presented to the Drupal community. I am looking for volunteers to help in this Analysis. Thanks for the responses! Cheers, Kieran
-- Dries Buytaert :: http://www.buytaert.net/
On 9-Aug-06, at 11:48 AM, Kieran Lal wrote:
On Aug 9, 2006, at 10:00 AM, Dries Buytaert wrote:
Thanks for doing this work, Kieran! It's important. I do have some questions though:
On 09 Aug 2006, at 04:54, Kieran Lal wrote:
Non-technical user developing community or social change web site forced to become "accidental technologist"
Does it matter what the user is developing? Maybe the following is sufficient:
Non-technical user forced to become "accidental technologist"
Yes it matters a lot. If you want to build an online community or social change website you are going to end up being recommended to use Drupal. Drupal's community knowledge extends beyond just CMS development but is now a expert community in online community building and social change websites. This is important to know because this growing user base has certain expectations and goals. This was clear from the interviews, maybe 4+/10 people fell into this category. We need to get a sense of how many Drupal sites are falling into this domain.
So...then add that as another category. You may be an accidental technologist building things for a death metal band. I suspect a lot of the folks you interact with fall into the social change category...and I agree it is important to capture. However, another huge category is "HTML Website developer adopting a CMS platform" or "Process consultant learning to us collaborative tools" (the former is more technical in terms of web stuff, the latter is less technical).
I don't understand how this answers above question. Whether the Drupal community innovates shouldn't be of concern to users of, say, twit.tv.
Many communities are limited by the commercial constraints of their community tools. For example, yahoo groups feature set hasn't changed much in 5 years. Users know that with Drupal they get more features faster to better meet their needs. For commercial communities like Twit this doesn't make much of a difference. But for more grassroots communities we are seeing a tipping point. "Go with Drupal they have the best and newest community stuff! "
And not just community stuff. Consultants (especially non-developer consultants) can pick Drupal and become experts at the modules available...the availability of configurable, themeable, well-written modules that perform a wide variety of things means they can deliver more functional websites at lower cost. Picking Drupal means always having well-written cutting edge options available. Client: "Hey, I heard new standard X came out...can we do that?" Consultant: "Why yes, the pants module implemented standard X just last week!" -- Boris
Yes it matters a lot. If you want to build an online community or social change website you are going to end up being recommended to use Drupal. Drupal's community knowledge extends beyond just CMS development but is now a expert community in online community building and social change websites. This is important to know because this growing user base has certain expectations and goals. This was clear from the interviews, maybe 4+/10 people fell into this category. We need to get a sense of how many Drupal sites are falling into this domain.
So...then add that as another category. You may be an accidental technologist building things for a death metal band. I suspect a lot of the folks you interact with fall into the social change category...and I agree it is important to capture.
Most of the folks were random people plucked out of #drupal-support or acquaintances of people in #drupal. I explicitly tried to not sample the CivicSpace folks and these results still came through.
However, another huge category is "HTML Website developer adopting a CMS platform" or "Process consultant learning to us collaborative tools" (the former is more technical in terms of web stuff, the latter is less technical).
Ok. Let me think about how to introduce that into the self description responses. I don't want to force stuff into the survey, that didn't come out in the interviews. But then this is more art than science and I do want public review to make sure the results make sense. Thanks for the feedback. Kieran
On Aug 9, 2006, at 12:12 PM, Boris Mann wrote:
On 9-Aug-06, at 11:48 AM, Kieran Lal wrote:
On Aug 9, 2006, at 10:00 AM, Dries Buytaert wrote:
Thanks for doing this work, Kieran! It's important. I do have some questions though:
On 09 Aug 2006, at 04:54, Kieran Lal wrote:
Non-technical user developing community or social change web site forced to become "accidental technologist"
Does it matter what the user is developing? Maybe the following is sufficient:
Non-technical user forced to become "accidental technologist"
Yes it matters a lot. If you want to build an online community or social change website you are going to end up being recommended to use Drupal. Drupal's community knowledge extends beyond just CMS development but is now a expert community in online community building and social change websites. This is important to know because this growing user base has certain expectations and goals. This was clear from the interviews, maybe 4+/10 people fell into this category. We need to get a sense of how many Drupal sites are falling into this domain.
So...then add that as another category. You may be an accidental technologist building things for a death metal band. I suspect a lot of the folks you interact with fall into the social change category...and I agree it is important to capture.
However, another huge category is "HTML Website developer adopting a CMS platform" or "Process consultant learning to us collaborative tools" (the former is more technical in terms of web stuff, the latter is less technical).
I'm going to put in my 2c since it was my answers that partially started this discussion. There is a pretty clear distinction between "online community" and "social change website". An online community has a feature set that is only partially coincidental with a social change or campaign website. An online community may have no social objective other than recreation. I'm thinking of sites like Flikr and MySpace. Social change websites are campaign-oriented, but often need to leverage a community. Therefore, quite often when building a social change/ campaign website one must first be able to capture, communicate with and empower a community of support, and therefore the same set of solutions that underpin community websites are a necessary condition for some successful social change/campaign websites. I would therefore say that Drupal in its essence (its core) is inherently suitable for online communities. It is vigorous activity in the contributions/distributions community (which has inflected back toward the core via the excellent CivicSpace team) that has driven Drupal's uptake in the realm of social change and campaigns. There are other burgeoning organic contrib communities there as well, either overtly or in nascent form, around music, publishing and other areas. IMHO it is critical for the health of Drupal that it continue to support the core functionality that its organic contrib/distrib communities depend on. Branding is another matter. "Online Community" is a safe brand for Drupal, and it is reflected in the "community plumbing" slogan. It is neutral and unencumbered (with the exception of some of the bad press MySpace has gotten) "Social Change/campaign websites" is a brand that should be driven by the contrib/distro community, given that it considerably narrows the scope of opportunity for Drupal, and inherits all the baggage of the highly polarized political climate (at least in North America). "Online community" and "social change/capaign website" should be distinct areas of concern/inquiry, since one is an order of magnitude removed from the other. evan (who realizes that, as new guy, he's probably just gone around some old blocks)
I don't understand how this answers above question. Whether the Drupal community innovates shouldn't be of concern to users of, say, twit.tv.
Many communities are limited by the commercial constraints of their community tools. For example, yahoo groups feature set hasn't changed much in 5 years. Users know that with Drupal they get more features faster to better meet their needs. For commercial communities like Twit this doesn't make much of a difference. But for more grassroots communities we are seeing a tipping point. "Go with Drupal they have the best and newest community stuff! "
And not just community stuff. Consultants (especially non-developer consultants) can pick Drupal and become experts at the modules available...the availability of configurable, themeable, well- written modules that perform a wide variety of things means they can deliver more functional websites at lower cost. Picking Drupal means always having well-written cutting edge options available. Client: "Hey, I heard new standard X came out...can we do that?" Consultant: "Why yes, the pants module implemented standard X just last week!"
1) great work putting this together, kieran! 2) i think gary and dries's concerns about the phrasing of the original draft of the survey are right-on, overall. 3) kieran, sounds like you agree with many of them -- how about an updated draft of the survey for futher reviews/edits so we don't duplicate suggestions that have already been incorporated? 4) a minor nit-pick in one of your replies... On Aug 9, 2006, at 11:48 AM, Kieran Lal wrote:
What do you mean with "modules that don't include dependencies"?
Example: If you want sign-up module: you download a tarball. But the tarball doesn't contain the dependent event module. If you build sites professionally, the need to get each dependency is expensive.
as the current maintainer of signup.module, i must protest. ;) signup does *not* require or depend on event.module. you can "signup- enable" any node you want. certain features of signup conditionally happen if the thing you're signing up for *is* an event, but that's optional. ;) cheers, -derek (dww)
participants (7)
-
Boris Mann -
Derek Wright -
Dries Buytaert -
Evan Leeson -
Gary Feldman -
Kieran Lal -
Stefan Borchert