Drupal Administration survey II: second draft
I would like to thank Mark Burdett for reviewing and editing the survey. Please review and edit the survey questions. The survey results will be used in helping to direct user experience improvements for the Drupal software, Drupal.org, and community processes. The survey will be posted in the top left block on Drupal.org and run for 45 days. This draft, http://groups.drupal.org/node/1326, has also been posted in the Usability Group: http://groups.drupal.org/usability If you complete all the questions in this survey and would like to be recognized as a contributor to improving Drupal user experience please indicate your Drupal.org username Drupal.org username_______________ How would you describe yourself as a Drupal administrator? (pick the best one) New user Non-technical user forced to become "accidental technologist" Inexperienced with Drupal but learning fast, based on other website-building experience Experienced administrator, comfortable with configuring site rapidly Module developer who understands what's going on behind the scenes Other_______________ Which types of Drupal websites do you administer (select all that apply) Personal Blog Community site Social Change site Brochure site News site E-Commerce site ...? Other site How many Drupal sites do you administer? (select one) 1 2-5 6-11 12-24 25 or more How frequently do you administer you your Drupal site? (pick the best one) Monthly Once every couple of weeks Several times a week Daily for 30-60 minutes of website management Daily for several hours in site development and deployment Other_______________ How long do you administer your Drupal site in a single sitting approximately? (pick the best one) Thirty minutes to an hour Few hours Half a day All day long Other_______________ How does Drupal help you accomplish your goals as a web site administrator? (select no more than 3) Rapid deployment of features and ability to meet customer requirements Web based content publishing is easy and allows for end user contributions Easy-to-learn website that can be taught to users Allows website developers to leverage previous experience when building new sites Allows for monitoring and logging of the website Module configuration, extensibility with new modules, and clean code make it easy to customize the website as you need it Dynamic and exciting developer community allows for rapid and fun learning Other_______________ Why do you use Drupal? (select no more than 3) Features, extensibility, integration of modules, clean code allow you to customize your site Speed of deployment and re-use of existing functionality Quality community is helpful, has smart people, is very active, and is working on community solutions not just technical capabilities Allows users to create content, categorize content, and add navigation to the site Other_______________ How does Drupal help your users? (select no more than 3) Gives them the features they want quickly Allows users to create web based content such as forum posts, or blogs Drupal community innovates and provides new and improved community-building tools Cost effective, easy to convince non-technical decision makers Other_______________ What are your *most common* Drupal administration tasks? (select no more than 3) Monitor site through reviewing logs, looking at user activity Manage spam through comments, trackbacks, forum, and user registration Configure modules Update modules, install modules, test patches, track fixes for modules Work on themes and edit theme templates to customize module output Learn about Drupal capabilities and features, understand terminology, and plan improvements Manage users accounts, change permissions Respond to user feedback during testing and make changes Create web pages through the creating content types Other_______________ What are your *least common* Drupal tasks? (select no more than 3) Add new features by adding modules or coding new features Manage spam, delete content and comment, and banning users Modify existing content such publishing to the front page, making content sticky, or changing titles Layout content in positions with blocks, views, or designing a content section Change site theme or theme a content section Modify site navigation Other_______________ When you administer your site you find it *easy* to: (1 is *not easy*, 5 is *very easy*) 1-5 Add new features and install modules 1-5 Automate tasks using cron 1-5 Manage user information by editing user account permissions 1-5 Change themes or make changes to layout with theme templates 1-5 Post, edit, and version web pages and other content types 1-5 General settings such as changing titles, or date format When you administer your site you find it *hard* to: (1 is *not hard*, 5 is *very hard*) 1-5 Discover where to configure site settings, or where all the different places where a site needs to be configured are 1-5 Understanding Drupal specific concepts or understand technical language without pictures 1-5 Upgrade modules manually, theme/customize module output, determine module dependencies 1-5 Manage content specifically creating new content types, viewing differences in node versions, importing content, or seeing multiple previews of content on a single page 1-5 Manage large groups or users, nodes, menu items, comments, blocks 1-5 Get overview of new content and activities on site 1-5 Configure user permissions with increasing granularity 1-5 Manage menus such as changing menu block weights for all menus on a page, or ordering menu items within a menu What are some *important* administration tasks that did not fit into the categories above for you? (select no more than 3) Analyzing logs to understand the state of your site Integrating modules, resolving incompatibilities, theming the output Setting up a testing process including creating a test role, soliciting feedback from users, configuring corrections, and working with module development process to get bugs resolved Work on the theme and templates to create structure and distinct looks to sections of the site Hide output from modules such as node links or browsing links Managing menus with the menu editing interface, bulk menu editing, and menu options Manage performance of the site Identify popular content Aggregate content from other sources into a site _______________ Other
Kieran Lal wrote:
How would you describe yourself as a Drupal administrator? (pick the best one)
It would be nice if each self identified administrative group filled out its own survey (same questions - just seperate surveys) - so that at the end apples could be compared with apples. It would be helpful for tracking things like: Of those who identified themselves as X - they find Y to be Z. But - those who identify themselves as W find Y to be much less Z. I don't know if that is feasible with the survey application you plan to use.
Which types of Drupal websites do you administer (select all that apply) Personal Blog Community site Social Change site Brochure site News site E-Commerce site ...? Other site
It may be helpful to define some of these: e.g. Personal Blog (example.com) - you are the primary author and administrator of all content on your site - users may register to post comments, but do not gain other privileges by signing up. Community Site (example.drupal.org) - you are one of many authors and administrators - etc. etc. etc.
What are your *most common* Drupal administration tasks? (select no more than 3) Monitor site through reviewing logs, looking at user activity Manage spam through comments, trackbacks, forum, and user registration Configure modules Update modules, install modules, test patches, track fixes for modules Work on themes and edit theme templates to customize module output Learn about Drupal capabilities and features, understand terminology, and plan improvements Manage users accounts, change permissions Respond to user feedback during testing and make changes Create web pages through the creating content types Other_______________
What are your *least common* Drupal tasks? (select no more than 3) Add new features by adding modules or coding new features Manage spam, delete content and comment, and banning users Modify existing content such publishing to the front page, making content sticky, or changing titles Layout content in positions with blocks, views, or designing a content section Change site theme or theme a content section Modify site navigation Other_______________
The list of 'most common' and 'least common' should be the same list. This question is leading and makes assumptions. Items in the least common list may turn out to be common or items in the common list may turn out to be uncommon.
When you administer your site you find it *easy* to: (1 is *not easy*, 5 is *very easy*) 1-5 Add new features and install modules 1-5 Automate tasks using cron 1-5 Manage user information by editing user account permissions 1-5 Change themes or make changes to layout with theme templates 1-5 Post, edit, and version web pages and other content types 1-5 General settings such as changing titles, or date format
When you administer your site you find it *hard* to: (1 is *not hard*, 5 is *very hard*) 1-5 Discover where to configure site settings, or where all the different places where a site needs to be configured are 1-5 Understanding Drupal specific concepts or understand technical language without pictures 1-5 Upgrade modules manually, theme/customize module output, determine module dependencies 1-5 Manage content specifically creating new content types, viewing differences in node versions, importing content, or seeing multiple previews of content on a single page 1-5 Manage large groups or users, nodes, menu items, comments, blocks 1-5 Get overview of new content and activities on site 1-5 Configure user permissions with increasing granularity 1-5 Manage menus such as changing menu block weights for all menus on a page, or ordering menu items within a menu
Once again - consider combining these two list - reword the question: When administering your site rate the ease with which you accomplish the following: Consider a 4 or 6 level scale to help avoid wishy washy responses (challenge people to decide if they really think its easy or hard - no middle answer): 6)very easy, 5)easy, 4)somewhat easy, 3)somewhat difficult, 2)difficult, 1)very difficult. Break apart questions the cover broad areas. e.g. > 1-5 Manage large groups or users, nodes, menu items, comments, blocks Consider defining 'large'. manage large numbers of user roles (10+) manage large numbers of users (100+) manage large numbers of nodes (1000+) manage large menus (75+ items) manage large number of comments (average of 3+ per node)
What are some *important* administration tasks that did not fit into the categories above for you? (select no more than 3) Analyzing logs to understand the state of your site Integrating modules, resolving incompatibilities, theming the output Setting up a testing process including creating a test role, soliciting feedback from users, configuring corrections, and working with module development process to get bugs resolved Work on the theme and templates to create structure and distinct looks to sections of the site Hide output from modules such as node links or browsing links Managing menus with the menu editing interface, bulk menu editing, and menu options Manage performance of the site Identify popular content Aggregate content from other sources into a site _______________ Other
I'm not sure what this question hopes to measure. andre
On Aug 31, 2006, at 9:37 AM, Andre Molnar wrote:
Kieran Lal wrote:
How would you describe yourself as a Drupal administrator? (pick the best one)
It would be nice if each self identified administrative group filled out its own survey (same questions - just seperate surveys) - so that at the end apples could be compared with apples.
SurveyMonkey allows this level of filtering. We are adding 4 additional questions to the survey to learn more about administrators.
It may be helpful to define some of these: e.g. Personal Blog (example.com) - you are the primary author and administrator of all content on your site - users may register to post comments, but do not gain other privileges by signing up. Community Site (example.drupal.org) - you are one of many authors and administrators - etc. etc. etc.
Ok, but we are adding a lot of questions about the users and I want to keep it short and focuses to increase the participation rates on the whole survey and each question high.
The list of 'most common' and 'least common' should be the same list. This question is leading and makes assumptions. Items in the least common list may turn out to be common or items in the common list may turn out to be uncommon.
While that makes logical sense that least common and most common lists are mirrored lists, we got these responses from user interviews. So I am sure there is someone with a dissertation in statistics or audience research under there belt who can weigh in on whether it's better to use actual responses versus logically consistent responses.
When administering your site rate the ease with which you accomplish the following:
Consider a 4 or 6 level scale to help avoid wishy washy responses (challenge people to decide if they really think its easy or hard - no middle answer): 6)very easy, 5)easy, 4)somewhat easy, 3)somewhat difficult, 2) difficult, 1)very difficult.
Ok. I'll check with a couple people about the downside of neutral responses.
Break apart questions the cover broad areas. e.g. > 1-5 Manage large groups or users, nodes, menu items, comments, blocks Consider defining 'large'. manage large numbers of user roles (10+) manage large numbers of users (100+) manage large numbers of nodes (1000+) manage large menus (75+ items) manage large number of comments (average of 3+ per node)
I like the numbers, they help give perspective.
I'm not sure what this question hopes to measure.
It gives the survey taker an opportunity to let us know what they consider important. For example, in the survey results last time we learned that internationalization, and mail management were important but didn't really get much consideration in the previous categories. In the live interviews, I find that the most interesting information comes when you ask people what is important to them. You can review the last question of the survey responses to see for yourself. http://groups.drupal.org/usability Thanks for you responses. Kieran
andre
Kieran Lal wrote:
The list of 'most common' and 'least common' should be the same list. This question is leading and makes assumptions. Items in the least common list may turn out to be common or items in the common list may turn out to be uncommon.
While that makes logical sense that least common and most common lists are mirrored lists, we got these responses from user interviews. So I am sure there is someone with a dissertation in statistics or audience research under there belt who can weigh in on whether it's better to use actual responses versus logically consistent responses.
This is a good opportunity to see if your interview results are consistent with results from a broader audience. The only way to do that is to allow users to choose on their own what is common/uncommon - not choose from a list of identified common/uncommon tasks. I have a particular interest since as you know with our usability report we had a very different list of common tasks performed by our stakeholders. So I am curious whether our stakeholder's common tasks are specific to them or a broader range of users.
When administering your site rate the ease with which you accomplish the following:
Consider a 4 or 6 level scale to help avoid wishy washy responses (challenge people to decide if they really think its easy or hard - no middle answer): 6)very easy, 5)easy, 4)somewhat easy, 3)somewhat difficult, 2)difficult, 1)very difficult.
Ok. I'll check with a couple people about the downside of neutral responses.
It all depends on what you are trying to measure. I think we are looking for items that either make the grade or don't and therefore need improvement. (Does a question that has a majority of people choosing the middle ground mean that resources should be directed at the feature or not?) Regardless if you choose this scale or not - i forgot to include a 'don't know' - for people that have never used a feature.
It gives the survey taker an opportunity to let us know what they consider important. For example, in the survey results last time we learned that internationalization, and mail management were important but didn't really get much consideration in the previous categories. In the live interviews, I find that the most interesting information comes when you ask people what is important to them. You can review the last question of the survey responses to see for yourself. http://groups.drupal.org/usability
Gotcha. Its for future research. In that case i would expect to see some questions about i18n and mail management in this survey. andre
On Thursday 31 August 2006 10:36 pm, Kieran Lal wrote:
(select no more than 3)
Why? You would get no less reliable result by allowing a form of Plurality Voting (choose 1 or more -without limit- from the list). I say: remove every such mention. A. -- http://www.wechange.org/ Because we and the world need to change. http://www.reuniting.info/ Intimate Relationships, peace and harmony in the couple.
I'm not a survey expert at all.. It seemed to me like we would get more useful data by asking people to choose the best answers, not all which could apply (with regards to why they use drupal, etc.). The original version did have "select all". --mark On 8/31/06, Augustin (Beginner) <drupal.beginner@wechange.org> wrote:
On Thursday 31 August 2006 10:36 pm, Kieran Lal wrote:
(select no more than 3)
Why? You would get no less reliable result by allowing a form of Plurality Voting (choose 1 or more -without limit- from the list).
I say: remove every such mention.
A.
-- http://www.wechange.org/ Because we and the world need to change.
http://www.reuniting.info/ Intimate Relationships, peace and harmony in the couple.
On Friday 01 September 2006 02:10 am, mark burdett wrote:
I'm not a survey expert at all.. It seemed to me like we would get more useful data by asking people to choose the best answers, not all which could apply (with regards to why they use drupal, etc.). The original version did have "select all".
"select all that apply" is good. I really doubt you'd get better result by giving an arbitrary limit. The overall top three would probably be the same items, but you'd get more detailed information towards the middle/bottom of the list, too. In reality, I doubt many people would chose more than 3 options within a short list, but it would be fairer to those who might have more than 3 answers not to limit the number of replies. Selecting any number from a list (0,1, or more) is called Approval Voting and there is a lot of technical papers showing that it is a very good way to find the most important item in a list (or the favorite candidate, etc.). I think the mention "select no more than 3" can be safely removed everywhere. Augustin. -- http://www.wechange.org/ Because we and the world need to change. http://www.reuniting.info/ Intimate Relationships, peace and harmony in the couple.
Augustin (Beginner) wrote:
On Friday 01 September 2006 02:10 am, mark burdett wrote:
I'm not a survey expert at all.. It seemed to me like we would get more useful data by asking people to choose the best answers, not all which could apply (with regards to why they use drupal, etc.). The original version did have "select all".
... I think the mention "select no more than 3" can be safely removed everywhere.
I talked with my friend who helped edit _Asking Questions: The Definitive Guide to Questionnaire Design_ [1]. He said: "If you want to know the most important thing, say 'all that apply.'" "If you want a good sample of the sorts of things people care about, go 'pick 3'" I'm going to go ahead with 'all that apply.' 1. http://www.wiley.com/WileyCDA/WileyTitle/productCd-0787970883,descCd-authorI... -- Neil Drumm http://delocalizedham.com/
participants (5)
-
Andre Molnar -
Augustin (Beginner) -
Kieran Lal -
mark burdett -
Neil Drumm