In June 2012, a major survey was conducted seeking users' opinions of the Wikipedia help system.
Survey method
editRegistered users asked to participate in the survey were selected randomly from database queries. They were divided into cohorts according to total edit count (server count) as of 11 June 2012. The groupings used were: 0 edits, 1–10 edits, 11–100 edits, 101–1000 edits, and over 1000 edits.
Apart from the 0 edits group, all participants have made at least one edit on Wikipedia since 12 May 2012, to filter out long-term inactive users. Blocked users and bots were excluded from the sample (to the best of our ability, a few may have slipped through). The 0 edit and 1–10 edit groups had the additional condition that their account was created since 3 June 2012.
250 users from each group were invited to participate through messages on their talk page.
Talk page invitation
|
|---|
|
Hi there, my name's Peter Coombe and I'm a Wikimedia Community Fellow working on a project to improve Wikipedia's help system. At the moment I'm trying to learn more about how people use and find the current help pages. If you could help by filling out [SURVEYURL this brief survey] about your experiences, I'd be very grateful. It should take less than 10 minutes, and your responses will not be tied to your username in any way. Thank you for your time, |
Unfortunately response rates among the 0 edits and 1–10 edit groups to the invitations on their talk pages were too low to be useful. Therefore 450 further users in each of these two groups were selected, with the same conditions as before and the extra requirement that they had enabled receiving Wikipedia e-mail from other users. These users were then sent invitations to participate in the survey via the Wikipedia e-mail system.
E-mail invitation
|
|---|
|
Subject: Wikipedia help pages survey Hi there, my name is Peter Coombe and I work for the Wikimedia Foundation, the non-profit organisation which operates Wikipedia. I'm currently working on a project to improve Wikipedia's help system. At the moment I'm trying to learn more about how people use and find the current help pages. If you could help by filling out a brief survey about your experiences, I'd be very grateful. It should take less than 10 minutes, and your responses will not be tied to your username or email address in any way. The survey is located here: SURVEYURL You can read more about the project here: https://en.wiki.x.io/wiki/Wikipedia:Help_Project/Community_fellowship Thank you for your time, Peter Coombe (Wikipedia User:The wub - https://en.wiki.x.io/wiki/User:The_wub) |
Results
editHelp topics
editUse
editHave you ever sought help on any of the following topics on Wikipedia?
| 0 | 1–10 | 11–100 | 101–1000 | 1000+ | |
|---|---|---|---|---|---|
| How to edit Wikipedia pages | 7 | 18 | 27 | 47 | 45 |
| How to add images to Wikipedia | 5 | 5 | 20 | 35 | 57 |
| How to create a new Wikipedia page | 11 | 7 | 15 | 23 | 30 |
| How to add references to a Wikipedia page | 6 | 11 | 27 | 45 | 58 |
| How to change the appearance of the Wikipedia interface | 4 | 3 | 7 | 10 | 34 |
| Other | 3 | 2 | 4 | 18 | 32 |
| Total responses | 20 | 26 | 42 | 70 | 93 |
"Other" responses
|
|---|
|
Satisfaction
editAnd how satisfied were you with the help you found on these topics?
- (Values given are mean of responses. 1 = Very Dissatisfied, 2 = Dissatisfied, 3 = Neutral, 4 = Satisfied, 5 = Very Satisfied)
| 0 | 1–10 | 11–100 | 101–1000 | 1000+ | |
|---|---|---|---|---|---|
| How to edit Wikipedia pages | 2.71 | 3.50 | 3.58 | 3.53 | 3.83 |
| How to add images to Wikipedia | 2.00 | 2.80 | 2.84 | 3.27 | 3.32 |
| How to create a new Wikipedia page | 3.00 | 2.71 | 2.87 | 3.20 | 3.63 |
| How to add references to a Wikipedia page | 2.33 | 3.18 | 3.16 | 3.71 | 3.50 |
| How to change the appearance of the Wikipedia interface | 2.75 | 4.33 | 2.71 | 3.13 | 3.44 |
| Other | 3.00 | 2.00 | 2.00 | 3.44 | 3.19 |
Reasons for dissatisfaction
editWhat in particular did you find dissatisfying about the help on these topics?
Extended content
|
|---|
|
Help methods
editAwareness
editWhich of the following methods of getting help on Wikipedia were you aware of?
| 0 | 1–10 | 11–100 | 101–1000 | 1000+ | |
|---|---|---|---|---|---|
| Wikipedia search | 15 | 27 | 34 | 51 | 80 |
| External search engine (such as Google) | 11 | 20 | 26 | 38 | 60 |
| Help link in left-hand sidebar | 6 | 10 | 20 | 30 | 49 |
| Tutorials (like Wikipedia:Introduction) | 4 | 10 | 20 | 31 | 50 |
| Wikipedia:Help desk | 2 | 6 | 8 | 24 | 59 |
| Wikipedia:Teahouse | 1 | 2 | 4 | 10 | 29 |
| IRC / "Live chat" | 1 | 3 | 5 | 15 | 31 |
| Asking questions on another editor's talk page | 4 | 5 | 11 | 30 | 73 |
| Using the {{Help me}} template on your own talk page | 2 | 5 | 10 | 10 | 38 |
| Other | 1 | 0 | 5 | 4 | 12 |
| Total responses | 25 | 35 | 43 | 68 | 95 |
"Other" responses
|
|---|
|
Use
editAnd which of these methods of getting help on Wikipedia have you used?
| 0 | 1–10 | 11–100 | 101–1000 | 1000+ | |
|---|---|---|---|---|---|
| Wikipedia search | 12 | 23 | 28 | 45 | 71 |
| External search engine (such as Google) | 10 | 16 | 20 | 29 | 38 |
| Help link in left-hand sidebar | 2 | 5 | 10 | 20 | 29 |
| Tutorials (like Wikipedia:Introduction) | 2 | 6 | 15 | 20 | 27 |
| Wikipedia:Help desk | 1 | 3 | 1 | 4 | 30 |
| Wikipedia:Teahouse | 0 | 1 | 0 | 3 | 4 |
| IRC / "Live chat" | 0 | 2 | 3 | 6 | 17 |
| Asking questions on another editor's talk page | 0 | 2 | 4 | 17 | 58 |
| Using the {{Help me}} template on your own talk page | 0 | 2 | 3 | 1 | 12 |
| Other | 1 | 0 | 4 | 4 | 11 |
Satisfaction
editAnd how satisfied were you with these methods for getting help?
- (Values given are mean of responses. 1 = Very Dissatisfied, 2 = Dissatisfied, 3 = Neutral, 4 = Satisfied, 5 = Very Satisfied)
| 0 | 1–10 | 11–100 | 101–1000 | 1000+ | |
|---|---|---|---|---|---|
| Wikipedia search | 3.64 | 3.87 | 3.75 | 3.70 | 3.47 |
| External search engine (such as Google) | 4.10 | 3.81 | 3.50 | 3.96 | 3.78 |
| Help link in left-hand sidebar | 3.50 | 4.00 | 3.70 | 3.55 | 3.29 |
| Tutorials (like Wikipedia:Introduction) | 3.00 | 3.33 | 3.67 | 3.90 | 3.46 |
| Wikipedia:Help desk | 4.00 | 2.33 | 4.00 | 4.00 | 3.62 |
| Wikipedia:Teahouse | - | 5.00 | - | 3.33 | 4.00 |
| IRC / "Live chat" | - | 5.00 | 5.00 | 3.50 | 3.94 |
| Asking questions on another editor's talk page | - | 5.00 | 4.00 | 4.41 | 4.20 |
| Using the {{Help me}} template on your own talk page | - | 2.00 | 3.33 | 4.00 | 3.27 |
| Other | 3.00 | - | 3.00 | 3.50 | 3.82 |
Reasons for dissatisfaction
editWhat in particular did you find dissatisfying about these methods?
Extended content
|
|---|
|
General comments
editFinally, do you have any other comments on Wikipedia's help pages?
Extended content
|
|---|
|
Conclusions
editHelp topics
editIt is clear there are marked differences between editors with different amounts of experience.
Strikingly among the 0 edit group, the most desired topic is "How to create a new page", whereas none of the other groups have this as their most popular topic. It is probable that in many cases, they have registered an account solely to make a new page, hence having no edits elsewhere. Their ratings of help in this area are on average neutral, although they rated this topic more highly than any of the others. This is likely because a lot of effort has been put into simple processes and documentation for creating a new page, as it is such a common desire for new users. Of course there is always room for improvement.
Among the users with some edits under their belt, "How to edit a Wikipedia page" and "How to add references to a Wikipedia page" are usually the most sought after topics (the sole exception being 1000+ editors, who are more interested in how to add images). Help on the admittedly broad topic of "How to edit a Wikipedia page" is generally rated more highly than the other topics. Again this is an area where a lot of effort has already been put in; despite the acknowledged complexity of wiki markup there exist some very basic guides such as Wikipedia:Cheatsheet linked from the edit page itself.
| “ | Images: too many, poorly structured help pages; information is scattered across too many pages, difficult to find | ” |
| — user with over 1000 edits | ||
Many users, even those with over 1000 edits, expressed their dissatisfaction with help on adding images. This is a difficult topic as successfully adding an image has three prerequisites: an understanding of copyright issues, understanding the upload process itself, and understanding how to edit the image into a page. All of these were flagged up as reasons for dissatisfaction:
- "I found the complex legal-like language in copyright and other related help-pages dissatisfying.", "Documentation on how to document copyright status is confusing."
- "It has not been explained properly how to add images, rather more focus is given on the prerequisites for that."
- "Not understanding placing pics onto pages."
It would certainly be beneficial to have more integrated information on adding images. This is somewhat complicated by the existence of Wikimedia Commons as a separate site, but this difficulty ought to be possible to overcome. The findings from this survey support the decision to focus on this as an area for improvement in the fellowship.
- Aside: One interesting suggestion from this section was "I wish Wikipedia had a review section for pictures BEFORE they are uploaded, just like we have for new articles. Sometimes I upload pictures thinking I have the proper copyright permissions, but I don't. I wish someone could review our pictures if we decided to put them for review before they actually uploaded." This is obviously outside the scope of the help fellowship, but is an interesting idea all the same (although the Articles for Creation queue regularly suffers huge backlogs, would an image equivalent face the same issues?)
A particular "other" topic frequently mentioned was finding templates, especially among users with higher edit counts. There already exists an index of templates at Wikipedia:Template messages, but this could perhaps use further attention.
Help methods
editThe results from this aspect of the survey also show significant differences between users by edit count.
Unsurprisingly given the prominence of the Wikipedia search form, using it to find help is the most popular method, and this is consistent among all user groups. The ratings for this seem to be somewhere in the middle compared to other methods. It is clear that the quality of results from searches will vary depending on the topic, and the search terms chosen. Since this method is so popular, and likely will remain so, it will be important to structure help pages well so that they can be found, provide useful titles and redirects where appropriate.
There appears to be no significant difference between the ratings of the Wikipedia search engine, and external search engines such as Google. The quality of results from the Wikipedia search engine has been a topic of complaints in the past, but this survey suggests that the situation has improved.
| “ | I was helped by people, not help pages. | ” |
Probably the most striking result is that asking questions on another editor's talk page is by some distance the most satisfactory method, at least among those who are aware of it as an option. It certainly seems to be a preferred method among the cognoscenti (1000+ editors), who use it at a rate only behind the Wikipedia search box.
Unfortunately new users seem to be largely unaware of this method, despite its obvious benefits. It would certainly be helpful to encourage it more. One step towards this has already been proposed in the more personalised messages at Wikipedia:Requests for comment/Level one user warnings which very prominently introduce the warning editor, and invite questions on their talk page. The proposed "Orientation" tutorial of the fellowship should also attempt to encourage this, by familiarising new editors with the talk page system. Software improvements such as Echo and LiquidThreads also aim to make talk pages easier for newbies, but these must be viewed as long-term projects and in the interim better explanations of the current system are needed.
The Help link in the left sidebar (leading to Help:Contents) appears to be quite often overlooked by new editors. As part of the overhaul of this page proposed for the fellowship project, ways of making it more obvious or linking it elsewhere should perhaps be examined. It is also rather unpopular and deemed less satisfactory among experienced editors, it will be important to try and address their concerns when working on it.
Full data files
editIn case anyone wants to do their own analysis, here are files containing all the raw survey responses in .csv format.
Thanks
editThanks are due to Siko Bouterse for feedback on the survey, Jonathan Morgan for further feedback and assistance with database queries, and Ayush Khanna for his help with the survey software. And of course thank you to everyone who participated!