Wikipedia:Community portal/Redesign 2012

The purpose of this page is to document an experimental redesign of the Community portal on English Wikipedia. In order to identify the elements that will make this a more useful space for new and/or existing editors, a series of short-term changes to the content will be performed and data gathered about their efficacy.

You can find updates about tests and findings here. More details about metrics and methodology are on the research page on Meta.

Background

Questions asked by new users at the Teahouse; the fifth most common question is where to find task suggestions or projects

The Community portal was created in 2004 on the English Wikipedia, as an editor-focused (as opposed to reader-focused) offshoot of the main page, and was originally a simple task list and a few links to fundamental policies and guidelines. Over time, as the number of editors grew and other pages like Village pumps and WikiProjects became more lively centers of community discussion, the Community portal became less important. However, the page is still linked from the sidebar and draws in around 10,000 pageviews a day.

A qualitative assessment of current Community portal viewers demonstrated that many of these pageviews probably come from readers and very new users who are looking for help or information from experienced editors, but most experienced editors no longer watch the page. As an additional datapoint, many of the new users who come to the Teahouse ask where to find task suggestions and/or general to-do lists and are guided to the Community portal's open task list. (See fig. right)

Unfortunately, the bot that updated the open tasks template with new tasks stopped running in February 2012.

These findings suggest that the Community portal might potentially fill a vital role for newer and less experienced users, but the current format is not serving this purpose effectively. Because the page is no longer used by most of the community of established editors, it may be an excellent space to iteratively test changes to design and layout and measure reader conversion and new editor engagement/retention.

Research questions

In this experiment, we are testing our hypotheses that:

1. It is mainly new and less experienced users of Wikipedia who are landing on the Community portal, and improving the information that is presented there will educate and empower these users to potentially become established Wikipedians.
2. Reducing and streamlining the information presented on the Community portal, focusing on one or two specific topics (e.g., tasks and collaborations; help and mentorship) will make it a powerful resource for new and current Wikipedians alike, and we will see the pageview-to-clickrate ratio go up significantly.

Assumptions

1. The Community portal could potentially be a useful starting place/landing page for newer and less experienced users, but it is currently not set up with any target audience in mind.
2. The overwhelming number of links on the Community portal makes it unnavigable and unusable.
3. The redundant and outdated links on the Community portal confuse and disorient newer and less experienced users.
4. The lack of focus on any one audience (e.g., readers, new editors, established Wikipedians) makes the Community portal not useful for anyone.
5. The lack of focus on any one aspect of community engagement (e.g., suggestions for tasks to do, projects to join, help spaces to visit, etc.) makes the Community portal not useful for any one of these calls to action.
6. The design/layout of the Community portal does not effectively highlight the important information that is contained there and needs to be updated.

Methodology

• Make weekly changes to the design/layout of the Community portal page, bringing one main page element into focus (e.g., task recommendation, Wikiprojects, help pages, adopt-a-user program...)
• Measure clicks and impressions on the page and rate of successful conversions (e.g., tasks attempted/completed, WikiProjects joined, help pages used, new users adopted, etc.)
• Identify successful changes and discuss incorporating them into a stable new version of the page

Control data-gathering

• Measure current clickrate on page for one week to determine which elements are being used

Initial results

After three days of clicktracking, there were a total of 3212 links clicked (approx. 9% of the pageviews for that period, roughly 36,000). As expected, the overwhelming majority of visitors who clicked on links on this page were unregistered editors, with very few experienced Wikipedians represented. Below is some more fine-grained data:

• 87% (2802) of the links came from unregistered users
• 13% (410) came from registered users
• As expected, there was no one overwhelming area where users clicked most. Among the more popular links were the Signpost (7%), the featured WikiProjects and Portals (each 6%), and, oddly, the Community portal itself (5%) – possibly because users were expecting something different on the page and clicked on the sidebar link again
• Just 3% of the clicks were to items in the Open tasks box, and only one of the articles clicked on was edited around the time of the experiment, meaning that none of these clicks were successful conversions
• Most of the clicks occurred on the top half of the page (and, within the Open tasks box, most of the clicks occurred on the top half of the list of suggested tasks, with virtually no clicks on the bottom rows). This suggests that there is simply too much information on the page, and most visitors to the Community portal rarely, if ever, scroll down all the way to the bottom.

One week results

After a week of clicktracking, there were a total of 9156 clicks on links in the Community portal, representing 13% of the approximately 70,000 pageviews for the period of July 27-August 2.

Users

• 86% (7847) of the clicks came from unregistered users
• 14% (1299) of the clicks came from registered users
• The majority of the registered editors (903, 70%) had made less than 1,000 edits to English Wikipedia

Links clicked

The majority of links clicked are located on the lefthand navigation toolbar. Some of the most popular included:

• The Community portal itself - 10% (888)
• Portals (e.g., Portal:Featured content) - 9% (818)
• The Main page - 9% (805)
• Special (e.g., Special:Random, Special:RecentChanges) - 9% (805)

Of the actual page content clicked, the most popular were:

• The Signpost content - 7% (635)
• Discussions (Requests for comment) - 6% (553)
• Articles in the open tasks area - 5% (418)
• WikiProjects - 4% (411)

Importantly, the general trend of clicks was highest at the top of the page and lowest at the bottom, and higher on the left than the righthand side, which is consistent with most eye-tracking studies of screen reading.

Conclusions

These quantitative findings support the qualitative feedback initially received: most visitors to the Community portal who click on any of its page elements are unregistered or relatively inexperienced editors. The haphazardness of elements clicked, and especially the very high number of clicks on the Community portal and other sidebar links, suggest that these users are not currently finding the page useful, and that there is much need for improved layout and design. The higher volume of clicks on links located in the top lefthand quadrant of the page suggests that more salient information should be placed there, so any topical redesign should be focused primarily in that area.

First iteration: tasking

Hypothesis: new users who come to this page might be looking for things to do/ways to help (per the Teahouse survey), but that information is not prominently displayed. This redesign will test the effect of placing a task list in the area where more visitors to the page are likely to see/interact with it.

Changes made:

• the header bars were removed/collapsed as much as possible
• the open tasks list was moved to the top lefthand corner of the page
• the task list was split into easier new-user-directed tasks (top) and more difficult experienced-user-directed tasks (bottom)

Initial results

The following was observed after one day (8/7/2012) of clicktracking:

• The overall number of clicks on the page increased (from an average of ~1308 a day during the control period to 1524 a day during the test period)
• The clickrate on open task articles went up from 3% of all clicks during the control period to 8% during the test period – a statistically significant increase
• However, the rate of task completion was also the same: no open task articles were edited during the experimental period
• Clicks on the open tasks list occurred mostly on articles in the first two task categories ("Add wikilinks" and "Copyedit", 18 and 20 of 123 open task clicks, respectively), followed in frequency by "Requested articles" (12 of 123) and "Requested images" (10 of 123)

Next steps

To test for interest-dependence in tasking, the list of tasks was reordered to move high-click topics to the middle and low-click topics to the top - if clicks on tasks by topic remain the same, this will suggest that task list order does not play a significant confounding role in what kinds of tasks users are choosing to click on.

Results

After one day of clicktracking, it appeared that people were still almost twice as likely to click on the two top-performing tasks, wikifying and copyediting, than on other tasks on the list, despite the fact that these two tasks no longer appeared in the eye-catching top area.

However, after 4 days, these results were reversed. Orphan and stub expansion, which were at the top of the new reordered list, became the most popular click area. The third most popular was AfD discussion, which was at the bottom.

Conclusion

It appears there is a weak relationship between task type and clicks, with people generally preferring wikifying and copyediting (two relatively simple and straight-forward tasks). However, there is a strong relationship between order of tasks and clicks—with items at the very top of the list receiving the most attention.

Possible next steps

Depending on the results of initial clicktracking, the following iterations may be tested:

• Task suggestions/recommendations: to make the open task list more prominent and useful
• Featured WikiProjects/collaborations: to surface current collaborations of note
• Help/mentoring: to surface useful help and mentoring spaces in an non-overwhelming way