User:SGrabarczuk (WMF)/sandbox

To be copied over the page Wikipedia:Requests for comment/Deployment of Vector (2022)

Questions for reviewers

Our goal for this RfC is to allow for the English Wikipedia community to communicate their readiness (or lack thereof) for the deployment of the skin. We would like to set the expectation here that the bar for this should be for the Vector (2022) skin to be significantly better than the legacy Vector (2010) skin, rather than the objective perfection/state of the Vector (2022) skin. We’re aware that no skin is perfect and are planning on continuing development and improvements with the collaboration of the community in the future. Do you agree this is a good approach? Do you feel like this sentiment is expressed within the RfC?
We tried to include a snapshot of our process, findings, and other relevant information about the project. Do you feel like this is a sufficient amount of information and resources to help people with forming their opinion? Are there parts that need to be added or that should be cut?

Hello everyone – this is an RfC written by the Web team at the Wikimedia Foundation, with help from a number of community members, after several weeks of preparation and discussion at the Village Pump.

TLDR

What blocks the deployment of the new Vector (2022) skin (preview) from becoming the default for desktop users on English Wikipedia?

The skin would be turned on for all logged-out users, and also all logged-in users who currently use Vector legacy (2010) (the current default for desktop). Logged-in users can switch to any other available skins, including the current version of Vector legacy (2010), Timeless, and Monobook. No changes are expected for users of these skins.

If the commenting community members think there are still significant blockers and that the skin is not ready for deployment at the current time, the Web team requests specific comments on when/what needs to be completed before a subsequent RfC is begun. These comments will be used as the basis of subsequent RfC’s on this topic.

RfC on Vector (2022)

Why are we having this RfC?

The new Vector (2022) skin is focused on making the interface more welcoming and comfortable for readers and useful for advanced users on desktop. The skin has been in development for the past 3 years, and thoroughly tested in collaboration with different audiences and communities on English Wikipedia and other wikis. For more details, see the project page on MediaWiki.org.

Since early July 2022, the English Wikipedia community has been discussing the changes the Vector (2022) skin will need prior to becoming the new default. The community decided on the changes necessary before deployment, which we will address before deploying. Within this conversation, the community also recommended that the next step would be this RfC, which requests feedback on whether the default skin for the desktop users may be switched at this time.

The WMF Web team would like to thank the many community members who have participated in this process and given their feedback and guidance, both within this current conversation, as well as during the three years of development of the skin. To name just about a dozen: Barkeep49, Certes, Enterprisey, Ganesha811, Izno, L235, Pelagic, Sdkb, Terasail, TheDJ, and xaosflux. Thank you!

What are the possible outcomes of this RfC?

If the community determines that the skin…

The Vector (2022) skin can be deployed once the tasks below are addressed: After this RfC, the implementation of the change will be contingent on the completion of the tasks listed in the section “Tasks which will be completed before the deployment”, identified through the previous conversation on Wikipedia:Village pump (proposals). Once these tasks are completed, the team will set a deployment date, inform the community and the readers, and continue with the deployment process. Any issues raised after the RfC closing will be triaged and addressed after deployment. Once again, the Web team would like to confirm that all logged-in users can opt-out of this skin at any time.

The Vector (2022) skin cannot be deployed until additional work is completed: The Wikimedia Foundation intends to deploy the new skin as the default to English Wikipedia in the future. We (WMF) believe this change will be crucial to making the contents of the project more readable, the projects' interfaces more welcoming to less-technical contributors, and thus, to the overall growth of our new readers and editors. However, we will not continue with this change without the explicit approval of the English Wikipedia community. Thus, we would like to request that those who do not find the skin ready at this time include a comment stating what factors are preventing their approval. We will review these comments, propose further potential changes based on this feedback, and begin a subsequent RfC for discussion.

Background on the Vector (2022) skin

About

The current skin, Vector legacy, has been in use since 2010. When it was developed, it reflected the needs of the readers and editors of the Wikimedia sites in that year. (See the Wikimedia Usability Initiative wiki for more information.) Since then, vast new audiences have begun using the Internet and Wikimedia projects. We are not currently meeting their needs. The Vector (2022) skin aims to change the interface in ways which include the needs of all of the current audiences - both those who have been using the projects for a long time, as well as those who have joined more recently, or have yet to join. We encourage you to read our recent blog post on equitable product development to gain more insight into the way we identified the needs and included the voices of our large set of audiences into the product development process.

The skin is focused on making the interface more welcoming and comfortable for readers and useful for advanced users. It builds upon the existing Vector legacy skin to introduce a series of feature changes that aim to improve problems new and existing readers and editors were having with the old skin. It also draws inspiration from previous user requests, the Community Wishlist Survey results, and existing gadgets and scripts across different projects.

The principles for the design were to make it easier to focus on the content of the page, to make commonly used tools and actions easier to find, use, and understand, and to improve the logic of the navigational hierarchy by bringing similar tools closer together. As a part of the project, the Web team has committed to not making any changes to the content itself and not removing any functionality.

The results of our A/B testing and qualitative testing show that these changes make it easier to read and learn, navigate within the page, search, switch between languages, use page and user tools, and more, without negative effects to pageviews, account creation, or edit rates. The team has been working on the new skin for the past 3 years, ensuring that every change is thoroughly tested and proven to work.

Process for developing the new skin

Learn more

Problem identification research with both readers and editors - During this phase, in 2019, we studied the way people used the site and identified the largest usability issues as well as issues to exploring the site further, becoming more engaged with reading or editing. We did this by interviewing readers and editors across multiple countries and locations. We also surveyed existing gadgets and use scripts across different wikis and communities to identify what types of changes and alterations to Vector legacy (2010) were made to improve readability and usability (See the links: Research and design: Phase 1, Research and design: Phase 2.)
Prototype development and testing - Over the course of 2019-2022, we built out the ideas for individual features and began showing potential solutions to our audiences. These ideas were built in the forms of usable prototypes. Each feature was tested with readers and editors through interviews and wider rounds of prototype testing. Generally, for testing with editors we used Central Notice banners across multiple languages and Wikimedia projects, including English Wikipedia, so that we could get the widest audience possible. Each prototype was tested by approximately 200 editors on average. (Example)
Refining and building - During 2020-2022, we took the feedback from the prototype testing and refined or changed the prototype based on what needs were identified in the prototype testing from both readers and editors. In some cases, we asked for additional feedback once changes and iterations, so that we were sure we were making the right decisions.
A/B testing and other quantitative testing on pilot (early adopter) wikis - For each feature, we performed quantitative tests for whether the features worked as expected based on the criteria of success we had defined. For example, the sticky header was designed to decrease scrolling to the top of the page. We gave the sticky header to 50% of users and compared them to the other 50% for two weeks. After two weeks we compared the results and identified that people who had the sticky header were indeed scrolling less to the top of the page in order to select any of the tools available there. If we received negative results from our test, we focused on identifying the reasons for these results across different audiences and made changes to the feature which addressed them. Then, we would test the feature again. This was the "beta" phase. During this phase, we also monitored usage across all wikis, including English Wikipedia, where many account holders have been already using the new skin.
Finally, now, we deploy Vector (2022) on more wikis and continue monitoring the way people are using it so that we can flag any issues. In this phase, Vector (2022) isn't "beta" anymore. It's more like a B-class article. Different wikis have different thresholds for B-class, and we believe that in the case of English Wikipedia, we'll be there when the visual refinements and other deployment blockers are ready.

New features

Learn more

The table of contents makes it easier to explore pages. It helps identify the current section and remains persistent while scrolling down the page, making it easier to navigate to various sections.
Through more prominently-placed language-switching tools, multilingual readers and editors can more easily find their preferred language and switch between languages as needed.
The content on the main namespace will now have a maximum line width. Research has shown that limiting the width of longform text leads to a more comfortable reading experience, and better retention of the content itself.
The search widget is placed in a more prominent location. It includes additional information about each query by adding images and descriptions, making it easier to find articles on Wikipedia.
A new collapsible sidebar allows for collapsing and opening the main menu of the page, depending on the context and task.
The sticky header allows for commonly-used tools such as access to the history and talk page, search, and user tools to be available while scrolling down the page.

Findings and results

Learn more

Throughout our process, we have measured many quantitative improvements of the Vector (2022) skin over the Vector legacy (2010) skin. Here is our key results at a glance:

On average, 88% of logged-in users on our first set of pilot wikis (incl. French Wikipedia) continue to use the new skin once they try it.
The sticky header makes it easier to access tools that editors use often. It decreases scrolling to the top of the page by 16%.
The new table of contents makes it easier to navigate to different sections. Readers and editors jumped between sections 50% more than with the old table of contents.
The new search bar is easier to find and makes it easier to find the correct search result from the list. This increased the amount of searches started by 30% on the wikis we tested on.
The collapsible sidebar allows readers and editors to choose whether they want to see the main menu or not. Our testing shows that this allows people to better focus on their current task - reading with a collapsed menu, or editing with an open menu.
The language switching menu makes it possible to switch languages without scrolling.
The user menu collects all user links in a single place, making it easier to understand what each link does. In testing, 71% of surveyed editors reported positive experiences with the new menu.
Overall accessibility of the skin is on-par or better than the Vector (2022) skin

Effects on other core metrics:

Slight increase in account creation observed on initial pilot wikis
No decrease in active editors, or edits attributed to the skin
No decrease in pageview attributed to the skin

Please visit the repository page for the full versions of our research and reports.

Projects part of our continued support for the Vector (2022) skin after deployment

Tasks which will be completed before the deployment

[DONE] Table of Contents collapsing and narrow screens behavior - This work is now completed. The table of contents is collapsible, and can be accessed from both the title of the page as well as from the sticky header.
Comment from the Web team: Please let us know if you have any concerns around the implementation here or additional requests around the ToC.
[To be completed by September 15] Visual refinements - The team is currently wrapping up the core parts of the visual refinements work. Please see the Qualitative Testing section on the project page dedicated to this part of the project for a full list of the changes Web plans on making.
Comment from the Web team: We appreciate your feedback on any and all of these.
[In Progress] Coordinates - Web is continuing to explore different solutions for coordinate alignment, including potentially adding coordinates directly into the styles of the skin in the future.
Comment from the Web team: Do you have any thoughts on this idea? Any immediate concerns? Let us know within the Phabricator ticket

Continued support for the Vector (2022) skin after deployment

Learn more

We plan to continue work on the desktop experience after this change is made, and into the future. Here’s a snapshot of the projects we will be working on within the next six months.

October-November 2022: Once the skin is deployed, we will be focusing on the page tools project, which will allow easier access to page tools and further customization or our menus and tools overall. We encourage you to check out the prototype for this work as well as our feature page.
March 2023: Navigation improvements - we are currently exploring ways to make it easier and clearer to understand how wiks and the content creation process works for readers, and specifically surfacing the fact that wikis are collaborative spaces anyone can get involved in. Our goal is to enable people to read content more critically and be able to recognize what is trustworthy, as well as to make it easier for those who might be interested to know that they can contribute and to begin their contributions journey. We will be collaborating on this work with the Growth and Editing teams (see: Core Experiences).

Discussion

The Vector (2022) skin can be deployed

~~~~

The Vector (2022) skin cannot be deployed until additional work, not mentioned in the section "Tasks which will be completed before the deployment", is completed.

Note: please do not state that the skin is not ready for deployment without adding context on what additional work is necessary

~~~~