Wikipedia:Pending changes/Metrics

Pending changes
Interface: Pages with pending edits · Pages under pending changes · Pending changes log ·
Documentation: Main talk · Reviewing guideline · Reviewing talk · Protection policy · Testing · Statistics

2010 Trial and 2012 Implementation

Historical: Trial proposal · Specifics · Reviewing guideline · Metrics · Terminology · Queue · Feedback · Closure · 2012 Implementation
Discussions:

Summary information for editors

Current status - Pending changes (level 1) was re-enabled on December 1st, 2012 by community consensus according to the 2012 RFC.
Logged in users – Logged in users (or users choosing to view pending changes) will see all edits as usual (unless the relevant setting has been changed in their preferences). All edits will still be added to the wiki and inappropriate edits must still be reverted or fixed as usual.
Logged out users – Until checked for obvious vandalism or superseded by appropriate editing, edits by new and unregistered users to "pending changes protected" pages will not be seen by users who are not logged in until approved. Edits by autoconfirmed users are approved automatically at level 1 when the prior revision is approved.
Policy – See the pending changes usage policy and the guideline on reviewing
Reviewer rights – Become a reviewer!.
Support and testing – Test page: Wikipedia:Pending changes/Testing. Bugs: Report them at WT:PC. For more information visit the IRC channel: #wikipedia-en-pc ^connect
Provide feedback and suggestion – Feedback page: Wikipedia talk:Pending changes. Your feedback and suggestions are appreciated.

This page will:

Document the list of metrics that have been collected
Document the metrics themselves

Goals of Pending Changes edit

To improve the quality of Wikipedia articles by making vandalism/unverified information/”bad” edits more difficult, some pages in EN wiki have various levels of protection status (e.g., semi-protection and full protection). Semi-protected and fully protected pages (ostensibly) have fewer “bad” edits that surface. For example, semi-protection prevents any anonymous and unconfirmed user to edit. But these articles suffer from the problem that potentially “good” edits by anonymous users can't get through. 'Pending changes' would be a way to allow anonymous edits but queue them for review before they show up in an article.

The current two month trial is being used to test Pending Changes. The test applies Pending Changes to pages that were previously under protection (see Scope. The predominant case involved converting a semi-protected article to Pending Changes Level 1 (PC1). There may be articles that were previously not under any form of protection that are now under Pending Changes, but these conversions are not a focus of the trial.

Expected Effects of Pending Changes edit

Given that Pending Changes is being applied to articles that were previously under protection, the following are hypothesized to happen:

By "opening up" pages previously under protection, anonymous and unconfirmed users are encouraged to edit. We hope these edits are constructive.
We will need to decide whether the increase in "good" edits justifies the amount of work to review all new edits.
There will be certain situations where edits from autoconfirmed users will require review (edits from autoconfirmed users never require review under semi-protection). We need to understand how often this occurs.

Some other areas worth evaluating:

System Throughput: We also need a better understanding of the ratio of reviewers to articles under Pending Changes (or edits requiring review). We don't want review queues to get too long, so for a certain number of articles under Pending Changes, there is probably a level of reviewers required to sustain a reasonable time-to-review. This may depend on the mix of the articles, but some broad guidelines will be helpful.
What types of pages Pending Changes seems to work for. There may be types/categories of pages for which Pending Changes works betterthan others.

Metrics edit

Here's a list of the metrics we've collected so far:

Per-page anonymous edit quality - table which breaks down per-article quality metrics, with summary
Per-page full stats - table which has a full list of per-article quality metrics. Superset of the anonymous edit quality table
Special:ValidationStatistics - Standard statistics page built into the FlaggedRevs plugin
Preliminary Analysis - Some cuts of the Per-page full stats to help the discussion along.

Notes edit

Here are a couple of places where this has been discussed: