Open main menu

Wikipedia:Wikipedia Signpost/2018-04-26/In focus

Admin reports board under criticism: A recent Community Health Initiative survey found only 27% of respondents are happy with the way reports of conflicts between editors are handled on the Administrators' Incident Noticeboard (ANI).

Zarasophos is currently working on everything related to Jadidism. He also recently took up Editing the Signpost.

Analysis of an Administrators Incidents Noticeboard Thread.png
The thread was found on AN/I April 20th, 2018, extending to more than double the shown length in total. The number of involved editors was close to twenty, and growing.

Out of over one hundred questioned editors, only twenty-seven (27%) are happy with the way reports of conflicts between editors are handled on the Administrators' Incident Noticeboard (AN/I), a recent survey by the Community health initiative on English Wikipedia found. The survey, which was undertaken by the Wikimedia Foundation Support and Safety and Anti-Harassment Tools teams, also found that dissatisfaction has varied reasons including "defensive cliques" and biased administrators as well as fear of a "boomerang effect" due to a lacking rule for scope on AN/I reports. Ideas for improvements included moderation of discussions by neutral clerks as well as bans of uninvolved editors in AN/I discussions. The survey also included an analysis of available quantitative data about AN/I.

53% avoided making a report due to fearing it would not be handled appropriately

Invitations to the survey were sent to editors who had recently contributed to the AN/I boards, but were also posted publicly on noticeboards and through Wikimedia affiliate mailing lists. Overall, 136 people completed the survey; 100 of those claimed to have been editors for longer than five years, which conforms with the teams' warning that the opt-in nature of the survey and its small sample would most likely result in a skew towards experienced editors.

Nearly three quarters (72.06%) of the participants reported being involved in an incident reported on AN/I in the last twelve months before the survey took place, while about as many (73.13%) said they were dissatisfied with the way reports are handled on AN/I. These do not necessarily have to be the same people – the survey was anonymous – but still, that's not a very good quota. There was also general consensus among answers that the AN/I process breaks down with increasing case complexity. However, while more than six in ten (62.5%) participants said they "sometimes" or "frequently" disagreed with the outcome of AN/I cases, nearly as many (51.13%) reported they "agreed" or "strongly agreed" with the general process of AN/I reports.

"Otherwise 'popular' users often avoid heavy sanctions for issues that would get new editors banned."

A specific problem raised by several answers is the discrepancy in the handling of new and old users – which is especially interesting considering the high self-reported experience of the participants. "Rarely is the discussion unbiased in these cases [...] where one of the users is new and the other one is a 'old hat' with plenty of friends", one editor writes. This bias of Wikipedia meta structures towards more experienced users – even in cases where that experience should not generally matter, such as in AN/I decisions that should be made according to Wikipedia policy – has already been reported on in other circumstances.

Another issue that could potentially further this clique-building was a perceived lack of admins actually active on the noticeboard – one participant reports seeing "the same old faces time after time." Participants speculated that this may be associated with the sometimes extreme complexity and long history of cases discussed on AN/I, as well as the "thanklessness of both the admin's and the involved editor's role." Finally, almost half (48.49%) of the participants said that discussions on AN/I are "almost never" or "rarely" focused and neutral.

"Discussions need to be clerked to keep them from raising more problems than they solve."

While there was no lack of criticism, there was also a consensus that AN/I in general was a positive thing in need of reform. This sentiment is also shared among admins active on AN/I, according to Oshwah:

The improvement to AN/I advocated by most editors was the introduction of moderators to keep discussions relevant to the discussed issue. These moderators would not have to be admins, as they would not be responsible for the final verdicts; instead, they would keep order so that admins could proceed with their investigations. Two other proposals that aimed in a similar direction were a ban on uninvolved editors getting involved in AN/I discussions and the introduction of a fill-in report form, which would allow more standardized procedures.

Oh, and there is a Harvard paper

The Wikimedia Foundation also reached out to the Harvard Negotiation and Mediation Clinical Program (HNMCP) in the autumn of 2017 to provide recommendations for reports and management of conflicts between editors. HMNCP observed several communities, interviewed experts and finally published an extensive catalogue of findings and recommendations.

The findings of the paper are similar to the ongoing internal criticism against AN/I: a discussion-based culture and a decentralized network of noticeboards without effective moderation do not lead to effective case management. For HMNCP, the result would be the replacement of the noticeboards with a single, centralized evaluation system. While this is harsher than what editors wanted in the internal survey, many of the proposed ideas seemed to build on similar ideas.

Another finding of HMNCP is a systemic inability of Wikipedia report structures to convert precedents into standards, with many cases being negotiated in very similar fashions time after time again. It is noticeable that Arbitration Committee (ArbCom) cases already function in a fashion of strictly enforcing and, if necessary, modifying prior verdicts. The status of ArbCom as Wikipedia "High Court" could inspire AN/I to adopt an analogous standardized way of conduct, in a fashion adopted to the generally lower profile of cases.

The HMNCP report applies the general idea of standardization in three recommendations:

Especially the call for better organization of complex discussions seems very much in line with the proposal of report forms and the exclusion of uninvolved editors made in the Wikimedia survey.

Finally, HMNCP recommends a better standardization and dissemination of systems and policies across Wikimedia communities and offers a bit of warning: Harvard "assumes no responsibility for the implementation of the recommendations expressed herein".