Talk:Inter-rater reliability

Latest comment: 2 years ago by Gandalf 1892 in topic Resolving inter-rater disagreements

Untitled

edit

Is Joint Probability the same as Simple agreement? Ben 23:53, 9 March 2007 (UTC)Reply

What are the consultation methods for asking a group of experts to reach an agreement that can be tested for inter-rater reliability? --131.170.90.3 06:21, 19 September 2007 (UTC)Reply

Multiple Issues

edit

I added a multiple issues cleanup tag focusing on (a) the need for neutral citations (there are citations which link to a sales page for the cited book); (b) the need for more reliable references, e.g., the first section (Inter-rater reliability#Sources of inter-rater disagreement) does not have any citations; and (c) the article lacks a logical flow (copy editing needed). If you disagree, let's discuss here first before removing the tag.   - Mark D Worthen PsyD (talk) 09:53, 26 December 2018 (UTC)Reply

And the references are a jumbled mess - they should probably be reworked from the beginning.   - Mark D Worthen PsyD (talk) 10:01, 26 December 2018 (UTC)Reply

Resolving inter-rater disagreements

edit

The page lacks a section on how to resolve discrepancies among many raters. There are at least two ways to solve them: if data are categorical, the most common decision among raters is chosen as the answer; if the number of raters who chose each option is even, then a random process like a coin toss decides. When the data are quantitative, the ratings are averaged to achieve the final score (arithmetic mean). Gandalf 1892 (talk) 22:03, 19 May 2022 (UTC)Reply