Wikipedia talk:Wikipedia Signpost/2020-09-27/Recent research

← Back to Recent research

Discuss this story

"social bias" on Wikidata

Why do you even bother with mentioning of this kind of extremely low-quality research? And undue promotion of sloppy researchers with weird results and no meaningful interpretation thereof. Staszek Lem (talk) 23:50, 27 September 2020 (UTC)

(For reference: Staszek Lem is talking about the paper by four researchers of Amazon Alexa, from Amazon's Cambridge Research lab.)

Out of curiosity, what is your opinion ("extremely low-quality", "sloppy") based on? Regards, HaeB (talk) 05:52, 4 October 2020 (UTC)

If decent researchers receive a weird result ("male list is lead by baritone"), they do something about this. As for the observation "women more likely to be homekeepers", I guess I have to add this example into my article "British scientists". Staszek Lem (talk) 16:32, 4 October 2020 (UTC)

Have you actually read the paper?

I'm not sure how this particular result is "weird" in the sense of likely to be wrong. Keep in mind that they are not saying that baritone was the most frequent profession among male Wikidata subjects; rather (as I indicated in the excerpt) that it was the profession ranked highest in a metric indicating that Wikidata subjects with this profession are more likely to be male than female. (The full description of this list in the paper is "Top 20 male professions in Wikidata relative to female using TransE embeddings".)

A more worthwhile question might be what Wikidata should and shouldn't do about these "harmful social biases related to professions" (to quote the paper's abstract). In the case of suffragettes (#4 on the top 20 female professions list), it's hard to imagine many benefits of adding male profession members until this disparity on Wikidata is resolved.

Regards, HaeB (talk) 06:12, 7 October 2020 (UTC)

Unless you are qualified to act as a peer reviewer and willing to place your reputation behind the article(s), it's a very bad idea to promote preprints that have not yet been peer reviewed. It's bad science and bad journalism. ElKevbo (talk) 00:28, 28 September 2020 (UTC)

This is really not correct. Preprints aren't as reliable as peer reviewed publications (but those aren't guaranteed either!), but they're distributed for people to read and review, and give feedback. Indeed, if there's a flaw in the work that might be obvious to editors here, it's good that they see it and contact the authors so it can be corrected before final publication; WilyD 09:19, 28 September 2020 (UTC)

This Wikipedia, not a peer review service. If they were decent researchers, they would have asked for feedback themselves. Signpost undeservedly puts them into the limelight just because they do some vivisection of our projects, and not for the merits. Staszek Lem (talk) 16:28, 28 September 2020 (UTC)

This is incorrect. Publishing preprints is an implicit invitation for feedback from interested parties. If we're reporting and discussing them, then we're interested parties. Not Peer reviewers (probably, perhaps someone is plausible referee, but I doubt I'd be asked to review a paper in any journal they're likely to submit to). Signpost is doing it's readers a service by letting us know what research is done on Wikipedia. Perhaps there's some room to improve how it's presented here to make it clearer it's a preprint, but the idea that the Signpost shouldn't be letting its readers what's going on with research on Wikipedia is totally indefensible. WilyD 09:24, 29 September 2020 (UTC)

The articles aren't being presented in the context of "these are unreviewed articles and readers should offer comments and suggestions to the authors at <mechanism to provide feedback and commentary>." They're being presented as authoritative research articles. Using this venue as a means to solicit feedback is a very interesting idea and one worth exploring but that is not what is currently being done. ElKevbo (talk) 16:55, 28 September 2020 (UTC)

No, nor should they be, nor did I suggest that. They're being presented as preprints describing research that's being done on Wikipedia. Which is of interest to Wikipedians. If you (or anyone) thinks they could use some feedback you (they) should offer it. That's one of the purposes of preprints. It's also of interest to people here to know what's being researched, by whom, and what they're finding. These are also purposes of preprints, and of plausible interest to people who read the Signpost. WilyD 09:27, 29 September 2020 (UTC)

It's irresponsible at best and dishonest at worst to present reviewed, published research alongside unreviewed, unpublished research as if they're equivalent. ElKevbo (talk) 14:22, 29 September 2020 (UTC)

• Since our objective is to make freely available the sum of all knowledge to all, it is on the face of it reasonable to check whether the English language version of Wikipedia is skewed towards coverage of floods in the parts of the world where English is spoken. Doing so without checking that other language versions have comparable skews does mean they are researching whether ursines defecate in a particular forest, without checking other forests with other species of ursines. It would be interesting to know whether the skew in EN Wiki coverage is better or worse than the coverage in secondary sources available online in English, that would be a bit of research that would tell us something new. ϢereSpielChequers 07:52, 30 September 2020 (UTC)

I don't think that it was stupid to write a Signpost article about this stuff (and certainly haven't read the paper myself), since it's nice to know what kinds of things are getting published about Wikipedia (for good or for ill). It does seem like kind of a facile observation that an English-language resource has more information on things that happened in English-speaking countries if they haven't controlled with foreign language resources − and if they didn't, I might be forced to echo the above comment about ursine defecation. That paper by the Alexa researchers seems like pure academic clickbait, though (and what a group of people to be making statements about "social responsibility", LOL!) jp×g 13:13, 15 October 2020 (UTC)

WikipediaBot

"we contacted the Wikipedia security team with the details and the inner workings of WikipediaBot prior to writing this publication as part of the responsible disclosure requirement" - What (within the parameters of what can be safely revealed in public) was the security team's response? Andy Mabbett (Pigsonthewing); Talk to Andy; Andy's edits 17:29, 20 October 2020 (UTC)