Open main menu


Sociological follow-upEdit

I have tried several times to address this issue here before taking action, but someone keeps deleting this thread from the page. I thus started just making the change, but someone keeps reverting that. Now an admin has asked that this come back to the talk page. I am trying this one last time, and if someone deletes it rather than responding, I am taking this issue to that admin.

The "sociological follow-up" subsection claims to provide quantitative proof of claims only suggested in Sokal's piece. In the footnote it only provides one source, and that source is self-published and not peer-reviewed. A self-published and non-peer reviewed piece doesn't provide quantitative proof of anything, and this should be sufficient grounds for removing the section anyway. More importantly, however, the piece isn't even available online anymore, as the footnote itself admits. So, this whole subsection is about one study that supposedly occurred, but there is not documentation of this study available anywhere. The footnote claims that there is a one-paragraph summary of the study on the researcher's website, but a) an abstract of a non-existent article doesn't mean anything and b) the link provided doesn't go to a page that even contains the quoted abstract. At this point, the study this section purports to quote doesn't seem to exist anywhere other than the footnote in this wikipedia article. There is no point for this wholly unsubstantiated section to be here, especially since the "Academic Criticism" section above provides sources that have further explored the phenomenon which have been peer reviewed and which do actually exist somewhere.

Edited to add that, if someone doesn't bother piping up in the next few days to explain why they think it should stay, I'm going to delete it. Happy to have the convo about why it should stay if someone wants to talk rather than deleting this thread and reverting edits.

— Preceding unsigned comment added by (talk) 19:43, 21 February 2015 (UTC) 
Note that the article is in fact online, under the title "The Intelligibility of Unintelligible Texts." For example, this link looks to be publicly accessible. Sunrise (talk) 22:03, 21 February 2015 (UTC)
  • Good find! However, it does not look likeit is anything else than a self-published manuscript, which would pretty much disqualify it as a source. Why should we regard this as a reliable source? --Randykitty (talk) 23:45, 21 February 2015 (UTC)
I haven't thought about it myself - I just saw the edit war and thought I would look it up. FWIW, I also found some indications that it was (part of?) his thesis work and that it was presented at a conference. I don't know what the standards are for this field though.
(I thought I had posted the above paragraph and came back to add more.) I was looking into this further and eventually my search terms found this by the same author, which cites it as a Master's thesis. Per WP:RS a Master's thesis must have had "significant scholarly influence" to be considered reliable, which I doubt we have. That link is to a published paper, though, and it does summarize the earlier work, so perhaps we could cite that instead if we decide it has sufficient WP:WEIGHT. Sunrise (talk) 05:13, 22 February 2015 (UTC)
From reading the abstract and the first few pages of that article, it doesn't have anything to do with the self-published study at issue here, nor does it pertain to the Sokal Affair. What exactly would be the point in citing it? The issue isn't whether Robb Willer has ever published peer reviewed work somewhere. The issue is whether he produced influential and reliable scholarship that supports Sokal's claims about peer review in humanities academia, as this wikipedia article claims that he did. The fact that an author published peer reviewed research on one subject doesn't validate his self-published research on another subject that is tangentially related at most. (talk) 17:38, 22 February 2015 (UTC)
From my reading, the previous work is a major component of the article. The phrase "an academic text evaluation" in the abstract refers to the work in question; it is one of the three studies being used to support their argument. The study is discussed in considerable detail on pages 472-476. Sunrise (talk) 18:51, 22 February 2015 (UTC)
None of that is relevant. Summarizing a self-published study for a few paragraphs in a separate peer-reviewed study does not validate the self-published study. The peer reviewers didn't evaluate that self-published study in depth, nor did they give approval for that study to be published in their journal. There isn't some transitive property of peer review where the respectability conferred by the approval of an article somehow transmits to every other source referred to in that article. Can anyone offer any good reason for trying so hard to defend this piece? I don't understand; it unambiguously does not meet the criteria for scholarly evidence. Why is this so contentious? Isn't it a little ironic that this conversation is happening on the talk page for an article about how peer review standards sometimes become lax on the basis of the researcher's reputation? (talk) 20:48, 22 February 2015 (UTC)
I don't see how you think I'm arguing for a "transitive property of peer review." If thesis work is published as a thesis before being published in a peer-reviewed journal, then we can cite the latter. In the new article, it's discussed for five pages, not a few paragraphs; it's a key component of the argument the paper presents and includes detailed description of the research methods. Unless the peer review was seriously deficient, it would have included this. (Or I suppose standards in this field could be very different from what I'm used to, which is completely possible.) This doesn't mean the article can remain as it is - we've agreed that the current citation can't be used, right? - but I think there is a case for including the information in some form. Of course, what form that is, and whether it should be less than the current version (or much less, or zero), is open for discussion. But once the information is cited to the new source, the argument that it's self-published, or not peer-reviewed, seems to me to be incorrect. Sunrise (talk) 22:30, 22 February 2015 (UTC)
Sincere apologies; I misunderstood your point. I only meant that this article doesn't make the master's thesis itself a valid source, but if you're suggesting to cite the claims made in this article itself, I would agree that this at least obviates the problem of the current reference to a self-published (and very difficult to find) source. Having read through pages 471-476, it does provide a method and results section that surely met muster for the peer reviewers, so I would think the citation stands if changed to refer to these pages of that article. Having read the description of that experiment now, I do have two follow up issues about relevance, though. 1) Sokal's main claim was that humanities luminaries like Jameson and Aronowitz would accept nonsense if it flattered their ideological biases. But that's really not what's at stake in this study is it? 2) The section as written says that Willer, et. al. wanted to address what they perceived to be a secondary claim of Sokal's: that research would be accepted on the basis of the researcher's reputation, rather than on merit. I'm not sure that was Sokal's point, really (did Jameson know who Sokal was?), but at any rate, Pages 471-476 of Willer, et. al.'s article don't say anything about such a motivation. More importantly, though, the study doesn't seem to be measuring the same phenomenon. It proved that junior and senior undergraduates intent on going to grad school will speak highly of nonsense articles when in a public setting where they are insecure of their intellectual standing. Does that really seem analogous to Sokal's claims about how famous intellectuals will privately evaluate a piece they are asked to peer review? Sokal seems relevant to Willer, et. al only insofar as it was convenient for their experiment about something tangentially related to have a text that was deliberately nonsensical while imbued with jargon. (talk) 00:57, 23 February 2015 (UTC)

───────────────────────── Well, they did use Sokal's original text, so there is a connection, but I do agree that the current version goes too far in what can be said. I think that if we just say what they did with Sokal's text, what they were measuring, and what the results were, that would probably be an improvement. What would you think of this?

In 2009, Cornell sociologist Robb Willer performed an experiment in which Sokal's paper was presented to undergraduate students and told that it was written by either another student or by a famous academic. He found that the students who believed that the author of the text was a high-status intellectual rated it as having higher quality and being more intelligible.[1]

  1. ^ Willer, Robb; Kuwabara, Ko; Macy, Michael (September 2009). "The False Enforcement of Unpopular Norms" (PDF). American Journal of Sociology. 115 (2): 451–90. Retrieved 21 February 2015.

About Willer's motivation, the point that the Sokal hoax played a role in inspiring the study could be sourced to the master's thesis (self-published sources are permissible when the conditions at WP:ABOUTSELF are met). I don't feel strongly either way on this, so we could leave it out unless other editors feel that it's important. Sunrise (talk) 08:03, 24 February 2015 (UTC)

I've added the above section into the article. Feel free to edit further. :-) Sunrise (talk) 02:56, 28 February 2015 (UTC)

Short on original subject matterEdit

I simply regret that so few links are available to the original papers in such cases. It's all commentary, reportage, analysis, and criticism. I would really like to gaze upon the original material that is the starting point for so much controversy. (talk) 21:32, 16 April 2015 (UTC)

Proposed section splitEdit

I've removed the {{split section}} tag from the bit about the paper -- nothing has been done about it, and I believe any split content would only end up being re-merged here, as the whole point of the paper was to perpetrate this hoax: it's nonsense, deliberately so, and so there's nothing to discuss about it that isn't more appropriate to include here. -- The Anome (talk) 14:45, 19 August 2015 (UTC)

Similar incidentsEdit

The section "Similar incidents" bears a notice from November 2015 that it may stray too far from the topic, and invites discussion here. Sadly, in the one year and more that has elapsed since that notice was posted, no other editor has seen fit to comment. Possibly the editor who posted the notice may have got more traction with an RfC. FWIW, I'll give my opinion of the question, and hope to see some other comments here, before ... let's see, 2020? Who knows, I may even start an RfC myself ...

  • Remove: I think that the list of "similar incidents" should be removed in its entirety, since it strays so far off-topic as to essentially constitute another article. That would leave only the reference to the Category:Academic scandals, which would then no longer be a "Main category" reference. Better still would be to replace that reference by one to a new article on Academic publishing scandals, incorporating the removed contents, also in the Category:Academic scandals. If I can make time – and barring any better suggestions – I'll do so myself. yoyo (talk) 17:25, 16 December 2016 (UTC)
  • Trim: I would support trimming the section for length and only retaining a few of the well-sourced and most closely similar incidents.  —jmcgnh(talk) (contribs) 00:33, 17 December 2016 (UTC)
  • Remove. If we trim it (or if we do nothing), it will continue to invite people to add more, and it will grow again. It could be a separate article though. I think Scientific publishing hoaxes or something like that would be preferable, since "scandals" would also include fraudulent papers and poor papers that had to be retracted - or were not but should have been. --Hob Gadling (talk) 19:09, 18 December 2016 (UTC)
Return to "Sokal affair" page.