Open main menu

2018 WikiCup points and rules discussionEdit

So once again, it's the time of the year that we discuss the possibility of changing the points for the WikiCup. So, what worked this year? What didn't work? Is the balance right between FAs and GAs? Should GARs score more? what about the "significant work" criterion? Are there new rules/methods of running the competition needed? Was having cash prizes a worthwhile addition to the competition? Feel free to open subsections on different subject matters under this section.

There were no changes in the scoring system for the 2017 WikiCup but there are now several changes that the judges are considering. One of these is the date that signups close; traditionally it has been part way through February but we are proposing making it the last day of January. We look forward to having your views on this and other matters. Cwmhiraeth (talk) 19:11, 11 November 2017 (UTC)

I do say that I think it is better to move the closing date for signups earlier. That way you stop the potential for people jump in at the last minute. The WikiCup is known enough in my opinion that we don't need such a wide grace period nowadays. The C of E God Save the Queen! (talk) 19:59, 11 November 2017 (UTC)
Scoring table definitely needs revision. There is a lot of variation between FA and FL. Editors almost put in the same effort to develop a FL compared to that they put in for a FA. They must be comparatively given an equal weight. Also the score for a GA must be hiked. Being our project is an encyclopedia, it is not reasonable to give almost equal weight to a picture contribution, which may be given importance when something is done on Wikimedia Commons. It is not same thing here, GA must given a higher score. Standardize the score of 10 for DYK. Hike ITN to 15 or 20, because unlike DYK or any other article, ITN has to be done within the time, before the news fades away. Good article review may be give a score of 10. Regards, Krishna Chaitanya Velaga (talk • mail) 14:34, 17 November 2017 (UTC)
I agree, there is a disparity between points for FA and FL, and to be honest FL often involves more work because of the formatting on some of them. They are both featured claims at the end of the day, and should be equally weight.  — Calvin999 15:03, 18 November 2017 (UTC)
FWIW, I disagree. The FA review process is much more in depth than a FL review, and generally the amount of content is vastly different too. I could easily create two or three FL quality lists per week, even with my current work/life/Wikipedia balance. Maybe a small adjustment could be made; 50 points for a FL, 40 for a GA, but I actually think the current ratio is about right. Generally, getting a Featured article is a significant amount of work, and deserves an appropriate reward. Harrias talk 16:02, 18 November 2017 (UTC)
From my experience of FLs, I've waited several months for them to be approved and I've also had a lot of input in the reviews. The reviewing process for FLs may not be as in depth as FAs, but FLs take far longer to format and create, especially the ones I've done.  — Calvin999 23:52, 18 November 2017 (UTC)
  • In general, I do believe the current scoring system is sound: it rewards greater quality (FA > GA > DYK) and also rewards the importance of the topic (through the bonus points system). My gut feeling is that GAs should receive a larger number of points, but I'm unwilling to support such a change unless its accompanied by an increase in scrutiny (see below). I also think we need to have a better system for judging the importance of a topic. For instance, during the course of the competition, I brough Paradises Lost to FA status, and received 200 points. I also brought Manual Noriega to GA status, and received 98. I'd say that the latter was both more work on my part, and a greater contribution to the encyclopedia, yet it received ~0.5 as many points. I dunno if there's a good way around this, but I thought I'd bring it up, and see if smarter people than I have a workaround. On a related note, I think it's clearly a result of this points structure that the clear "winning strategy" for five years now has been to expand articles on wildlife. I do not think this is necessarily a problem, as I certainly consider wildlife a "serious" topic, but I do wonder if there's a way to ensure that other equally central topics (history, geography...) do not get neglected. Vanamonde (talk) 11:42, 21 November 2017 (UTC)

Offering cash/vouchersEdit

I only just saw this because of the message left on my talk. I think this was, is, a terrible idea, because people are only doing it for the money. I'm actually quite shocked that this has happened or even put forward as an idea in the first place which got as far as getting the green light. I feel like this really goes against Wikipedia's principles actually. People contribute for free because they care about the topics they edit. This is essentially payment for a select few, as it's always the same names who end up in the final everywhere anyway, so it's not like anyone else ever really has a chance of getting to the final, yet alone winning money or vouchers. Having made it to the final three times, I've always felt the competition has been really good without a monetary incentive. I was happy just get a participation badge! But I've always done it because I really care about the topic I edit. It's making something good or featured that I hope others will enjoy reading that makes me want to do it well.  — Calvin999 14:52, 17 November 2017 (UTC)

I like the idea - and have run the Core Contest on that basis. However, the interesting thing would be to look over the years and see if people "gamed" the system harder to gain the vouchers. My gut feeling is probably not as there are controversies of varying degrees most years. Cas Liber (talk · contribs) 21:06, 17 November 2017 (UTC)
I think giving money for a friendly competition to improve Wikipedia because they care about it is just wrong. If anything I think it will make the gaming even worse. You could get someone who works on a lot more articles and make lots more contributions compared to someone to does a few FAs with someone bonus points and wins. It's not a fair incentive. Usually you get financial incentives in the real world because you've worked harder, not because you're done less when compared to others. Furthermore, a cash prize is effectively taxable income because you've worked for it, but I doubt the receivers would be declaring it as such. But I don't really expect someone who has profited from it to argue against it.  — Calvin999 22:59, 17 November 2017 (UTC)
"Profited"? Really now? I've been out of the loop for a while but if memory serves, the third place winner would have been awarded something like $50... literally cents per hour of editing to promote several FAs. It's absurd to discredit a participant's take on the issue on the grounds that he spent 10 months to earn the equivalent of a tank of gas. – Juliancolton | Talk 05:32, 18 November 2017 (UTC)
Yeah, I agree with Calvin, give monetary benefits is not a good practice. However, we would change the idea a bit. Instead of gift vouchers, we can give, say 5 books of choice to the winners, 4 to second place, 3 to third. 2 books to category-wise winners. I feel that this would be good because it'll further help to improve their knowledge and contribute to encyclopedia. --Regards, Krishna Chaitanya Velaga (talk • mail) 06:30, 18 November 2017 (UTC)
That is the point of using people can choose their own books and it isn't cash. It is patronising and wasteful to choose specific books for people you don't know. Cas Liber (talk · contribs) 07:03, 18 November 2017 (UTC)
I used vouchers won in the Core Contest to get books, including Invertebrate Zoology, a textbook that I find invaluable for writing articles. Cwmhiraeth (talk) 10:41, 18 November 2017 (UTC)
But that doesn't necessarily say that everybody will buy books from the vouchers. My point is, instead giving them vouchers, let them choose books, and the organizers can directly buy them for the winners. Regards, Krishna Chaitanya Velaga (talk • mail) 13:20, 18 November 2017 (UTC)
I'd have no problem with that. Cas Liber (talk · contribs) 13:29, 18 November 2017 (UTC)

─────────────────────────My only real objection is that it would put an extra burden on the judges/sponsors and it wouldn't be as quick to get done as if the winner did it themselves. I've been awarded several book grants and it takes longer to get the books that way than if I did it myself. Plus, there's a certain amount of duplication of effort as the awardee has to look up the books wanted versus their price to decide between books and then the judge/sponsor has to look them up again to order them. So I think that the voucher/gift card is the best compromise.--Sturmvogel 66 (talk) 16:05, 19 November 2017 (UTC)

I remember a couple of years ago that someone suggested that people who got to a certain stage would be awarded with a T-shirt with WikiCup on it. Personally I think if we offer prizes, we should offer something like this as a token award rather than a monetary award. The C of E God Save the Queen! (talk) 12:03, 21 November 2017 (UTC)

I guess I'm the "doing it for the money" guy. It's why I signed up in 2017, and it motivated me to contribute more new content than I would have otherwise. I'm not sure I understand Calvin's point about how it undermines people contributing for free because they care about a topic. It's not like the WikiCup dictated what subject matter participates could work in... Argento Surfer (talk) 18:57, 21 November 2017 (UTC)

I don't have much opinion on financial reward or not, but I do think if we have it, then it should be aimed towards the continued benefit of the encyclopaedia. To that extent, I think t-shirts would be a complete waste of money; whereas Amazon vouchers (or similar) can clearly be spent on books for more research. Harrias talk 19:09, 21 November 2017 (UTC)

Good Article ReviewsEdit

I need to chime in on something that has irked me for several years and it will probably sound like blasphemy... I think we should not award points for GARs. I know this has been where the vast majority of the submissions is from, but that may be the problem- and a problem for Wikipedia as a whole. Because people identify it as the best get-rich-quick scheme, we've had loads of half-assed reviews that are passed prematurely, which only have to be re-reviewed at a later time, because editors got greedy and just wanted points. I understand this will be an unpopular suggestion, so it will likely get struck down immediately, but I see it as more of a detriment than an incentive. I believe people should work for their WikiCup rankings, as the highest-ranking participants typically have done. DARTHBOTTO talkcont 23:04, 17 November 2017 (UTC)

I agree, I've seen so many who spend half of their review not really reviewing it, just waffling. I had someone tell me how nice an article of mine was written in about 10 different ways bullet pointed, and actually didn't list anything that needed changing. I flagged it but it was ultimately kept (looked over) because the reviewer was presumably too high level to remove it from. But there are a minority who do give good reviews. I'm not for scrapping points, I wouldn't oppose it behind reduced back to 3 points, but I do think there should be a minimum length requirement or something similar. Like 15 bullet points of negatives/things than need improving, correcting, changing etc., without all the positive "wow this is great" waffling to just fluff it up more.  — Calvin999 23:11, 17 November 2017 (UTC)
The reason I am in favor of completely giving the boot to it is that I feel like having too much leverage with determining the criteria or having too many criteria itself could be a slippery slope. I see any incentive for rewarding reviews as leeway for sloppy GAs. Then again, I think that should be only a single perspective in the face of the overarching theme that something with GARs needs to change. DARTHBOTTO talkcont 02:51, 18 November 2017 (UTC)
After Darth's comment, I feel that GAs must be moved out of the way. Because depending upon the character count or no. of bullet points, you can't judge the quality of the review. I reviewed more than 100 GANs, it is not the same game for every article, even if they are from the same topic. If I want to increase the review size, I can easily do that. For example, instead of writing "Link Vice Admiral, torpedo tubes, allied forces", I can split that into "Link Vice Admiral" "Link torpedo tubes" "Link allied forces", in this way both no.of points and as well the size gets increased significantly and easily. This is just one improvement, the same can be done for many others. So I feel that reviewers may get hurried in mining points, and compromise on the reviews. This'll not only effect the review, but also ultimately the article, which will a GA tag. Also GA Cup is anyways there for this part, I don't think we need to do it here again. Regards, Krishna Chaitanya Velaga (talk • mail) 06:39, 18 November 2017 (UTC)
But the GA backlog is soooooo long. Cas Liber (talk · contribs) 07:09, 18 November 2017 (UTC)
I have frequently judged reviews, told the judges which ones and most of them were removed because they agreed with me. I'd even be happy to take on the role of judging reviews.  — Calvin999 09:20, 18 November 2017 (UTC)
The inclusion of GARs encouraged me to educate and involve myself with the process, however this may have led to premature action. I loved being able to take part in an area of Wikipedia I did not previously know about, but I may have taken part before I was ready and fully qualified to. That being said, I have learned a lot about the process from the WikiCup, and without it I would not have known anything about it, but I also wouldn't have been able to improve my skills in being able to judge a 'Good Article'. ThomDevexx ॐ (talk) 15:28, 18 November 2017 (UTC)
The WikiCup adds dramatically to the already huge GA backlog, so if it doesn't in turn help (by providing points) then I suggest that GA points should only be awarded if the nominator also provides a quid pro quo GA review; so in essence GA points are only awarded for 1 GA + 1 GAR. Otherwise the GA backlog is going to get even more out of control than it already is. Harrias talk 16:05, 18 November 2017 (UTC)
I don't think you should only get points for a GA if you also review a GAR. I've done two and a half times the amount of GARs than I have submitted GAs.  — Calvin999 23:54, 18 November 2017 (UTC)

─────────────────────────One thing about GARs, though, is that it is the only category where you can get points on your own schedule. This was my third year participating and the first two times I was eliminated despite having enough points worth of nominations to get me to the next round (and it wasn't like I nominated them at the last minute). This year I was eliminated under the same circumstances, except that I made a few nominations in the last couple of weeks before the end of a round. And yes, I've included all noms in Wikipedia:WikiCup/Reviews Needed. GARs are a way for many editors to earn their way to the next round, instead of leaving them at the mercy of waiting for reviews.

I would much rather see the abuse of GARs for points addressed, rather than eliminate GARs entirely. Things that can be addressed would include specifying what should be included in a GAR that can receive points (even if a shorter review would pass at GAR) and stop awarding points for quick fails and any fail that doesn't allow a set period of time for the nominator to respond (see this discussion), eg. minimum 7 days and require reviewers to act in good faith to extend that time if issues are being addressed by the nominator. Requirements for passing should include 1) requiring reviewers to use a template to make sure all GA criteria are met 2) capping the length of comments on issues with prose that satisfies the length requirement (current criteria is simply: "As a rough guide, no review shorter than 1000 bytes will be considered") because I find a lot of reviews pad the length with nitpicking the quality of the prose and even addressing issues that are not part of the GA criteria. [Note: I'm busy and may not respond to responses for a few days. Sorry in advance.] AHeneen (talk) 10:28, 19 November 2017 (UTC)

The Cup places a huge burden on the GA review process and without awarding points for GARs, the list of reviews needed would be at least twice as long as it already is, which I would find insupportable. I pushed for points for GARs back in the day because I wanted the problem to go away and I'm pleased that you guys have responded very well as we've reviewed more than we've submitted for at least the past two years.
I'm more concerned about crappy reviews being accepted, but that's on us as the judges. I don't mind people pointing those out to us, but I think it would be more productive for contestants to focus on their own work instead.--Sturmvogel 66 (talk) 16:13, 19 November 2017 (UTC)
I can't say I've looked through a significant amount of other people's reviews, but I do think the concerns about gaming are overblown. Instead of eliminating points for GAR, I'd prefer to see the addition of points for comments on FACs. I've had more than one fail due to lack of participation. Argento Surfer (talk) 19:30, 21 November 2017 (UTC)
Points for chiming in at FACs? Yikes, that's a slippery slope if ever I've read one. DARTHBOTTO talkcont 20:56, 23 November 2017 (UTC)

I think encouraging GARs is good to help reducing the GAR backlog. Even during this year when GARs are rewarded, my GA nominations often took months before being looked at. True that GARs are somewhat more prone to abuse than other submission types, so in order to not attract abuse, we should keep the score low like this year. I also believe there should be more scrutiny on GAR submissions, including judge's scrutiny as well as the judges' willingness to act on complaints filed against low quality GARs. HaEr48 (talk) 05:47, 28 November 2017 (UTC)

Then, perhaps WikiCup should have one more judge, specifically designated for monitoring the intake of GARs. This is by far the most disruptive side effect of the competition and most of the input on this entire page is about how things with it need to change. DARTHBOTTO talkcont 04:47, 29 November 2017 (UTC)
Per this edit below, I don't think the issue is a lack of judges, but a shortage of active judges in 2017. Argento Surfer (talk) 13:43, 29 November 2017 (UTC)

Judge's discretionEdit

The judges have to look at all submissions to make sure they qualify. The vast majority do, but there are occasions when they do not. One of these concerns whether a GAR is of sufficient quality: the rules state "Only reviews of a sufficient length will be counted; quick fails and very short reviews will generally not be awarded points." As a judge, this is not something I care for deciding. I particularly dislike someone else, perhaps another contestant, coming along and telling me that a GAR is inadequate and should not qualify. I would prefer to accept all GARs, and if people have concern that a contestant is reviewing inadequately, they should bring the matter up at the appropriate (GA) discussion board. Cwmhiraeth (talk) 11:15, 18 November 2017 (UTC)

Cwmhiraeth The problem is in several of the years this did not happen. It definitely got better in recent years though, being patrolled more. I was first aware of it when someone did a review of my nomination and the 15 points weren't actually things for me to change, just commenting on how nicely formatted things were. That shouldn't be allowed, it's just waffle to look like a long review, when you actually read it, nothing was needed to be changed. I then saw that the editor had amassed about 50 GARs he was collecting points for, I went through them and emailed a judge about which ones I felt were being wrongly claimed, and the judge struck off about half of them for claiming reviews which were too short and/or not critically constructive enough to warrant any changes by the nominator. It is a minority thing, most people like me do (did) claim for worthy reviews. I often spent up to an hour reviewing just one nomination, and I'm only online for a matter of single digit hours a week doing editing as well, so it's not really like I can even put the time in to get loads of reviews done, that's why I don't think points should be scrapped because people like me actually work really hard on them. Another reason why it went on was because obviously people don't want to write on a judge's talk page or a GA discussion publicly calling out someone for cheating, email is the only way really.  — Calvin999 15:02, 18 November 2017 (UTC)
I'm more comfortable reviewing people's reviews than I think Cwmhiraeth is, but it's not one of my favorite things to do as it can be a real time suck trying to decide whether a review is acceptable or not. And sometimes it's a pretty fine line between the two. Maybe we (us judges)'ll have to have an internal discussion about some changes in how we handle them. In my experience, crappy reviews are really only a problem in the first round or two as newbies try to game the system.--Sturmvogel 66 (talk) 16:19, 19 November 2017 (UTC)
That's why I think it's a good idea to impose conditions beforehand, such a certain length requirement in the form of bullet points where each point is not a continuation of the former, so each one is a different issue that needs improving. Or a certain byte size. Also perhaps no 'positive' bullet points, all points need to be critical and constructive. Highlight any positives in a summary at the end which doesn't count towards the review length etc. I'd be happy to do it, I've done it unofficially in the last few times I competed anyway.  — Calvin999 12:09, 21 November 2017 (UTC)
The problem with a strict "minimum" is that when you honestly come across a well-written article, you would be penalised, and not be able to count that review. I think just muddling along is best. After all, 90% of GANs are 90% of the way to being a GA already. Without feeling overly guilty, I could pass half of the current nominations without any suggestions. Sure, there are improvements that could be made, but most of them already meet the GA criteria, technically. Harrias talk 19:12, 21 November 2017 (UTC)
If an article is well-written, then the review you conduct wouldn't be worthy of submitting for points anyway, as there wouldn't be anything or hardly anything to change to write a lengthy review for. With my experience of having reviewed 256 GANs or something, I've actually found that 90% are not ready to be passed without suggestions. It is worrying that you think that such a high percentage is instantly passable. I've issued countless instant fails, but I think only two or three of the 256 I've reviewed have been instant passes, and by instant there have still been about three bullet points for corrections which they've done straight away. There is no nomination that is exempt from at least one point of correction, ever, not even my nominations.  — Calvin999 10:18, 22 November 2017 (UTC)
Cwmhiraeth I think GARs are different from other types of submissions because no one else is checking whether the GAR is "good", unlike FA, GA, or DYK which has to be checked by other editors before passing. Therefore, I think judges should scrutinize it more than other submission types. It's also easy to abuse, e.g. someone rushing a lot of low-quality reviews to get extra points. I believe there was an example like that at the end of one of the 2017 rounds, someone prematurely failed a GAR so that they got their 4 critical points in time. For this reason, I think "someone else coming along and telling me that a GAR is inadequate" is a good thing, because having to look at so many submissions, it's easy for judge to miss these abuses, and the "people coming along" provide extra eyes. The fact that they're other contestants gives them the right incentive to be critical, although maybe not the incentive to be fair. HaEr48 (talk) 05:38, 28 November 2017 (UTC)
Most GARs were done in good faith and were perfectly satisfactory, but not all decisions on their adequacy are clear cut. That's why a panel of judges is best, and during 2017 that system broke down when real life issues caused two of the judges to be largely unavailable and I had to make decisions on my own. Cwmhiraeth (talk) 06:40, 28 November 2017 (UTC)

Weightage to FPOsEdit

Since we are having fewer FPOs from Wikicup, I think something should be done to encourage it. Not only FPOs, but also other categories which receive less attention. As far as I have seen, even FLs have been given less importance, as there are very few FLs made. Perhaps I could be wrong, so just an opinion. I am not familiar with these two categories, but making FPOs seem difficult. Maybe not point-wise, but if any other thing could be done to encourage more contribution to categories which are given less weight would be great. Adityavagarwal (talk) 17:46, 19 November 2017 (UTC)

More scrutiny neededEdit

I find myself in the unusual position of disagreeing with Cwmhiraeth's point above, not just about GARs, but all submissions. The strength of the Wikicup is that it provides an incentive to improve content. Its corresponding flaw is that it provides an incentive to cut corners to get points more quickly. I was generally happy about the promoted content that I looked at during the course of the cup. But, there were a number of occasions on which I discovered copyvios, content which failed verification, and inappropriate use of sources. This is less of a problem with featured content, which receives intense scrutiny from multiple editors, but it is certainly a problem with GAs, and even more so with GARs. I do not think it is okay for us to brush these off as problems with those processes that need to be sorted out elsewhere. At the same time, I do not think it fair to pile more work on the judges. Therefore, I'd suggest the following: a panel of secondary judges, whose only job is to check submissions. I am in no way suggesting that all submitted content should be perfect; but at the very least, it should be compliant with WP:V, WP:BLP, and free of copyvios. A team of judges should not find it terribly onerous to perform spotchecks for most contestants. If this idea gains favor, I would be willing to serve on this team, as I am not likely to contest next year. Vanamonde (talk) 11:32, 21 November 2017 (UTC)

Whether it's a panel or not, things need to be tighter. Pages like Counter Logic Gaming were rushed through, only because people nominating and people reviewing were eager for points. DARTHBOTTO talkcont 20:59, 23 November 2017 (UTC)
@DarthBotto: Thanks for chiming in. I (obviously) agree that this is a problem; but if there are alternative solutions to a panel checking submissions, I'd be happy to hear and discuss them. Regards, Vanamonde (talk) 11:34, 24 November 2017 (UTC)
@Cwmhiraeth, Godot13, and Sturmvogel 66: I'd really like to hear your thoughts on this, as the alternative to my suggestion would be folks concerned about quality sending all our concerns to you three. Regards, Vanamonde (talk) 05:12, 29 November 2017 (UTC)
Well, the obvious solution would be for you to join the judging panel. Cwmhiraeth (talk) 06:04, 29 November 2017 (UTC)
@Cwmhiraeth: I'm hesitant for a simple reason: for the first part of the year my wiki-presence will be sporadic. I will be around, but I may not be able to predict when or for how long. Thus I'd feel able to check submissions; but not always to fulfill the other responsibilities of the judges with respect to setting up rounds, passing submissions listed by the bot, answering questions/addressing conflicts on the talk page, and so forth. If these issues are not a big deal, and if the judges are able to reach a consensus that we shouldn't accept submissions that are not compliant with WP:V, WP:BLP, and copyright (as independently checked by the panel, at least in some cases) then I would be happy to join. Regards, Vanamonde (talk) 06:31, 29 November 2017 (UTC)
My major concern would be your limited availability at the beginning of the year when we typically have the biggest problem with substandard articles and GARs as newbies haven't fully grasped the requirements. Having you focus solely on GARs would be fine, IMO, as that would allow the rest of us to concentrate on the other submissions.--Sturmvogel 66 (talk) 16:42, 1 December 2017 (UTC)
@Sturmvogel 66: I understand your concern: but there's little I can do about my availability. If you believe that is too much of an issue for me to join the judging panel, that's perfectly reasonable, and I will not lose sleep over it. What I can promise is that I will be around for a minimum of two weeks during a given round. Regardless of whether I become a judge, I fully intend to check submissions when I can, and not just GARs. Some of the most serious concerns I had this time were with GAs that had verifiability and copyright problems. I understand that GARs are a common source of trouble, and I would be happy to check them as far as I am able. I think it might help to say that in addition to being a certain length, all GARs must explicitly check off the criteria. Just a though. Regards, Vanamonde (talk) 18:18, 1 December 2017 (UTC)

Bonus pointsEdit

I noticed that the Bonus points section says December 2017. Should have be updated to 2018? Guettarda (talk) 13:12, 20 February 2019 (UTC)

Indeed, it should have been updated. I have done it now. Cwmhiraeth (talk) 13:47, 20 February 2019 (UTC)
Return to the project page "WikiCup/Scoring".