Setting DYKUpdateBot to run only twice a day: it's time

Shubinator, we've been having a discussion at WT:DYK#Crisis looming about going to twice a day, and I think we're finally there: when I first suggested it on July 7 at about this time we had 140 nominations; while there was an increase during the day to 150 hooks, we're now down to 127, and we have seven empty queues and preps which could take up 49 of these hooks ... or would if we had that many approved.

Can you make the necessary changes? We typically do the twice-a-day hooks at noon and midnight UTC, so if you get to this after the 0800 update, you may need to wait until the after the 1600 update goes up and let the twice a day start with midnight. Thank you very much. BlueMoonset (talk) 06:13, 9 July 2013 (UTC)

Orlady took care of the changeover; as it happens, the hooks are at 08:00 and 20:00 UTC, which I can't imagine would be a problem: 12 hours is 12 hours. Hope you're having a great time, wherever you are! BlueMoonset (talk) 14:22, 10 July 2013 (UTC)
Glad to hear Orlady took care of it. The bot will drift the update times over to 00:00 UTC and 12:00 UTC. (And sorry for not being around the past couple of days.) Shubinator (talk) 14:47, 10 July 2013 (UTC)
Yeah, people would think that you have a real life or something. SL93 (talk) 15:11, 10 July 2013 (UTC)

DYKUpdateBot late

Shubinator, as far as I can tell, the DYKUpdateBot should have run at 21:00 UTC, and didn't; it's currently 21:35 as I write this. Can you please see what's up with the bot? Housekeeping seems to be running properly, or was as of 20:49; there wasn't anything to provoke at 21:19 update, and we won't know for sure until 21:49 update runs and does (or doesn't) reflect the 21:20 addition. Thanks! (Note: I'll be posting to WT:DYK in case there's an admin who can update by hand. BlueMoonset (talk) 21:38, 12 July 2013 (UTC)

Recovery code kicked in for the 12:00 (UTC) update. --Allen3 talk 12:05, 13 July 2013 (UTC)

DYKUpdateBot missed 12:00 15 July 2013 (UTC) update

FYI, DYKUpdatebot failed to perform the 12:00 15 July 2013 (UTC) update. A manual update has been performed. ‎DYKHousekeepingBot performed an update at 11:51 15 July 2013 (UTC), so this does not appear to be a toolserver outage. --Allen3 talk 12:46, 15 July 2013 (UTC)

DYKUpdateBot has just missed the 00:22 16 July 2013 update. Looks like the recovery code failed to perform a restart. Could you look into giving the bot a kick when you get a chance. --Allen3 talk 00:39, 16 July 2013 (UTC)
Well that's a new one. DYKUpdateBot was behaving oddly (not outputting to logs), but recompiling it fixed it. Which is quite weird. Anyways, should be back to normal now. Shubinator (talk) 05:14, 16 July 2013 (UTC)

Funny glitch

[1]. Materialscientist (talk) 00:07, 18 July 2013 (UTC)

Looks like a side effect from recent events involving Dr. Blofeld. Earlier today Blofeld redirected his user talk page.[2] Since then, Theo's Little Bot (talk · contribs) has twice added things to the redirect([3][4]) instead of following the redirect and editing the new user talk page. DYKUpdateBot appears to be reading the entire message and then trying to use it as an alternate user talk page name. --Allen3 talk 00:36, 18 July 2013 (UTC)--Allen3 talk 00:36, 18 July 2013 (UTC)
Should we pull all DYKmake templates with Dr. Blofeld from current queues and preps, noting that we're only doing so because he has closed his talk page, and the redirect is breaking the bot? Or is this a harmless glitch that doesn't stop the bot from finishing its task, in which case it's merely annoying, and we don't need to take any specific action on those templates? BlueMoonset (talk) 01:19, 18 July 2013 (UTC)
Credit templates with bad user names is a moderately common error. When the bot encounters one it logs an error and moves on to the next action. The difference between this and past instances is the unusually long "user name" caused by the redirect. DYKUpdateBot completed the update that generated the error and then cleared the message from the error page during it's next status check (the bot's normal behavior). --Allen3 talk 01:47, 18 July 2013 (UTC)
Allen3's right, the weirdness won't stop the bot from updating. The negative side effects are 1) the user doesn't get credited automatically, and 2) noise at DYKUpdateBot/Errors. Having said that, I see the bug in the code and can fix it, if you folks think it's worth fixing. Shubinator (talk) 02:52, 18 July 2013 (UTC)
As no one has spoken, I will through my support behind fixing the bug. There are still a number of DYK nominations from Dr. Bloefeld that have not reached the Main page and it would be nice to avoid the negative side effects. --Allen3 talk 11:51, 20 July 2013 (UTC)
Done. I just noticed that DYKHousekeepingBot isn't following redirects, so I've fixed that too. We should see the new redirect code at work on the July 21st 12:00 UTC update. Shubinator (talk) 16:06, 20 July 2013 (UTC)

DYK RfC

  • As a listed DYK participant, you are invited to contribute to a formal Request for Comment on the question of whether Good Articles should appear in the Did You Know? slot in future. Please see the proposal on its subpage here, or on the main DYK talk page. To add the discussion to your watchlist, click this link. Thank you in advance. Gilderien Chat|Contributions00:23, 1 August 2013 (UTC)

DYK image protection

I know this is not a DYKUpdateBot task, and you don't have to look into this, but it would be great if you did (as well as talk page stalkers, particularly Allen3). Most DYK images are protected by KrinkleBot on Commons. I rarely check that, but when I do, I often find that the bot removes a DYK image from the protection cascade shortly after it is posted on en.wiki main page, thereby fooling DYKUpdateBot (and the DYK promoting admin). I've posted about it to Krinkle in January (here [5] is a link, but ignore it in favor of next one), and only today got a reasonable reply on the failure mechanism [6]. I was hoping that someone knowledgeable could suggest to Krinkle how to avoid this recurring problem. Materialscientist (talk) 06:11, 1 August 2013 (UTC)

My suspicion is that this problem is being caused by delays in updating the Help:What links here information related to pages containing transcluded templates. A few weeks ago, I noticed that (while logged into the English Wikipedia) flushing the server cache for Wikipedia:Main Page/Tomorrow would allow KrinkleBot to protect the image for a newly promoted DYK set on its next scan. Previous to this discovery it normally took KrinkleBot between 30 minutes and 6 hours to notice a change. With this insight there are two courses of action that spring to mind. The first is to have KrinkleBot request a purge of the server cache as part of it's normal scans. As the bot does its work on Commons, this may or may not effect pages that reside on the English Wikipedia and are being accessed by a cross-wiki link. The second possibility is for DYKUpdateBot to provide an assist by forcing a server cache purge at the end of an update. The second option is less preferable as it is tangential to DYKUpdateBot's purpose and would only benefit one of the language Wikipedias that KrinkleBot works with. --Allen3 talk 10:49, 1 August 2013 (UTC)
Have another data point. ITN had an image swap shortly before 19:00 (UTC) today. I noticed the change 10 to 15 minutes after it was made, loaded a browser tab with a copy of the Main page, and attempted to flush the server cache. All this was completed by 19:16 (UTC). KrinkleBot did not notice the new image until 20:18 (UTC) (the bot normally runs at 3, 18, 33, and 48 minutes past the hour). Apparently my trick to get a new image noticed at Main page/Tomorrow does not work when applied to the Main page. --Allen3 talk 23:27, 2 August 2013 (UTC)
We can't have a bot to continuously purge a page. I was hoping there is a more reliable way for a bot to check an image presence on a page than the imageusage database. Materialscientist (talk) 02:56, 3 August 2013 (UTC)
One possible way to bypass the imageusage database is to take a copy of the Main page source code (or other page with images to protect), run it through Special:ExpandTemplates, and then search the results of the expanded source for links to either the File or Image namespaces. This process works when performed manually. I will let one of our bot operators tell us how hard it would be to code. --Allen3 talk 03:21, 3 August 2013 (UTC)
Odd. DYKUpdateBot has always purged the Main Page right after updating it, I wonder if a MediaWiki change is causing this now? Regardless, parsing the Main Page itself is definitely an option. However, because the bot won't be using the same code as the MediaWiki parse engine, the bot's parse code will probably have to be fine-tuned over time (like DYKUpdateBot's). The first iteration of the parse code shouldn't take much time; it's relatively simple. Shubinator (talk) 05:30, 5 August 2013 (UTC)

00:00 20 August 2013 (UTC) DYKUpdateBot update

FYI,

Looks like there have been some issues with the toolserver. DYKUpdateBot missed the 00:00 20 August 2013 (UTC) update (manual update has been performed). Additionally, DYKHousekeepingBot last updated Wikipedia:Did you know/DYK hook count at 14:31, August 19, 2013 (UTC). More interesting was the 16:01 (UTC) update to the hook count by an IP that matches the address for willow.toolserver.org. If you get the chance, please check both bots. I suspect that at least one needs to be restarted. --Allen3 talk 00:49, 20 August 2013 (UTC)

Thanks for the heads up, I've restarted DYKHousekeepingBot. DYKUpdateBot's logs show it was having issues, but it looks like the bot recovered properly. Shubinator (talk) 01:49, 20 August 2013 (UTC)

Cleanup of uploaded images by DYKUpdateBot

I have noticed that Category:Uploaded from Commons main page images has been accumulating a number of old {{C-uploaded}} images from DYK. Examples include File:George St Lo.jpg, File:Mill church.jpg, and File:San Francisco Civic Center Historic District 09.jpg. When your schedule permits, could you look into why DYKUpdteBot did not delete the local copies of these images when their time on the Main page was completed? All current examples where uploaded by a relatively new DYK admin, so it is possible that they are somehow different from previous C-uploaded images (differences in white spaces inside the file names perhaps). The bot was however able to locate the images well enough to add a {{DYKfile}} to all of the image description files. This suggests the bot should be able to identify the file name well enough to delete the local copy of the uploaded image. --Allen3 talk 12:10, 27 August 2013 (UTC)

Those files are tagged with {{c-upload}} instead of {{c-uploaded}}; currently the bot only recognizes the latter. The bot is finding the files, but it doesn't think they're marked as temporarily uploaded for the Main Page. Shubinator (talk) 05:56, 29 August 2013 (UTC)

00:00 29 August 2013 (UTC) DYKUpdateBot update

FYI,

DYKUpdateBot missed the 00:00 29 August 2013 (UTC) update. Additionally, ‎DYKHousekeepingBot last performed an update at 20:24 28 August 2013 (UTC) even though there have been modifications to the nominations page since that time. Could you check things when you get a chance? --Allen3 talk 02:22, 29 August 2013 (UTC)

DYKUpdateBot's automatic reset code kicked in, and it looks like it has made a full recovery. DYKHousekeepingBot was not faring as well, so I restarted it. Thanks! Shubinator (talk) 05:54, 29 August 2013 (UTC)
Hmmm, this is looking more complicated than I thought. DYKHousekeepingBot is still erroring out when attempting to edit Wikipedia, and I wouldn't be surprised if DYKUpdateBot does the same. Feels related to the Mediawiki update that just rolled out. Shubinator (talk) 06:06, 29 August 2013 (UTC)
Figured it out, we're back in business with DYKHousekeepingBot editing without issues :) Shubinator (talk) 06:14, 29 August 2013 (UTC)
DYKUpdateBot just missed the 12:00 29 August 2013 (UTC) update. Looks like there may still be a problem. --Allen3 talk 12:12, 29 August 2013 (UTC)
Restarted DYKUpdateBot, it's now editing without problems. Hopefully that fixed things... Shubinator (talk) 15:21, 29 August 2013 (UTC)
Thanks. --Allen3 talk 15:32, 29 August 2013 (UTC)