Wikipedia:Bots/Requests for approval/The Anomebot2
- The following discussion is an archived debate. Please do not modify it. Subsequent comments should be made in a new section.
A long time ago, I ran User:The Anomebot, implemented in Python, which was one of the first Wikipedia bots. I now want to register a new bot account for running a simple bot, based on the Wikipediafs Debian package and a simple Python editing script, that will add geotags to allready-existing geographical articles without geotags. As part of this work, I have compiled a list of just under 10,000 new geotags, which has already undergone spot checks for validity. The bot will check for existing tags during its run, and will not attempt to replace any existing tags.
The following data is the result of taking Wikipedia's category links and the public domain NIMA GNS data, and rubbing vigorously. With quite cautious checks applied to both datasets, this gives an unambigious location for 12660 out of a possible 28628 (44%) articles about non-US cities, towns and villages.
The results are sorted by country, then place, and binned into four files. They have also been compared to the data in Koordinaten_en_CSV.txt, and labelled by whether they are new coordinates (NEW), or duplicate coordinates already in articles (dup), and, if so, whether they are exact duplicates, or if not, roughly how many km out they are. (The distance calculation uses several approximations, so treat it only as an order-of-magnitude figure).
- User:The Anome/Geodata 1: countries A-F
- User:The Anome/Geodata 2: countries G-I
- User:The Anome/Geodata 3: countries J-M
- User:The Anome/Geodata 4: countries N-Z
Where this data differs from the existing Wikipedia data, the new data has been found to be correct in almost every case: where the data was in error because of bugs in the list compilation, I fixed the bugs, and regenerated the output data to remove any similar errors.
The bot is intended to run at a limited edit rate, and is intended to self-check its edits independently of Wikipediafs, and to stop if its edits are not saved, or differ in any way from the intended content: it should thus automatically stop if blocked, or if the Wikipediafs or the Wikipedia servers are malfunctioning. -- The Anome 02:16, 11 August 2006 (UTC)[reply]
- One week trial period approved, please limit speed to no more than 2 edit/min; and limit the test run to 100 pages or less. — xaosflux Talk 16:05, 12 August 2006 (UTC)[reply]
- Thank you. I've now completed testing this bot: I've checked a number of possible error scenarios, including restarts and auto-shutoff, and it seems to be working OK. -- please see [1] for the results. -- The Anome 11:35, 13 August 2006 (UTC)[reply]
- Following Eugène van der Pijll's feedback, I've now made several more improvements, with a number of other filters being used to suppress edits in cases where geotags are already present either directly or through transclusion, and better placement of the tag within the article. I think I'm ready to run. I'd like to perform another limited test run of 100, just to make sure everything works OK, if you can approve that. If manual review of those 100 edits shows no problems, I think I'll be ready to run on the full dataset, subject to approval. -- The Anome 15:29, 13 August 2006 (UTC)[reply]
- Additional 100 article run is OK, please post results here. — xaosflux Talk 15:59, 13 August 2006 (UTC)[reply]
I've now done another few small test runs, the most recent of which started at 23:44 with Aparan, and ended at 00:36 with Berry, New South Wales (with a slight tweak to the edit comment string in the middle for a couple of entries). The bot seems to be working OK in this run:
- Articles with categories, interwikis and Unicode characters are handled OK, and the tag is added in the correct place: see Goris (as a side effect, the category and interwiki links are also sorted into the standard order after all other page content, and are sorted in alphabetical order)
- Articles marked with a variety of pre-existing geodata template styles seem to be caught now: see Augusta, Western Australia, and Yerevan for examples.
- Restart is working OK
- Blocking the bot shuts it down as soon as it detects the write error
- New feature: as requested by Eugene, the geodata tags now have region and feature codes, for example {{coor title dm|34|47|S|150|42|E|region:AU_type:city}}
Please let me know what you think, and whether you would like me to do any more testing. -- The Anome 23:46, 13 August 2006 (UTC)[reply]
Update: I've regenerated the input dataset using the latest dump data, and performed a few more bot edits to validate the new list, staying within my existing test allowance. I now have geodata available for more than 15,000 towns, cities and villages alone. I can easily do the same for other classes of geographic features later, using the same bot, and the same data-generation code, just by changing the GNS and category filtering parameters. Please let me know if/when I can proceed with adding the city data. -- The Anome 13:14, 18 August 2006 (UTC)[reply]
Looks very solid to me -- permissions is granted - do you want a botflag? -- Tawker 15:30, 18 August 2006 (UTC)[reply]
- Thanks! No, I'm fine without a botflag for now, as I'm only making one edit every 30 seconds. There's lots more geodata goodness coming: I've been working on on modularizing the data preparation, in readiness for being able to process mountains, hills, railway stations, mine workings, woodlands, etc, etc. at a later date. -- The Anome 15:27, 19 August 2006 (UTC)[reply]
- The above discussion is preserved as an archive of the debate. Please do not modify it. Subsequent comments should be made in a new section.