Module talk:Footnotes/Archive 1

This doesn't work properly in Template:Sfn

Please see Template talk:Sfn#Lua version is broken. --Redrose64 (talk) 22:43, 2 April 2013 (UTC)

Broken parameter

Please see Template talk:Harvard citation no brackets/Archive 1#Broken parameter. Similar templates are probably similarly broken. Wtmitchell (talk) (earlier Boracay Bill) 07:13, 28 April 2013 (UTC)

This should be fixed. Dragons flight (talk) 10:18, 28 April 2013 (UTC)

More than six parameters

It would help if the code was more compatible with CS1. At the moment it does not handle more than four authors elegantly. See Module talk:Citation/CS1#CITEREF and harvnb and CITEREF and CS1. -- PBS (talk) 13:09, 21 December 2013 (UTC)

Markup

The Lua CS1 templates strip markup, such as bold or italics, where this module does not. Thus if markup is used in {{sfn}} or variant, then the link and anchor will not match. --  Gadget850 ^talk 11:28, 6 March 2014 (UTC)

Protected edit request on 27 June 2014

Module:Citation/CS1 runs mw.uri.anchorEncode() on options.id before generating the <span id="... line. For compatibility between generated id in that module and the #CITEREF generated here, I think this change needs to be applied on this module.">

(Here is an example footnote which is not linked to it's citation and I think it's because of the above mentioned problem.) Dalba 07:38, 27 June 2014 (UTC)

Done Thanks for the fix! — Mr. Stradivarius ^{♪ talk ♪} 12:15, 27 June 2014 (UTC)

Template:sfnp

{{sfnp}} could usefully be implemented within this module as well as {{sfn}}. As far as I can see, function f.sfnp( frame ) would simply duplicate function f.sfn( frame ) with the addition of:

args.bracket_year_left = "(";

args.bracket_year_right = ")";

An oddity at present is that |postscript=none works for {{sfn}} but not for {{sfnp}}. Peter coxhead (talk) 20:21, 15 December 2014 (UTC)

It does in a sort of round-about way:

{{sfnp}} calls {{Harvard citation/core}} calls Module:Footnotes function harvard_core().

Is |postscript=none necessary for {{sfnp}}? The code seems to indicate that {{sfn}} uses '.' for a default of postscript character. If |postscript= is empty or omitted, then {{sfnp}} uses an empty string as the default postscript.

—Trappist the monk (talk) 20:38, 15 December 2014 (UTC)

Nope, {{sfnp|James|2004}} generates the following footnote.

(After the tweak noted below, {{sfnp|Smith|2005|ps=none}} generates the following footnote.)

References

So as a CS2 user, I end up with {{sfnp|ps=|...}} but would prefer |ps=none for same reason as you (well, actually I'd prefer another template, and did once create one, but the consensus was to delete it).

Doing things in a "round-about way" is a sure recipe for software errors (at least that's what I used to teach in first-year software engineering). The complexity of the current code is shown by the fact that it isn't easy to work out what {{sfnp}} actually does. A separate function is clearer and easier to maintain. Or perhaps better still (because more modular), pass appropriate parameters to f.sfn(). Peter coxhead (talk) 21:13, 15 December 2014 (UTC)

Right, it gets it's postscript from the template call to {{Harvard citation/core}}. Module:Footnotes tweaked and I tweaked your example.

You'll get no argument from me in support of round-about code. Every line must be maintained. I'll think about integrating {{sfnp}} in the module.

—Trappist the monk (talk) 22:02, 15 December 2014 (UTC)

Ok, tweaked {{sfnp/sandbox}} to pass the bracket defaults as |bracket_year_left=( and |bracket_year_right=) to f.sfn() in Module:Footnotes/sandbox where those parameters override the defaults normally used by {{sfn}}:

Some text using {{sfnp}}.

Some text using {{sfnp/sandbox}}.

Some text using {{sfnp/sandbox|ps=none}}.

References

—Trappist the monk (talk) 13:08, 16 December 2014 (UTC)

A much more elegant approach than the current version; hopefully soon to be deployed. Peter coxhead (talk) 18:56, 16 December 2014 (UTC)

Done.

—Trappist the monk (talk) 00:38, 17 December 2014 (UTC)

Thanks! Peter coxhead (talk) 16:54, 17 December 2014 (UTC)

Dashes and hyphens, oh my

A number of threads have shown up on WT:VPT (e.g. WP:VPT#Help in fixing a "cite error") and elsewhere regarding the change in MediaWiki to output an error where the same names are used in a reference but where the reference is defined differently.

It seems to me that a basic level of checking for that could be implemented in this module, either (namely) to treat (example) |p= and |pp= the same before outputting the reference for Template:sfn or to do some basic level of sanity checking e.g. where |p= uses a dash/hyphen (and should not) or where |pp= doesn't use one of the set: (,|-) (or similar).

Thoughts? --Izno (talk) 17:17, 14 November 2015 (UTC)

Only that there's a need to be careful not to exclude electronic publications which increasingly have "page" numbers with non-numerical characters in them. Peter coxhead (talk) 17:26, 14 November 2015 (UTC)

My suggestion would be only to look for certain punctuation. --Izno (talk) 17:40, 14 November 2015 (UTC)

And then do what once the detector detects whatever it is set to detect?

This is difficult. |p=A-1 or |p=17-6 are legitimate page numbers.

—Trappist the monk (talk) 18:10, 14 November 2015 (UTC)

Precisely. Peter coxhead (talk) 22:05, 14 November 2015 (UTC)

Auto-formatting dashes in parameter "pages"

Module:Citation/CS1 based templates automatically replace hyphens (-) with endashes (–) when hyphens are typed in the |pages= parameter. Is it possible to implement this feature into this module?
Αντιγόνη (talk) 07:24, 28 September 2016 (UTC)

Yes, we can do that here if there is sufficient support for it. If we do add it, I would like to see us adopt some sort of mechanism that allows editors to override the auto-conversion. Elsewhere I suggested a couple of ways that this might be accomplished. Those proposals were not taken up.

—Trappist the monk (talk) 10:40, 28 September 2016 (UTC)

So what would happen with something like |pp=A-1–A-15? Entirely possible with electronic publications. Peter coxhead (talk) 14:15, 28 September 2016 (UTC)

Which is why I suggested |pp=((A-1–A-15)) or |pp=A--1-A--15 as ways to tell the module to 'do it this way': pp. A-1–A-15.

—Trappist the monk (talk) 14:48, 28 September 2016 (UTC)

Well, either there has to be some special mechanism, as you suggest, or no automatic replacement, which I prefer. Peter coxhead (talk) 15:24, 28 September 2016 (UTC)

Last change broke date=n.d.

Please add a condition in function core to avoid adding the ampersand before the date when date=n.d.: {{harv|Author|n.d.}} results in (Author n.d.). 72.43.99.146 (talk) 00:30, 18 October 2016 (UTC)

Amendment: even though it should be obvious anyway, I should have stated that any iteration of an undated reference gets similar treatment ("undated", "no date" etc.) since the module thinks that these are author names. 72.43.99.146 (talk) 00:40, 18 October 2016 (UTC)

test cases

Hi. I've just run the test cases against some small changes to the code in the sandbox, with lamentable results. I reverted my changes and the test cases still fail. Is this expected? Alternatively, should I copy the main code into the sandbox to ensure that the sandbox code is up to date? I don't want to damage anyone else's work. Regards, Martin of Sheffield (talk) 22:40, 21 August 2017 (UTC)

The {{sfn}} tests have never 'passed' as far as I know because two {{sfn}} templates with identical input parameters must be separately distinguishable from each other else the back-links won't work. Two instances of the same template:

{{sfn|brown|red|orange|2017}}

produce:

<sup id="cite_ref-FOOTNOTEbrownredorange2017_1-0" class="reference"><a href="#cite_note-FOOTNOTEbrownredorange2017-1">&#91;1&#93;</a></sup>

<sup id="cite_ref-FOOTNOTEbrownredorange2017_1-1" class="reference"><a href="#cite_note-FOOTNOTEbrownredorange2017-1">&#91;1&#93;</a></sup>

I suspect that the values used in the comparison made by the test code are the values of two stripmarkers:

?'"`UNIQ--ref-0000000A-QINU`"'?

?'"`UNIQ--ref-0000000C-QINU`"'?

Stripmarkers are replaced with actual content long after the template and module code has run.

—Trappist the monk (talk) 09:48, 22 August 2017 (UTC)

Page needed

Have recent changes to this template caused the use of {{page needed}} as a value for the parameter "p=" caused a break. If so it has broken many pages. See for example this edit (Revision as of 09:56, 2 February 2017) mentioned in this section Talk:Second Boer War#Fixing style/layout errors.

• {{sfn|Haydon|1964|p={{page needed|date=February 2017}} }}

References

If so please revert the and discuss how to fix it, as the change as this will have broken many pages, and the fix of placing page needed outside the short citation is not appropriate. -- PBS (talk) 20:03, 7 December 2017 (UTC)

No recent changes. Last change was 18 September as you can see from the history. What is interesting is that if you were to rewrite that example without a date, it appears to function correctly. {{sfn|Haydon|1964|p={{page needed}} }}

References

Something mysteriously wrong with {{page needed}}?

—Trappist the monk (talk) 21:20, 7 December 2017 (UTC)

{{Page needed}} needs a date (a bot will added it). It seems to work now, so (as always) as we do not use version numbers the broken text no longer shows an error. So for the record/archive what was fixed? -- PBS (talk) 19:54, 10 December 2017 (UTC)

This; described here.

—Trappist the monk (talk) 03:13, 11 December 2017 (UTC)

Thanks. For the record now the above is now archived: Wikipedia:Village pump (technical)/Archive 161#Either Template:Sfn or some subtemplate of it appears to be broken. -- PBS (talk) 18:41, 18 December 2017 (UTC)

Logic error in handling of "n.d."

The conditional statement in lines 70-72 ("prevent double periods when date is 'n.d.'"). It forces |ps=none ("then args.postscript = ''"). Please fix. 108.182.15.109 (talk) 13:26, 19 June 2018 (UTC)

The error is harder to see because it is only visible when the in-source location is absent. 108.182.15.109 (talk) 13:33, 19 June 2018 (UTC)

Can you provide a clear example of what you mean?

—Trappist the monk (talk) 13:41, 19 June 2018 (UTC)

{{sfn|Author|2018|ps=, postscript}}

{{sfn|Author1|n.d.|ps=, postscript1}}

65.88.88.127 (talk) 20:33, 19 June 2018 (UTC)

fixed

—Trappist the monk (talk) 11:14, 22 June 2018 (UTC)

Nice, thank you. 72.43.99.138 (talk) 15:43, 23 June 2018 (UTC)

consolidating and abandoning Template:Harvard citation/core

Module:Footnotes has, at present, these functions:

harvard_core() – support for {{harvcol}}, {{harvcolnb}}, {{harvcoltxt}}, {{harvtxt}}, {{Harvp}} via a call through {{Harvard citation/core}}

harvard_citation() – support for {{harv}}

harvard_citation_no_bracket() – support for {{harvnb}}

sfn() – support for {{sfn}} and {{sfnp}}

The separate function sfn() is required because it is notably different in functionality from the others. All the rest can be combined to use a common function. As part of that, there is no need for the call to {{Harvard citation/core}}. In Module:footnotes/sandbox, and the associated template sandboxen, I have created a common function harvard_citation() and removed the now unused code. The 'core' part of these templates, the creation and rendering of the author-date is not changed. What has changed is the method for specifying bracket and page separator used by the various templates. Here are the modified template sandboxen compared to the current live versions of the same templates:

{{harv}}

(Blue, Blanc & Rouge 1794, p. 5) – live

(Blue, Blanc & Rouge 1794, p. 5) – /sandbox

{{harvnb}}

Blue, Blanc & Rouge 1794, p. 5 – live

Blue, Blanc & Rouge 1794, p. 5 – /sandbox

these use {{Harvard citation/core}}:

{{harvcol}}

(Blue, Blanc & Rouge 1794:5) – live

(Blue, Blanc & Rouge 1794:5) – /sandbox

{{harvcolnb}}

Blue, Blanc & Rouge 1794:5 – live

Blue, Blanc & Rouge 1794:5 – /sandbox

{{harvcoltxt}}

Blue, Blanc & Rouge (1794:5) – live

Blue, Blanc & Rouge (1794:5) – /sandbox

{{Harvard citation text}}

Blue, Blanc & Rouge (1794, p. 5) – live

Blue, Blanc & Rouge (1794, p. 5) – /sandbox

{{Harvp}}

Blue, Blanc & Rouge (1794), p. 5 – live

Blue, Blanc & Rouge (1794), p. 5 – /sandbox

And the sfn templates:

{{sfn}}

live

sandbox

{{sfnp}}

live

sandbox

References

In the sfn templates above, different years are used so that there are four different renderings (brackets are not made part of the FOOTNOTES identifier).

I think I've got all of the harv templates did I miss any?

If there are no objections, I shall update the live module and templates. When I do that, there will likely be temporary script errors because it is not possible to simultaneously update the module and all of the templates. At the end of this, I think that {{Harvard citation/core}} can be deleted.

—Trappist the monk (talk) 15:55, 8 August 2018 (UTC)

These are purely internal changes, with no affect on the use or resulting display of these templates? ♦ J. Johnson (JJ) (talk) 21:30, 8 August 2018 (UTC)

Yes. I expected that the above examples would have illustrated that. Did they not?

—Trappist the monk (talk) 00:19, 9 August 2018 (UTC)

Appearances can be tricky. I prefer to avoid making assumptions where a definitive statement is curiously absent. ♦ J. Johnson (JJ) (talk) 23:38, 9 August 2018 (UTC)

There having been no further comments, updated. Remember, there will likely be temporary script errors because it is not possible to simultaneously update the module and all of the templates.

—Trappist the monk (talk) 12:30, 11 August 2018 (UTC)

implementing sfnm and sfnmp

{{sfnm}} and {{sfnmp}} are more-or-less clones of each other and each calls its own {{sfnm/core}} and {{sfnmp/core}} which also are more-or-less clones of each other – {{sfnmp/core}} parenthesizes the year whereas {{sfnm/core}} does not.

I have hacked Module:footnotes/sandbox to support {{sfnm}} and {{sfnmp}} so that these templates use the same core code that supports the {{sfn}} and {{harv}} templates. The implementation should be indistinguishable from the current version with minor exceptions.

this example uses positional parameters that include white space which the live templates do not ignore:

{{Sfnm |One |2001 |1p=15 |Two |2002 |2p=2}}

live – creates this: FOOTNOTESmith_2005_15Jones_2004_2

sandbox – creates this: FOOTNOTESmith200515Jones20042

References

the arbitrary 10-source limit is removed:

{{sfnm|One|2001|Two|2002|Three|2003|Four|2004|Five|2005|Six|2006|Seven|2007|Eight|2008|Nine|2009|Ten|2010|Eleven|2011|Twelve|2012|Thirteen|2013|Fourteen|2014|Fifteen|2015}}

live

sandbox

References

combinations of named and positional parameters are supported:

{{sfnm|One|2001|2a1=Two|2a2=Three|2y=2000|Four|2004}}

live – Four's work is omitted

sandbox

{{sfnm|1a1=Two|1a2=Three|1y=2000|One|2001|3a1=Two|3y=2002|Four|2004}}

live – One's work is omitted; Two and Four are not in proper order

sandbox

References

accepts without comment |nps=none and supports |ps=none:

{{Sfnm|One|2001|1ps=none|1p=15|Two|2002|2p=2|ps=none}}

live

{{Sfnm/sandbox|Two|2002|1ps=none|1p=15|Three|2003|2p=2|ps=none}}

sandbox

References

all of the above have illustrated {{sfnm}} though the same applies to {{sfnmp}}:

{{Sfnmp|One|2001|1p=15|Two|2002|2p=2}}

live

sandbox

References

• One (2001). Title. {{cite book}}: Invalid |ref=harv (help)
• Two (2002). Title. {{cite book}}: Invalid |ref=harv (help)
• Three (2003). Title. {{cite book}}: Invalid |ref=harv (help)
• Four (2004). Title. {{cite book}}: Invalid |ref=harv (help)
• Five (2005). Title. {{cite book}}: Invalid |ref=harv (help)
• Six (2006). Title. {{cite book}}: Invalid |ref=harv (help)
• Seven (2007). Title. {{cite book}}: Invalid |ref=harv (help)
• Eight (2008). Title. {{cite book}}: Invalid |ref=harv (help)
• Nine (2009). Title. {{cite book}}: Invalid |ref=harv (help)
• Ten (2010). Title. {{cite book}}: Invalid |ref=harv (help)
• Eleven (2011). Title. {{cite book}}: Invalid |ref=harv (help)
• Twelve (2012). Title. {{cite book}}: Invalid |ref=harv (help)
• Thirteen (2013). Title. {{cite book}}: Invalid |ref=harv (help)
• Fourteen (2014). Title. {{cite book}}: Invalid |ref=harv (help)
• Fifteen (2015). Title. {{cite book}}: Invalid |ref=harv (help)
• Two; Three (2000). Title. {{cite book}}: Invalid |ref=harv (help)

Without objection I shall update the live versions of these two templates and the module.

—Trappist the monk (talk) 11:08, 13 August 2018 (UTC)

For compatibility with {{sfn}} and {{harv}}, added support for |ref=:

{{Sfnmp|One|2001|1p=15|Two|2002|2p=2|2ref=some other reference id}}

sandbox

References

• One (2001). Title. {{cite book}}: Invalid |ref=harv (help)
• Two (2002). Title.

—Trappist the monk (talk) 12:18, 13 August 2018 (UTC)

There having been no comments, I have updated the live versions of these two templates and the module.

—Trappist the monk (talk) 15:22, 16 August 2018 (UTC)

Hi @Trappist the monk:! I'm concerned these changes to the {{sfnm}} template are causing new error messages about duplicate reference definitions (like "Cite error: Invalid <ref> tag; name "FOOTNOTE" defined multiple times with different content (see the help page).") in articles that were previously fine. One example is Cum occasione], and the other is Birinus.

These articles have multiple identical {{sfnm}} invocation which previous resolved to the same reference tag. However, whatever has changed seems to have cuased the template to produce slightly different reference definitions, and now the error message is produced.

While the duplicate usage of {{sfnm}} isn't idela, it's pretty common -- and there's no documentation that I've ever found about avoiding it (or how to avoid it). Was this side-effect of the edits intentional? What can be done for the articles that will be affected? -- Mikeblas (talk) 16:03, 16 August 2018 (UTC)

Other articles apparently affected include Grahame Clark, Judiciary of Austria, Cretan War (205–200 BC), Filioque, and Declaration of the Clergy of France. I'm sure there are more. -- Mikeblas (talk) 16:10, 16 August 2018 (UTC)

Typo on my part; fixed, I think. Thanks for the notice. Grahame Clark has a legitimate error; not my fault.

—Trappist the monk (talk) 16:43, 16 August 2018 (UTC)

Thanks! Looks better, and I figured out the error in the Clark article. -- Mikeblas (talk) 23:37, 16 August 2018 (UTC)

@Trappist the monk: {{sfnmp}} have used the sandbox module since August 2018, while {{sfnm}} have used the main module since the transition to use lua. Why is sfnmp still using the sandbox? --Dipsacus fullonum (talk) 05:56, 11 June 2019 (UTC)

Copy pasta error probably. Fixed.

—Trappist the monk (talk) 09:47, 11 June 2019 (UTC)

Volume before page number

The |loc= parameter seems to be mainly used to include volume or chapter numbers. However, currently the {{sfn|Smith|2005|loc=chpt. 3|p=12}}text displays as "Smith 2005, p. 12, chpt. 3.". I think it would be most logical as "Smith 2005, chpt. 3, p. 12. with the page last. Unless others disagree, I therefore suggest the following edit:

Current

if args.page ~= '' then
result = table.concat ({result, args.page_sep, args.page});
elseif args.pages ~= ''then
result = table.concat ({result, args.pages_sep, args.pages});
end      

if args.location ~= '' then
result = table.concat ({result, ', ', args.location});
end

Suggested

if args.location ~= '' then
result = table.concat ({result, ', ', args.location});
end

if args.page ~= '' then
result = table.concat ({result, args.page_sep, args.page});
elseif args.pages ~= ''then
result = table.concat ({result, args.pages_sep, args.pages});
end      

T.Shafee(Evo&Evo)^talk 11:07, 17 December 2019 (UTC)

 Not done for now: this module is highly used so we need to tread carefully and test thoroughly. Please can you WP:SANDBOX your changes and refer to Module talk:Footnotes/testcases. I suggest advertising this discussion more widely, e.g. Template talk:Harvard citation and Template talk:Sfn — Martin (MSGJ · talk) 11:42, 17 December 2019 (UTC)

Template-protected edit request on 15 February 2020

The indent of line 186: local escaped_uri; looks incorrect. Most probably escaped_uri is intended to be a local variable inside strip_url function? If yes, can we add the correct indent there? Thanks. ネイ (talk) 11:58, 15 February 2020 (UTC)

Is that indent bothering you ;) I'm not convinced it's worth editing a highly transcluded module for such a minor change, but I have added it to the sandbox copy, so can be deployed with a more substantive change — Martin (MSGJ · talk) 20:57, 15 February 2020 (UTC)

Excelsior! EEng 22:09, 15 February 2020 (UTC)

Thanks - I was mistaken in that I thought it makes it a global variable (too used to Python) - if it's just an indent problem then there's nothing to hurry.--ネイ (talk) 13:06, 18 February 2020 (UTC)

Getting error messages when things are working fine

There's some kind of problem going on with the sfn templates at Problem of two emperors; some of the links are saying "Harv error: no target", but they are all working fine. For instance citation #6 says "Harv error: no target: CITEREFVan_Tricht2011", but the citation itself does correctly get to its target (Van Tricht's 2011 text). I can't seem to fix this. Ichthyovenator (talk) 21:14, 27 March 2020 (UTC)

This condition is described at the help link in the error message. Does that help text not answer your question? I'm known to suck at writing help text and documentation, so if the help text doesn't sufficiently answer your question, tell me what it needs to make it more understandable.

—Trappist the monk (talk) 21:30, 27 March 2020 (UTC)

Trappist the monk: I have worked on fixing some of these errors, and the false positives are bothersome. Do you have a way to know whether there could possibly be a solution that eliminates the false positive errors? In other words, is it provably impossible to eliminate them from this module, or could we possibly fix the false positive error messages by appealing to more programmers who might have a clever idea about how to get rid of them? I hope that my question is clear. – Jonesey95 (talk) 05:22, 28 March 2020 (UTC)

At the very least, there should be a check if what |ref= has is equivalent to |ref=harv. It could already be the case though. Headbomb {t · c · p · b} 05:28, 28 March 2020 (UTC)

I don't understand what it is that you wrote. The module has support for

• |ref=harv in cs1 templates
• |mode=cs2 in cs1 templates
• |ref={{sfnref|...}} also {{harvid}} which is a redirect to {{sfnref}}
• |ref=<CITEREF...> – hand-crafted CITEREF anchor IDs
• |ref=<text> – plain-text anchor IDs

But, the module must be able to see these in the article's wikitext along with author / editor names and date / year as dictated by the value assigned to |ref= (if it is visible)

—Trappist the monk (talk) 14:13, 28 March 2020 (UTC)

I don't have a better solution than |ignore-err=yes but am open to suggestions. The module reads the raw wikitext (the same text as you see in an edit window) so if the necessary names and dates are not there, then this module cannot create an anchor ID to add to the list of anchor IDs. I have said all along that this is a limitation to this scheme.

It has occurred to me that we might create a whitelist for templates that take no parameters, the wrapper template {{Churchman & Hurst Railways of New Zealand}} for example, internally creates its anchor ID from |ref={{sfnref|Churchman & Hurst|2001}}. But, the whitelist may be problematic because there are a lot of wrapper templates, not all of them suitable to such a scheme. And I haven't fully thought about what other problems might be encountered.

—Trappist the monk (talk) 14:13, 28 March 2020 (UTC)

I think we should try a whitelist in order to get the most obvious and widespread wrapper-related errors out of the category. User:Ucucha's script will still show errors for the diehard editors who want to see them, and the whitelist will buy us some time and goodwill while we try to think of a better (less bad?) way to reduce false positives. I am willing to work through the category to look for whitelist possibilities; they are not too hard to find when you have User:Ucucha's script enabled, because you get only one red error message instead of two. – Jonesey95 (talk) 14:32, 28 March 2020 (UTC)

I'm not really sure this is a workable solution if this amount of 'fixes' is what is required to deal with basic situations. Something like this would be much easier to handle (for each offending source), if that's possible. Or alternatively something like {{reflist|anchor-errors=no}}. Headbomb {t · c · p · b} 05:39, 28 March 2020 (UTC)

Also why are those errors even visible to start? These should be hidden to users that don't explicitly set them visible in their skins when things are still at a very early experimental stage. Headbomb {t · c · p · b} 05:41, 28 March 2020 (UTC)

When editors do not know that the error checking present, they will not act as additional eyes to find problems with the module. Because the category is a hidden category it is likely that most editors would never know that it exists. I believe that editors are rather accustomed to red error messages in reference sections; cs1|2 having led the way. As long as there are no glaring script errors and no major uprising like there was with the cite-web-requires-website kerfuffle, I am content to show the error messages. So far, there have been few complaints or questions.

—Trappist the monk (talk) 14:13, 28 March 2020 (UTC)

@Trappist the monk: Yeah the help text didn't really answer what was going on, thankfully Headbomb fixed the issue on the page in question, but (as can be seen from the talk section right below this one) this error doesn't just affect the article I linked. It's a recent issue as well, the citations in the Problem of two emperors article didn't give error messages until very recently (should be something in the last few days), and that's without any edits being made to the citations themselves. Ichthyovenator (talk) 12:16, 28 March 2020 (UTC)

What do you suggest should be done to improve the help text? I'm all pointy-eared for suggestions.

You're right, there are a lot of articles with error messages (47,403 as I write this). Most of those articles have legitimate errors that need fixing. These errors have been with us for a rather long time. The changes to this module merely show that the old way of making sure that short-form citations using this module actually link to their matching long-form citations was not wholly adequate; if it were, then there would be far fewer errors.

The visible error messaging is new (17:39, 27 March 2020 (UTC)) so of course, the Problem of two emperors article didn't give error messages until very recently.

—Trappist the monk (talk) 14:13, 28 March 2020 (UTC)

I'm not saying that there is much wrong with the help text, it just didn't explain why there suddenly were errors when there weren't any previously - but you've explained it now so that's fine. I also agree that errors need to be fixed - I was just writing here to inform you that there actually weren't any errors in the Problem of two emperors article to begin with - the refs linked just fine to their intended destination even when the error messages showed up (because some of those destinations used other templates than "cite book" or "cite journal" etc.). It's of course good if editors can see more clearly that there are errors in the work they're doing. Ichthyovenator (talk) 16:56, 28 March 2020 (UTC)

There is something very wrong, in that these error messages should not have been enabled at this early a stage when there are still massive limitations and an ungodly false positive rates. Hell, CS1 generating ref=harv anchors isn't even in the CS1 sandbox. Headbomb {t · c · p · b} 19:34, 28 March 2020 (UTC)

I'm not going to get too involved in this discussion since this is way out of what I can help with, I'm just happy to see the issue with the article I had been working on resolved. I do agree that a huge amount of red in the references of so many articles is awful - especially in cases where there actually aren't any errors. Looking things over, it appears that there wasn't much discussion preceding whatever change was implemented in the template? If that's the case I agree that that is very problematic as well. Ichthyovenator (talk) 21:28, 28 March 2020 (UTC)

Error messages

I have reverted the error messages being enabled by default. It is way too early for them given

1. The extremely short time for community input
2. The massive false positive rate
3. The difficulty of suppressing erroneous messages on several FAs and other articles
4. The lack of |ref=harv being default in CS1 (see discussion), which would cut down on the number of cases that need fixing by probably 50% or more
5. The lack of any general bot strategy to fix errors/false positives

Headbomb {t · c · p · b} 20:10, 28 March 2020 (UTC)

This is what should be done going forward, IMO

1. Enable |ref=harv being default in CS1 templates. This would fix the vast majority of legitimate errors.
2. See what remains for legitimate errors. Some patterns will emerge, which can then be fixed through bots or AWB runs.
3. See what remains for false positives. Think of alternatives to [1].

Once CS1 templates are updated, that bots and AWB have cleaned up the low-hanging fruits, and that we have a better way of handling positives than humans going around doing this, then we can think about enabling error messages. Headbomb {t · c · p · b} 21:18, 28 March 2020 (UTC)

Headbomb: Many, many thanks for reverting this change. Much appreciated! I can see that the aim is desirable. So... (having no knowledge of the Wikimedia software at all) might one possibility be to link the behaviour to the edit function? We preserve the traditional appearance (no error message) for the general reader, but when the edit function is invoked then something, somewhere, somehow triggers a display of the problem for the editor. There already seem to be analogous functionality "if condition X applies, then display X.message at the editing box" for conditions such as WP:BLP. Then the conscientious editor (who has gone there for other reasons) can also attempt to patch up this particular issue at the article. Might something like that be one way (of perhaps several parallel ways) to take this forward? Feline Hymnic (talk) 21:32, 28 March 2020 (UTC)

There's a possibility of displaying the error messages on previews only, yes. This is one of many possible things that could be done. Headbomb {t · c · p · b} 21:34, 28 March 2020 (UTC)

Errors-in-preview-only might be a short-term workaround but it's useless as an ongoing strategy. I fix {{convert}} errors most days, and that template displays a hideous error message when previewed demonstrating that editors introducing the problems don't see it, and subsequent editors don't fix the error possibly because it's a bit hard to understand and is not their problem. Johnuniq (talk) 01:15, 29 March 2020 (UTC)

This isn't a "select one, and only one" exercise. (Multiple, independent, parallel approaches are possible.) Nor is it "I've seen a one instance of a bad implementation, therefore the whole underlying principle is bad". Might you have a positive proposal to contribute to the mix? Thanks. Feline Hymnic (talk) 10:34, 29 March 2020 (UTC)

Thank you Headbomb. Levivich ^{[dubious – discuss]} 21:38, 28 March 2020 (UTC)

+1 David Brooks (talk) 18:11, 29 March 2020 (UTC)

harv* adding trailing spaces and false positives

*_{{harv|Lee|1999}}_
*_{{harvp|Lee|1999}}_
*_{{harvtxt|Lee|1999}}_
*_{{harvcol|Lee|1999}}_
*_{{harvcoltxt|Lee|1999}}_
*_{{harvnb|Lee|1999}}_
*_{{harvcolnb|Lee|1999}}_

becomes

• _(Lee 1999)_
• _Lee (1999)_
• _Lee (1999)_
• _(Lee 1999)_
• _Lee (1999)_
• _Lee 1999_
• _Lee 1999_

A trailing space is added after each template in some namespaces (edit: I don't know why but this isn't the case in this namespace). I don't remember this being the case.

The spaces are followed by invisible "no target" errors (<span class="error harv-error" style="display: none; font-size:100%">Harv error: no target: CITEREFLee1999 ([[:Category:Harv and Sfn template errors|help]])</span>), which are now generated even when the target does exist, as the following {{citation}} will demonstrate: Lee (1999), Title. Nardog (talk) 09:12, 7 April 2020 (UTC)

Moved the space inside the error message <span>...</span>.

I don't see a false positive error message from your example. All of the Lee 1999 short-cites that you listed above link to the Lee 1999 citation. Did you write what it was that you really meant to write? Can you point me to an article that exhibits what it is that you mean?

—Trappist the monk (talk) 10:46, 7 April 2020 (UTC)

@Trappist the monk: I said what I meant. Copy the source of this thread and paste it on Special:ExpandTemplates or on a non-existent page and then preview it (but not save), and then check the source of the bullet list. I think you'll see errors. It seems that they disappear once you save the page. Nardog (talk) 10:54, 7 April 2020 (UTC)

Well, I guess it makes sense if the module is looking for anchors only in the saved source. But that's still a false positive. Nardog (talk) 10:57, 7 April 2020 (UTC)

(edit conflict)

Ok, I copied this section from this edit window and pasted it into a non-existent mainspace article and previewed. No error messages. I viewed the page source and found only two instances of the words "harv-error", both in your sentence that begins The spaces are followed by invisible "no target" errors...; one for the preview version and one for the source.

I then pasted this section into Special:ExpandTemplates where I do see the errors. The errors exist there, I think, because there is no article text for the Module:Footnotes/anchor id list to read. Special:ExpandTemplates may use the same 'preview' mechanism that ve uses. Are you using ve? ve does not 'preview' in the same way that the source editor previews (I learned this at phab:T221625)

—Trappist the monk (talk) 11:18, 7 April 2020 (UTC)

I viewed the page source and found only two instances... Did you use Developer Tools? view-source: won't work if the live preview is on because it uses JavaScript to show the result. No, I'm not using VE (unless it's NOT short for VisualEditor). Nardog (talk) 11:21, 7 April 2020 (UTC)

I used my browser's right-click → View page source. I don't know what Developer Tools is nor do I know what you mean by live preview.

—Trappist the monk (talk) 11:38, 7 April 2020 (UTC)

Developer Tools: right click -> "Inspect" (Chrome) or "Inspect Element" (Firefox), or press F12. Live preview: Preferences -> Editing -> "Show previews without reloading the page". (If you're missing out these things you're missing out a lot btw.)

Oh, this is interesting. If you disable JavaScript (or just live preview), you don't see the errors. Given <span>...</span> is invisible anyway, maybe this is a false positive trivial enough to not need fixing (Category:Harv and Sfn template errors will be visible at the bottom when doing this in a section of an existing page instead of a non-existing page and while live preview is on, though). Nardog (talk) 12:18, 7 April 2020 (UTC)

Right, I know what F12 is and use it occasionally but for the most part, I want to see what the normal non-logged-in reader sees so for that View page source is a pretty good indication of what that reader sees. Yeah, because I'm logged in, MediaWiki serves me my personal copy of a page but there is little or nothing getting in the way of seeing what the non-logged-in reader sees. I turned on 'live preview' and edited a non-existent article in mainspace. With that tool turned on, I do see the error messages. I suspect that this is the same, or a substantially similar, 'preview' mechanism to that used by ve. I'll be turning it off.

When editing and previewing a section with the generic preview, there will be false-positive harv errors just like the false-positive errors that MediaWiki places in the references preview subsection that it appends to the preview:

Cite warning: <ref> tag with name ... cannot be previewed because it is defined outside the current section or not defined in this article at all.

I haven't tried, so I don't know if it is possible for lua to detect a section preview; I rather expect that it isn't possible so this module will not be able to replicate what MediaWiki does.

—Trappist the monk (talk) 13:06, 7 April 2020 (UTC)

Opportunities to fix multiple articles in batches using AWB

See also: Category:Harv_and_Sfn_template_errors

There are some groups of articles that each have an identical problem that should be fixable with AWB or a similar find-and-replace tool. I do not have a computer suitable for use with AWB, unfortunately. I will list batches of articles here when I find them. – Jonesey95 (talk) 18:07, 9 April 2020 (UTC)

CITEREFFrancillon1979

CITEREFFrancillon1979: 50 airplane-related articles. Search results. Fixable with an edit like this. – Jonesey95 (talk) 18:07, 9 April 2020 (UTC)

CITEREFGröner1991

1,500+ articles. Bot request here. – Jonesey95 (talk) 18:07, 9 April 2020 (UTC)

 Done

CITEREFHackman2001

About 200 ship-related articles. Search results. Sample fix. Note that some pages have just "Hackman" and some have "Hackman 2011" in the short refs. As far as I can tell, there is only one version of this book, from 2001. – Jonesey95 (talk) 04:17, 10 April 2020 (UTC)

Am fixing these.Acad Ronin (talk) 16:22, 12 April 2020 (UTC)

Done.Acad Ronin (talk) 19:05, 12 April 2020 (UTC)

 Done

CITEREFWeatherill1908

About 50 ship-related articles. Search results. Sample fix. – Jonesey95 (talk) 04:40, 10 April 2020 (UTC)

I have fixed these.Acad Ronin (talk) 17:48, 11 April 2020 (UTC)

 Done

CITEREFPhipps1840

About 150 ship-related articles. (There are 3,100 articles in the category, or about 9% of the articles in the category, that transclude {{Infobox ship career}}.) Search results. Sample fix. – Jonesey95 (talk) 04:53, 10 April 2020 (UTC)

I have fixed these by hand. I prefer to do this by hand because although it is tedious, it enables me to catch some typos and other mistakes.Acad Ronin (talk) 16:51, 11 April 2020 (UTC)

 Done

CITEREFWinfield2008

About 170 ship-related articles. This one is a bit complicated, since there are four volumes in this book series, and you have to make sure you are templating the right one and skipping the ambiguous ones for humans to fix later. This one may not be amenable to a simple find-and-replace fix. Search results. Sample fix. – Jonesey95 (talk) 05:06, 10 April 2020 (UTC)

So far I have whittled the list down to about 34 articles.Acad Ronin (talk) 17:58, 15 April 2020 (UTC)

Thank you! Here's a fix for WinfieldRoberts2015. – Jonesey95 (talk) 18:38, 15 April 2020 (UTC)

 Done. I will add the last remaining error to the whitelist. Nice work. – Jonesey95 (talk) 15:02, 16 April 2020 (UTC)

CITEREFButt

About 100 rail-related articles. Search results. Some short refs are missing the year, which is 1995, and others have erroneous parentheses in the sfn template. – Jonesey95 (talk) 17:00, 10 April 2020 (UTC)

• This is mostly done, aside from a few cases where the source confusion requires expert attention. Mackensen (talk) 20:25, 16 April 2020 (UTC)

  Let's mark this as  Done. I think there are only two left. – Jonesey95 (talk) 22:25, 16 April 2020 (UTC)

CITEREFBradshaw1922

About 50 rail-related articles. Search results. Sample fix. It links to a template with a |year= value of 1985 and |orig-year= of 1922; the CITEREF generator uses 1985, but editors apparently put 1922 in the sfn template by mistake. – Jonesey95 (talk) 17:09, 10 April 2020 (UTC)

This group is  Done. – Jonesey95 (talk) 22:37, 19 April 2020 (UTC)

East African campaign (World War II)

There is a Harv and Sfn template errors on this article but I can't find any mistakes. Would someone mind having a look to be sure it's a false positive? Regards Keith-264 (talk) 11:36, 22 April 2020 (UTC)

 Done Martin of Sheffield (talk) 11:53, 22 April 2020 (UTC)

Thank you, does this mean that further reading sections with the template will all need the same amendment? Regards Keith-264 (talk) 12:15, 22 April 2020 (UTC)

Yes. There has been a change to the cite XYZ templates so that they include by default |ref=harv. Martin of Sheffield (talk) 12:18, 22 April 2020 (UTC)

Thought so, thanks, I'll alter them as I drive by. Regards Keith-264 (talk) 12:24, 22 April 2020 (UTC)

@Keith-264: "Need" is too strong a word. Those are warnings that are only of use to those that have Ucucha's script enabled and want to do advanced citation review. You can certainly add |ref=none when appropriate, but in general the warnings don't affect anything a reader would encounter so there's nothing wrong with leaving them on either. Headbomb {t · c · p · b} 12:50, 22 April 2020 (UTC)

OK thanks for that. Keith-264 (talk) 12:52, 22 April 2020 (UTC)

Fornaçon/Müller false positives in An Wasserflüssen Babylon

Seems related to the use of {{NDB}} Headbomb {t · c · p · b} 12:56, 22 April 2020 (UTC)

 Fixed. I still have NDB and a few other cite encyclopedia templates on my list to add to the whitelist. – Jonesey95 (talk) 13:41, 22 April 2020 (UTC)

False positives template:Drury-North American Steam

This one generates a bunch of false positives too. Headbomb {t · c · p · b} 13:03, 22 April 2020 (UTC)

I have added it to the whitelist, but I don't see any articles where it would have been generating false positive "no target" errors. Can you please link to one? If you are talking about the brown warning messages from Ucucha's script, those are not errors, and we should not be "fixing" them by modifying anything. – Jonesey95 (talk) 13:44, 22 April 2020 (UTC)

It was creating several false positives in many articles, 2-10-0 was the one I came across, but there were many others like it. It's fixed now though. Headbomb {t · c · p · b} 14:05, 22 April 2020 (UTC)

Somehow I missed that one in my scan of rail articles. Thanks for the link; article links are always helpful so that I can verify my work. – Jonesey95 (talk) 15:07, 22 April 2020 (UTC)

Whitelist of templates to prevent false positives?

The above exchange is copied from the section #Getting error messages when things are working fine above. I am pulling it out to start a separate discussion about whether creating a whitelist of template wrappers is a good idea to mitigate the false positive problem. Opinions and alternatives are welcome.

It has occurred to me that we might create a whitelist for templates that take no parameters, the wrapper template {{Churchman & Hurst Railways of New Zealand}} for example, internally creates its anchor ID from |ref={{sfnref|Churchman & Hurst|2001}}. But, the whitelist may be problematic because there are a lot of wrapper templates, not all of them suitable to such a scheme. And I haven't fully thought about what other problems might be encountered.

—Trappist the monk (talk) 14:13, 28 March 2020 (UTC)

I think we should try a whitelist in order to get the most obvious and widespread wrapper-related errors out of the category. User:Ucucha's script will still show errors for the diehard editors who want to see them, and the whitelist will buy us some time and goodwill while we try to think of a better (less bad?) way to reduce false positives. I am willing to work through the category to look for whitelist possibilities; they are not too hard to find when you have User:Ucucha's script enabled, because you get only one red error message instead of two. – Jonesey95 (talk) 14:32, 28 March 2020 (UTC)

end of excerpted section

I have created Module:Footnotes/whitelist. In it, I have placed a handful of anchor IDs from templates in Category:Rail transport book citation templates. I have tweaked Module:Footnotes so that it queries the whitelist when it can't find a match in the article's anchor ID list.

Here is an example: 40th Street station (Market–Frankford Line). Before the change to use the whitelist, all three of that article's {{sfnp}} templates emitted errors. All of that article's long-form citations are wrapper templates and all of their anchor IDs are in the whitelist. The Cox and Williams {{sfnp}} templates are correctly written; Springirth is not (different dates).

But, you see the problem that this introduces. Now, along with the false-positive errors that this mechanism attempts to address, it will likely introduce false-negative errors as well. A false-negative error is the failure to emit an error message for a short-cite template because an anchor ID for a common name (Cox, Williams, Smith, ...) is not in the article but the anchor ID for a wholly unrelated template is found in the whitelist.

So, how to get around false-negative errors? We could, I suppose, since it's doing it anyway, have the article reader create a list of templates used in the article. Then at error message time, first check the article's anchor ID list and then the whitelist. If found in the whitelist, the template associated with the whitelisted anchor ID must also be in the article's template list.

—Trappist the monk (talk) 14:17, 30 March 2020 (UTC)

I'm not convinced by your "false negative" solution. Take an example I'm familiar with, {{Cite EB1911}}, which is one of those many {{Cite encyclopedia}} users. Sfn'ing to it is an error, but if I kindly include ref=harv (which I usually do anyway with an AWB hotkey), it's not an error. Yesterday's solution would have marked both as an error. Your solution sounds like a lot of effort, requiring detailed understanding, of something that is already automated with those js files. David Brooks (talk) 16:22, 30 March 2020 (UTC)

ETA: When I walked away and thought about the above, I wonder if you can eliminate even more false positives with the whitelist along with checking for "ref=xxxx" in any non-whitelisted template. I think that's a fairly common syntax. But that would unfairly excuse templates that don't use ref in the same way, as well as those with incorrectly formatted ref's. I'm still opposed to the whole approach, but offering this in a spirit of cooperation. David Brooks (talk) 16:27, 30 March 2020 (UTC)

Whitelist issues

I have been unable to get the redirect {{butt-stations}} to work in the whitelist. It is used at Victoria line. Am I doing something wrong? – Jonesey95 (talk) 18:57, 30 March 2020 (UTC)

The official name of the redirect is Butt-stations; first letter uppercase. Because en.wiki accepts both upper and lower case initial letter in template and redirect names and because all of the rest of the letters in a template / redirect name must be the case that the template/redirect page has, and because Module:Footnotes will use the name in the template names table as an index into the templates table that ~/anchor id list makes, I chose to force first character upper case. For the purposes of the whitelist, template and redirect names are the same as their 'official' page shows them: butt-stations.

—Trappist the monk (talk) 19:12, 30 March 2020 (UTC)

That worked, thanks. – Jonesey95 (talk) 19:29, 30 March 2020 (UTC)

I just looked at the whitelist. If I understand how it works... you list the tag CITEREFChisholm1911 as being emitted by {{EB1911}} and {{Cite EB1911}} This is a problem for two reasons. {{Cite EB1911}} can only be trusted if the editor has added ref=harv (I know of many without). And if the template uses the optional author name, say Edmund Gosse, then the generated reference is CITEREFGosse1911. I think both of these cases would cause a problem for the whitelist. And, again, I believe 1911 is only a paradigm of many other templates designed at the same time by the same people. I fear this would descend into a cascade of ad hocs. David Brooks (talk) 19:49, 30 March 2020 (UTC)

At the risk of piling on, another case that occurred to me during our daily mile walk. {{Cite book}}, used on over 1M pages (not always with footnotes of course). When cited in a footnote, to be a usable anchor it usually needs at least ref=harv (which is frequently missing, a genuine error). Then you'd need to pull out the author last name(s) and year to be able to generate the expected anchor. And if instead it's ref=grarble, another case. Or, as you note, an anchor in an embedded span. Blah. David Brooks (talk) 20:39, 30 March 2020 (UTC)

I don't understand the second comment, but for the EB1911 case, yes, there will need to be many CITEREFXXXX1911 entries on the whitelist. I added maybe half a dozen a few hours ago (they all end in "1911"), and they appear to work fine. It is indeed an ad hoc workaround until something better comes along. As I have been poking through the category looking for templates to add to the whitelist, I have found and fixed many broken short citations, so this category is proving useful despite the initial cleanup of false positives that we are needing to take care of. – Jonesey95 (talk) 22:13, 30 March 2020 (UTC)

Module:Footnotes/anchor_id_list looks at |ref= in cs1 templates ({{cite book}} etc.) and acts accordingly. When |ref=:

• is empty or missing or has none, ~/anchor_id_list does nothing
• has harv, ~/anchor_id_list extracts contributor / author / editor names and year portion of date to make anchor ID image
• has {{sfnref}}, ~/anchor_id_list extracts the anchor ID information from that template to make anchor ID image (also its redirect {{harvid}})
• has <other text>, ~/anchor_id_list uses that <other text> as the anchor ID image (may match a short-cite template's |ref= parameter)

~/anchor_id_list does not process <span id="<anchor ID>"></span> in it's html form like that but does process {{anchor}}.

—Trappist the monk (talk) 23:43, 30 March 2020 (UTC)

Umm, why would {{EB1911}} have any other date than 1911? For the 1922 supplements we have {{EB1922}}. I don't understand this:

I believe 1911 is only a paradigm of many other templates designed at the same time by the same people.

—Trappist the monk (talk) 23:43, 30 March 2020 (UTC)

{{EB1911}} accepts |year= so that there can be year disambiguation. A better parameter for that template would probably be |ydab=<lowercase-dab-letter>.

—Trappist the monk (talk) 11:53, 31 March 2020 (UTC)

Some editors have in the past taken to adding year=1910 if they have reason to believe a particular volume was published then. I've generally changed them for consistency as I come across them. More seriously, I have a list of first/last/authorlink parameters for authors that likely have more than one EB1911 article; it's 99 entries long (I can send it to you) and misses many one-offs. Again, this problem exists equally in DNB (or more so; ISTR it has more named authors than EB). 14:29, 31 March 2020 (UTC)

ETA: a deeper review shows 539 distinct author lastnames in the two EB1911 templates (although I can already see one obvious typo :-( ) By "paradigm", I meant templates that cite DNB, Catholic, and probably others. Will scan DNB later; have to go now. David Brooks (talk) 17:10, 31 March 2020 (UTC)

I have been working on adding all of the EB1911 authors to the Whitelist, and I am almost done. The actual count of whitelist entries thus far is 138, with probably a dozen or two left to add. In the meantime, if you are looking for actual problems to fix, as I am, here's a search for CITEREFEB1911, which is an error that typically needs to be changed to Chisholm 1911, but not always. I am planning to work on DNB next. – Jonesey95 (talk) 17:58, 31 March 2020 (UTC)

I began wondering overnight if an alternate solution exists. We might create article-local whitelists by making a non-functioning template that holds anchor IDs for false-positive errors on the local page. So, for Problem of two emperors, we might add:

{{sfn whitelist|CITEREFBrand1968|CITEREFFine1994|CITEREFGeanakoplos1959|CITEREFVan_Tricht2011}} – {{harv whitelist}} as an alias

The template itself does nothing. It is read by the article reader to make a list of messages to suppress in that article; |ignore-false-positive= and |ignore-err= remain as options if necessary to fine-tune at the short-cite level. This local mechanism would go a long way toward alleviating the false-negative problem incurred by the 'global' whitelist which, I'm beginning to think, will grow until it becomes unwieldy.

—Trappist the monk (talk) 11:53, 31 March 2020 (UTC)

So I implemented it and deployed an experiment in Problem of two emperors.

—Trappist the monk (talk) 13:30, 31 March 2020 (UTC)

Interesting idea, but I would not want to spend the rest of my editing days adding it to articles and explaining it to editors. In my effort to fix all of the EB1911 false positives (still in progress; you can see my additions at the whitelist), I have found that it is MUCH easier to add templates and their matching CITEREFs to the whitelist than to add an unrendered, customized template to each article that has false positives. Adding CITEREFChisholm1911 to the whitelist probably fixed hundreds of false positives with a single edit. There are many templates in the railroad space that are used 50 to 100 times; adding one entry to the whitelist is much preferable to editing a hundred railroad articles and bothering the railfans. (A numeric example: this edit, which took no more than ten minutes to research and prepare, added eight templates that are transcluded in about 1,369 articles.) – Jonesey95 (talk) 14:54, 31 March 2020 (UTC)

Back to my bêtes noires: currently EB1911 templates have 538 distinct authors (I fixed the typo I spotted) and DNB templates have 370. Here they are: User:DavidBrooks/EBAuthors and User:DavidBrooks/DNBAuthors respectively. David Brooks (talk) 22:40, 31 March 2020 (UTC)

See my note below on DNB. As for EB1911, I believe that I have whitelisted all of the valid CITEREFs, though I may have missed a couple. If you find an EB1911 false positive, please let me know. There are still some erroneous uses of short references to EB1911 that are putting articles in the error category, especially CITEREFEB1911 (search link above) and misspellings of "Chisholm". – Jonesey95 (talk) 22:47, 31 March 2020 (UTC)

(Hoping you are still reading this section) While examining EB1911 and DNB, which are both stacked on top of {{Cite encyclopedia}}, I thought I'd look at that template itself. There are actually very few direct uses, but the first possible problem I came across was in Scarlet Memorial: Tales of Cannibalism in Modern China (sorry about that choice), which has a {{cite encyclopedia}} in its end matter, with author Leung, and an {{sfnb}} citing Leung 1994. Next on the list was Harold Isaacs (author Rand). I'm losing track of whether such usage is correctly diagnosed by your basic code, but I'm beginning to feel you're on a hiding to nothing trying to whack-a-mole all of these instances. Or does your regular expression approach cover them more easily? As I said, I think Catholic Encyclopedia is going to have many cases, and I'll analyze those next. David Brooks (talk) 22:36, 1 April 2020 (UTC)

It is not clear to me what it is that you think is wrong with Leung 1994. What is the possible problem?

As for Rand 1995, its problem is that the target uses a redirect {{encyclopedia}}. Most of {{cite encyclopedia}}'s 10 redirects are {{cite <something>}} so they are properly handled because the code is looking for {{cite <something>}}. I am aware that redirects exist to the cs1 templates but have deferred addressing that particular issue. I'm pretty sure that I will not do anything with other-language redirects ({{Citace elektronické monografie}} and the like).

—Trappist the monk (talk) 23:34, 1 April 2020 (UTC)

To the first: so much has happened since I first noticed the false positive error messages that I can't be sure of the exact conditions that cause them, but from what I understand the original implementation would have marked this as an error, because it is structurally like the EB1911/DNB class. I may be wrong in that, but I wanted to be sure your new work handles this case. I can research Catholic tomorrow. David Brooks (talk) 00:38, 2 April 2020 (UTC)

By first, I assume you mean Leung 1994. Because that citation is {{cite encyclopedia}} as itself (nothing wrapping it), all of it parameters must be in the article wikitext and so visible to Module:Footnotes' article reader. As a native cs1 template it is structurally unlike {{EB1911}} and {{DNB}}.

—Trappist the monk (talk) 19:41, 2 April 2020 (UTC)

I'm not following the updated Footnotes code because frankly it scares me, but I'm still thinking about the "hidden" but valid general reference/sfn/harv situations (that would have produce a false positive when this all started. More date: {{Catholic Encyclopedia}}, which has 6 redirects (one of which apparently has no uses in main space). Using those 7 versions I found 59 articles that have a corresponding sfn or harv with the correct lastname and year. If I read it correctly, those would have been errors. Some canonical examples: Massacre of the Innocents and Henri Valois (uses the redirect {{CE1913}}). Am I right in thinking that these would have to be separately whitelisted?

I'm sure there are many other wrappers for {{cite encyclopedia}} that could tell the same story. And what's to stop someone next week creating another general reference template with its own rule for generating a semantically valid CITEREF? As I said, I'm not following your effort to build escape hatches in detail, but this probably isn't easy. I still think the Svick/Ucacha solutions are the right ones to direct at the class of editor who also has the skill to fix the error (if only they worked in preview!) David Brooks (talk) 18:32, 2 April 2020 (UTC)

I have added {{Catholic Encyclopedia}} to the whitelist, along with all of the author/date pairs that I found in use in the error category. Massacre of the Innocents and Henri Valois display without error messages now. If you find an article that is trying use a short cite to link to {{Catholic Encyclopedia}} but does not link or shows an error with Ucucha's script, check to ensure that the dates match. I came across about half a dozen (out of about 220 pages) with date mismatches between the short citation and the full citation. I did not attempt to dig into the sources to resolve them. – Jonesey95 (talk) 19:28, 2 April 2020 (UTC)

(edit conflict)

Yep, not easy, and this whole exercise may yet collapse under the the weight of the whitelist. And yep, the user-scripts solution is a pretty good solution when it is the only solution available – except that given the number of true errors in Category:Harv and Sfn template errors, the scripts solution, no matter how good and no matter how long the scripts have been around, clearly has not worked. I suspect that the reason the scripts solution has not worked is likely because, according to Wikipedia:User scripts/Most imported scripts, fewer than 500 editors use one or both of the scripts. That there are so few editors using these scripts surprises me; but it does help to explain why the error category has so many true errors. I've never used Svick but I do know that Ucucha does work in preview.

—Trappist the monk (talk) 19:41, 2 April 2020 (UTC)

I hope it's clear that I'm working on this in good faith, although I guess I'm making it equally clear that I don't think it is on the right path. On Cath: my 59 articles are those where both the lastname and year do match (i.e. false positives in the original implementation). I don't think it's easy to invert the search for actual mismatches (e.g. {{sfn|Hoverman|1913}} could well be correctly referring to another book published at the same time). That really is a job for the js tools. Maybe there's a way of promoting their use among interested editors, although it did take me some years for me to realize that the bluelinks in footnotes weren't going anywhere and that was a problem. And, yeah, Ucucha works for me in preview but Svick doesn't. However, it doesn't seem to operate in AWB preview! David Brooks (talk) 20:01, 2 April 2020 (UTC)

Why would you think that I think that you are not working on this in good faith? Yeah, I know that you think that continuing to persue this idea is a colossal waste of time. And yeah, I know that this experiment is far from a complete success and will likely end badly when the ANI kraken bestirs itself and sinks my little boat.

Ucucha does not work in awb because preview there shows you the article as a non-logged-in reader will see it. Ucucha is a personal user script fetched via User:DavidBrooks/common.js which does not load when the preview loads. Why? Don't know. Clearly, you must be logged in to use awb so perhaps there is some performance or security reason. Have you reported this as a bug?

—Trappist the monk (talk) 22:24, 2 April 2020 (UTC)

Just being diffident, maybe overly so. And I was mistaken: Ucucha works in preview, but Svick doesn't. I can't see why they're different. Ucucha also provides a flag that lets you suppress the check for "general refs without a corresponding sfn". As to AWB, perhaps it's to do with using the Internet Explorer rendering engine directly. Yes, I'll report it, but I suspect it may be "by design, unfortunately". David Brooks (talk) 22:35, 2 April 2020 (UTC)

How to insert a whitelist entry with an apostrophe?

A simple syntax question: How do I insert a whitelist entry where the cite ref is CITEREFD'Arcy1898? I get an error caused by the apostrophe in the name. Thanks. – Jonesey95 (talk) 22:44, 31 March 2020 (UTC)

Two ways:

['CITEREFD\'Arcy1898'] = '...',

["CITEREFD'Arcy1898"] = '...',

—Trappist the monk (talk) 22:55, 31 March 2020 (UTC)

Considering using wildcards for DNB cite refs

I have scanned 76 out of 2,619 articles in the error category that use {{DNB}} or {{cite DNB}}, and I have ended up with 54 cite refs. As I go down the list, I am finding a few repeats, but not nearly as many repeats as in {{EB1911}}, which had about as many articles in the error category and ended up with only 175 cite refs in the whitelist. My gut feeling is that we might end up with 500+ DNB whitelist entries. I would like to open a discussion about whether we might want to create a wildcard system for DNB. It's a slippery slope, I am aware, and there will probably be some false negatives, but we only had the user script before, and we'll still have the script to highlight errors after this mess is cleaned up. I welcome your thoughts (and technical possibilities). We could limit the year range to the publication range of the DNB and limit the "last name" characters to something reasonable in order to catch typos. – Jonesey95 (talk) 22:44, 31 March 2020 (UTC)

Show me what you think that would look like. Do you have a list of contenders for wildcarding?

—Trappist the monk (talk) 23:05, 31 March 2020 (UTC)

I'd like to be able to do something like (pseudocode with pseudoregexes): ['CITEREF[a-z\-_\']+1[89][890][0-9]'] = template_names['DNB'], where valid years are 1885–1900 and valid author names (so far) contain any of the ASCII letters (case insensitive, no diacritics found yet, and none listed at User:DavidBrooks/DNBAuthors) plus hyphen, underscore (possibly; I haven't found one yet), and apostrophe. – Jonesey95 (talk) 23:29, 31 March 2020 (UTC)

I added DNB_special_patterns which is tried only when a DNB template is in the article.

—Trappist the monk (talk) 00:48, 1 April 2020 (UTC)

Thanks! The patterns appear to work well. I will look through the articles to see if I notice anything odd. – Jonesey95 (talk) 01:52, 1 April 2020 (UTC)

Now that I look more closely, it appears to work fine with {{DNB}}, which has |ref=harv embedded in the template code, but not with {{cite DNB}} when |ref=harv is used as an optional parameter. I demonstrated this in two versions of my sandbox: [2], [3]. Both versions have some deliberate errors (invalid years) to contrast with the valid years that should pass the tests. Am I doing something wrong? I copied the citations in question from Anne de Mortimer, if it makes a difference. – Jonesey95 (talk) 02:59, 1 April 2020 (UTC)

It was only looking for {{DNB}}.

—Trappist the monk (talk) 11:18, 1 April 2020 (UTC)

That looks better. – Jonesey95 (talk) 13:11, 1 April 2020 (UTC)

Did you see my note above: the DNB pair of templates last parameter has 370 distinct authors mentioned. Does your wildcard solution work with any author name? David Brooks (talk) 15:48, 1 April 2020 (UTC)

I believe that it does. Let me know if you find one that is not working (you may have to purge or null edit the article to make the error message go away). – Jonesey95 (talk) 16:42, 1 April 2020 (UTC)

I don't understand how wildcard solution wouldn't end up with the danger of false negatives (e.g. mis-spelled lastname in the footnote). Do you parse the general ref and make sure it matches the cite? David Brooks (talk) 18:35, 2 April 2020 (UTC)

It is possible that there will be false negatives, as I mentioned in my original post in this section. User:Ucucha/HarvErrors should catch them. If you have a better option, we are all ears. – Jonesey95 (talk) 19:25, 2 April 2020 (UTC)

Enabling ref=harv by default in CS1 templates

A thing that would make things much easier to review and understand is to enable |ref=harv as a default in CS1 references. It's been weeks that this have been requested, with no progress. As of now, the category is still polluted with zillions of those cases, making it hard to determine what sort of whitelisting needs to be done, if anything. Headbomb {t · c · p · b} 02:00, 2 April 2020 (UTC)

I agree, unless there are major downsides that I am not thinking of. We may end up with a few more "multiple target" errors, but based on my viewing of a few thousand articles in a non-random sample of the current sfn/harv error category, I would guess that 20 to 30% of the current errors would be fixed by this change. We have already cleared about 20% of the errors with the new whitelist, and there are many more templates that still need to be whitelisted. – Jonesey95 (talk) 02:15, 2 April 2020 (UTC)

One effect that you may consider a downside: the Ucucha script would tag these as "citeref with nothing referring to it". In many cases, it isn't an error because these templates are specifically intended to be a general "see also", as opposed to those that, by the "footnote everything" rule, should be referenced inline. David Brooks (talk) 20:05, 2 April 2020 (UTC)

Which is easily solved by not using Ucucha's script, or learning to accept those extra warnings. Svick's script will be fully functional at only highlighting the errors. Headbomb {t · c · p · b} 00:24, 3 April 2020 (UTC)

Or, as I just noticed, by inserting window.checkLinksToCitations = false; before importing the Ucucha script. David Brooks (talk) 00:58, 3 April 2020 (UTC)

The lack of progress on this is highly frustrating. Headbomb {t · c · p · b} 20:36, 8 April 2020 (UTC)

Actually it seems Trappist added this to the CS1/2 template sandboxes on April 2nd. That's great news. Now to wait for the rollout... Headbomb {t · c · p · b} 20:45, 8 April 2020 (UTC)

I think that you will be disappointed. That same day, I made this edit which anticipates the cs1|2 change. From that point on, Module:Footnotes assumes that all cs1|2 templates have an anchor ID so does not create an error message or categorize templates like these ({{sfn}} in DSB Class MT; {{harvnb}} here for simplicity):

{{harvnb|Christensen|Poulsen|2009|pp=204–205}} → Christensen & Poulsen 2009, pp. 204–205

{{cite book |last1=Christensen |first1=Peter |last2=Poulsen |first2=John |year=2009 |title=MotorMateriel 7 |publisher=Bane Bøger |isbn=978-87-91434-23-5 }}

Christensen, Peter; Poulsen, John (2009). MotorMateriel 7. Bane Bøger. ISBN 978-87-91434-23-5.

The short-form citation link does not work of course, because cs1|2 does not yet create an anchor ID for every rendering. If you have script error detection turned on, you will see that. But, there is no error message from Module:Footnotes.

Since the time of that change, Editor Jonesey95 has made significant changes to Module:Footnotes/whitelist which reduced the number of false-positive errors by some amount though I can't say how much.

The peak number of articles in Category:Harv and Sfn template errors occurred on 2020-03-27: ~47,700. That number had declined to ~47,300 at the time you reverted the error messaging (2020-03-28). At the time I made the change that causes Module:Footnotes/anchor id list to assume that all cs1|2 templates have an anchor ID (as if |ref=harv either explicitly or implicitly), the category listed 40,585 articles. At the time of this writing, the category lists 35,779 articles.

—Trappist the monk (talk) 21:42, 8 April 2020 (UTC)

I'm not sure why I would be disappointed that several thousand of currently non-working anchors will be working. Changing cite book to cite book/sandbox above gives exactly the desired/expected behaviour. Headbomb {t · c · p · b} 22:34, 8 April 2020 (UTC)

Possible multiple target false positive

Trappist, if you have time, can you please look at the multiple target errors in Declaration of the Clergy of France? There is some nesting of templates and duplication of years and sfnref years, and I don't know which attribute might be causing the error message. Thanks. – Jonesey95 (talk) 19:55, 2 April 2020 (UTC)

That happened because {{citation-attribution}} wraps {{cite book}}. To minimize the number of pattern matches and so that the article reader doesn't have to have a separate pattern for every possible {{cite ...}} and {{citation}} and redirect and variant, the article reader looks only for the pattern '{{%s*[Cc]it[ae]'. So, it found cita and then |ref={{Sfnref|Jervis|1872a}} so it added CITEREFJervis1872a to the list and then found cite and again found |ref={{Sfnref|Jervis|1872a}} so it bumped the tally. The article reader does know the template name so it is a simple matter of adding a new list that holds templates that are to be skipped; {{citation-attribution}} is now skipped.

—Trappist the monk (talk) 21:59, 2 April 2020 (UTC)

Template:Cite letter causing false positive?

I did my best to look at the code in Module:Footnotes/anchor id list, but I was unable to figure out why {{cite letter}}, a CS1 wrapper, is causing a false positive error at D&KR Burgoyne Class. – Jonesey95 (talk) 17:05, 3 April 2020 (UTC)

{{cite letter}} uses |last= for the letter writer and |subject= for the subject of the letter. In cs1|2, |last= and |subject= are aliases. I had the loops that extracts names inside out so the article reader found |last= and then found |subject= so used the latter (Dublin and Kingstown Railway No. 29). Fixed now, I think.

—Trappist the monk (talk) 19:35, 3 April 2020 (UTC)

Errors not showing or adding category at DSB Class MT

It looks like errors should be showing at DSB Class MT, but they are not showing for me. The error category is also not displayed. – Jonesey95 (talk) 17:13, 3 April 2020 (UTC)

|ref=harv will be the cs1|2 default so the article reader is anticipating that.

——Trappist the monk (talk) 19:40, 3 April 2020 (UTC)

False positive in 2 Thessalonians 1 - split category?

Anchors are working, no wrapper template is invoked... and yet there's a module-reported error here. What's going on? Headbomb {t · c · p · b} 22:41, 8 April 2020 (UTC)

Nevermind, it would be a multiple target error assuming the ref=harv default in CS1 rollout. Which very likely means there needs to be two categories, one for multiple target errors, one for broken anchor errors. Headbomb {t · c · p · b} 22:45, 8 April 2020 (UTC)

I too would prefer a separate category for each error before we go live with error messages. The "no target" errors are where all of the false positives live; I have been working on fixing those, and having a separate category for them would make searching with petscan more efficient. The "multiple target" errors are, in my experience, many times due to the same reference being repeated or to two different references with the same CITEREF, which IMO are less urgent, since a reader interested in verification will have one or two full sources to look at for information.

I do recognize that some articles will appear in both categories, which is fine if they legitimately have both errors. (Also I am boldly changing the name of this section heading, since I think Headbomb has found the crux of this section's issue.) – Jonesey95 (talk) 01:27, 9 April 2020 (UTC)

Trappist the monk, if you have the time and energy, can you please modify the module to assign two different categories, perhaps "Category:Harv and Sfn template no target errors" and "Category:Harv and Sfn template multiple target errors", eliminating the current catch-all category? The errors turn out to be pretty different in character, and in my hunt for templates to be whitelisted, the "multiple target" articles end up causing a lot of noise in my searches. I looked at the module code, and I was unable to parse the logic that assigns the current category well enough to propose a specific code change. – Jonesey95 (talk) 13:50, 22 April 2020 (UTC)

Done as subcats of Category:Harv and Sfn template errors.

—Trappist the monk (talk) 14:08, 22 April 2020 (UTC)

Thanks. That will help. – Jonesey95 (talk) 15:09, 22 April 2020 (UTC)

Category:Harv_and_Sfn_template_errors

@Trappist the monk: the remaining few in that category don't seem to want to fall into either subcategory. Null edits and purges don't fix this either. Headbomb {t · c · p · b} 15:33, 26 April 2020 (UTC)

{{harvc}} errors are not split into two categories.

—Trappist the monk (talk) 16:49, 26 April 2020 (UTC)

Any particular reason why this is so? Headbomb {t · c · p · b} 21:38, 27 April 2020 (UTC)

broken harv link reporting

There has always been a problem with the {{harv}} and {{sfn}} families of templates. There is no native mechanism (like redlinked wikilinks) to indicate that links created by these templates do not link to the matching long-form citation as they should. To know for sure, editors must test each and every short-form link. An aid for that has been user-scripts like User:Ucucha/HarvErrors. But, not all editors use that script so broken harv links can, and do, go undetected for a long time. At the moment, there is a prospective bot that would mark broken-link harv templates by adding {{broken footnote}} after the broken-link harv template.

I have another possible solution. I have created a lua module that extracts reference linking data from a variety of source templates (cs1|2 templates, cs1|2-like templates, vcite-family templates, wikicite templates, and harvc templates). I have modified Module:Footnotes/sandbox and Module:Harvc/sandbox to use the extracted data to determine if:

• the short-form template does not have a matching source template (target)
• more than one source template is the target of the short-form template

When either of these conditions exist, the modified Footnotes or Harvc module emits an error message with a help-text link, and places the article into the maintenance category Category:Harv target errors (categorizes article namespace only).

harv errors

template	rendering	comment
{{Harvard citation no brackets/sandbox|Black|Black|Brown|2020}}	Black, Black & Brown 2020	no target; if you have the Ucucha/HarvErrors script, its message confirms that there isn't a target for this short cite
{{Harvard citation no brackets/sandbox|Black|Black|Red|2020}}	Black, Black & Red 2020	when there are multiple targets, a link to the first target works but the second goes unlinked
citation templates
{{cite book |title=Title |last=Black |last2=Black |last3=Brown |date=2020}}	Black; Black; Brown (2020). Title.	not a target because |ref=harv not included in the template
{{cite journal |journal=Journal |title=Title |last=Black |last2=Black |last3=Red |date=2020 |ref=harv}}	Black; Black; Red (2020). "Title". Journal. {{cite journal}}: Invalid |ref=harv (help)	target #1: the short cite links to this target
{{citation |title=Title |editor-last=Black |editor-last2=Black |editor-last3=Red |date=2020}}	Black; Black; Red, eds. (2020), Title	target #2: this target goes unlinked

Alas, there is a limitation. Because the sandbox module reads page wikitext, and because wrapper templates often set parameter values inside the wrapper, the sandbox Footnotes and Harvc modules may emit false error messages for wrapped templates. {{cite EB1911}} wraps {{cite encyclopedia}}. The necessary parameters |editor-last=Chisholm and |date=1911 are set inside {{cite EB1911}} and not in this page that you are reading so those necessary parameters are invisible to the Footnotes and Harvc modules. In this example, the Chisholm 1911 link works but is marked as an error:

false harv errors

template	rendering
{{Harvard citation no brackets/sandbox|Chisholm|1911}}	Chisholm 1911
citation templates
{{Cite EB1911|wstitle=Alfred the Great|ref=harv}}	Chisholm, Hugh, ed. (1911). "Alfred the Great" . Encyclopædia Britannica (11th ed.). Cambridge University Press. {{cite encyclopedia}}: Invalid |ref=harv (help)

So that false error messages don't linger in an article, I have added |ignore-err= to the Footnotes and Harvc modules. This parameter causes the false error messages and categorization to be suppressed. The multiple target error may not be suppressed because it does not result from the wrapper-template problem. In this example, {{cite DNB}} wraps {{cite encyclopedia}} and internally sets the necessary parameter |date=1901:

suppressing false harv errors

template	rendering
{{Harvard citation no brackets/sandbox|Power|1901}}	Power 1901
{{Harvard citation no brackets/sandbox|Power|1901|ignore-err=yes}}	Power 1901
citation templates
{{Cite DNB|first=D'Arcy|last=Power|pages=331–332|supplement=1|volume=1|wstitle=Bucknill, John Charles |ref=harv}}	Power, D'Arcy (1901). "Bucknill, John Charles" . In Lee, Sidney (ed.). Dictionary of National Biography (1st supplement). Vol. 1. London: Smith, Elder & Co. pp. 331–332. {{cite encyclopedia}}: Invalid |ref=harv (help)

I have commandeered Module talk:Sandbox/trappist the monk/harv link test as a testcases page for the variety of source templates that create anchor IDs. There, you can see that, for these test cases at least, the sandboxen do not emit error messages except in the cases illustrated above.

I have also copied the content of Module:Footnotes/sandbox into Module:Footnotes and then previewed quite a few articles that use that module. Those experiments caused me to fix a few bugs but, in general, indicate that the modules are working as they should. I intend to sync the live Footnotes and Harvc modules from their sandboxen and create the error category with help text. If all goes well, then great; if not (when does live code ever survive first contact with reality?), revert the sync, fix the problems and try again.

Comments? Objections?

The following will be have been notified of this discussion. If others should be notified, please do.

• Template talk:Harvard citation no brackets
• Template talk:Harvard citation
• Template talk:Harvard citation text
• Template talk:Harvcoltxt
• Template talk:Harvcol
• Template talk:Harvcolnb
• Template talk:Harvard citations
• Template talk:Harvp
• Template talk:Sfn
• Template talk:Sfnp
• Template talk:Sfnm
• Template talk:Sfnmp
• Template talk:Vcite book
• Template talk:Vcite journal
• Template talk:Wikicite
• Help talk:Citation Style 1
• Wikipedia talk:Citing sources

—Trappist the monk (talk) 17:46, 16 March 2020 (UTC) 17:59, 16 March 2020 (UTC)

I don't think I see anything particularly objectionable at this point, but I feel that having opt-in message (e.g. class HarvErrors) after the initial roll out will let us find many corner cases (and see just how many of those there are), and polish the system before enabling things for everyone. Headbomb {t · c · p · b} 18:32, 16 March 2020 (UTC)

Hiding the error messages during debug can be done.

—Trappist the monk (talk) 19:30, 16 March 2020 (UTC)

The css to show error messages from Module:Footnotes will be:

.harv-error {display: inline !important;}					/* display Module:Footnotes errors */

—Trappist the monk (talk) 21:46, 16 March 2020 (UTC)

Having the {{harv}} series templates generate the errors is a cleaner solution than my bot, as it gives editors immediate feedback. I am concerned about the false positives though, with the noted wrapped templates issue as well as non-template citations with manually-created anchors. However, it shouldn't be extremely difficult to adapt the bot to suppress false positives instead of tagging errors. --AntiCompositeNumber (talk) 18:42, 16 March 2020 (UTC)

While I was writing this, it occurred to me that were it to be rejected, your bot might add a parameter to the target-less harv or sfn template in place of the {{broken footnote}} template. That would let Module:Footnotes render an error message, help text link, and category and it would never be separated from that separate template.

This is not a solution for non-template short cites like [[#Bubba_2012_ref|Bubba 2012]] p. 35 or the like because there is no template that would read the anchor link Bubba_2012_ref and do something with it. We could perhaps search for such constructs and have Module:Footnotes add a category if it couldn't find a matching anchor somewhere. This though, is not the time to consider adding that kind of functionality to these modules.

—Trappist the monk (talk) 19:30, 16 March 2020 (UTC)

(edit conflict) My only real feedback is on the content of the error messages themselves. I think "no target" is a little "programmer-y", which is not a criticism, just a description. I would prefer plainer English, perhaps something like "Harv error: no full citation matching CITEREFPower1901 (help)", or maybe remove the jargon word "Harv" entirely: "Error: no full citation matching CITEREFPower1901 (help)". I would rewrite the "multiple targets" message as something like "Error: multiple full citations match CITEREFPower1901 (help)".

Are there plans to identify full citations that have no matching short citation, as Ucucha's script does? I don't think you asked, but I think that would be undesirable, since many citation lists and reading lists use {{citation}} without matching short citations. – Jonesey95 (talk) 18:46, 16 March 2020 (UTC)

And CITEREFPower1901 isn't programmer-y jargon-ish? I used the Ucucha/HarvErrors messaging as a guide. Though not implemented now, it isn't too difficult, I think, to have Module:Footnotes emit a template specific error message. So, for {{sfn}} templates without targets the message would read . I'm open to suggestions. Terse, I think, is better.

No plans for a full-cite-missing-short-cite version of this.

—Trappist the monk (talk) 19:30, 16 March 2020 (UTC)

I created a provisional redirect to Template:Harvard citation documentation#Possible issues in the meantime. The ideal landing page should be to Help:Harvard errors instead of Help:Harv errors, I feel. And then we can move the relevant template documentation there, with advice on how to fix things, and how to handle corner cases. Headbomb {t · c · p · b} 19:44, 16 March 2020 (UTC)

For this experiment I plan on putting the error message help text directly on the category page. If this experiment is not rejected then I expect to move the help text. Still, the category page seemed to have worked well enough for error message emitted by Module:Lang; see Category:Lang and lang-xx template errors. The current category name is Category:Harv and Sfn template errors.

—Trappist the monk (talk) 21:46, 16 March 2020 (UTC)

Category:Harv and Sfn template errors created and first hack of help text is there. I have updated Module:Footnotes from its sandbox. I have fixed a couple of Lua script errors that occurred in the lua module that reads article text.

—Trappist the monk (talk) 14:24, 21 March 2020 (UTC)

Hi Trappist. I enabled the new tests as above and have found a couple of problems. On the page Strood the three {{National Heritage List for England}} citations and the {{Cite ODNB}} (Rhodes, 2004) references flag up with a "Harv error: no target:" error and yet the citerefs reported are correct and the blue link functions as before. I've tried this with no error check, Ucucha/HarvErrors.js, .harv-error and .harv-error + Ucucha/HarvErrors.js. Only when .harv-error is switched on do the reports occur.— Preceding unsigned comment added by Martin of Sheffield (talk • contribs) 21:31, 21 March 2020 (UTC)

Did you read the help text or the discussion above?

—Trappist the monk (talk) 22:25, 21 March 2020 (UTC)

As of this writing, error message display is enabled.

—Trappist the monk (talk) 17:39, 27 March 2020 (UTC)

Please see page Livonian language, bottom: syntax {{wikicite|ref={{harv|Schätzung Vääri|1966}}|reference=...}} is broken. Staszek Lem (talk) 21:40, 29 April 2020 (UTC)

Not the fault of this module. This (and the others) is malformed:

{{wikicite|ref={{harvid|Mosley|2002}}|reference=Mosley, Christopher (2002) ''Livonian'', München: LINCOM EUROPA (English)}}

Mosley, Christopher (2002) Livonian, München: LINCOM EUROPA (English)

Why? Because it uses {{harv}} in |ref=. Should be |harvid=:

{{wikicite|ref={{harvid|Mosley|2002}}|reference=Mosley, Christopher (2002) ''Livonian'', München: LINCOM EUROPA (English)}}

Mosley, Christopher (2002) Livonian, München: LINCOM EUROPA (English)

Using {{harv}} gives this:

'"`UNIQ--templatestyles-000000B5-QINU`"'<cite class="citation wikicite" id=([[#CITEREFMosley2002|Mosley 2002]])<span class="error harv-error" style="display: inline; font-size:100%"> harv error: multiple targets (3×): CITEREFMosley2002 ([[:Category:Harv and Sfn template errors|help]])</span>>Mosley, Christopher (2002) ''Livonian'', München: LINCOM EUROPA (English)</cite>

but using {{harvid}} gives this:

'"`UNIQ--templatestyles-000000B7-QINU`"'<cite class="citation wikicite" id=CITEREFMosley2002>Mosley, Christopher (2002) ''Livonian'', München: LINCOM EUROPA (English)</cite>

note the differences in the <cite> tag's id= attribute.

—Trappist the monk (talk) 21:52, 29 April 2020 (UTC)

False positives at Abu Qir

Caused by something I can't really put my finger on. Headbomb {t · c · p · b} 13:43, 30 April 2020 (UTC)

Do this:

{{sfnp|''EB''|1911a}} → {{sfnp|Chisholm|1911a}} and {{harvid|''EB''|1911a}} → {{harvid|Chisholm|1911a}}

{{sfnp|''EB''|1911b}} → {{sfnp|Griffith|1911b}} and {{harvid|''EB''|1911b}} → {{harvid|Griffith|1911b}}

—Trappist the monk (talk) 14:36, 30 April 2020 (UTC)

(edit conflict) Fixed like this. EB 1911 citations use the author or editor's last name. – Jonesey95 (talk) 14:37, 30 April 2020 (UTC)

Guilty of failing to clean that up properly 4 years ago :-) But the references section is still a mess: the EB9 reference shows a Ucucha warning (needs ref=none), and the Canopus reference should be {{Cite EB1911}} (there's no actual text to attribute; it's just a reference, so it's really Further Reading). Will fix. David Brooks (talk) 14:50, 30 April 2020 (UTC)

False positive at Zucchetto#CITEREFBraun1912

Related to Template:Cite Catholic Encyclopedia Headbomb {t · c · p · b} 08:54, 2 May 2020 (UTC)

 Fixed in the whitelist. – Jonesey95 (talk) 17:04, 2 May 2020 (UTC)

ref=none whitelist suppression

@Trappist the monk and Jonesey95: If a CS1/CS2 template is on the whitelist, and has a sfn/harv reference pointing to it, but the template has |ref=none, then HarvErrors.js (or equivalent) correctly throws an error, but the new Footnotes code doesn't show "sfn error: no target". Anyway, I think that's what my test was telling me. Is that a known omission? David Brooks (talk) 14:40, 2 May 2020 (UTC)

@DavidBrooks: link to said error/lack of error? Headbomb {t · c · p · b} 14:44, 2 May 2020 (UTC)

@Headbomb: User:DavidBrooks/sandbox. ETA: More properly, permalink to current sandbox. David Brooks (talk) 15:11, 2 May 2020 (UTC)

(edit conflict)

You are describing this?

{{harvnb|Chisholm|1911}} → Chisholm 1911 – this is a whitelisted anchor ID

and this matching template with |ref=none:

{{cite EB1911|ref=none}}

Chisholm, Hugh, ed. (1911). Encyclopædia Britannica (11th ed.). Cambridge University Press. {{cite encyclopedia}}: Missing or empty |title= (help) – a wrapper template that creates an anchor ID that Module:Footnotes/anchor id list cannot see

{{harvnb}} does not show because that anchor ID is whitelisted; we have instructed Module:Footnotes to mute its error messages for anchor IDs listed in the whitelist. This is the false-negative that we discussed beginning with post .

For your sandbox: spelling?

—Trappist the monk (talk) 15:20, 2 May 2020 (UTC)

@Trappist the monk: That's why it's in the sandbox: deliberate errors to test error messages. Yes, I know {{EB1911}} is whitelisted. The point I was making is that a usage of {{EB1911}} which might have an editor erroneously adding |ref=none (who knows why; we're not all experts) should not be protected by the whitelist, any more than a mis-spelt "Chisholm" should (the second test verified that mis-spell is caught). Can you change the code so the ref=none is trapped? And I'm actually using {{EB1911}}, which I know wraps {{Cite EB1911}}; one is CS1 and the other CS2, so until recently they had different defaults. The prescript of {{EB1911}} means it really should be referenced somewhere, but there are plenty of old articles that don't observe that (yet). David Brooks (talk) 15:58, 2 May 2020 (UTC)

{{EB1911}} wraps {{cite EB1911}} which wraps {{cite encyclopedia}}, a cs1 template. {{EB1911}} internally set |ref=harv; {{cite EB1911}} never did but neither of those templates was cs2. If you want one or the other to render in cs2 style, you must explicitly provide |mode=cs2.

Neither of {{EB1911}} or {{cite EB1911}} are whitelisted. The whitelist holds the anchor ID CITEREFChisholm1911, which is the default anchor ID for both of {{EB1911}} and {{cite EB1911}}.

Module:Footnotes/anchor id list collects anchor IDs for the templates using the common {{cite ...}} naming scheme, the various {{vcite ...}} templates, {{citation}}, and most of their associated redirects. It does not attempt to deal with the large variety of other names that editors use for wrapper templates. So, the module attempts to extract an anchor ID from {{cite EB1911}} (and fails most times because the template, as written in the article, doesn't provide enough information); the module does not attempt to extract an anchor ID from {{EB1911}}. When |ref=none, the module abandons processing of that template.

When Module:Footnotes processes {{sfn}}, it looks in the list created by ~/anchor id list for an anchor ID that matches the link that was created from the various {{sfn}} parameters. If it fails to find that anchor ID, it looks in the whitelist for a match. When the anchor ID is not found in the whitelist, Module:Footnotes emits the error message.

What I think you want is for Module:Footnotes to know, somehow, that:

• it is linking to a whitelisted anchor ID
• the anchor ID is not in the anchor ID list because the template:
  • is not present in the article – emit an error message
  • is a wrapper template – do not emit an error message
  • has |ref=none – emit an error message

I don't think that I would hold my breath were I you.

—Trappist the monk (talk) 18:34, 2 May 2020 (UTC)

Yes, something along those lines, thank you, although I thought {{EB1911}} was in the whitelist itself despite being a wrapper. Not holding breath. The error is rare in my experience. David Brooks (talk) 00:55, 3 May 2020 (UTC)

Compilation?

It would be cool if someone could go through Category:Harv and Sfn no-target errors/Category:Harv and Sfn multiple-target errors and compile a list of citation with generate those errors, and sort them by times used. Headbomb {t · c · p · b} 18:14, 7 May 2020 (UTC)

It may be possible for someone to do a database report on articles in the categories and export the CITEREF values from the error messages, since "CITEREF" is not a string that should appear anywhere in articles except in those error messages. You could ask at Wikipedia talk:Database reports, or there may be a more appropriate venue.

If you are looking for batches of things to fix, take a look at the "CITEREFFrancillon1979" section above. If you find a CITEREF string that you think may be common, you can do a search for it to see how many articles it appears in. – Jonesey95 (talk) 19:03, 7 May 2020 (UTC)

False positives at Spherical harmonics due Template:Springer

See Spherical_harmonics#cite_note-32. Headbomb {t · c · p · b} 18:42, 30 April 2020 (UTC)

Fixed in the whitelist. You are welcome to edit the whitelist yourself. It's not too tricky. – Jonesey95 (talk) 20:43, 30 April 2020 (UTC)

I'll need to internalize how it's structured first. Headbomb {t · c · p · b} 20:54, 30 April 2020 (UTC)

@Jonesey95: This does not seem to work at Bayazid of Sylhet. What am I doing wrong here? Headbomb {t · c · p · b} 19:48, 7 May 2020 (UTC)

This is the diff that will make it work. You don't need to use the template_names section unless a template has a bunch of redirects in common use. Basically, copy the CITEREFNameYYYY section for the left side of the equals sign, and insert the template name (always using an initial capital letter) into the curly braces. – Jonesey95 (talk) 20:41, 7 May 2020 (UTC)

False positive at Gregentios#CITEREFFiaccadori1991

This doesn't seem to work. I swear one day I'll wrap my head around this crap... Headbomb {t · c · p · b} 20:44, 8 May 2020 (UTC)

Likewise for [4] and Kinaidokolpitai#CITEREFvon_Wissmann1960. Headbomb {t · c · p · b} 22:40, 8 May 2020 (UTC)

Seems to be related to caching issues... nevermind. Headbomb {t · c · p · b} 22:41, 8 May 2020 (UTC)

(edit conflict)

You added:

['ODB'] = {'ODB', 'Oxford Dictionary of Byzantium'}

to a lua table named template_names. You also added:

['CITEREFFiaccadori1991'] = {'ODB'}

to a table called whitelist. The tables are made of key / value pairs. The key in whitelist is the anchor ID with the value being another table that, most commonly, holds a single template name. There is a second form that I'll come back to in a moment.

The key in template_names is a shorthand string associated with the template aliases with the value being another table that can hold as many template aliases as there are, in this case two:

['ODB'] = {'ODB', 'Oxford Dictionary of Byzantium'}

The first form is the most common and the simplest:

['CITEREFA\'Becket1907'] = {'Cite CE1913'},

Module:Footnotes gets the table assigned to the anchor ID key from whitelist with an assignment operator something like this:

local t_tbl = whitelist['CITEREFFiaccadori1991']

in the actual code, the key to the whitelist table is a variable wl_anchor_id.

When there are multiple aliases for a template, those aliases are collected in a table with an assigned key:

['ODB'] = {'ODB', 'Oxford Dictionary of Byzantium'}

This k/v pair is then kept in template_names. In order to assign the aliases table to the anchor ID key, ['CITEREFFiaccadori1991'] in whitelist it is necessary to point to the table in template_names using the assigned key ['ODB']. In whitelist you should write:

['CITEREFFiaccadori1991'] = template_names['ODB']

That assigns the table in template_names at key ['ODB'] to key ['CITEREFFiaccadori1991'] in whitelist so that Module:Footnotes can find the appropriate template name.

—Trappist the monk (talk) 22:58, 8 May 2020 (UTC)

False positive at HMS Wallace (1918)#CITEREFLondon Gazette 6 April 1920

Not in the mindspace to figure out how to whitelist this myself. Headbomb {t · c · p · b} 20:22, 9 May 2020 (UTC)

I'm not sure that it should be whitelisted. This particular anchor ID exists only at HMS Wallace (1918) so perhaps what should be done is to locally whitelist that anchor ID:

{{harv whitelist|CITEREFLondon Gazette 6 April 1920}}

—Trappist the monk (talk) 21:41, 9 May 2020 (UTC)

Whitelist and dynamic implementation

I freely acknowledge that the following comment is several weeks late, and apologize right up front. I've been on other projects and not paid attention to the whitelist implementation except knowing that for the most part it's working; I see few false positives. Now I look at it (and using my usual {{Cite EB1911}} and {{EB1911}} as examples), I see that it contains a long list of known authors that might be in any given article, which gives an OK to any short footnote that references one of those authors (and year, always 1911 in this case). Discussing with Jonesey95, I noted that the two flaws (a) the list of authors isn't complete (b) the module doesn't notice if the sfn/harv's target citation is erroneously not actually represented in the article, only affects a small, manageable number of cases.

Still, I had assumed that a more "dynamic" approach could have been used, at least for those citation templates that pass normally-named parameters down to CS1/2:

• Look up the default values for the citation template(s) in the article. In this case, Chisholm and 1911.
• Read the parameters in the template. If there's a |ref= that isn't harv, use that and skip the next steps (if it's none, bail)
• If there's a |last=, and maybe |last2= (etc), substitute for Chisholm. If there's a |year=, substitute that for 1911.
• Stick them together with CITEREF on the front.
• Use the result(s) to validate the sfn's in that article.

I realize there are edge cases here, such as disambiguators for more than one EB1911 reference, but as outlined I think this would cut down on whitelist maintenance and reduce the (again, admittedly few) false positives and negatives. Or am I overestimating the capabilities of module coding? Needless, I don't speak LUA. David Brooks (talk) 19:00, 9 May 2020 (UTC)

Actually - never mind. After a little thought I realized that this module probably only has access to the contents of the template itself, and isn't really up to looking for whitelisted templates elsewhere in the wikitext. Blame the mental decline during this lockdown for some muddled thinking. David Brooks (talk) 21:50, 9 May 2020 (UTC)

False positive at French ship Brillant (1690)#cite note-Troude p209-212-1

Seems to be en-dash related. Headbomb {t · c · p · b} 13:39, 14 May 2020 (UTC)

@Trappist the monk:, any tweak for this? Headbomb {t · c · p · b} 21:10, 15 May 2020 (UTC)

That short-cite / long-cite pair has been fixed so what is it that you are wanting from me?

—Trappist the monk (talk) 21:22, 15 May 2020 (UTC)

The Patterson ref in the link above. There's possibly a fixable issue with date recognition in Season YYYY–YYYY type of dates. Headbomb {t · c · p · b} 22:08, 15 May 2020 (UTC)

What link? The text 'Patterson' does not occur in any revision of French ship Brillant (1690).

If you are talking about the short cite in this old version, then this format:

{{Harvard citation no brackets|Troude|1867-1868}}

[[#CITEREFTroude1867-1868|Troude &amp; 1867-1868]]<span class="error harv-error" style="display: none; font-size:100%"> harvnb error: no target: CITEREFTroude1867-1868 ([[:Category:Harv and Sfn template errors|help]])</span>

Troude & 1867-1868

has malformed output ('1867-1868' is not the second author) and the anchor link doesn't match the anchor ID in:

{{Cite book|last=Troude|first=Onésime|url=https://gallica.bnf.fr/ark:/12148/bpt6k6540038z/f9.item|title=Batailles navales de la France|publisher=Challamel aîné|year=1867–1868|isbn=|ignore-isbn-error=|volume=|location=Paris|page=|pages=453|language=fr|access-date=}}

Troude, Onésime (1867–1868). Batailles navales de la France (in French). Paris: Challamel aîné. p. 453. {{cite book}}: Cite has empty unknown parameter: |ignore-isbn-error= (help)

because Module:Footnotes requires values in a range to be separated with an ndash character (that is how Footnotes knows that '1867–1868' – with an ndash – is a date and not an author. cs1|2 is less stringent and much more sophisticated; it allows a hyphenated range by overwriting the hyphen with an ndash character which then requires the short-form to use an ndash; cs1|2 also puts the citation into a maintenance category so that the template can be fixed. Writing both of the two dates correctly, as you did, fixes the problem.

—Trappist the monk (talk) 23:36, 15 May 2020 (UTC)

I am talking about possible tweaks for Paterson 1972–1973 in Elmina R. Lucke, as linked in the 2nd post of this section. Headbomb {t · c · p · b} 23:44, 15 May 2020 (UTC)

Trappist the monk, I worked around the problem in Elmina R. Lucke with a manual harvid template, but it looks like a tweak to the module might be needed to recognize year ranges. The Ucucha script showed no error, but the module showed a "no target" error. The link worked fine. Check the version before my edit to see the error related to Paterson 1972–1973. – Jonesey95 (talk) 00:39, 16 May 2020 (UTC)

Fixed.

—Trappist the monk (talk) 13:33, 16 May 2020 (UTC)

Excellent. Thanks. – Jonesey95 (talk) 13:58, 16 May 2020 (UTC)

False positives at Help:References and page numbers

AFAICT, there's no reason why any false positives should be flagged there. Headbomb {t · c · p · b} 11:50, 1 May 2020 (UTC)

The multiple target errors on that page are not false positives.

—Trappist the monk (talk) 12:08, 1 May 2020 (UTC)

AFAICT, each reference has a unique anchor, or suppressed anchor. There's no two references with 1972c for year, for example, and yet it says there's multiple targets. That seems like a false positive to me. Headbomb {t · c · p · b} 15:00, 1 May 2020 (UTC)

I also believe that the error messages on this page are incorrect. I see only one full source for each unique CITEREF id. I wonder if the module's error-checking code is confused by full citations wrapped in nowiki tags. – Jonesey95 (talk) 15:27, 1 May 2020 (UTC)

It's indeed related to the nowiki stuff. Headbomb {t · c · p · b} 15:39, 1 May 2020 (UTC)

Before counting anchor IDs, the code was looks for and removes templates of any type when they are wrapped in <nowiki>...</nowiki> tags. In this case, the tags wrap more than just a single template. I've relaxed the requirement so that <nowiki>...</nowiki> tags and all of their content is removed before the counting begins.

—Trappist the monk (talk) 16:03, 1 May 2020 (UTC)

@Trappist the monk: Similar thing at Wikipedia:WikiProject_Plants/Template#References but with <pre></pre> tags. Headbomb {t · c · p · b} 14:16, 6 May 2020 (UTC)

content of <pre>...</pre> tags now ignored.

—Trappist the monk (talk) 14:40, 6 May 2020 (UTC)

@Trappist the monk: <syntaxhighlight lang=""></syntaxhighlight> also cause false positives, e.g. Wikipedia:Parenthetical_referencing#Linking_inline_and_full_citations. Headbomb {t · c · p · b} 01:12, 18 May 2020 (UTC)

Hyphens in page ranges

Would it be possible to have hyphens in the page range of {{Sfn}} display as en dashes or have CitationCleanerBot replace them? Seems to me that’s an easily fixable mistake. (originally posted at User talk:Headbomb) Eddie891 ^Talk _Work 19:44, 18 May 2020 (UTC)

Botreq to get help to plow through errors

See WP:BOTREQ#Enlist help to clear Category:Harv and Sfn no-target errors. Headbomb {t · c · p · b} 23:19, 18 May 2020 (UTC)

"No target" error fixed by changing redirect Template:Encyclopedia to Template:Cite encyclopedia

I don't know if this can be fixed in the code, but I just made a "no target" error go away by changing {{Encyclopedia}} to {{Cite encyclopedia}} at the article Hoppo. This was a false positive error message. – Jonesey95 (talk) 18:17, 19 May 2020 (UTC)

I intentionally limited the Module:Footnotes/anchor id list article reader to cs1 template names that begin with cite; that catches most templates. For the whole list of lua patterns for all cs1 templates and their redirects see Module:Ref info/data cs1_template_patterns{}. As it works now, the article reader trolls through the article once picking up each cite ... template as it goes. To handle redirects, the code would have to troll multiple times. I think that would cost too much of the limited time available.

—Trappist the monk (talk) 19:04, 19 May 2020 (UTC)

Makes sense. I have updated the "no target" error-fixing instructions. – Jonesey95 (talk) 20:18, 19 May 2020 (UTC)

Ignore non-numeric named parameters 5+ / recognize first numeric parameter

If you have something like

{{sfn|Ab|Ba|Ca|Da|Bra|Hoc|Us|Poc|Us|1999|p=22}}

you get #CITEREFAbBaCaDa for an anchor. Most people / CS1/2 templates would expect that to output #CITEREFAbBaCaDa1999 for anchor.

The module should be smart enough to figure this out on its own. Headbomb {t · c · p · b} 18:06, 17 May 2020 (UTC)

Ping @Jonesey95 and Trappist the monk: as the resident LUA experts familiar with this module. Headbomb {t · c · p · b} 13:37, 18 May 2020 (UTC)

@Trappist the monk: Am I correct in thinking the recent updates took care of this? Headbomb {t · c · p · b} 18:56, 21 May 2020 (UTC)

{{harvnb|Ab|Ba|Ca|Da|Bra|Hoc|Us|Poc|Us|1999|p=22}} → Ab et al. 1999, p. 22

—Trappist the monk (talk) 18:58, 21 May 2020 (UTC)

Seems to be partly addressed then... {{sfnref|Ab|Ba|Ca|Da|Bra|Hoc|Us|Poc|Us|1999}} → CITEREFAbBaCaDaBra instead of CITEREFAbBaCaDa1999. Headbomb {t · c · p · b} 19:02, 21 May 2020 (UTC)

{{sfnref|Ab|Ba|Ca|Da|Bra|Hoc|Us|Poc|Us|1999}} → CITEREFAbBaCaDa1999

—Trappist the monk (talk) 22:09, 21 May 2020 (UTC)

Many false positives at 23rd Street (Manhattan)

Don't have time to review in detail, but the category seems to have received an increase in false positives today / in the last few hours. Headbomb {t · c · p · b} 00:29, 22 May 2020 (UTC)

This is due to the change noted in the section above. Trappist is refining the tests. Adding a few more entries to the whitelist would also have worked. – Jonesey95 (talk) 00:59, 22 May 2020 (UTC)

False error messages everywhere

Hi, a recent change to Template:Sfn is throwing up false red error messages everywhere and needs to be reverted. For example, at Hearts (card game) there are no less than 22 'sfn' footnotes with red error messages when every single one has a full reference in the Literature section. What is going on? Bermicourt (talk) 09:28, 28 March 2020 (UTC)

At Hearts (card game), while the {{sfn}} short form citations may all have matching long-form citations, none of the wikilinks from the {{sfn}} templates work because none of the long form templates in Hearts (card game) § Literature are configured to provide the necessary anchor IDs that the {{sfn}} templates need in order to function correctly. The error messages on that page are not false and are correctly announcing that something is broken.

—Trappist the monk (talk) 12:44, 28 March 2020 (UTC)

Reporting this too. Some pages don't have it and other pages have it. Mixed bag. Ominae (talk) 13:13, 28 March 2020 (UTC)

@Trappist the monk:. I'm sorry, but I don't think that's acceptable to impose new rules retrospectively that result in redlink chaos. When I and other editors started using Template:Sfn there was no requirement to link it to another specific template and it worked perfectly well. So the change should not have been introduced without a wide consensus and an agreed plan to change the thousands of new red links that have appeared. Please revert your edit until those are in place. Bermicourt (talk) 15:07, 28 March 2020 (UTC)

Of course there is no requirement to use templates. If the purpose of our work here is to produce a quality encyclopedia, and we care about how articles that we work on appear to readers, then wikilinks that link to nothing detracts from that appearance of quality. Why would you want articles that you care about to have links that go nowhere?

{{sfn}} was created in June 2009. User:Ucucha/HarvErrors.js was created in June 2011 as a method of locating broken {{sfn}} and {{harv}} links. The availability of the script was announced on the {{sfn}} talk page in July 2011. I just trolled through the Template talk:Sfn archives. I found one instance of an editor asking that there be a way for {{sfn}} to not link to a long-form citation; see Template talk:Sfn/Archive 2 § Group as namespace vs. visual style selector. Of course it is entirely possible that I missed something in my trolling; I did not stop to thoroughly read every post that contained the word 'link'. Still, this suggests to me that the links to long-form citations are, to most editors, important reasons for using the {{sfn}} and {{harv}} templates. Were it important, I would have thought that there would have been discussion about adding |no-link=yes or some-such or that someone would have created a separate non-linking version of the template – if they have, I do not know about it.

The template documentation has had a mention of linking to a long-form citation since at least this edit in September 2014 (updated to the current form in August 2019.

I don't see this as an imposition of new rules but instead, is a tool to help editor find and fix broken links as an aid to the creation and maintenance of a quality encyclopedia.

—Trappist the monk (talk) 16:38, 28 March 2020 (UTC)

Except of course, when there's a bunch of non-errors flagged as errors. These should be maintenance messages, not errors. Because requiring editors to do this is not reasonable. That's a false positive rate of 34 out of 73 on that article alone. Headbomb {t · c · p · b} 19:29, 28 March 2020 (UTC)

I just noticed this error on Frasier Crane, and I lost too much precious time discerning the cause. I see this conversation died two months ago. Did the issue remain or resurface? More importantly, how can this be corrected to prevent others losing time on it, please? Thank you, WikiWikiHigh (talk) 01:00, 22 May 2020 (UTC)

@WikiWikiHigh: There's no error on Frasier Crane. However, there was something weird going on with the template or Wikipedia (database hiccup possibly, WP:ITSTHURSDAY possibly) for a few minutes. Seems resolved now. Headbomb {t · c · p · b} 01:23, 22 May 2020 (UTC)

Unless this was the fix? Headbomb {t · c · p · b} 01:26, 22 May 2020 (UTC)

There was a Lua error with red text sprinkled repeatedly throughout the Frasier Crane article yesterday. It appears now to be resolved, thank you. WikiWikiHigh (talk) 19:10, 22 May 2020 (UTC)

Hobey Baker#CITEREFCollege Football Hall of Fame1975

How do you handle these ones? Same as the old ones? Headbomb {t · c · p · b} 00:56, 25 May 2020 (UTC)

Multiple invalid harvnb's: preview hangs

I was trying to fix the sfn error in 15th century (I found the cited Harvey book), and Show preview hangs indefinitely with the spinning icon. It seems to be caused by footnote 3 in the current version, which has a ref with four {{harvnb}} in a row, all of them erroneous. If I reduce them to just one, then Preview works. Maybe it's just me; the article was last saved 2 days ago. I could try to save without previewing, but I thought I'd leave it as-is in case it's a side-effect of the current changes and some expert can (a) repro and (b) debug? David Brooks (talk) 14:24, 26 May 2020 (UTC)

Works for me here. Try again? Headbomb {t · c · p · b} 14:48, 26 May 2020 (UTC)

@Trappist the monk: Turns out removing the import of User:Trappist the monk/HarvErrors.js from my common.js made it work. Do you use that? And why that should cause a hang in Preview but not a normal read beats me. David Brooks (talk) 15:07, 26 May 2020 (UTC)

I do use my own script but I cannot duplicate the problem when I preview either the section or the whole page. I'm using the most recent version of chrome.

—Trappist the monk (talk) 15:20, 26 May 2020 (UTC)

Firefox and no issues here. Headbomb {t · c · p · b} 15:41, 26 May 2020 (UTC)

Chrome and Edge (the new Edge so basically Chrome too). But another clue: my Preferences/Editing had "Show preview on first edit" unselected and "Show previews [sic] without reloading the page" selected. Reversing either of those settings fixes the bug. Also, I just noticed (using my original settings) that the Preview function after a few seconds displays the Parser Profiling Data, grayed out, while the spinner keeps spinning. David Brooks (talk) 16:40, 26 May 2020 (UTC)

I don't have those selected. I can see the problem when I select 'Show previews without reloading the page'. The problem happens with the Ucucha version of the script from which mine derives. I do not have sufficient js knowledge to know how to debug this peculiar interaction between the previewer (another js script I think) and the harv errors script.

—Trappist the monk (talk) 17:13, 26 May 2020 (UTC)

@Jonesey95 and SD0001: which I think both have good JS knowledge. Headbomb {t · c · p · b} 17:39, 26 May 2020 (UTC)

No idea about the JS. I have the reverse settings to David Brooks (i.e. "Show preview on..." selected, the other two unselected). The article worked fine for me. I was able to find the missing full citations with a quick search. Like many of these no target errors, someone copied the short refs without copying the full citations. The trick is to search for "Smith 2005" or whatever, including the quote marks, to see if you can find a matching citation supporting the same or similar text. – Jonesey95 (talk) 17:49, 26 May 2020 (UTC)

Ouch this was a tricky one. Can be fixed by replacing lines 13-16 with :

$(elem.parentNode).append(" ", $("<span style=color:DarkOrange>Harv error: link from " + href + " doesn't point to any citation.</span>"));

It's always to better to make explicit appends rather than modify the innerHTML directly. SD0001 (talk) 19:07, 26 May 2020 (UTC)

@SD0001: That works, thank you.

—Trappist the monk (talk) 19:40, 26 May 2020 (UTC)

-----

<inserting with apologies for forking the thread> @SD0001: Presumably the same fix would apply to User:Ucucha/HarvErrors.js, and even User:Svick/HarvErrors.js? I do realize the number of their users who are impacted by this bug tends towards zero. David Brooks (talk) 14:59, 27 May 2020 (UTC)

-----

Thanks, team, for the quick fix. I was hoping it would also fix the bug I reported on May 3 -- see the end of that section -- but that's still not working for me (harverrors show up in Further Reading, but only in preview). That bug goes away if I deselect "Show previews without reloading the page". I'll keep my original settings in case I trip over something else. David Brooks (talk) 20:52, 26 May 2020 (UTC)

That, I don't think is fixable. I added a temporary test probe that included the number of citations found in §Further reading to the warning message. When 'Show previews without reloading the page' is enabled, and the page is previewed, in this case William Allan (painter), the script emits the warning message for every citation in §Further reading (6×). The test probe for each of these indicated that there were none found. I tested the test probe by disabling 'Show previews without reloading the page' and copying one of the §Further reading citations out of §Further reading. With 'Show previews without reloading the page' disabled, none of the citations in §Further reading showed the warning message. The one citation that I copied out of §Further reading showed the warning message and the test probe showed a count of 6.

This suggests to me that the rendering accomplished by 'Show previews without reloading the page' while previewing uses some other mechanism for headings than the html <h2>...</h2> tags. This can be confirmed by enabling 'Show previews without reloading the page' and then using your browser to look at the page's source (in chrome right mouse click → View page source). For this article, in live view mode, there are seven <h2>...</h2> tags and the article content is obviously present as html text. In the preview, there is one <h2>Navigation menu</h2> tag, and the article content is wikimarkup (the edit box content) but there is no html article content. Compare preview html source with 'Show previews without reloading the page' disabled: html and wikisource article content; seven <h2>...</h2> tags. The HarvErrors script is looking for <h2>Further reading</h2> and <h2>Bibliography</h2> tags. When these are not found, the script assumes that the citations are not in §Further reading or §Bibliography so it emits the warnings.

—Trappist the monk (talk) 22:52, 26 May 2020 (UTC)

Trappist, this is not correct. Using the "view page source" option, you're actually seeing the page source before it was modified by any javascript. MediaWiki's Live Preview uses javascript to dynamically change the page html and insert the preview. You can see this new html from the "Inspect" option in the right click menu. The "console" option of the devtools is also very useful for live-testing javascript, without needing to save edits to a script page first.

This bug can be fixed by changing $('#Further_reading') and $('#External_links') to $content.find('#Further_reading') and $content.find('#External_links'). Sorry my bad I suggested them originally. SD0001 (talk) 04:37, 27 May 2020 (UTC)

Yes, that works thank you.

What am I missing? I could find and change my script in the console. What I couldn't figure out was how to run the script to see that the changes worked. How do I do that?

—Trappist the monk (talk) 12:02, 27 May 2020 (UTC)

Year check

I just fixed a Smith 2103, p. 23 type of situation. There should be a visible error message to double check the date, given it's a clearly impossible/future year. Headbomb {t · c · p · b} 00:34, 5 June 2020 (UTC)

I don't think we need that. cs1|2 does not include bad dates in its anchor ID and does emit a bad-date error message so the short-form cite with a bad date will emit the no-target error message. Adding a bad date error message on top of that is just redundant.

—Trappist the monk (talk) 11:08, 5 June 2020 (UTC)

The no target error message is not visible, and this is an obvious error to check for and flag. Headbomb {t · c · p · b} 12:52, 5 June 2020 (UTC)

"No Target" error appearing in search results

Although the harv/sfn errors are suppressed for most users, they still appear in search results if the searched text is close enough to the error. Admittedly, this is only likely if you are on a citation hunt. For example, I was trying to identify the source for the bad footnote in Solar cycle, and searching for the string "WS12007" produces a number of snippets with "harvnb error: no target..."; it turns out there are several pages with the same dangling footnote. This of course did not surprise me, but I wonder if an editor not otherwise exposed to the error messages, and coming across text from a footnote, might be unduly puzzled. That said, I think the real issue is that the search engine shows spans that are display:none by default, and Footnotes can't fix that. Also, I concede the likelihood of hitting this with a generic search is very low.

There are other problems with that footnote, like a dead link, but that's for another day. David Brooks (talk) 14:32, 12 June 2020 (UTC)

You're right, not something that Module:Footnotes can fix. I would argue that this 'anomaly' is useful when searching for specific broken anchor ID links or specific variants of an error message so even if we could 'fix' it, we shouldn't.

—Trappist the monk (talk) 14:43, 12 June 2020 (UTC)

I think that's a good call. BTW, referencing a suggestion by Jonesey95 a little above, I find that I can often identify "full citation in a related page, but only sfn duplicated here" with a search along the lines of insource:/Smith *\| *2005/. — Preceding unsigned comment added by DavidBrooks (talk • contribs)

+Catholic Encyclopedia author

Can someone add ['CITEREFCleef1907'] = template_names['Catholic'], (see Hans Baldung). Also, while here, I see that CITEREFBurns1907 is duplicated, and not in the specialized Catholic list. Finally, @Trappist the monk: I see you started experimenting with a dynamic solution for Cath in the sandbox; was that proving too complex? David Brooks (talk) 15:50, 15 June 2020 (UTC)

I have updated the live whitelist for CE. The problem that must be solved for the other wrappers in your list is the non-standard default anchor IDs as I think I noted in the other discussion.

Any anomalies? Report here.

—Trappist the monk (talk) 16:32, 15 June 2020 (UTC)

Actually Hans Baldung has now regressed. Cleef 1907 still shows an sfn error, but now so does Baynes&Smith 1880. David Brooks (talk) 17:34, 15 June 2020 (UTC)

And now fixed. Thanks. David Brooks (talk) 20:49, 15 June 2020 (UTC)

False positive at Hedi Schoop and Margarete Luise Schick

I can't seem to clear or whitelist those correctly. Headbomb {t · c · p · b} 18:10, 16 June 2020 (UTC)

The article reader expects to find named parameters from which is can build an anchor ID: |contributor=, |author=, |editor=, |year=, and |date=. Failing that, for templates named {{cite ...}}, the article reader looks for |ref= which may hold plain-text CITEREF<name-list><year> or {{sfnref}}}} or {{harvid}} or just plain text. Because these templates do not have named parameters, the article reader must rely on the content of |ref=.

from Hedi Schoop:

{{NDB|23|468|469|year=2007a|Schoop, Trudi|Kühn, Volker |116918756|ref={{sfnref|Kühn|2007a}}}} –

Because this template uses {{sfnref}} and because this template has a redirect {{cite NDB}}, rewriting it should work:

{{cite NDB|23|468|469|year=2007a|Schoop, Trudi|Kühn, Volker |116918756|ref={{sfnref|Kühn|2007a}}}}

Kühn, Volker (2007a), "Schoop, Trudi", Neue Deutsche Biographie (in German), vol. 23, Berlin: Duncker & Humblot, pp. 468–469; (full text online)

from Margarete Luise Schick:

{{cite ADB|31|167|169|Schick, Margarete Luise|Heinrich Welti|ADB:Schick, Margarethe Luise}}

This template does not use |ref={{sfnref}} but it can:

{{cite ADB|31|167|169|Schick, Margarete Luise|Heinrich Welti|ADB:Schick, Margarethe Luise|ref={{sfnref|Heinrich Welti|1890}}}}

Heinrich Welti (1890), "Schick, Margarete Luise", Allgemeine Deutsche Biographie (in German), vol. 31, Leipzig: Duncker & Humblot, pp. 167–169{{citation}}: CS1 maint: ref duplicates default (link)

—Trappist the monk (talk) 19:23, 16 June 2020 (UTC)

False positives at Fishing industry in North Korea

Caused by {{country study}}. There is likely a smarter way to handle it than custom whitelists. Headbomb {t · c · p · b} 19:28, 19 June 2020 (UTC)

Likely, but the dumb way works too. Fixed. – Jonesey95 (talk) 20:37, 19 June 2020 (UTC)

That seems to follow the simple rule, like many other wrapper templates, by just forwarding to cite book. If last or last1 is specified, generate a CITEREF. Add year if it is specified. Honor a specific |ref= I think there are several wrappers that could be listed to use the same logic. David Brooks (talk) 21:04, 19 June 2020 (UTC)

Similarly {{Cite Australian Dictionary of Biography}} and its redirectors, vide John Macarthur (wool pioneer), which I just came across. Could probably be handled with the same code path. David Brooks (talk) 21:47, 19 June 2020 (UTC)

Sorry, I copied the wrong link. {{Cite Australian Dictionary of Biography}} works correctly; I remembered overnight the LUA code that looks for [Cc]it[ae]. It's {{Australian Dictionary of Biography}} (a redirect to the above) that causes the false positive in John Macarthur (wool pioneer). I'm sure there is a class of templates including this and {{Country study}} that behave like, or redirect to, Cite.* templates, and could benefit from a lookup table. Probably an inexpensive way of knocking down some more exceptions. David Brooks (talk) 13:44, 20 June 2020 (UTC)

Encyclopaedia of Islam, New Edition whitelist

Now that the Catholic auto-whitelist is working, I'm wondering if it is also possible to have dynamic whitelisting for Template:Encyclopaedia of Islam, New Edition and its redirect EI2. There are 129 special cases currently in the whitelist, but that still leaves some false positives: Ziyad ibn Abihi for example has three.

The rule is: use explicit |ref= if provided. Else, if |last= or |last1= is provided, use a standard harvid using a year inferred from the volume (any explicit |year= is ignored). Else, use CITEREFEI2, which I expect may require a coding change. David Brooks (talk) 19:47, 17 June 2020 (UTC)

I have code that can support EI2. The problem, as I mentioned elsewhere, is the default case where volume is not specified (short-form linking to a long-form citation for the whole encyclopedia – shouldn't be doing that but ...) or when an author is not named.

• volume is not specified and author is not named: {{EI2}} uses {{harvid|EI2}}.
• volume is specified and author is not named: {{EI2}} uses {{harvid|EI2}}.
• volume is not specified and author is named: {{EI2}} does nothing; cs1|2 forms an anchor ID from the author-name list and the year preset |year=1960–2005
• volume is specified and author is named: {{EI2}} does nothing; cs1|2 forms an anchor ID from author-name list and the year preset |year=<volume year>

A couple of changes at EI2 might be contemplated:

1. include the collective editor's names in the editor-parameter presets for when |volume= is not set (Brill gives Bearman, Bianquis, Bosworth, van Donzel, and Heinrichs as the collective editors) and then remove the |ref={{harvid|EI2}} preset so that cs1|2 can create volume-wise and collection-wise anchor IDs from the author-name list (if present) or the editor-name list – this is normal cs1|2 functionality; preset |display-editors=0 when authors and volume are both not specified, en.wiki editors may override this
2. leave the editor-name list presets alone; replace the whole |ref= preset with the collective editor list:

   |ref={{{ref|{{#if:volume||{{harvid|BearmanBianquisBosworthvan Donzel1960–2005}}}}}}}
   

Either of these changes are acceptable to me (though push coming to shove, I would take the first) because both will make the short-form citations for the no-author cases be just like any other where cs1|2 falls back to the editor name list. Additionally, special secret codes like EI2 are meaningless to the reader (see at Itakh). Bearman et al. 2014 is more easily understood because Bearman is easily found in an alpha-sorted list of citations.

Perhaps I'll play around in the EI2 sandbox

—Trappist the monk (talk) 12:11, 18 June 2020 (UTC)

I would not support the use of "CITEREFEI2", just as we should not support "CITEREFEB1911". Are there EI2 entries with no named author or editor? If all EI2 entries have a discoverable author or editor, CITEREFEI2 should continue to display an error so that people know there is something to fix. – Jonesey95 (talk) 13:48, 18 June 2020 (UTC)

I am not suggesting that we should use anything that creates an anchor ID that looks like CITEREFEI2. If you read anything in what I wrote that suggests that we should be using CITEREFEI2, show me where it is and I will correct my writing.

The only article that I know of that uses the CITEREFEI2 anchor ID is Itakh. If {{EI2}} gets fixed in either of the ways I described above then the short-cites at Itakh change from {{sfn|EI2}} to {{sfn|Bearman|Bianquis|Bosworth|van Donzel|1960–2005}}.

—Trappist the monk (talk) 14:04, 18 June 2020 (UTC)

I agree that nuking the CITEREFEI2 default is a good solution, but Itakh may not be the only case. For example, Al-Abbas ibn Ahmad ibn Tulun looks similar, as does one of the 4 references in Enderûn. But there are few enough that they can be fixed. I did also find some constructs like ref = {{harvid|EI2|1960–2004}} (in Abbasid Samarra) but that's probably correct syntax albeit weird. David Brooks (talk) 17:14, 18 June 2020 (UTC)

Template talk:Encyclopaedia of Islam, New Edition § proposed changes to this template

—Trappist the monk (talk) 23:40, 18 June 2020 (UTC)

Many thanks to Trappist the monk for reforming the EI2 template, and reducing the load on the whitelist file. I'm trying to find a list of templates like {{country study}} (see below) that behave like the Cite* templates (pass-through, no defaults), which should be easy to handle with a simple list and extending the test for cs1-like in anchor_id_list_make(), correct? David Brooks (talk) 18:07, 26 June 2020 (UTC)

I have been reverted.

—Trappist the monk (talk) 11:12, 28 June 2020 (UTC)

Unfortunate, after your apparently good work in the sandbox. Did Cplakidas also undo the harv error exemption code? I can't make out their changes and forget where to find a valid testcase. David Brooks (talk) 17:28, 29 June 2020 (UTC)

Dynamic whitelist revisited

I know this should have been suggested earlier, or maybe it was considered and rejected, but I'd like to kick around some more the idea about making the whitelist mechanism tidier, with particular reference to citation-generating templates that provide default name, date, or both. As previously discussed (especially on the basis of Jonesey95's intensive work), the current implementation does a pretty good job of eliminating false positives. But the way it's done is still susceptible to false positives (a previously unseen author/year is specified in the template and correctly referenced in the short footnote) and false negatives (a familiar author/year is used in the short footnote but not provided in any citation). Admittedly there are now very few of either of these types, and we've eliminated some classes completely, but the potential still exists.

I think it should be possible, instead, to identify a list of citation-style templates along with their default name/year values. Specifically, and with some hand-waving:

• A list of template names that collapse to classes, either through redirect or invocation (e.g. {{1911}}, {{EB1911}}, {{Cite EB1911}} are all in the 'EB1911' class).
• A table of class,name,year tuples. Examples: EB1911/Chisholm/1911, ODLA/Nicholson/2018. This is an oversimplification, see note below.
• Code in Module:Footnotes/anchor id list, in anchor_id_list_make, modified to look up the template name in the class map, and look up the default values. Or maybe that code belongs in anchor_id_make.
• anchor_id_make modified to use those defaults in cases where the template parameters render an empty date or anchor_id.
• If the template name has been thus processed, don't validate the citeref against the current whitelist implementation.

I believe this would

• not impact performance
• algorithmically eliminate the false pos and neg now and in the future
• cut down the size of Module:Footnotes/whitelist and reduce its ongoing maintenance load
  • although the tables of defaults I identified need to be created, and probably belong to the whitelist page
• not cover every known cite-class template, of which there are hundreds
• allow the existing whitelist mechanism to continue to work as a backup

There are many citaiton-class templates that have a simple default name and year ({{EB1911}}, {{ODLA}}, {{ODB}}, {{Aldrich-LocosGER7}} and many more minor ones like that). Some have only one of the values defaulted, but the existing code would continue to expect an explicit setting for the missing value.

Here is the oversimplification. Some common templates have name, year, or both, that depend on another parameter, usually volume. Examples are {{CE1913}}, {{EB9}}, {{The History of al-Tabari}}. That would require an additional level of indirection. In some cases, the lookup is via a subpage indexed on page number, but usually it's algorithmic, which I admit may be a bridge too far.

Am I overthinking this? I happily concede that the existing whitelist, generated using an ad-hoc process, covers most of the false positive problem, which is where I personally first came in. David Brooks (talk) 18:25, 19 May 2020 (UTC)

As currently written, Module:Footnotes/anchor id list only recognizes the existence of {{cite EB1911}} from the several that you mention.

To make this work it would be necessary to have the code test all templates in an article. That means at least a partial rewrite of how the article reader reads the article. That rewrite would likely have some benefits because the article reader reads the article multiple times looking for cs1 templates, for cs2 templates, for vcite templates, for harvc templates, for anchor templates, for the several html-like tags that are to be ignored. That means that the code would need a list of all of those templates (and their redirects) plus your list of wrapper-templates (and their redirects). For each template in an article (no matter what it is) the article reader would first consult the primary list of citation templates; if found process as it does now. When not in the primary list, consults the wrapper-template list and makes an anchor ID from info provided in the template call or from the default info from the wrapper-template list.

There is benefit from doing it this way because to look at every template in the article could be done with a single scan of the article instead of the multiple scans required with the current architecture. Downside is creation and maintenance of the cs1, cs2, vcite, ... lists; creation and maintenance of the wrapper-template list.

—Trappist the monk (talk) 16:19, 20 May 2020 (UTC)

Thanks for giving this serious consideration. Yes, this is work in creation of new structures; I was partly motivated to suggest this by thinking the subsequent, ongoing maintenance load would be lighter, and easier to do incrementally and, as I said, alongside the current implementation. As we software developers know, it's always more fun to create in the first place than maintain.

On the topic of scanning the entire content for every template: doesn't template_list_make() already make that list, or is it not always executed? I don't know if LUA has a dictionary lookup, which would make it cheaper to tease out the target templates, and I think a full implementation would find dozens or even hundreds with the same pattern of dropping a default year/name into CS1/2. On a detail, I forgot to note that the "list of template names that collapse into classes" already exists in the footnotes/whitelist template_names table. ANd it seems that it would be conceptually easy to create the template-class/name/year tuples and use them in anchor_id_make(), but it might be more effort to map template/volume through volume->name/year.— Preceding unsigned comment added by DavidBrooks (talk • contribs) 18:43, 20 May 2020 (UTC)

I have created Module:Footnotes/anchor id list/data. That module creates lists of known cs1, cs2, vcite, harvc, wikicite, anchor, and sfn whitelist templates and their redirects.

I have rewritten Module:Footnotes/anchor id list so that it scans the article only once when searching for citation templates that are known. For each template found in an article, the template name is added to template_list. Then the article reader looks in the tables created by ~/anchor id list/data for a matching template name. When found, the article reader attempts to construct an anchor ID for that template.

I have updated the live version of ~/anchor id list. At the time of the update, the error categories had:

Category:Harv and Sfn multiple-target errors 4,558 (live: 3)

Category:Harv and Sfn no-target errors 28,153 (live: 4,597)

There is nothing really new except that the article reader checks a slightly broader variety of templates so I don't expect dramatic change. Still to do, figure out how to support wrapper-template defaults.

—Trappist the monk (talk) 18:52, 21 May 2020 (UTC)

If you wish, I can start to research and build a list of templates' default year/name values (and the volume dependencies). I don't think there is a mechanical way to do that because it depends on the vagaries of various editors' template coding style. The templates mentioned in the current whitelist are a starting point, and I believe the lists that I worked up for Jonesey95 contain more possibilities. David Brooks (talk) 22:17, 21 May 2020 (UTC)

The "no target" error count appears to have increased by 3% or so since this update. Any idea why that would have happened? – Jonesey95 (talk) 22:36, 21 May 2020 (UTC)

I would have been surprised if the count went the other way. The list of templates that the previous version considered was more restricted; the patterns are in Module:Footnotes/anchor id list/data. Of course there could be bugs in the code ... If you find anything untoward, speak.

—Trappist the monk (talk) 22:52, 21 May 2020 (UTC)

In the previous version, this:

{{cite citygrid|ref={{harvid|Koeppel|2015}}}}

would not have caused an error because the template name begins with cite. Now, {{cite citygrid}} is not listed in one of the known-templates lists so the article reader doesn't add CITEREFKoeppel2015 to the article's anchor_id_list as it did before. I'll have to think about that.

—Trappist the monk (talk) 23:10, 21 May 2020 (UTC)

Referring to that research: I started to analyze the templates mentioned in the whitelist, in order of occurrence. At least so far, many more than I had expected have defaults that depend on volume (EI2, al-Tabari, ADB, NDB...). Some have a single "name" but year that depends on volume (Catholic, EB9, Cite Biblica...). Some have no defaults (ODNB for a start). A few big ones do have simple default pairs (EB1911 family, ODB, EGA...). As I suggested, any code that builds default name/year pairs will have to be able to look up in a volume-based table if required, and those tables would have to be reconstructed in code. And I'm sure some of the lesser-used templates have single defaults. I have all this in a spreadsheet and will work some more on the list Sunday. David Brooks (talk) 20:14, 22 May 2020 (UTC)

I don't know exactly what the language supports (O(1) dictionary-based lookup would be nice) but based on what I've seen so far, in data structure terms this would need

• A table or tables of template names to look up the default data, defined below
  • Because some of the most-used templates have multiple redirecters/transcluders, a secondary class-based lookup would be nice, but just duplicating the data per template would avoid that complication
• Default data in one of three forms. There may be a way of combining them in one table:
  • A simple name/year tuple (example {{Cite EB1911}}). In some templates there is no default year, and CITEREFSmith would result unless the year is provided explicitly.
  • A constant name, with year dependent on volume ({{Cite EB9}}). The input parameter is usually volume but obviously could vary.
  • A name and year both dependent on volume, and sometimes with a default if no volume provided.

Volume keys should not be required to only be digits ({{Cite EB9}} allows roman numeral) or, sadly, can be a positional parameter (({{Cite ADB}}).

Names are often a concatenation of multiple authors, especially in the multi-volume opuses, and I'd anticipate just recording that long string, not its multiple parts. It's possible for the transcluder to provide a last2 but not a last1, resulting in an unforeseen combination, but I suspect that would probably turn out to be a typo.

Explicit names can also be a positional parameter ({{Cite ADB}} again). This is all pretty complex, I know. David Brooks (talk) 14:01, 23 May 2020 (UTC)

The sandboxed modules support simple name/year. Module:Footnotes/whitelist/sandbox is a clone of the more-or-less live module (there have been additions to the live module since I cloned the sandbox). All of the EB1911 whitelisted anchor IDs in the sandbox are commented out. The sandbox has two new tables: wrapper_template_defaults and wrapper_templates. Each entry in wrapper_template_defaults is a k/v pair where the key is the canonical name of the wrapper template and the value is a sequence table where [1] is the default anchor ID name-list as it would appear in the anchor ID and [2] is the date portion of the anchor ID.

Module:Footnotes/anchor id list/sandbox (article reader) first attempts to extract an anchor ID from the various cs1, cs2, vcite, wikicite, anchor templates. Failing that, article reader queries wrapper_templates to see if template_name (the name of the template it is processing) is listed. If template_name is listed, article reader attempts to extract a name list from the template. When it does, it uses what it finds; failing that, article reader uses the default name list from wrapper_template_defaults[<template_name>][1]. The same process holds for the date portion of the anchor ID.

Examples of this are at Module talk:Footnotes/anchor id list § dynamic whitelisting. CITEREFAdamsonMitchell1911 is listed in the whitelist (commented out); CITEREFJonesBryant1911 is not listed in the whitelist. Also near the top of that page there is a working CITEREFChisholm1911 example ({{cite EB1911}} without authors or date parameters).

—Trappist the monk (talk) 15:13, 23 May 2020 (UTC)

and now live.

—Trappist the monk (talk) 18:45, 23 May 2020 (UTC)

Thanks for doing all of this while I wasn't paying attention :-) Questions:

• Was it necessary to give {{EB1911}} (and its redirects) a separate table from {{Cite EB1911}} (and its redirects)? The one transcludes the other. I could see the point if you want to anticipate their getting different defaults some day, or if you want to simply reflect that they are not actually redirects of each other.
• Any objections to me adding other name/year pairs when I find them? If the year is not provided, should that be represented by an empty string or by nil?
• Do you see a way to implementing per-volume defaults?

The whitelist weighs less now, which is good. David Brooks (talk) 21:14, 23 May 2020 (UTC)

{{EB1911}} and {{Cite EB1911}} are not separate tables. They are merely grouped by canonical name with redirects to those canonical names given an extra indent to distinguish the redirect names from the canonical names.

{{cite Catholic Encyclopedia}} and {{Catholic Encyclopedia}} and their redirects have default date determined by |volume= see Module talk:Footnotes/anchor id list § dynamic whitelisting.

—Trappist the monk (talk) 14:34, 24 May 2020 (UTC)

This is great to see. Three other template types have >20 entries in the "old-style" whitelist: EI2, the al-Tabari group, and ODNB (which has no defaults so uses explicit name/year). EI2 just generates CITEREFEI2 unless an explicit |ref= or |last[1]= is given. So many variants... Are you still in testing mode? David Brooks (talk) 14:43, 25 May 2020 (UTC)

EI2: That CITEREFEI2 anchor ID apparently has only 1 hit

al-Tabari: similar; for the CITEREFAl-Tabari1985–2007 anchor ID there are 19 hits. But, when given |ref=harv without specifying a volume, this template uses the series editor Yarshater and series dates 1985–2007 for the anchor ID: one hit.

ODB: similar; no hits

ODLA: similar; no hits

I'm inclined to think that these wrapper templates should be tweaked so that when |volume= is omitted, the series editor(s) should be rendered in the citation and used for the anchor ID so that such citations are consistent with template renderings when |volume= is supplied. The code that passes the value from |ref= to {{cite encyclopedia}} should be simplified to just pass whatever is in the wrapper-template's |ref= to the underlying cs1|2 template so that Module:Citation/CS1 can sort it out using the author/editor/date/ref parameter values as it does for all cs1|2 templates.

—Trappist the monk (talk) 16:43, 25 May 2020 (UTC)

For prioritization I was counting the distinct appearances in Module:Footnotes/whitelist. There are 121 total for EI2, which I guess reflect actual author names coded into the template uses. For example, Banu Judham uses |last=Al-Bakhit and the template code generates CITEREFAl-Bakhit1995, which is the year for volume 8. If I read the source right, ODB defaults to ODB/1991 unless the transcluder includes another lastname and/or year. Similarly ODLA defaults to ODLA/2018. I think those two could just be plugged into wrapper_template_defaults. Deferring to you suggestion, we could make ODB default to Kazhdan/1991, and there may be other cases where it would be useful to do so, as long as it isn't taking too much on board at once.

And, in case it's not clear, I'm now a full convert and I'd concur with turning the sfn errors on by default - except I'd like a wording that's marginally more meaningful to the general user, like "citation link error: Chishom/1911 not found". And I now realize the value of the error category, looking at some cases where the editor probably was slightly confused about how sfn's parameters work. David Brooks (talk) 18:15, 25 May 2020 (UTC) -- ETA: for example. David Brooks (talk) 18:47, 25 May 2020 (UTC)

There's still a bit more bot-ish work to be done before those can be enabled by default IMO. There are many citation with something like {{harvnb|Smith|1654|23-24}} that could be fixed to {{harvnb|Smith|1654|pp=23-24}} or similar. And a couple more false positives to be handled first. But we're getting close. Headbomb {t · c · p · b} 14:03, 27 May 2020 (UTC)

And personally, I'd rather get WP:BOTREQ#Enlist help to clear Category:Harv and Sfn no-target errors this bot going first so we can tackle the problem with people that will very often know what is meant to be cited, and have an interest in digging around for solutions. Headbomb {t · c · p · b} 14:12, 27 May 2020 (UTC)

Looking at Jonesey95's recent changes to the whitelist, I'm realizing (what should have been obvious) that dynamic-with-defaults CITEREF construction would overkill in many or most cases. For example, {{Neukirch ANT}} is hard-wired to generate CITEREFNeukirch1999 so the pre-dynamic implementation works. {{Baghdad During the Abbasid Caliphate}} will generate CITEREFLe_Strange1900 or CITEREFLe_Strange1922 depending on |edition=, and chances of a false negative are minimal. Unless Jonesey95 has already done it, I'll try to surface some more templates that do forward explicit name/year parameters downwards.

Also, +1 on Headbomb's preference above. David Brooks (talk) 15:26, 27 May 2020 (UTC)

I'm going through the list at User:DavidBrooks/CiteBookStats. See the note at the bottom about what has been removed from that list already. User:DavidBrooks/EncycStats is the larger challenge, since encyclopedia entries often use CITEREFs containing author names, not editor names, so there is a lot more variety. – Jonesey95 (talk) 16:36, 27 May 2020 (UTC)

I'm currently clearing {{harvnb|Smith|1654|23-24}} type of stuff. I think I have about half done. Will probably do the rest tomorrow morning. Headbomb {t · c · p · b} 00:50, 28 May 2020 (UTC)

Template talk:The History of al-Tabari § proposed changes to this template similar to the update to {{EI2}}.

—Trappist the monk (talk) 14:47, 1 July 2020 (UTC)

I have updated {{The History of al-Tabari}} and made appropriate changes to the ~/whitelist.

—Trappist the monk (talk) 16:30, 8 July 2020 (UTC)

Russian occupations of Beirut and {{Naval Wars in the Levant 1559–1853}}

It's showing up as an error. What's the solution? Renata (talk) 05:39, 15 July 2020 (UTC)

This addition to the whitelist appears to have fixed it. – Jonesey95 (talk) 06:33, 15 July 2020 (UTC)

False positives with Template:EI3

See Abu Sa'id Gardezi. Headbomb {t · c · p · b} 13:41, 30 April 2020 (UTC)

Neither {{Encyclopaedia of Islam, THREE}} nor its redirect {{EI3}} is recognized as a citation template because the template names do not begin with cite. Bosworth 2013 can be added to the whitelist.

—Trappist the monk (talk) 14:26, 30 April 2020 (UTC)

Added to the whitelist. – Jonesey95 (talk) 14:43, 30 April 2020 (UTC)

Another instance: Safavid occupation of Basra. Renata (talk) 07:29, 15 July 2020 (UTC)

Fixed. – Jonesey95 (talk) 13:18, 15 July 2020 (UTC)

False positive with Template:Cite DCB

False positive with {{Cite DCB}} e.g. Alexander_Mackenzie_(politician)#cite_note-FOOTNOTEForster1990-2 – Finnusertop (talk ⋅ contribs) 22:30, 15 July 2020 (UTC)

Fixed. – Jonesey95 (talk) 00:41, 16 July 2020 (UTC)

False positives with Template:Rybczynski2000

False positives with {{Rybczynski2000}} e.g. in Screw#cite_note-Rybczynski2000pp34,66,90-63. – Finnusertop (talk ⋅ contribs) 21:31, 17 July 2020 (UTC)

 Fixed. – Jonesey95 (talk) 22:34, 17 July 2020 (UTC)

Module:Footnotes/whitelist/sort

I have hacked a tool that aids the maintenance of whitelist{} in Module:Footnotes/whitelist. See ~/sort for an explanation.

—Trappist the monk (talk) 22:29, 21 July 2020 (UTC)

This is amazing. Thank you. It will make my (occasional) work on this whitelist slightly less tedious. – Jonesey95 (talk) 22:42, 21 July 2020 (UTC)

More false positive

Produced by {{Appletons'}}. See this before I changed it. Renata (talk) 06:41, 25 July 2020 (UTC)

Also {{Americana}} in Francisco José Furtado. Renata (talk) 07:18, 25 July 2020 (UTC)

And {{Cite LarousseXIXe}} in François Antoine Teste. Renata (talk) 07:27, 25 July 2020 (UTC)

And {{ODNBweb}} in Francis Grose. Renata (talk) 07:53, 25 July 2020 (UTC)

False positive, and Cite ADB

Jakob Friedrich von Rüchel-Kleist has a false positive: CITEREFBernhard_von_Poten1889 correctly links to an {{Allgemeine Deutsche Biographie}} reference. However, this brings up two observations:

• {{Cite ADB}} (606 transclusions) has redirects: {{Allgemeine Deutsche Biographie}} (421) and {{De-ADB}} (396) so perhaps they should be linked as a synonym group using one of the whitelist mechanisms. Only {{Cite ADB}} currently appears in the whitelist (11 entries).
• I don't know how many of these 1,423 transclusions are used in short references, but could get an estimate. If the number gets significant, perhaps the parameter inference could be automated? The template transcludes {{Citation}} directly and:
  • if |ref= is provided, it is respected, otherwise:
  • |author= is positional parameter 5. It's usually author full name.
  • |date= is indexed by positional parameter 1 (volume). There are 56 volumes and the lookup is a simple index in {{ADB/Jahr}}.

Or does Trappist's 16 June comment above say that this can't be done with positional parameters?

If it is possible, perhaps also look at the similar {{NDB}} (and its redirect {{Cite NDB}}) (1,167 articles), which also have 11 entries in the current whitelist. David Brooks (talk) 19:47, 29 July 2020 (UTC)

It looks like there are 68 articles using Template:Cite ADB in the "no target" category. Not all of them will have errors with Cite ADB. – Jonesey95 (talk) 22:51, 29 July 2020 (UTC)

Yes, and 15 using "Allgemeine...", plus four using "De-ADB". However, one of those four is Karl Koberstein, which doesn't claim to be a member of the error category. I guess that was because of the Rines 1920 citation, which Trappist whitelisted 4 days ago. What has to happen to refresh the category after a whitelist update? Anyway, main question: is it worth while to do a deep-dive textual analysis to fish out the actual errors? David Brooks (talk) 23:20, 29 July 2020 (UTC)

Well... of the 1,394 articles that use one of the three synonym citation templates:

• Most of them have no relevant short citation anyway
• 25 have an explicit |ref=, which in every case is a last name and year. But 3 of them have the year/name reversed, and one has the wrong year for the volume. I assume that whoever made those citerefs also created a matching sfn; I didn't check yet.
• 9 have |ref=none and 1 has a redundant |ref=harv.
• 33 have a sfn that matches the cite. Of those, 11 are whitelisted and 22 aren't (19 distinct values).

Given that, is it better to add those 19 values to the whitelist or add code to calculate the citerefs? David Brooks (talk) 19:58, 31 July 2020 (UTC)

Should have added: some of these use {{Allgemeine Deutsche Biographie}}, of which only one (Friedrich Wilhelm, Graf von Wylich und Lottum) uses a reference that's already whitelisted for {{Cite ADB}}. If the decision is not to change the code for this set of citation templates, I can publish the (current) list of 19 whitelist candidates. David Brooks (talk) 15:01, 1 August 2020 (UTC)

False positives with Template:Fernandez1983

False positives with {{Fernandez1983}} e.g. in George J. Mead#cite_note-Fernandez1983pp28-29-2. Is someone working toward a general solution toward templated refs? I've never been a huge fan of them to begin with, and this one only has mere 10 transclusions. – Finnusertop (talk ⋅ contribs) 16:53, 13 August 2020 (UTC)

No one is working toward a general solution toward templated refs because there isn't one; Module:Footnotes can't see into those templates. All that Module:Foonotes sees at George J. Mead is the wikitext for the template invocation: {{Fernandez1983}}. Were all such template names the same as the anchor ID, then yeah, a solution would be to simply decode the template name but, alas, they are not and we allow redirects ... I have added Fernandez1983 to the whitelist

—Trappist the monk (talk) 17:38, 13 August 2020 (UTC)

Perhaps we should enforce a strict naming convention, or even consider deprecating such template refs, say below a certain transclusion count. – Finnusertop (talk ⋅ contribs) 17:46, 13 August 2020 (UTC)

Some ADB false positives

I dropped bugging people about {{Cite ADB}} and its redirects, but for now can someone add to the whitelist:

• CITEREFWinkler1883, used by {{Allgemeine Deutsche Biographie}} in Karl Christian Erdmann von Le Coq and Karl Ludwig von Lecoq
• CITEREFBernhard_von_Poten1889, also by {{Allgemeine Deutsche Biographie}}: Friedrich Erhard von Röder and Jakob Friedrich von Rüchel-Kleist
• CITEREFBernhard_von_Poten1898, used by {{Allgemeine Deutsche Biographie}} in Friedrich Wilhelm, Graf von Wylich und Lottum and by {{Cite ADB}} in Carl Friedrich Heinrich, Graf von Wylich und Lottum

Thanks David Brooks (talk) 20:21, 13 August 2020 (UTC)

What's wrong in Hey Baby (New Rising Sun)

What's wrong with ref #37 McDermmott 2010? Is it because {{cite AV media notes}}? Renata (talk) 01:51, 19 September 2020 (UTC)

It was a typo in the last name within the sfn template. I fixed it. – Jonesey95 (talk) 02:05, 19 September 2020 (UTC)

Ha! Duh! I stared at it for few minutes and could not see it. Thank you! :) Renata (talk) 13:20, 19 September 2020 (UTC)

False positives with Template:The Governors and Judges of Egypt

As seen on e.g. Abd al-Rahman ibn Khalid al-Fahmi. – Finnusertop (talk ⋅ contribs) 18:31, 20 September 2020 (UTC)

 Fixed. – Jonesey95 (talk) 19:47, 20 September 2020 (UTC)

False positives with Template:Biblica

As seen on e.g. Adami-nekeb. – Finnusertop (talk ⋅ contribs) 18:34, 20 September 2020 (UTC)

 Fixed. – Jonesey95 (talk) 19:47, 20 September 2020 (UTC)