User talk:Kaniivel/Reference Organizer

Works nicely

Tested here and works very well; one step up from References segregator. Thank you, Cumbril! Did you consider using {{R}}? IMO it makes the text even easier to read in edit mode, especially when it wraps multiple refs. — Sam Sailor 18:53, 8 February 2017 (UTC)

@Sam Sailor: I'm glad you like it! Didn't know about {{R}}, Will think about adding it as an option. (I think most wikis don't have this template.) Cumbril (talk) 07:25, 12 February 2017 (UTC)

@Sam Sailor: Added support for {{R}}. No support for wrapping multiple refs though yet. If you want to turn it on, put the following into your common.js (before the line that calls the script):

if ( typeof( refConsolidateConfig ) === 'undefined' ) refConsolidateConfig = {};
refConsolidateConfig.usetemplateR = true;

Cumbril (talk) 15:45, 2 March 2017 (UTC)

​Sounds great, thank you; looking forward to test it. Thank you again for the time you put into scripts like these. Best, Sam Sailor 12:10, 9 April 2017 (UTC)

@Sam Sailor: I notice now that the script can ony write in {{R}} format, it does not read the format. Once references are converted to {{R}} format, the script no longer detects them. I don't know when or if I will add full {{R}} support. This is what it is for now. Cumbril (talk) 19:31, 9 April 2017 (UTC)

​Yeah, you are right, but I don't think it's a big problem. Sure, there's likely a bunch of slightly different scenarios that could result in some sort of mess, say if you run References Consolidator on an article that already has LDRs and uses {{R}}, and you change the name of ref, then References Consolidator will change the ref=name in the ref tags but not change the corresponding R template. But that ought to be very noticeable as it will result in error messages like

Cite error: The named reference BUNNYBUNNY was invoked but never defined (see the help page).

Cite error: A list-defined reference named "BUNNYBUNNY" is not used in the content (see the help page).

in the preview, and I expect that users who are aware of LDRs and are using References Consolidator will notice this before saving the page. :-) N'est-ce pas? Sam Sailor 18:18, 10 April 2017 (UTC)

Suggestion

Read about this in the Signpost. I use a jury rigged version myself and would love to switch to something better maintained. Re: If a reference does not have a name, the script gives it a randomly generated name of five characters. I opted against this, since five random characters doesn't do much good for identifying a ref without needing to scroll down to the list-defined refs. What about instead using the publication name and the most interesting/identifiable word from the title. This keeps the ref name identifiable/useful. E.g., "Edson Chagas' photographs are simple and striking" → <ref name="Design Indaba: striking">. "Most interesting" is subjective, of course. I currently use whatever word is the longest in the title, and just hope it's the most identifiable, but it would be nice to filter out words that are also in the WP article title, in case those are the longest words. czar 04:20, 9 February 2017 (UTC)

Thanks for a feedback, Czar! I agree that randomly generated five characters are not descriptive about the ref. But the problem with choosing a word from ref is that all refs are different – some use templates, others are plain, etc. So it wouldn't work very well in all of the instances. (Though it could be made to work in the instances where reference was described using a template and that template has filled title field. Giving all other refs that don't have title field, a random name?) Another problem to solve is making sure that the name is unique. Titles can repeat. This could be solved by using title + random string, or verifying that the title is indeed unique.

I am thinking about adding pop-up window to the script, where user can see all reference names and their content, and can give the refs meaningful names. The script would check if the names are unique and wouldn't let it go forward until they are. I would add this as an option. So a user can take extra time and give the refs meaningful names. Another user can still run it without any input, if he is OK with randomly named references. What do you think about this solution? Cumbril (talk) 07:25, 12 February 2017 (UTC)

The pop-up could work! It'd be nice if it was even stored in memory (able to be refreshed), so users can change the ref names without having to lose their place in the text to find the right list-defined ref at the bottom of the page. In the pop-up, you could even specify whether you prefer random characters or some amalgam from the source's title. In my experience, I've found few conflicts with the ref naming method I described, and the pop-up could help users verify the automated process to their liking. But the key benefit is that it's human readable and rememberable. When you're in an article with a hundred refs, it becomes a pain to remember which one is which. czar 17:03, 20 February 2017 (UTC)

@Czar: Now the tool has pop-up screen, where user can change reference names. I don't understand what you mean by "stored in memory". Changing references in the edit window while the pop-up screen is open is not possible – the screen has the state of article at the moment of the script's execution; changing the article text while the script has not finished processing would make the screen and article go out of sync. If you change the article text, you just have to run the script again. Cumbril (talk) 18:58, 17 March 2017 (UTC)

Nice. Would it be possible to text wrap so that the window doesn't have to scroll to the right? By stored in memory, I meant something like that pop-up for recalling the ref names without having to scroll to the bottom of the text (I said memory because I didn't expect this to parse so fast on the fly). Now the bigger question... what if the script detected bare URLs, and added a filled checkbox in a third column of the pop-up, and filled them with Citoid (script)? czar 21:58, 17 March 2017 (UTC)

@Czar: Scrolling to the right happens because of long unbroken reference lines without any spaces. There are some ways around it, like introducing space character into the string somewhere, or putting each reference into its own scrollable area. Still thinking about the best solution.

I'm also planning to make reference text itself editable, so introducing spaces into references might not be the best idea. Also want to make columns sortable (and give user option to save references in the chosen sorting order). Plus, I want to give user an option to choose the number of citations (uses) at which reference is moved into bottom list. And I want to give option to use Template:R (already possible, but it would be more user friendly if it was a checkbox on the screen). I see this script's purpose more oriented towards organizing references and moving them around in bulk inside article versus editing individual references. There is already a tool that is geared towards editing references individually – ProveIt. Integration with Citoid would be nice option, but I feel it's not central to the script's purpose. Maybe if all this other planned stuff is done, I would tackle this. Unless someone else wants to write code for it :). Cumbril (talk) 22:50, 17 March 2017 (UTC)

I think it would be fine to break on spaces between the parameters and perhaps do a soft wrap on URLs. I prefer {{R}} personally, so kudos for that. I linked Salix alba's Citoid script above, and I could help prepare that processing if you could help hook it into yours. (I use my own jury-rigged script to export citations from Zotero.) Does your script stack refs when identical and duplicated? Also two stylistic questions: Why is there a space between the close quotes in the ref name and the slash? I also usually put a new line between list-defined refs (makes it much easier to read)—any chance that might be supported? I wouldn't worry so much about giving options to order refs in different ways. I've rarely seen list-defined refs (LDR) used non-alphabetically except when it was thrown together by accident. If the page's editors want a specific kind of grouping, they will have some kind of notice to let editors know (and I imagine would eventually contact you to have some kind of template flag to opt-out). Thanks for your work on this—looking forward to using it more czar 03:24, 18 March 2017 (UTC)

@Czar: Please see User:Cumbril/References_Consolidator#User_options to turn on support for {{R}} and alphabetical sorting. Yes, the script stacks duplicate refs. For unnamed refs it compares ref content with named refs content and puts refs under existing name, if content is the same. For named refs, it compares with other named refs, if it finds two refs with the same content, one name gets dropped. The behavior is more thoroughly outlined here. The space between ref name and slash is consistent with ProveIt. I understand that different users have their own accustomed ways of formatting references and reference lists. Things also differ in different Wikipedias. For example, in my home wiki I haven't yet seen LDR references ordered alphabetically, ordering refs by putting the most used refs first makes as much sense to me as alphabetical ordering. Most refs formatting options would be easy to add, but I think its best add only the variants used by significant number of users, to not get overwhelmed in options. Cumbril (talk) 18:38, 18 March 2017 (UTC)

I think the two modifications I mentioned (no space before reference slash and extra line break between refs) are relatively popular on enwp, but I suppose I could modify a copy myself if the mod doesn't fit the scope czar 21:20, 18 March 2017 (UTC)

@Czar: I think support for these two formatting options is justified (I have seen both styles used in enwiki), but I want to add general code that would allow every script user to feed in formatting formula according to his/her likes. Because adding every imaginable option individually would soon lead to horrible code. For every user that likes one style, there will be another that likes another style. Cumbril (talk) 23:48, 18 March 2017 (UTC)

To make matters short: there will be support for both options. Bear with me. :) Cumbril (talk) 00:13, 19 March 2017 (UTC)

Little bug

Hi Cumbril! It's been a while, so I hope this find you well. Have a look at this diff: Special:Diff/816315199/822949573. Thank you once again for a really nice gem of a script! Sam Sailor 13:08, 29 January 2018 (UTC)

Another thing, not really a bug: in Special:Diff/822985623 I had inserted two references, but had not created a / References / section with a * {{reflist}}. In that case Reference Organizer will simply exit, when you try to run it. User:TheJJJunk/ARA.js detects and inserts a missing reflist fine, perhaps that code can be of inspiration? Thanks, Sam Sailor 17:28, 29 January 2018 (UTC)

User defined variables for Reference list template

Hi again Cubril.

Would it be possible to set the "References location:" and "Other choices:" in the Reference list template as a variable in one's common.js so, say, "all references are located in reference list template(s)" and "save in the selected sort order (click on headers to sort)" are set by default?

Thanks, Sam Sailor 12:02, 30 January 2018 (UTC)

/ in refname

Hi again Cumbril. It's rare to see a forbidden character in the WP:REFNAME, but here was a case, look for <ref name="Chantelle/Brian Bromfords">. Fixed in the subsequent diff. Probably very rare scenario, just wanted to mention it. Sam Sailor 12:24, 12 February 2018 (UTC)

Enable in ns:118

Hi Cumbril, I hope all is well with you. It would be nice to have RO enabled in draft space. Thank you, Sam Sailor 17:52, 14 July 2018 (UTC)

Feature request--alphabetization

Is it possible to make the script also alphabetize organized references? This would make them much easier to find in the {{reflist}}. Thanks. ―Justin (koavf)❤T☮C☺M☯ 04:31, 9 September 2018 (UTC)

bug in 'The script keeps user given reference names': it deleted many names

The intended feature list states: "The script keeps user given reference names." But in this edit a whole lot of user-defined reference names were removed. I assume that this is a bug. Just because a reference is not presently used more than once does not mean it will not be needed to be used multiple times in the future (e.g. because a paragraph that is initially justified by a single end-of-paragraph source gets split up into individual sentences justified from different sources - the first and last sentence might both need to be individually referenced, even though no new info is used from the source). (I restored these by hand, though I guess maybe I should have undone the script-based edit...) Boud (talk) 15:53, 15 April 2019 (UTC)

Pausing on confirm on larger articles

Hi! I love your script, it's ultra helpful. I have recently been having issues when attempting to use this on articles with quite a few references (50 or more), such as the 1985 World Snooker Championship. The script works, and reads all of the references in the article. However, when clicking on confirm, the box just hangs on screen, and nothing happens. I've tried the script on smaller articles (works fine), and other longer articles (has issues). Is this a known fault, or am I doing something stupid? Best Wishes, Lee Vilenski ^{(talk • contribs)} 07:55, 9 September 2019 (UTC)

I just encountered this issue with this article that has 87 references. The script loaded fine, I renamed most of the references, selected "all references are located in reference list template(s)", and pressed Continue. Nothing happened, the Continue button was now greyed out so I couldn't press it again. I waited several minutes but nothing happened.

I'm using Firefox 69.0.1 64-bit on Linux Mint 19. I have several extensions installed, uBlock Origin, uMatrix , and HTTPS Everywhere however I have disabled on Wikipedia as they are unnecessary on the site. Other extensions I have installed are Decentraleyes, Enhanced Steam, Firefox Multi-Account Containers, Google search link fix, I don't care about cookies, Old Reddit Redirect, Reddit Enhancement Suite, Send to MPV player, and Temporary Containers. --Veikk0.ma 20:54, 2 October 2019 (UTC)

Small formatting improvements

Very useful script. It'd be great if:

1. The citations didn't include a space between the name of the reference and the forward slash. Currently citations of list-defined references are formatted as <ref name="test1" /> instead of <ref name="test1"/>, which is a bit annoying. It can be changed manually with the search-and-replace tool, which is what I'm currently doing, but it's an extra step in the process. In RefConsolidate.js, this part of the string seems to be in line 1625.
2. When moving to list-defined references, references inside the reflist template were separated from each other by two newlines for better readability (see this article's reference section for an example). Currently only a single newline is used, which causes all the references to look like they're are lumped together as a single block of text. I've not figured out how to do this with the search-and-replace tool, so I'm currently doing it manually, which can be time-consuming when there are lots of references.

--Veikk0.ma 14:54, 19 December 2019 (UTC)

if it is not broke, don't "fix" it

There is a principle in engineering and other technical disciples often summarized as "if it ain't broke, don't 'fix' it". I see this tool in alarming violation of this principle, as in this edit

I think policy strongly discourages human contributors from unnecessarily rewriting metadata that work, for purely aesthetic reasons.

The unnecessary rewriting of metadata, or computer programs, opens the possibility that poorly thought out rewrites, or typos, will turn stuff that works into stuff that doesn't work.

The standard diff engine relies heavily on newlines. When a paragraph was initially written as one long logical line, inserting one of more newlines into that paragraph will blind the diff rendering engine from recognizing other changes. When a paragraph was initially written with multiple lines, and newlines are removed, for aesthetic reasons, this again blinds the diff rendering engine from recognizing other changes, to grammar, spelling, punctuation, word order.

It is best for human editors to restrict themselves to corrections with as light a hand as possible, in order to let the standard diff rendering engine work properly.

The same holds true for templates like the {{cite}} template. Human contributors who fix actual errors in template metadata should do so with as light a hand as possible. In particular, they should not combine multiple lines. It best if a diff from the revision where a template was added, to the current revision, only shows actual corrections, and is not obfuscated by merely aesthetic changes.

This edit I used as an example added unnecessary quotes around the name value in instances of <ref name=> and <ref name=/> It is a lapse from the "if it ain't broke, don't 'fix' it" principle.

It also rewrote all references from using one field per line to placing all fields on a single logical line. Woah! Did the script make any actual genuine corrections? Who knows? Messing around with newlines means the standard diff rendering engine is blind to those changes. This was another lapse from the "if it ain't broke, don't 'fix' it" principle.

I strongly urge the author of this tool, and all other tool authors, to make sure their tools make changes to articles with as light a hand as possible. Geo Swan (talk) 23:46, 11 August 2020 (UTC)

There is another principle used in engineering and other technical disciples often summarized as "don't do things in different ways in different places unless you have a good reason", AKA "Be consistent". This makes life easier for automated tools and search functions.

If you have any actual examples of Reference Organizer "turning stuff that works into stuff that doesn't work", please post them so the bug can be fixed.

If you see any example where you think Reference Organizer made the wrong decision, as opposed to just opposing any change, please post them so we can discuss them.

If, like me, you think our diff engine sucks, I suggest asking the W?F to fix it rather than asking everyone else to accommodate the suckiness. --Guy Macon (talk) 19:21, 11 November 2020 (UTC)

I agree. Stop making pointless and unhelpful mass changes to references. Especially stop making them based on mis-readings of guidelines such as thinking it is mandatory to place reference names in quotations. This has been discussed ad infinitum and quotations are required only if the names contain spaces (and some other strange characters?). Please don't just add them for the sake of it, and please don't add reference names that are inconsistent with article naming conventions (and completely unintelligible) for no reason. Lithopsian (talk) 20:26, 11 November 2020 (UTC)

The edit linked at the top of this section is a clear violation of WP:CITEVAR. If individual editors have control over the formatting of reference names and citation template parameters when using this tool, then the editor needs to be admonished to preserve CITEVAR. If the tool preserves CITEVAR only in certain situations, then its documentation should clearly state those limitations. – Jonesey95 (talk) 22:49, 11 November 2020 (UTC)

Does this script improve Wikipedia? Verifiability and future editing seem more difficult after the script is run on a article.

Some points to ponder

• perhaps the best modification to the Reference Organizer script would be a different output mode. The requested inline mode would put the references inline, rather than accumulate them for deposit in the {{reflist}}. By default, the first use of the reference would also contain the definition of the reference. This alternate output mode would be a very helpful debugging tool for articles with funky source wikitext.
• An inline reference immediately follows the facts it supports. This script moves the reference away from the facts. Future editors, examining the article via visual or source/wikitext editing now have to edit the whole article, rather than using article section editing. For large pages, editing the whole article can tax both one's patience (find the area to change) and also one's browser resources (editing and rendering the entire article).
• Unlike inline refs, VisualEditor currently (15 November 2020) does not support list-defined references in the "Visual editing mode".

Attempting an edit gives the message:

Reference
This reference is defined in a template or other generated block, and for now can only be edited in source mode.
This reference is used twice on this page.

or if attempting to edit the References list

References list
List of general references
This reference list is generated by a template, and for now can only be edited in source mode

• From Wikipedia:Citing_sources#To_be_avoided

  To be avoided:
  When an article is already consistent, avoid:
  ...
  • changing where the references are defined, e.g., moving reference definitions in the reflist to the prose, or moving reference definitions from the prose into the reflist.

  This script has been applied to articles resulting in prose-embedded reference definitions being moved in bulk into the reflist.
• Confusing to novice editors: For instance, VisualEditor, a.k.a. 2017 wikitext editor, produces inline ref when adding using the Cite tool. Also when editing in "Source Editor" (wikitext) mode, the novice editor has to move between the prose and its following ref tag and the reference definition in the References section.
• Future manual removal of refs embedded in prose text may cause an error, as the corresponding ref definition in the reference list also needs to be removed after the last use has vanished. The help addresses this Help:List-defined_references#Guidelines ...any unused references will generate an error ... Future editing that removes the last (or only) instance of an inline ref will cause this error. While removing the defining named ref causes a similar problem for inline references, in my experience, many references are only used once.
• Future edits to prose or supporting reference are not linked. Current inline changes to prose or supporting reference are visible in current, built-in, Wikipedia version diff tool. Particularly affects edits to quotes embedded in a reference, or the corresponding prose.
• Automatic rearranging from inline references to list-defined references makes validation difficult with existing tools. For example, this set of 29 changes is hard to check using the standard diff as shown : https://en.wiki.x.io/w/index.php?title=Sterling_Hall_bombing&diff=next&oldid=988644735
• Personally, I tend to monitor the watchlist following my last edit to an article. This helps me see when I've generated mangled citations, e.g. duplicate parameters in a template. This script makes it difficult to see if a following edit is a fix to something I've messed up, a revision makes me rethink my edit or citation.
• Organization is changed for a questionable benefit. The rendered page looks the same whether the refs are list-defined or inline.
• The list of references rendered is the same for both inline and list-defined references.Prose and references are separate when rendered: this allows the reader to absorb the content. Supporting sources are visible through a click or mouse-over preview, and allow the user to read the content without undue distraction. Unless, of course, the prose has too many citations :-) The References section remains what the encyclopedia user will need to learn more about the source.
• Encourages editing of reflist separate from the prose text, via edit the references section on its own. This is probably will lead to a divergence between the prose and the references.

The ref element is designed to be inline, surrounding a citation with a starting tag and and ending tag:<ref>Source name, access date, etc.</ref>. The list-defined reference removes the inlining, and mandates a unique name <ref name="just_used_once" /> even when a source is only used once. The unique name must then be matched to the corresponding defining ref in the reflist entry. All the names in the list are now enclosed in the reflist, placing a reference amongst many other, possibly similar, references.

So in summary:

• Added complexity, previously unneeded in the case of a single in-line reference
• Separation of facts from sources, making future edits more difficult, including prose and source deletion
• Organization changes to article with no clear benefit
• No difference in the appearance of the rendered article
• Poor or no support by current diff and visual editing tools

Certainly the script is BOLD in its actions :-). Lent (talk) 06:07, 15 November 2020 (UTC)

I can agree with some of the OP's complaints, but name="..." in a ref should always have quotes, because people change the value sometimes and if they are not wikicoding experts paying close attention they will break citations by doing something like name=Johnson 2020 which is invalid.  — SMcCandlish ☏ ¢ 😼  21:27, 25 August 2023 (UTC)

Rounding

For some reason this tool doesn't work on the Rounding page. --Guy Macon (talk) 19:06, 11 November 2020 (UTC)

RfC announce: Citation tools

There is an RfC at Wikipedia talk:Citing sources#RfC: Citation tools regarding whetyer citation tools should allowed. Your input on this question is welcome. --Guy Macon (talk) 22:01, 11 November 2020 (UTC)

That's now at Wikipedia talk:Citing sources/Archive 50#RfC: Citation tools for anyone looking for it. It closed quickly (the gist is that there is no consensus against using such tools, and someone reverting non-erroneous edits made with them was being disruptive).  — SMcCandlish ☏ ¢ 😼  06:12, 25 January 2024 (UTC)

Bug report

In this edit[1] Reference organizer replaced <ref name="Zhang6April2020"> with <ref>, presumably because the "keep unnecessary article body reference names" wasn't checked. But it also changed <ref> to <ref name="ECDCQA">. adding an unnecessary article body reference name. --Guy Macon (talk) 23:50, 20 November 2020 (UTC)

That edit removed tons of reference names and removed commented-out references, which a script should not do. – Jonesey95 (talk) 06:22, 21 November 2020 (UTC)

I could not find any removed commented-out references, Could you tell me which ones you think were removed?

There is a checkbox for "keep unnecessary article body reference names". I normally check the box, something which I have been harshly criticized for doing. The user who made the edit we are discussing went with the default, which is to remove unnecessary article body reference names. Is there any discussion where there was a consensus for one or the other? --Guy Macon (talk) 08:01, 21 November 2020 (UTC)

Hi@Guy Macon:, thanks for the notification. The diff that you linked to is of the edit that undid the changes that were done by the script user. The string "ECDCQA" can be seen already in this version of the article. As for removing commented-out references, the script does remove all references in article body that are not used anywhere. I think that removing commented out references is not necessary, but this is how it works now. Maybe I will fix it in the code, if I can summon enough motivation. :) Kaniivel (talk) 12:32, 21 November 2020 (UTC)

Thanks! Right now there is a spirited discussion going on, mostly about whether to add quotation marks to single words. I think you can ignore that one for now. If there is ever a consensus to Wikipedia:Establish the bikeshed needs painting before you decide which color to paint it I will report here. I suspect that the answer will involve either doing nothing or changing the default so that "keep unnecessary article body reference names" is checked by default. --Guy Macon (talk) 17:18, 21 November 2020 (UTC)

2017 Wikitext editor incompatibility

This script does not appear to work with the 2017 Wikitext editor. I attempted to use this script while having the 2017 WT editor enabled, and, finding that it did not work, disabled the 2017 WT editor in my preferences. After doing so, this script worked perfectly fine as the documentation page described it to. It appears this script would need an update to be compatible with the 2017 WT editor. Joyce-stick (talk) 10:07, 29 August 2022 (UTC)

Forked it

I got annoyed that it output a random string of characters, so I modified it to output : followed by 6 numbers. You can then use RefRenamer to get logical names. The fork is here and the modified start.js over here. I hate JavaScript. ~ Argenti Aertheri^(Chat?) 06:06, 1 August 2023 (UTC)

@Argenti Aertheri

Hey there. I appreciate you forking the script, however I've encountered an error (both with the original script as well as yours), in that it fails to parse the $ character in the title parameter of citation templates. I've only tested this on cite web templates, so I'm unsure if this behaviour is consistent throughout all templates.

Sink Cat (talk) 20:56, 20 April 2024 (UTC)

Names

Please tell me it is not doing this If a reference does not have a name, the script gives it a randomly generated name of five characters to every single reference. That would be very unhelpful. A ref should not have a name="..." unless that ref is reused. The very presence of a name="..." is what tells other editors that the ref is being reused (i.e., so don't remove that cite or replace it without doing other cleanup, like moving the original citation details to where the cite has been reused, or changing the content at the later citations to agree with the new cited source). Even aside from that, assigning name="..." values to refs that are only used once is just excessive code bloat.  — SMcCandlish ☏ ¢ 😼  21:23, 25 August 2023 (UTC)

Using the User:Argenti_Aertheri/RefConsolidate_alt.js fork mentioned above, I found that it would assign such names to single-use references, in the script's own window, but not impose them in the actual content without doing extra stuff to force that to happen. Hopefully that's how the original works as well. The fork version even removed unnecessary ref names from sources used only once, which is good.  — SMcCandlish ☏ ¢ 😼  05:39, 25 January 2024 (UTC)