Module talk:Arguments

Iterator corruption

@Mr. Stradivarius: I found a subtle iterator corruption bug in this module.

local args = require('Module:Arguments').getArgs(frame)
for k, v in args do
 mw.log(k .. '=' .. (v or 'nil') .. ' ')
 if args[k .. 'somesuffix'] then
  mw.log('Found suffix for ' .. k)
 end
end

Attempting to read the somesuffix argument causes it to be memoized, adding it to the internal table, which apparently can corrupt the iterator and causes some arguments to be skipped. I've noticed this is only reproducible some of the time. Jackmcbarn (talk) 02:58, 13 April 2014 (UTC)

@Jackmcbarn: That's a good find. (I assume that should be pairs(args) on line 2 rather than just args?) We're running into the undefined behaviour mentioned in the next function docs: "Behavior is undefined if, when using next for traversal, any non-existing key is assigned a value." The way that the __pairs metamethod works in this module means that all the existing arguments will have been memoized before the user gets a chance to index the args table. So if a user queries an existing argument during the pairs iteration, there will be no problem, as it will already be present in the metaArgs table. The error occurs when the user queries a non-existing argument. The __index function is set up to memoize this in metaArgs as nilArg, a blank table. That means that it is possible to add these blank tables as new values to the metaArgs table, even after all the non-nil values have been copied over from the frame objects. I've put a fix in the sandbox for this that uses the metatable.donePairs flag to check whether or not the arguments have been copied across. If they have already been copied across, then the __index metamethod won't memoize nils at all. While this fixes the bug, not memoizing the nils might cause adverse performance for some modules. Take a look and see what you think. Also, maybe Anomie would like to check the fix before we put it up live? — Mr. Stradivarius ^{♪ talk ♪} 16:02, 13 April 2014 (UTC)

@Mr. Stradivarius: Yes, that should have been pairs(args). What about a flag that gets set while you're inside the pairs method, and while it's set, it memoizes nils to some other table, then when the flag gets unset, it moves them to where they really go? Also, related, if an argument is an empty string, it gets iterated over even if empty strings get converted to nils, which is unexpected. Jackmcbarn (talk) 17:47, 13 April 2014 (UTC)

@Mr. Stradivarius: I realized it's impossible for an iterator function to tell when it stops iterating (since the function calling it can return early, etc.), so that idea was out. Instead, I changed the way nils are memoized. They go to a different table now, which should solve that problem and the other problem at the same time. Thoughts? Jackmcbarn (talk) 23:14, 13 April 2014 (UTC)

Once I got that implemented, I had another idea. Once pairs runs, we don't need to worry about memoizing at all anymore, because everything from argTables we'll ever look at is already part of metaArgs at that point. Jackmcbarn (talk) 23:51, 13 April 2014 (UTC)

I think we should memoize after pairs runs, because users might query new keys that have nil values, and also because memoizing things the same way every time is simpler. I like your idea of using a nilArgs table rather than just putting a blank table in metaArgs. That will solve the iterator problem and allow us to use the same memoization scheme whether we have used pairs or not. Also, blank strings shouldn't be iterated over unless they are explicitly allowed, due to the way the mergeArgs function works (unless you found a bug in that as well?) — Mr. Stradivarius ^{♪ talk ♪} 05:25, 14 April 2014 (UTC)

After running pairs, though, you don't need to check argTables anymore, so it's not worth memoizing nil to nilArg, since you can just return nil either way. Won't the code in the sandbox right now work right? Jackmcbarn (talk) 18:28, 14 April 2014 (UTC)

Ah yes, you're quite right. I wasn't registering the fact that the new check meant that we bypassed the argTables check. I've added a comment and updated the module - hopefully everything should work now. — Mr. Stradivarius ^{♪ talk ♪} 08:12, 15 April 2014 (UTC)

@Jackmcbarn: Oops - we have been forgetting the problem of arguments being iterated over even if they are empty strings which get converted to nils. This would be solved by a nilArgs table, but is still present in the current version. I'll try and switch back to the nilArgs table version while keeping the formatting. — Mr. Stradivarius ^{♪ talk ♪} 19:47, 15 April 2014 (UTC)

@Mr. Stradivarius: Now that I think about nilArgs, I don't really like it since it's an extra table lookup. Maybe if nilArg is found while iterating, just skip it and go on to the next element (or change all nilArg to nil once we're in pairs). Jackmcbarn (talk) 19:49, 15 April 2014 (UTC)

I've implemented the nilArgs version in the sandbox. I think it is quite an elegant solution, despite being an extra table lookup. Skipping nilArg tables while iterating isn't easy, as we would need to implement an iterator inside of an iterator for each of __pairs and __ipairs. And changing all nilArg tables to nil once we are in pairs would mean we would have to run pairs on metaArgs after running mergeArgs to catch all of the nilArg tables that have been introduced by __index and __newindex. Using nilArgs to memoize avoids these problems and makes the code quite a bit shorter (take a look at the new __pairs and __ipairs functions). — Mr. Stradivarius ^{♪ talk ♪} 20:24, 15 April 2014 (UTC)

@Mr. Stradivarius: Okay, I guess I'm sold on it. I think I see a few subtle bugs, though; let me see if I can track them down. Jackmcbarn (talk) 20:35, 15 April 2014 (UTC)

Thanks for taking a look at it. If I have time tomorrow, I may rewrite the test cases in the way foretold in the fine manual. That should make tracking these subtle bugs slightly less hit-and-miss. — Mr. Stradivarius ^{♪ talk ♪} 20:59, 15 April 2014 (UTC)

Also, I found a bug in my code: __newindex wasn't properly overwriting nil arguments in metaArgs, which would have caused problems for both __pairs and __index. — Mr. Stradivarius ^{♪ talk ♪} 21:11, 15 April 2014 (UTC)

@Jackmcbarn: I've finished rewriting Module:Arguments/testcases, and I've also added some bad input tests and some iterator tests. I've tried my best to break it, but all the tests have passed so far. As expected, the main module fails four of the iterator tests. Are there any other ways you can think to break it? If not, I think it is time to update the main module. — Mr. Stradivarius ^{♪ talk ♪} 13:10, 17 April 2014 (UTC)

@Mr. Stradivarius: Looks good. I did add one extra check for performance reasons. Jackmcbarn (talk) 18:47, 17 April 2014 (UTC)

Wrapper templates

Please make the changes at Special:Diff/604718144/611675481. This adds support for the "wrappers" option. When set, it causes it to process parent arguments only if the parent is a wrapper, or frame arguments only otherwise. Jackmcbarn (talk) 00:25, 5 June 2014 (UTC)

Perhaps User:Mr. Stradivarius could check your code and apply this. — Martin (MSGJ · talk) 09:12, 5 June 2014 (UTC)

@Jackmcbarn: I'm not quite understanding what it means to say "if the parent is a wrapper". What kind of wrapper are we talking about? I can see that it would make sense to not try and index frame:getParent() if it's going to return nil sometimes, but the only time I can see this happening is if you call frame:getParent() on the current frame and then pass the parent frame to getArgs. Then again, there is probably something I'm missing, and I imagine that getting my head round this wrapper business will clear things up. As for general code review, local title, found = parent:getTitle(), false seems a little dangerous to me. That would break if for some reason frame:getTitle ever switches to outputting two values (unlikely, but possible), so I would put those statements on separate lines. Also, we should probably check that options.wrappers is a table, so that we can give people a more informative error message if they specify something like {wrappers = true}. — Mr. Stradivarius ^{♪ talk ♪} 09:43, 5 June 2014 (UTC)

@Mr. Stradivarius: A wrapper is a template that just calls a module, like Template:Infobox is a wrapper for Module:Infobox and Template:Edit protected is a wrapper for Module:Protected edit request. That's unrelated to the nil issue; I just fixed that at the same time since I had to modify that part of the code anyway. You're right that the main time getParent() would be nil is if you'd already called getParent() once, but the other time is if you call a module with a real frame through the console. I fixed the locals on the same line. Instead of throwing an error on non-tables, I made it turn it into a table, to handle the (very) common case where a module only has one wrapper. New diff is Special:Diff/604718144/611678252. Jackmcbarn (talk) 13:31, 5 June 2014 (UTC)

Ah, I see what this is doing now. So if getArgs is called from a wrapper template, and that wrapper is listed in options.wrappers, it only loads the parent args, thereby saving a lookup in the frame args each time a new argument is requested from the client module. And if the parent frame isn't listed in options.wrappers it assumes that a user is calling the client module directly through #invoke. That sounds like a useful feature to add. One thing I was wondering - would it complicate the code too much to not call frame:getParent() if options.frameOnly is set? I'm not sure how expensive frame:getParent is to call, but I think it would make sense to not call it if we don't have to. (But then again, frameOnly isn't used that much as an option in my experience.) — Mr. Stradivarius ^{♪ talk ♪} 00:57, 6 June 2014 (UTC)

@Mr. Stradivarius: I've made it do that. New diff is Special:Diff/604718144/611759842. Jackmcbarn (talk) 01:11, 6 June 2014 (UTC)

I've found one more optimization. Special:Diff/604718144/611760186. Jackmcbarn (talk) 01:13, 6 June 2014 (UTC)

I've added some comments: Special:Diff/604718144/611784069. The code and the test cases look good to me, so if you're happy with this then I think we're ready to update the main module. — Mr. Stradivarius ^{♪ talk ♪} 06:23, 6 June 2014 (UTC)

@Mr. Stradivarius: I'm happy with it. Jackmcbarn (talk) 13:58, 6 June 2014 (UTC)

Done Ok, it's updated. Let me know if you spot any issues with it. — Mr. Stradivarius ^{♪ talk ♪} 14:25, 6 June 2014 (UTC)

Protected edit request on 5 July 2014

Please make these changes. This allows wrappers to still give both sets of arguments in either of the cases if such behavior is explicitly requested, while still preventing the double lookup in the other case. Jackmcbarn (talk) 03:30, 5 July 2014 (UTC)

Done If you could update the documentation too, that would be great. — Mr. Stradivarius ^{♪ talk ♪} 03:54, 5 July 2014 (UTC)

Integrating with Lua

I'm thinking of integrating this module into Scribunto, the same way Module:HtmlBuilder was, but to do that, it needs to be released under a different license. @Mr. Stradivarius: @Anomie: Do you both agree to release your contributions to this module under the GNU General Public License v2 or newer (GPL v2+)? Jackmcbarn (talk) 14:58, 3 September 2014 (UTC)

Sure. Anomie⚔ 15:07, 3 September 2014 (UTC)

Yes, that's fine with me. — Mr. Stradivarius ^{♪ talk ♪} 21:50, 3 September 2014 (UTC)

I've submitted gerrit:158323 that will add this to Scribunto. Note the following differences between this module and what I submitted:

• Instead of taking a frame and an options table, it now takes only an options table, and frame is one of its options. This makes it a standard named-arguments function.
• When wrappers aren't in use, it behaves as if frameOnly were set by default. Indiscriminate mixing of frame and parent arguments without knowing what the parent is has caused subtle bugs in the past, and it doesn't appear to have any legitimate use cases.
• If you want just the parent arguments, pass frame:getParent() in place of frame when calling it. The parentOnly option has been removed.
• When wrappers are in use, if the caller specifically requests frame arguments in addition to parent arguments (via wrappersUseFrame), the parent arguments always have precedence.

@Mr. Stradivarius: ping. Jackmcbarn (talk) 21:09, 4 September 2014 (UTC)

Implement i18n

Should this module implement i18n? Eg. by allowing a second parameter (boolean), which will make it try to load a name-map from a sub-module. Eg. Bananas could use

args = getArgs(frame, true)['firstname']

and Bananas/i18n_de could contain

return { vorname = 'firstname' }

(I admit a mapping in the opposit direction is more intuitive, but this allows multiple parameternames to be mapped to the same lua-parameter.) Poul G (talk) 09:40, 1 November 2014 (UTC)

The danger with this idea is that it makes it more difficult to use the module in different languages, unless everyone uses the canonical name anyway. For example, "Spezial:Beobachtungsliste" works if you go to dewiki but doesn't here or most other-language wikis, while "Special:Watchlist" will work everywhere. Anomie⚔ 14:38, 1 November 2014 (UTC)

WhyTF do we have a soft-redirect at Spezial:Beobachtungsliste? Anomie⚔ 14:45, 1 November 2014 (UTC)

Well, our user-editors in non-english languages should have access to templates in their native language. But at the same time it would be a great advantage to share the logic in the Lua-modules. Which implies that a translation is needed; it could be in the template {{#Person:name|firstname={{{vorname|}}}|...}} or be hidden in i18n within the modules configuration. (Maybe it was a mistake to open this on the site, where translation isn't needed.) Poul G (talk) 12:45, 3 November 2014 (UTC)

Is it possible this module to handle named parameters with Unicode names like:

{{my_template | unnamed_1 | параметър = 123 | named_2 = ... etc.}}

--Pl71 (talk) 15:32, 24 February 2016 (UTC)

Pairs bug

I've just found a bug in the pairs code of this module. It turns out that if you delete a value in the args table by setting it to nil, the the value is still present if you access the args table with pairs. There's a demonstration of the bug in my sandbox, and I've added two new test cases which are currently failing.

To fix this, it looks like we would need a new table to memoize nils. We need to check whether an argument has been expressly deleted when calling mergeArgs, but at the same time, values in nilArgs need to be overwritable for precedence whitespace arguments to work properly. I don't see how one table can fulfil both functions.

Alternatively, we could get away with using one nilArgs table if we change the module to only ever check the frame args or the parent frame args, and never both. If I remember rightly, this is what the proposed getArgs function inside Scribunto does, so if that solution seems better we could wait for that function to be deployed and then switch all of our existing modules over to it. — Mr. Stradivarius ^{♪ talk ♪} 06:59, 9 December 2014 (UTC)

@Mr. Stradivarius: There is one edge case in the new getArgs function that would still read both, so that won't save us. However, I did find a way to make it work without adding an additional table. It's in the sandbox. Jackmcbarn (talk) 15:11, 9 December 2014 (UTC)

@Jackmcbarn: Yes, that looks like a good approach to solving it - definitely better than introducing another table. Instead of using trinary logic, how about using strings to denote the status, similar to what Lua does with __mode? I think that would make the code more readable. We could use 'hard' and 'soft' for hard and soft nils, or just 'h' and 's' if we want to be concise. — Mr. Stradivarius ^{♪ talk ♪} 15:55, 9 December 2014 (UTC)

@Mr. Stradivarius: Okay, that's done. Jackmcbarn (talk) 04:11, 10 December 2014 (UTC)

@Jackmcbarn: Looks good. Unless there's anything else you would like to change, I think we're ok to update the main module now. — Mr. Stradivarius ^{♪ talk ♪} 04:37, 10 December 2014 (UTC)

@Mr. Stradivarius: Actually, there is, but I can't do it yet. Once the inexpensive mw.title.new change gets here, I want to make this use mw.title.new to normalize wrapper names (to make less stuff break when our modules get transwikied to wikis with different namespace names). Jackmcbarn (talk) 03:57, 11 December 2014 (UTC)

@Jackmcbarn: Ok, but I don't think there's any need to wait for that before we fix the current bug. I'll go and update the module now. — Mr. Stradivarius ^{♪ talk ♪} 04:27, 11 December 2014 (UTC)

Okay. Jackmcbarn (talk) 04:28, 11 December 2014 (UTC)

Ipairs bug

@Mr. Stradivarius: I discovered that calling ipairs() on args and then breaking out of the loop early will unnecessarily result in all of the numeric arguments being expanded, instead of just the ones that were iterated over. I added a test case for this and implemented a fix in the sandbox. Can you take a look at it? If it looks good, I'll add it (along with the other change waiting in the sandbox) to the main module. Jackmcbarn (talk) 05:32, 28 December 2014 (UTC)

@Jackmcbarn: Sorry for the delay in replying. Actually, yesterday and today I've been a bit ill, so I don't really trust myself to do code reviews right now. I'll take a look at this when I have my higher brain functions back, or if you want to go ahead and implement your fix anyway, that's fine by me. — Mr. Stradivarius ^{♪ talk ♪} 06:27, 29 December 2014 (UTC)

Document argument translation system

Hi Jackmcbarn !

Could you please add documentation about this? Helder 11:12, 1 September 2015 (UTC)

I'd rather not encourage its use right now, since a better but incompatible way will become available soon. Jackmcbarn (talk) 19:23, 1 September 2015 (UTC)

@Jackmcbarn: can you provide more details about that? Where is that new way being developed? Helder 19:30, 13 September 2015 (UTC)

@He7d3r: It's already written; it's just awaiting approval. You can see it at gerrit:158323. Jackmcbarn (talk) 20:58, 13 September 2015 (UTC)

Jackmcbarn, Helder, now that we have tabular data support, we can easily implement global translations. I already started on doing it with the TNT module. It allows a module or a template to be copied anywhere without modifications, and all localization is done in one place on Commons. This means we can introduce parameter localization as well, without any core changes. Let me know if you want to help with it :) --Yurik (talk) 03:46, 13 January 2017 (UTC)

Cool! I'll keep an eye on that. Helder 12:01, 18 January 2017 (UTC)

Using ipairs

I have not seriously used Module:Arguments so now that I'm looking at how it is used in Module:Team appearances list, I am puzzled about the default options and ipairs. What happens if a module does the following?

local getArgs = require('Module:Arguments').getArgs
local args = getArgs(frame)  -- where frame is from a template invoke
for i, v in ipairs(args) do
    print(i, v)
end

I gather that works as expected with something like {{example|one|two|three}} (and it would trim any leading/trailing whitespace from each parameter).

However, it would only process "one" in {{example|one||three}} because the blank second parameter would be converted to nil, and that would terminate ipairs.

Does that mean that anything using Module:Arguments with a variable number of numeric arguments must use something like compressSparseArray from Module:TableTools (or set options to not trim/remove parameters)? If that is true, I would have thought it would be mentioned in the documentation here. Did an early version of Module:Arguments default to removing blank parameters so ipairs processes each provided numeric parameter (that's what I thought happened)? Johnuniq (talk) 03:25, 18 November 2016 (UTC)

@Johnuniq: Have you found a solution for this problem? — UnladenSwallow (talk) 01:58, 15 May 2020 (UTC)

@UnladenSwallow: I haven't looked for a solution because I don't like obscure layers. Module:Arguments appears to be very efficient although it appears to do quite a lot of work, yet it seems unnecessary overhead to me. I've never needed the module and I don't know if the above is a problem now. Johnuniq (talk) 03:38, 15 May 2020 (UTC)

Help in writing better testcases

Hello developers, I am working with mw:Multilingual Templates and Modules and to convert this module into a shared one, we need better mw:Module:Arguments/testcases. Can anyone please help? @Frietjes, RexxS, Johnuniq, Mr. Stradivarius, Anomie, Xaosflux, Ans, Jackmcbarn, and Jonesey95: Capankajsmilyo^{(Talk | Infobox assistance)} 10:39, 22 May 2019 (UTC)

The Arguments cannot contain "="?

see test case: Module:Sandbox/shizhao/test，Module talk:Sandbox/shizhao/test。 If arguments contain "="， Lua error: bad argument #1 to 'match' (string expected, got nil).--Shizhao (talk) 15:45, 21 January 2020 (UTC)

@Shizhao That's just standard procedure: in a template or module invoke, a parameter like aaa=bbb appears as a named parameter with value bbb for parameter aaa. See the fix in diff. I haven't looked at what Module:Sandbox/shizhao/test is for but please don't fuss with a signature—keeping them simple is best and editors should not need a module for a private purpose. Johnuniq (talk) 22:43, 21 January 2020 (UTC)

Using frame and parentFrame simultaneously

There is a wrapper template Template:Authority control (arts) which uses {{#invoke:Authority control|show=arts}}

And on articles, I want to use {{Authority control (arts)|show=CZ,ES}} to show additional things.

I would like to concatenate both of these values in a comma-separated list, i.e. show = arts,CZ,ES} Is that something which is possible with this module? — Martin (MSGJ · talk) 13:17, 20 January 2023 (UTC)

@MSGJ: No - this module can get values from both the frame and the parent frame, but one will take priority and overwrite the other. You will need custom logic to do what you want to do. — Mr. Stradivarius ^{♪ talk ♪} 13:56, 20 January 2023 (UTC)

Okay, thought so. Thanks for the reply. — Martin (MSGJ · talk) 15:16, 20 January 2023 (UTC)

Case-insensitive arguments

I've had a request for case-insensitive argument names in a client module of Module:Arguments over on enWS (that imports from enWP). Specific case requested was treating |Volume= as equivalent to |volume= (there are historical reasons why contributors on enWS expect that to work). At first blush that looks really messy to implement in the client module, and at the same time it seems like something that 1) would be best implemented as an option (ala. |trim= or |removeBlanks=) in Module:Arguments and 2) would be generally useful for other clients of Module:Arguments. Without digging too deep I imagine it'd really be "canonical casing", i.e. just internally lower-casing all provided argument names when the option is set.

Thoughts? Xover (talk) 06:25, 31 May 2023 (UTC)