Wikipedia:Guide to Scribbling
{{Information page|WP:Scribbling|WP:Luafication}}
{{Nutshell|Scribbling involves converting templates to use Lua scripts for improved performance and maintainability by having thhe template invoke Lua-based modules. This enables efficient handling of logic and reduces the reliance on complex parser functions. For example,
File:Wikipe-tan the Library of Babel.png
{{toc right}}
This is a guide to Scribbling, also known as Luafication. Scribbling is an informal term used to describe the process of creating a new template or converting an existing one so that it invokes a module containing functions written in Lua script. The Scribunto{{efn|The name "Scribunto" is Latin. "scribunto" is third person plural future active imperative of "scribere" and means "they shall write". "scribble" is of course an English word derived from that Latin word, via Mediaeval Latin "scribillare".{{sfn|MW|2003a|p=1116}}}} extension enables the embedding of scripting languages in MediaWiki. Currently, Lua is the only supported scripting language. This guide provides an overview of Scribbling and points to additional resources for further learning.
Scribbled templates consist of two components: The template itself, and one or more back-end modules, stored in the Module:
namespace. These modules contain Lua programs that are executed on the wiki servers to generate the wikitext that the template expands to.
Templates invoke a function within a module using the
The main purpose and primary advantage of Scribbling is improved template processing performance, as Lua scripts are significantly more efficient than traditional template parser functions. Another benefit is simplified code. By using Lua, Scribbling eliminates the need for complex parser function programming, such as {{tlc|#if}}, {{tlc|#ifeq}}, {{tlc|#switch}} and {{tlc|#expr}}, which were originally designed as extensions to a template system rather than a true programming language.{{efn|For an idea of what "bolted-on" connotes when it comes to software design, see the Flintstones cartoons where the rack of ribs from the Drive-Thru is so heavy that it causes the Flintstones' car to fall on its side.}} By handling complex logic within the module, Scribbling reduces clutter in the template itself, making it easier to maintain and update. Additionally, Scribbling addresses the issue of expansion depth limits by eliminating the need for templates to transclude other templates.{{efn|It may need, until such time as the whole of the specified API for Scribunto is available to modules, to transclude magic words. See the tips and tricks section. Magic words are not templates, however.}}
Lua
{{further|Lua (programming language)|Help:Lua for beginners|{{FULLPAGENAME}}/Programmers' Quick start Guide to Lua}}
The language in which modules are written is Lua. Unlike the template parser function system, Lua was actually designed not only to be a proper programming language, but also to be a programming language that is suitable for what is known as embedded scripting. Modules in MediaWiki are an example of embedded scripts. There are several embedded scripting languages that could have been used, including REXX and tcl; and indeed the original aim of Scribunto was to make available a choice of such languages. At the moment, however, only Lua is available.
The official reference manual for Lua is {{harvnb|Ierusalimschy|de Figueiredo|Celes|2006}}. It's a reference, not a tutorial. Consult it if you want to know the syntax or semantics for something. For a tutorial, see either {{harvnb|Ierusalimschy|2006}} ({{harvnb|Ierusalimschy|2003}} is also available, although it is of course out of date.) or {{harvnb|Jung|Brown|2007}}. The downsides to these books are that quite a lot of the things that they tell you about have no bearing upon using Lua in MediaWiki modules. You don't need to know how to install Lua and how to integrate its interpreter into a program or run it standalone. The MediaWiki developers have done all of that. Similarly, a lot of the Lua library functions are, for security, not available in modules. (For example, it's not possible to do file I/O or to make operating system calls in MediaWiki modules.) So, much of what these books explain about Lua standard library functions and variables that come with the language is either irrelevant or untrue here.
The original API specification — the Lua standard library functions and variables that are supposed to be available in modules — is given at MW:Extension:Scribunto/API specification. However, even that is untrue. What you'll actually have available is documented in MW:Extension:Scribunto/Lua reference manual, which is a cut down version of the 1st Edition Lua manual that has been edited down and modified by Tim Starling to bring it more into line with the reality of Scribbling. Again, though, this is a reference manual, not a tutorial.
The things in Lua that you will mostly be concerned with, writing Scribbled templates, are tables, strings numbers, booleans,
Template structure
This is simple. Your template comprises one expansion of
{{#tag:syntaxhighlight|{{#invoke:Page|getContent|Template:Harvard citation|as=raw}}|lang="wikitext"}}
If you find yourself wanting to use other templates within your template, or to use template parser functions, or indeed anything at all other than
Module basics
= Overall structure =
Let's consider a hypothetical module, Module:Population. It can be structured in one of two ways:
== A named local table ==
local p = {}
function p.India(frame)
return "1,21,01,93,422 people at (nominally) 2011-03-01 00:00:00 +0530"
end
return p
== An unnamed table generated on the fly ==
return {
India = function(frame)
return "1,21,01,93,422 people at (nominally) 2011-03-01 00:00:00 +0530"
end
}
==Execution==
The execution of a module by
- The module is loaded and the entire script is run. This loads up any additional modules that the module needs (using the
require() function), builds the (invocable) functions that the module will provide to templates, and returns a table of them. - The function named in
{{#invoke:}} is picked out of the table built in phase 1 and called, with the arguments supplied to the template and the arguments supplied to{{#invoke:}} (more on which later).
The first Lua script does phase 1 fairly explicitly. It creates a local variable named p
on line 1, initialized to a table; builds and adds a function to it (lines 3–5), by giving the function the name India
in the table named by p
(p
. It could be named any valid Lua variable name that you like. p
is simply conventional for this purpose, and is also the name that you can use to test the script in the debug console of the Module editor.
The second Lua script does the same thing, but more "idiomatically". Instead of creating a named variable as a table, it creates an anonymous table on the fly, in the middle of the India
. To expand such a script with more (invocable) functions, one adds them as further fields in the table. (Non-invocable local functions can, again, be added before the
In both cases, the template code that one writes is India
from the module Module:Population. Also note that
One can do more complex things than this, of course. For example: One can declare other local variables in addition to p
, to hold tables of data (such as lists of Language or country names), that the module uses. But this is the basic structure of a module. You make a table full of stuff, and return it.
= Receiving template arguments =
An ordinary function in Lua can take an (effectively) arbitrary number of arguments. Witness this function from Module:Wikitext that can be called with anywhere between zero and three arguments:
Functions called by frame
in the parameter list of the function). It's called a frame because, unfortunately, the developers chose to name it for their convenience. It's named after an internal structure within the code of MediaWiki itself, which it sort of represents.{{efn|In MediaWiki proper, there are more than two frames.}}
This frame has a (sub-)table within it, named args
. It also has a means for accessing its parent frame (again, named after a thing in MediaWiki). The parent frame also has a (sub-)table within it, also named args
.
- The arguments in the (child, one supposes) frame — i.e. the value of the
frame
parameter to the function — are the arguments passed to{{#invoke:}} within the wikitext of your template. So, for example, if you were to write{{#invoke:Population|India|a|b|class="popdata"}} in your template then the arguments sub-table of the child frame would be (as written in Lua form){ "a", "b", class="popdata" } . - The arguments in the parent frame are the arguments passed to your template when it was transcluded. So, for example, were the user of your template to write
{{Population of India|c|d|language=Hindi}} then the arguments sub-table of the parent frame would be (as written in Lua form){ "c", "d", language="Hindi" } .
A handy programmers' idiom that you can use, to make this all a bit easier, is to have local variables named (say) config
and args
in your function, that point to these two argument tables. See this, from Module:WikidataCheck:
function p.wikidatacheck(frame)
local pframe = frame:getParent()
local config = frame.args -- the arguments passed BY the template, in the wikitext of the template itself
local args = pframe.args -- the arguments passed TO the template, in the wikitext that transcludes the template
Everything in config
is thus an argument that you have specified, in your template, that you can reference with code such as
Everything in args
is thus an argument that the user of the template has specified, where it was transcluded, that you can reference with code such as /doc
page.
See {{tl|other places}} and {{tl|other ships}} for two templates that both do x
, thereby obtaining different results from one single common Lua function.
For both sets of arguments, the name and value of the argument are exactly as in the wikitext, except that leading and trailing whitespace in named parameters is discounted. This has an effect on your code if you decide to support or employ transclusion/invocation argument names that aren't valid Lua variable names. You cannot use the "dot" form of table lookup in such cases. For instance: |author-first=
argument, but a reference to an |author=
argument and a first
variable with the subtraction operator in the middle. To access such an argument, use the "square bracket" form of table lookup:
Named arguments are indexed in the args
table by their name strings, of course. Positional arguments (whether as the result of an explicit 1=
or otherwise) are indexed in the args
tables by number, not by string.
Finally, note that Lua modules can differentiate between arguments that have been used in the wikitext and simply set to an empty string, and arguments that aren't in the wikitext at all. The latter don't exist in the args
table, and any attempt to index them will evaluate to
= Errors =
Let's get one thing out of the way right at the start: Script error is a hyperlink. You can put the mouse pointer on it and click.
We've become so conditioned by our (non-Scribbled) templates putting out error messages in red that we think that the Scribunto "Script error" error message is nothing but more of the same. It isn't. If you have JavaScript enabled in your WWW browser, it will pop up a window giving the details of the error, a call backtrace, and even hyperlinks that will take you to the location of the code where the error happened in the relevant module.
You can cause an error to happen by calling the
Tips and tricks
= Arguments tables are "special". =
For reasons that are out of the scope of this Guide,{{efn|If you want to know, go and read about how MediaWiki, in part due to the burden laid upon it by the old templates-conditionally-transcluding-templates system, does lazy evaluation of template arguments.}} the args
sub-table of a frame is not quite like an ordinary table. It starts out empty, and it is populated with arguments as and when you execute code that looks for them.{{efn|Don't be surprised, therefore, if you find a call backtrace showing a call to some other module in what you thought was an ordinary template argument reference. That will be because expansion of that argument involved expanding another Scribbled template.}} (It's possible to make tables that work like this in a Lua program, using things called metatables. That, too, is outwith the scope of this Guide.)
An unfortunate side-effect of this is that some of the normal Lua table operators don't work on an args
table. The length operator, #
, will not work, and neither will the functions in Lua's table
library. These only work with standard tables, and fail when presented with the special args
table. However, the
= Copy table contents into local variables. =
A name in Lua is either an access of a local variable or a table lookup.{{sfn|Ierusalimschy|de Figueiredo|Celes|2011|loc=§EVAL AND ENVIRONMENTS}} math
table, for example. Table lookups are slower, at runtime, than local variable lookups. Table lookups in tables such as the args
table with its "specialness" are a lot slower.
A function in Lua can have up to 250 local variables.{{sfn|Ierusalimschy|2008|p=17}} So make liberal use of them:
- If you call
math.floor many times, copy it into a local variable and use that instead:{{sfn|Ierusalimschy|2008|p=17}}
local floor = math.floor
local a = floor((14 - date.mon) / 12)
local y = date.year + 4800 - a
local m = date.mon + 12 * a - 3
return date.day + floor((153 * m + 2) / 5) + 365 * y + floor(y / 4) - floor(y / 100) + floor(y / 400) - 2432046
- Don't use
args.something over and over. Copy it into a local variable and use that:local Tab = args.tab (Even theargs
variable itself is a way to avoid looking up"args" in theframe
table over and over.)
When copying arguments into local variables there are two useful things that you can do along the way:
- The alternative names for the same argument trick. If a template argument can go by different names — such as uppercase and lowercase forms, or different English spellings — then you can use Lua's
or operator to pick the highest priority name that is actually supplied:
local Title = args.title or args.encyclopaedia or args.encyclopedia or args.dictionary
local ISBN = args.isbn13 or args.isbn or args.ISBN
This works for two reasons:
nil is the same asfalse as far asor is concerned.- Lua's
or operator has what are known as "shortcut" semantics. If the left-hand operand evaluates to something that isn'tfalse ornil , it doesn't bother even working out the value of the right-hand operand. (So whilst that first example may at first glance look like it does four lookups, in the commonest case, where|title=
is used with the template, it in fact only actually does one.) - The default to empty string trick. Sometimes the fact that an omitted template argument is
nil is useful. Other times, however, it isn't, and you want the behaviour of missing arguments being empty strings. A simpleor "" at the end of an expression suffices:local ID = args.id or args.ID or args[1] or ""
= Don't expand templates, even though you can. =
If local variables are cheap and table lookups are expensive, then template expansion is way above your price bracket.
Avoid
Similarly, avoid things like using w:Template:ISO 639 name aze (deleted August 2020) to store what is effectively an entry in a database. Reading it would be a nested parser call with concomitant database queries, all to map a string onto another string. Put a simple straightforward data table in your module, like the ones in Module:Wikt-lang.
Notes
{{Notelist|50em}}
References
= Cross-references =
{{reflist|30em}}
= Citations =
{{refbegin}}
- {{cite dictionary|ref={{harvid|MW|2003a}}|dictionary=Merriam-Webster's Collegiate Dictionary|entry=scribble|page=1116|edition=11th|publisher=Merriam-Webster|year=2003|isbn=9780877798095|title=Merriam-Webster's Collegiate Dictionary: Eleventh Edition}}
- {{cite journal|title=Passing a Language through the Eye of a Needle|journal=Queue|publisher=Association for Computing Machinery|first1=Roberto|last1=Ierusalimschy|first2=Luiz Henrique|last2=de Figueiredo|first3=Waldemar|last3=Celes|date=12 May 2011|volume=9|issue=5|url=http://queue.acm.org/detail.cfm?id=1983083|id=ACM 1542-7730/11/0500}}
- {{cite book|title=Lua Programming Gems|editor3-first=Roberto|editor-last3=Ierusalimschy|editor-first1=Luiz Henrique|editor-last1=de Figueiredo|editor2-first=Waldemar|editor-last2=Celes|publisher=Lua.org| date=December 2008 |isbn=978-85-903798-4-3|chapter=Lua Performance Tips|first=Roberto|last=Ierusalimschy|chapter-url=http://lua.org./gems/sample.pdf}}
{{refend}}
Further reading
= Lua =
{{refbegin|indent=y}}
- {{cite book|title=Lua 5.1 Reference Manual|url=http://lua.org./manual/5.1/|author-first1=Roberto|author-last1=Ierusalimschy|author-first2=Luiz Henrique|author-last2=de Figueiredo|author3-first=Waldemar|author3-last=Celes|publisher=Lua.org| date=August 2006 |isbn= 85-903798-3-3}}
- {{cite book|title=Programming in Lua|edition=Second|author-first=Roberto|author-last=Ierusalimschy|language=English|publisher=Lua.org| date=March 2006 |isbn=9788590379829}}
- {{cite book|title=Programming in Lua|first=Roberto|last=Ierusalimschy|publisher=Lua.org| date=December 2003 |isbn=85-903798-1-7|url=http://lua.org./pil/#1ed|edition=First|language=English}}
- {{cite book|title=Beginning Lua Programming|first1=Kurt|last1=Jung|author2-first=Aaron|author2-last=Brown|isbn=978-0-470-06917-2| date=February 2007 |publisher=Wrox}}
{{refend|indent=y}}
{{Wikipedia technical help|collapsed}}