Module:module documentation

The following documentation is located at Module:module documentation/documentation.

Useful links: subpage list • links • transclusions • testcases • sandbox

This module automatically generates documentation for other modules. It fetches in-line comments in Lua code and converts them into a form that can be used on a documentation page via {{module documentation}}. This module's documentation is in fact an example of it in action.

It is helpful to do documentation this way, as it means function/method documentation is available in two places: at the top of the module page (as conventional Wikitext), and above the function itself (as a Lua comment). Each suits a different kind of editing style, and doing it this way keeps them synchronised.

A section of documentation is given using Lua's multi-line comment syntax, which looks something like this: --==]. The number of equal signs should normally be two in order for the documentation to be properly snarfed by {{module documentation}}. The following conventions can be used inside of the documentation:

Long lines other than in list items can be broken by newlines optionally followed by spaces or tabs, and long lines in list items can be broken by newlines followed by a space or tab. This is especially useful in lists, to make the raw comment more readable. In such a case, the newline and any following whitespace (i.e. spaces or tabs) is converted to a space. Use two newlines in a row to break paragraphs. In general, it's recommended to break lines after at most 120 characters, to facilitate reading the raw comment.
Template calls (using two braces) can be inserted literally and will be expanded.
Single braces can be used to surround literal text, and will automatically be syntax-highlighted as Lua code. Nested braces inside of this literal text will be properly handled as long as they're balanced. If the first character of the literal text is itelf a brace, put a space before it (but not at the end), and it will be ignored.
Backquotes can be used to surround literal text, which will be displayed using <code>...</code>. The stuff inside of backquotes cannot contain a backquote or extend to multiple lines (see triple backquotes below for this).
Double backquotes can be used to surround placeholder variable names, which will be displayed using monospaced italic (using <code><var>...</var></code>, or just <var>...</var> in a monospaced context). The stuff inside of backquotes can only contain letters, numbers, underscores, hyphens and periods.
Triple backquotes can be used to delimit a multiline block of literal text, which will be displayed monospaced and will not join lines broken by newline or interpret lists or other special symbols. Inside triple backquotes, double backquotes still work to delimit italicized placeholder variables, and single backquotes can be used to surround text that should not be monospaced.
Outside of a backquote or code block context, use double underscores to surround a type, which is displayed italicized underlined. Only ASCII letters, numbers and underscore can occur inside the double underscores.

Certain special directives can follow the opening multiline comment indicator, if placed on the same line as the indicator. In particular, the following directives are currently recognized:

The directive intro: by itself signals introductory text, which will be placed at the beginning, prior to function documentation. This is useful to give a general introduction/overview of the module.
The directive var: by itself at the beginning of a doc comment preceding a top-level variable definition causes the variable (before the assignment = sign) and its documentation to be included in the exported documentation. This is useful for documenting tables of property values and other important settings.
The directive func: export.function(arg1, arg2, ...) can be used when documenting a function declared in a nonstandard way (e.g. through a metatable, through an anonymous or locally-declared function assigned to the export table, etc.). The directive indicates the desired way for the function to appear, and the remainder of the comment describes the function's operation, as usual.

export.show

function export.show(frame)

The main entrypoint for {{module documentation}}. The frame object can contain 3 optional arguments:

|comment_level=: The number of equals signs (=) a given section uses. Default: 2 (i.e. --==])
e.g. The value 4 means --====].
|section_level=: The header level used for each function/method. Default: 3 (i.e. L3: === ... ===).
|identifier=: A Lua string pattern. Only the comments of functions whose names match this pattern are used. When not given, all function are accepted.
This is useful when giving object methods, using a pattern such as ^object_name:.

local export = {}

local parameters_module = "Module:parameters"
local m_str_utils = require("Module:string utilities")

local codepoint = m_str_utils.codepoint
local concat = table.concat
local insert = table.insert
local u = m_str_utils.char
local rsplit = m_str_utils.split
local dump = mw.dumpObject

--[===[ intro:
This module automatically generates documentation for other modules. It fetches in-line comments in Lua code and
converts them into a form that can be used on a documentation page via {{tl|module documentation}}. This module's
documentation is in fact an example of it in action.

It is helpful to do documentation this way, as it means function/method documentation is available in two places:
at the top of the module page (as conventional Wikitext), and above the function itself (as a Lua comment). Each
suits a different kind of editing style, and doing it this way keeps them synchronised.

A section of documentation is given using Lua's multi-line comment syntax, which looks something like this:
{--==]}. The number of equal signs should normally be two in order for the documentation to be properly
snarfed by {{tl|module documentation}}. The following conventions can be used inside of the documentation:
# Long lines other than in list items can be broken by newlines optionally followed by spaces or tabs, and long lines
  in list items can be broken by newlines followed by a space or tab. This is especially useful in lists, to make the
  raw comment more readable. In such a case, the newline and any following whitespace (i.e. spaces or tabs) is converted
  to a space. Use two newlines in a row to break paragraphs. In general, it's recommended to break lines after at most
  120 characters, to facilitate reading the raw comment.
# Template calls (using two braces) can be inserted literally and will be expanded.
# Single braces can be used to surround literal text, and will automatically be syntax-highlighted as Lua code. Nested
  braces inside of this literal text will be properly handled as long as they're balanced. If the first character of the
  literal text is itelf a brace, put a space before it (but not at the end), and it will be ignored.
# Backquotes can be used to surround literal text, which will be displayed using {<code>...</code>}. The stuff inside of
  backquotes cannot contain a backquote or extend to multiple lines (see triple backquotes below for this).
# Double backquotes can be used to surround placeholder variable names, which will be displayed using monospaced italic
  (using {<code><var>...</var></code>}, or just {<var>...</var>} in a monospaced context). The stuff inside of
  backquotes can only contain letters, numbers, underscores, hyphens and periods.
# Triple backquotes can be used to delimit a multiline block of literal text, which will be displayed monospaced and
  will not join lines broken by newline or interpret lists or other special symbols. Inside triple backquotes, double
  backquotes still work to delimit italicized placeholder variables, and single backquotes can be used to surround text
  that should ''not'' be monospaced.
# Outside of a backquote or code block context, use double underscores to surround a type, which is displayed italicized
  underlined. Only ASCII letters, numbers and underscore can occur inside the double underscores.

Certain special directives can follow the opening multiline comment indicator, if placed on the same line as the
indicator. In particular, the following directives are currently recognized:

* The directive `intro:` by itself signals introductory text, which will be placed at the beginning, prior to function
  documentation. This is useful to give a general introduction/overview of the module.
* The directive `var:` by itself at the beginning of a doc comment preceding a top-level variable definition causes
  the variable (before the assignment `=` sign) and its documentation to be included in the exported documentation.
  This is useful for documenting tables of property values and other important settings.
* The directive `func: export.<var>function</var>(<var>arg1</var>, <var>arg2</var>, ...)` can be used when documenting a
  function declared in a nonstandard way (e.g. through a metatable, through an anonymous or locally-declared function
  assigned to the `export` table, etc.). The directive indicates the desired way for the function to appear, and the
  remainder of the comment describes the function's operation, as usual.
]===]

local TEMP_LEFT_BRACE = u(0xFFF0)
local TEMP_NEWLINE = u(0xFFF1)

local function format_doc(str)
	local code_blocks = {}
	local code_blocks_i = 0
	local private_use_start = 0x100000
	-- Put a newline at the very beginning and end to facilitate further replacements. We will remove them later.
	str = "\n" .. str .. "\n"
	local subbed_str = (str
		-- Multiline literal text between backquotes; you can't use <pre> or <syntaxhighlight> because that
		-- disables Wikitext parsing for <var>...</var>, italics, <span>...</span> etc. Instead use the trick of
		-- putting a space at the beginning of each line, which yields monospace text without disabling Wikitext
		-- interpretation.
		:gsub("```(.-)```", function(inside)
			return inside
				 -- placeholder variable names between double backquotes; we need to repeat this here to avoid
				 -- the following pattern for single backquotes from clobbering double backquotes
				:gsub("``(+)``", "<var>%1</var>")
				 -- single backquotes undo monospacing
				:gsub("`(+)`", '<span style="font-family: sans-serif;">%1</span>')
				 -- text on the first line should be monospaced
				:gsub("^()", " %1")
				 -- text after a newline should be monospaced, and temp-escape the newline so later replacements
				 -- to join continued lines in a paragraph don't take effect
				:gsub("\n", TEMP_NEWLINE .. " ")
				-- escape { so it won't be interpreted as a code block
				:gsub("{", TEMP_LEFT_BRACE)
		end)
		:gsub("``(+)``", "<var>%1</var>") -- placeholder variable names between double backquotes
		:gsub("`(+)`", function(inside) -- literal text between backquotes, set using <code>...</code>
			-- Escape { so it won't be interpreted as a code block.
			inside = inside:gsub("{", TEMP_LEFT_BRACE)
			return "<code>" .. inside .. "</code>"
		end)
		 -- {} blocks: blocks of code
		 -- Escape to avoid removing line breaks.
		:gsub("%b{}", function(m0)
			local next_char = m0:sub(2, 2)
			if next_char == "|" then
				-- Wikitable; don't try to parse it as code. But we do want to parse special syntax in them (in
				-- particular {...} syntax for embedded code snippets), and if we return nil that won't happen.
				-- Instead, we call format_doc() recursively on the innards.
				return "{" .. format_doc(m0:sub(2, -2)) .. "}"
			end
			if next_char == "{" and m0:sub(-2, -2) == "}" then return nil end
			local text = "<syntaxhighlight lang=lua" .. (m0:match("\n") and "" or " inline") .. ">" .. m0:sub(2, -2):gsub("^ +", "") .. "</syntaxhighlight>"
			-- Prevent any further processing by storing the desired text into the `code_blocks` array and replacing
			-- the whole thing with a single private-use-area character.
			code_blocks_i = code_blocks_i + 1
			code_blocks = text
			return u(private_use_start + code_blocks_i)
		end)
		-- Undo escaping of left brace to prevent code block interpretation.
		:gsub(TEMP_LEFT_BRACE, "{")
		-- Remove indentation for list items.
		:gsub("\n+%f", "\n")
		-- Prevent a list item (beginning with #, *, : or ;) from being joined to the next line unless
		-- the next line starts with whitespace.
		:gsub("%f(*)\n%f", "%1" .. TEMP_NEWLINE)
		-- Join continued lines in a paragraph. We don't want to do that if there are two newlines in a row,
		-- and not if the second line begins with a list item (#, *, : or ;) or | indicating a wikitable
		-- item. In the process, compress any whitespace indentation at the beginning of the line.
    	:gsub("%f\n*()", " %1")
		-- Double-underline to indicate types (displayed as italicized underlined).
		:gsub("__(-)__", function(inside)
			return "<u><i>" .. inside .. "</i></u>"
		end)
		-- Undo escaping of newline to prevent joining of continued lines.
		:gsub(TEMP_NEWLINE, "\n"))

	-- Put <code>...</code> around <var>...</var> invocations that don't already occur inside of
	-- <code>...</code> blocks.
	local split_on_code = rsplit(subbed_str, "(<code>.-</code>)")
	for i = 1, #split_on_code, 2 do
		split_on_code = split_on_code:gsub("(<var>.-</var>)", "<code>%1</code>")
	end
	subbed_str = concat(split_on_code)
	local retval = (subbed_str
		-- Undo code-block stashing.
		:gsub("\244", function(char)
			return code_blocks
		-- Remove extra newlines added at beginning and end.
		end))
	retval = mw.text.trim(retval)
	
	-- mw.log(("Returning: %s"):format(dump(retval)))
	return retval
end

--[===[
The main entrypoint for {{tl|module documentation}}. The frame object can contain 3 optional arguments:
* {{para|comment_level}}: The number of equals signs (=) a given section uses. Default: 2 (i.e. {--==]})
*: e.g. The value 4 means {--====]}.
* {{para|section_level}}: The header level used for each function/method. Default: 3 (i.e. L3: {=== ... ===}).
* {{para|identifier}}: A Lua string pattern. Only the comments of functions whose names match this pattern are used. When not
  given, all function are accepted.
*: This is useful when giving object methods, using a pattern such as {^object_name:}.
]===]
function export.show(frame)
	local parent_args = frame:getParent().args
	
	local params = {
		comment_level = {type = "number", default = 2},
		section_level = {type = "number", default = 3},
		identifier = {},
	}
	local args = require(parameters_module).process(parent_args, params)
	
	local comment_level = args.comment_level
	-- `typeid` matches the identifier at the beginning of the comment that marks this comment as of a special type
	-- that is snarfed by this module. It can be `nil` for regular function comments, which don't have a preceding
	-- special identifier (they just have to match in the number of equal signs, which is usually two). This function
	-- generates a Lua pattern that matches a comment with the right number of equal signs and possibly with the
	-- special identifier at the beginning. The comment has one or more captures: any captures in `typeid` come at
	-- the beginning, followed by a capture for the comment text (after any introductory special identifier).
	local function make_comment_pattern(typeid)
		if typeid then
			typeid = "%s*" .. typeid
		else
			typeid = ""
		end
		return "%-%-%*%]" .. ("="):rep(comment_level) .. "%]"
	end
	local fn_comment_pattern = make_comment_pattern(nil)
	local intro_comment_pattern = make_comment_pattern("intro:")
	local metafunc_comment_pattern = make_comment_pattern("func:%s*((+)+%))")
	local var_comment_pattern = make_comment_pattern("var:")
	local section_mark = ("="):rep(args.section_level)
	local pattern_identifier = args.identifier or ""
	
	local mod_title = mw.title.getCurrentTitle()
	if mod_title.text:match("/documentation$") then return "(<i>The generated documentation is located at the module page.</i>)" end
	local mod_text = mod_title:getContent()
	if not mod_text then return "(<i>The module page does not exist now.</i>)" end

	-- This contains function and intro documentation. Each element is a two-element list of {POSITION, DOCS} specifying
	-- the generated documentation for a function and the character position in the file where it was found (for sorting
	-- purposes).
	local docs
	
	local intro_comment = mod_text:match("^.-" .. intro_comment_pattern)
	if intro_comment then
		docs = { {1, format_doc(intro_comment) }}
	else
		docs = {}
	end

	-- Look for actual functions.
	for p0, f, fn in mod_text:gmatch("()\n*function +((+)+%))") do
		if fn:match(pattern_identifier) then			
			local c = mod_text:sub(1, p0 - 1):match("^.*" .. fn_comment_pattern .. "%s*$")
			insert(docs, {p0, section_mark .. fn .. section_mark .. "\n\n" .. "<syntaxhighlight lang=lua inline>function " ..
				f .. "</syntaxhighlight>\n\n" .. format_doc(c or
				'<strong class="error">This function lacks documentation. Please add a description of its usages, inputs and outputs, ' ..
				"or its difference from similar functions, or make it local to remove it from the function list.</strong>" ..
				"]")})
		end
	end

	-- Now look for comments with the function declaration inside them (used for metatable functions etc.).
	for p0, f, fn, comment in mod_text:gmatch("()" .. metafunc_comment_pattern) do
		insert(docs, {p0, section_mark .. fn .. section_mark .. "\n\n" .. "<syntaxhighlight lang=lua inline>function " .. f ..
			"</syntaxhighlight>\n\n" .. format_doc(comment)})
	end

	-- Now look for variables with var: comments preceding.
	for p0, comment, p1 in mod_text:gmatch("()" .. var_comment_pattern .. "()") do
		local after_text = mod_text:sub(p1)
		local first_var = after_text:match("\n(.-)%s*=")
		if not first_var then
			first_var = ('<strong class="error">Unable to locate variable definition starting at position %s</strong>'):format(p1)
		end
		insert(docs, {p0, section_mark .. first_var .. section_mark .. "\n\n" .. "<syntaxhighlight lang=lua inline>" ..
			first_var .. "</syntaxhighlight>\n\n" .. format_doc(comment)})
	end

	table.sort(docs, function(a, b) return a < b end)
	
	local chunks = {}
	for i, decl in ipairs(docs) do
		insert(chunks, decl)
	end

	return frame:preprocess(concat(chunks, "\n\n"))
end

return export

Module:module documentation

export.show

Wikious

Boobota

Sagapedia