User:Scsbot/wikised documentation

Because wikised is a framework for performing many sorts of editing tasks, there are several pieces of information it must be given on startup to tell it what to do. It has to know which wiki to make changes to and who to log in as. It has to know which actual editing script to run and whether there are any check scripts. And it needs a list of pages to edit.

This information can be given to wikised either on its command line or via a couple of configuration files.

We'll start with a quick list of all the command-line flags, followed by some more detailed explanations of what they mean and how they're used.

-host h

set host of wiki

-url u

set full base URL of wiki

-editsummary m

set edit summary message

-minor

mark edits as minor

-data f

set driver/data file (list of pages to edit, + parameters)

-okaytocreate

okay to create new pages

-mustcreate

don't overwrite old pages (must create new)

-sleep s

sleep for s seconds between edits

-precheckscript s

set pre-check script

-postcheckscript s

set post-check script

-editscript s

set main edit script

-filter

edit script is a pure filter

-checkdiffs

check diffs

-ninsert n

expected number of inserted lines

-ndelete n

expected number of deleted lines

-user u

set username to log in as

-pass p

password for that username

-loginconfig c

use predefined login configuration c (sets host, url, and user)

-f s

use master setup script s (sets edit script and all related parameters)

-help

print a brief help message

To specify which machine to log in to, use either the -host or -url options. If you use -host to specify a host h, the script assumes that the base URL is http://hhttps://en.wiktionary.org/w/index.php. Or you can specify the full base url explicitly using -url (in which case you don't need to use -host).

You must also specify a username and password. If you don't want to put the password on the command line, you can omit the -pass option, and the script will prompt for the password.

The cornerstone of the process is the edit script, specified with --editscript. By default, this script will be invoked as

script filename

where filename is a temporary file containing the wikitext of the page being edited, and parms is a list of any additional parameters (see below). However, if the -filter flag is specified, the script will be invoked as

script

and it will receive the current wikitext on its standard input, and is expected to write the edited text to its standard output.

Typically the bot is run to perform edits on many pages. The action of which pages to edit is determined by a driver script. The driver script is an ordinary text file consisting of one or more columns. The first column is the page name to edit. Any remaining columns are parameters to be passed to the edit script when editing that particular page. For example, when adding rhymes, the driver script might look like

bag    -æɡ
dawn   -ɔːn
fawn   -ɔːn
thorn  -ɔː(r)n
...

You specify the name of this driver file with -data.

You can provide an edit summary for the bot to use with -editsummary. You can have the bot mark its edits as minor with -minor.

Normally, it is an error if the bot tries to edit a page that does not exist. If your script might be creating new pages (perhaps because it is specifically an upload script), you should use --okaytocreate which will suppress the error if a page does not exist. If you're uploading pages which might already exist and which, if so, you do not want to overwrite, you can specify the -mustcreate option which will cause an error if the page does exist.

If there is a pre check script, it is specified with the -precheck option. The pre check script is invoked without arguments, with the (unedited) wiki text presented on standard input.

If the -checkdiffs option is present, wikised will effectively press the "Show changes" button and inspect the diff output. The analysis is currently rudimentary, just a count of the number of lines inserted and deleted. Specify the expected number of inserted lines with -ninsert and the expected number of deleted lines with -ndelete. (It is also possible for the edit script to communicate a variable, per-edit ninsert back to wikised.)

If there is a post check script, it is specified with the -postcheck option. The post check script is invoked without arguments, with the (edited) wiki text presented on standard input.

To avoid overwhelming the Wikimedia servers, the script normally waits a minute between each page edited. This delay can be configured with the -delay option.

To streamline the specification of the several different wikis and logins you might be using this script with, you can use the -loginconfig option which automatically sets the host, base url, username, and password based on a small database of frequently-used values.

To streamline the specification of all the information pertaining to a particular edit task, you can use a "master script". A master script is specified with the -f option, or as the first (plain) argument on the command line.

So there are potentially three different configuration scripts you might want to set up: the login config script, the master script, and the driver script. These three scripts are distinct: the login config script controls which wiki is logged in to, the master script controls which editing task is performed, and the driver script controls which specific pages are edited.

These three things are separated because I want to be able to control them separately. I want to be able to test a new edit script on my home test wiki and then, once it's working, without changing anything else, perform exactly the same edits on the live Wiktionary. Then, once I've performed some edits for one set of pages, I want to be able (again, without changing anything else) to run exactly the same edits on another (perhaps larger) set of pages.

The master script is a set of keyword/value pairs, one per line. The keywords are:

host

set the host, just like the -host command-line option

baseurl

set the base url, just like the -url command-line option

driverscript

set the data driver script, just like the -data command-line option

reason

set the edit summary message, just like the -editsummary command-line option

minoredit

indicate minor edits, just like the -minoredit command-line option

okaytocreate

allow page creation, just like the -okaytocreate command-line option

mustcreate

force new page creation, just like the -mustcreate command-line option

delay

set the delay between edits, just like the -delay command-line option

checkscript

set the pre check script, just like the -precheckscript command-line option

editscript

set the edit script, just like the -editscript command-line option

filter

indicate that the edit script is a filter, just like the -filter command-line option

postcheckscript

set the post check script, just like the -postcheckscript command-line option

checkdiffs

request checking diffs, just like the -checkdiffs command-line option

expecteddelete

set the expected number of insertions, just like the -ninsert command-line option

expectedinsert

set the expected number of deletions, just like the -ndelete command-line option

username

set the user name, just like the -user command-line option

password

set the password, just like the -pass command-line option