User:KeithTyler/mwpush.pl

mwpush.pl is a w:Perl script which submits any data provided on standard input (or in filenames given as the last argument(s) on the command line, see w:Perl diamond operator) to a given Wiki page on a given MediaWiki wiki. It was developed by Keith D. Tyler ¶ and uses code taken from upload.pl by User:Eloquence, adapted for page editing. This page is listed at Wikipedia:Tools/Editing tools#mwpush .

Of course, it is up to the operator of this script to:

Ensure that the edits of other users are not lost
Ensure that the script does not prove a burdensome load on the infrastructure. A good measure of hour-to-hour loads is indicated by the "job queue length" in Special:Statistics. In 2007, if that value is more than 500,000, then the servers are probably rather busy.

See also Wikipedia:Bot policy and the Python-based tool m:Using the python wikipediabot which is more up-to-date.

Usage

cat wikitext.txt | mwpush.pl -l UserName -p PassWord -w WikiSite -t WikiPage [-eh]

(or, alternately:)

mwpush.pl -l UserName -p PassWord -w WikiSite -t WikiPage [-eh] wikitext.txt

where:

UserName is the name of a user on the target Wiki
PassWord is the Wiki password for that user
WikiSite is the hostname and path to a given wiki (for example, www.wikipedia.org or intranethostname/mediawiki). Do not include http:// or a trailing slash.
WikiPage is the verbatim name of the page you want to submit content to
wikitext.txt is the name of the file containing your Wiki-ready content

Optional arguments:

-h provides a brief usage guide
-e only updates the target page if it does not already exist

Source

Source of mwpush.pl as of 00:51, 6 March 2008 (UTC)



!/usr/bin/perl
mwpush.pl - Push page data to a Wikimedia server
By WikiPedia:User:KeithTyler
Portions largely taken or based on upload.pl by WikiPedia:User:Eloquence


call requirements

use Getopt::Std;
use LWP::Simple;
use LWP::UserAgent;
use HTTP::Request;
use HTTP::Response;
use HTTP::Cookies;

use warnings;

getopts('l:p:w:t:he');
if ($opt_h) { die "Usage: mwpush.pl -w  -t  -l  -p  [-e] [files ...]\n"; }
if (!$opt_l) { die "Provide Wiki login with -l"; }
if (!$opt_p) { die "Provide Wiki password with -p"; }
if (!$opt_w) { die "Provide Wiki hostname and path with -w"; }
if (!$opt_t) { die "Provide Wiki page name with -t"; }
my $username=$opt_l;
my $password=$opt_p;
my $WIKI_PATH=$opt_w;
my $WIKI_PAGE=$opt_t;
my $onlyifempty=$opt_e;

Login to wiki


Set up connection data

my $browser=LWP::UserAgent->new();
my @ns_headers = (
'User-Agent' => 'MediaWiki Pusher 0.1 by KeithTyler',  #Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7) Gecko/20041107 Firefox/1.0',
'Accept' => 'image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, image/png, */*',
'Accept-Charset' => 'iso-8859-1,*,utf-8',
'Accept-Language' => 'en-US',
);

Hold cookies

$browser->cookie_jar( {} );

Make login request

$response=$browser->post("http://".$WIKI_PATH."/w/index.php?title=Special:Userlogin&action=submitlogin",
@ns_headers, Content=>[wpName=>$username,wpPassword=>$password,wpRemember=>"1",wpLoginAttempt=>"Log in"]);

After logging in, we should be redirected to another page.
If we aren't, something is wrong.


if($response->code!=302) {
print
"We weren't able to login. This could have the following causes:

The username ($username) or password may be incorrect.

Solution: Re-run script with correct credentials.

The MediaWiki software on the target host has been upgraded.

Solution: Go to http://commons.wikimedia.org/wiki/Commons:File_upload_service
and get a new version of the upload script.

You are trying to hack this script for other wikis. The wiki you

are uploading to has cookie check disabled.
Solution: Try setting \$ignore_login_error to 1.
Regardless, we will now try to write the output from the server to
mwpush.debug.out....\n\n";
open(DEBUG,">mwpush.debug.out") or die "Could not write file.\n";
print DEBUG $response->as_string;
print
"This seems to have worked. Take a look at the file for further information or
send it to moeller AT scireview DOT de if you need help debugging the script.\n";
close(DEBUG);
exit 1;
}

Get a wpEditToken


We need to load our target page first in edit mode to capture a
wpEditToken.  Without this the submit page will not submit.

$response=$browser ->
get("http://".$WIKI_PATH."/w/index.php?title=".$WIKI_PAGE."&action=edit",
@ns_headers);
my $startTime;
my $editToken;
my $content;
$content = $response->as_string;

Get EditToken
Note: wpEditToken can contain "+" and "\"

($editToken) = ( $content =~ m/value\=\"([0-9a-f\+\\]*)\" name\=\"wpEditToken\"/ );
($editTime) = ( $content =~ m/value\=\"([0-9a-f]*)\" name\=\"wpEdittime\"/ );
($startTime) = ( $content =~ m/value\=\"([0-9a-f]*)\" name\=\"wpStarttime\"/ );

Determine page existence state
If we find "selected new" (a CSS class set), it is a new (empty) page
This is preferable to depending on message text (which is changeable)

if ($onlyifempty) {
if ( ! ($content =~ m/class=\"selected new\"/ )) {
print "Existing page is not empty. Aborting due to -e.\n";
exit 1;
}
}

Collect input data into string

my $INPUT_DATA;

Read data

while (<>) {
$INPUT_DATA.=$_;
}

Post data to Wiki

$response=$browser ->
post("http://".$WIKI_PATH."/w/index.php?title=".$WIKI_PAGE."&action=submit",
@ns_headers,
Content_Type=>'application/x-www-form-urlencoded',Content=>
[ wpTextbox1 => $INPUT_DATA,
wpSummary => "Automated page entry using MWPush.pl",
wpSave => "Save page",
wpSection => "",
wpStarttime => $startTime,
wpEdittime => $editTime,
wpEditToken => $editToken,
]);

Evaluate response from Wiki

if($response->code!=302) {
print "Upload failed! Response was:\n";
print $response->as_string;
exit 1;
} else {
print "Uploaded successfully.\n";
}

Evaluate operation

print "Everything seems to be OK. Log will be written to mwpush.log.\n";
open(LOG,">mwpush.log") or die "Could not write file.\n";
print LOG $response->as_string;
close(LOG);

mwpush