Silicon Valley Perl Message Board › using Perl to access a webpage that is behind a login-required page

using Perl to access a webpage that is behind a login-required page

Ivan G.
user 35228152
Fair Oaks, CA
Post #: 23
Hi Guys and Gals,

I am not sure if you allow technical questions here, but here goes.

I have a webpage that we monitor over simple http. Its been working ok, until this
week, when the middleware team put up a login-page to block open access to
that and other URL's.

My plan is to create a perl script, that gets in, authenticates, and sends back the results of the webpage for text filtering.

I am using LWP::UserAgent and others. My plan is to
1)first authenticate thru the login-page, and hold on to the session cookie.
2)post an additional GET after the fact, to grab the real webpage I want. The script always seems to just grab the contents of the login-page source-code and thats all.

I wonder if anyone could help me here. I'm not a Perl novice, just a perl-to-web novice.

Here is my paraphrased code:
use lib '/app/perl64/lib/site_perl/5.8.8/HTTP';
use lib '/app/argus/.cpan/build/WWW-Mechanize-1.­73-Jz8ZpY/lib';
use HTML::Form;
use HTTP::Request;
use LWP::UserAgent;
use WWW::Mechanize;
use HTTP::Cookies;


my $brows = LWP::UserAgent->new;
my $cookie_jar = HTTP::Cookies->new();
$brows->cookie_jar( $cookie_jar );

#first, to authenticate against the login page
my $req = HTTP::Request->new(
POST => 'http://ep-myserver:9180/solr-master/pro­vSearch/admin/j_security_check');
$req->content_type('application/x-www­-form-urlencoded');
$req->content('j_username=Tzzzzz&­j_password=alphabeta');

#now, to go and get the real page I want....
my $nextreq = HTTP::Request->new(
GET => 'http://ep-myserver:9180/solr-master/adm­in/cores?action=STATUS&core=provSear­ch');
my $final = $brows->request($nextreq);
print $brows->request($nextreq)->as_stri­ng . "\n";

=======> Is there something blatant that I've missed ??


Thanks in advance
-IvanG.
Lambert L.
user 29190182
Group Organizer
Milpitas, CA
Post #: 2
I don't think anyone answered this one. Looks like you have a preference for WWW::Mechanize. Keep reading WWW::Mechanize. When you get to the section about form submission, use login and password to login. The web server should bring mechanize to the next page and you can instruct mechanize to follow whatever link you need to go to.
Powered by mvnForum

People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy