Silicon Valley Perl Message Board › using Perl to access a webpage that is behind a login-required page
Fair Oaks, CA
Hi Guys and Gals,
I am not sure if you allow technical questions here, but here goes.
I have a webpage that we monitor over simple http. Its been working ok, until this
week, when the middleware team put up a login-page to block open access to
that and other URL's.
My plan is to create a perl script, that gets in, authenticates, and sends back the results of the webpage for text filtering.
I am using LWP::UserAgent and others. My plan is to
1)first authenticate thru the login-page, and hold on to the session cookie.
2)post an additional GET after the fact, to grab the real webpage I want. The script always seems to just grab the contents of the login-page source-code and thats all.
I wonder if anyone could help me here. I'm not a Perl novice, just a perl-to-web novice.
Here is my paraphrased code:
use lib '/app/perl64/lib/site_perl/5.8.8/HTTP';
use lib '/app/argus/.cpan/build/WWW-Mechanize-1.73-Jz8ZpY/lib';
my $brows = LWP::UserAgent->new;
my $cookie_jar = HTTP::Cookies->new();
$brows->cookie_jar( $cookie_jar );
#first, to authenticate against the login page
my $req = HTTP::Request->new(
POST => 'http://ep-myserver:9180/solr-master/provSearch/admin/j_security_check');
#now, to go and get the real page I want....
my $nextreq = HTTP::Request->new(
GET => 'http://ep-myserver:9180/solr-master/admin/cores?action=STATUS&core=provSearch');
my $final = $brows->request($nextreq);
print $brows->request($nextreq)->as_string . "\n";
=======> Is there something blatant that I've missed ??
Thanks in advance
I don't think anyone answered this one. Looks like you have a preference for WWW::Mechanize. Keep reading WWW::Mechanize. When you get to the section about form submission, use login and password to login. The web server should bring mechanize to the next page and you can instruct mechanize to follow whatever link you need to go to.