Server response - Perl Programming

Posted by John_Moylan (jfp), 14 February 2003

Morning all
any clues on how to get just the server response from a http request in a cgi script?

Looking at redirecting if the response is good ie: 200 or somewhere else if bad (404 etc)
Theres got to be an easy way!

jfp

Posted by admin (Graham Ellis), 14 February 2003

Are you looking at one server from another? Do you have the LWP module loaded on your server? If so, you can use that. The example that I have here is a stand along program, but I've done the same thing wrapped in a CGI script to have various servers talking to each other.

Code:

#!/usr/bin/perl

use LWP::UserAgent;

$agent = LWP::UserAgent->new;
$agent->agent("Well House Consultants/$0 ");

foreach $url ("http://www.wellho.net/index.html",
"http://www.wellho.net/greatsite.html") {
$request = HTTP::Request->new(GET => $url);
$response = $agent->request($request);
$status = $response->code();
print "Response from $url was code $status\n";
}

runs:

bash-2.04$ ./gresp.pl
Response from http://www.wellho.net/index.html was code 200
Response from http://www.wellho.net/greatsite.html was code 404
bash-2.04$

Posted by admin (Graham Ellis), 15 February 2003

Came back this morning and actually read your question

If you replace the GET with HEAD in my code, your script will read back just the header from the server and not the body ... then you'll be able to make your decision whether or not the server has good information at the present time

Posted by John_Moylan (jfp), 16 February 2003

Thanks Graham, thats great.

Are you aware of a problem with useragent though? it throws a 500 error when the request is valid.

What I'm trying to do is get the server repsonse from a password protected page, so the url looks like:
Code:

http://username:password@www.mysite.co.uk

(yes I excape the \@ in the code) to pass the username & pwd with the url.

Though it works in the browser the script using useagent throws a 500 eror? strange!
Have you encountered this?

jfp

Posted by admin (Graham Ellis), 17 February 2003

Not a form of URL that I use personally ... but I'm very suprised at the error 500 as it usually indicates a server-side problem and not a fault in the request. However - I would have a detailed look at the request.

I've stepped back into some older examples - here is an "echo server" that takes whatever is thrown at it on port 8080 and echos it back as a web page. If you run this against your user agent script, and again using your regular browser, you might spot what's different in the requests and so be lead towards a solution.

Code:

#!/usr/local/bin/perl

# httpd requests - echo server

use Socket;
use whc_filter;

# Set up listener

$on_port = 8080;
$proto = getprotobyname("tcp");

socket (Server, PF_INET, SOCK_STREAM, $proto) ||
die ("socket");
setsockopt(Server, SOL_SOCKET,SO_REUSEADDR,1) ||
die ("setsockopt");
$pbind = sockaddr_in($on_port,INADDR_ANY) ||
die ("sockaddr_in");
bind(Server,$pbind) ||
die ("bind");
listen(Server,SOMAXCONN) ||
die ("listen");

# await a contact

@response = <DATA>;

while ($paddr = accept(Client, Server)) {
while (<Client>) {
push @gotten,$_;
if ($_ =~ /^\s*$/) { last;}
($f,$v) = split(/\s+/,$_,2);
$param{lc($f)} = $v;
}

if ($gotten[0] =~ /^POST/) {
read(Client,$data,$param{"content-length:"});
}

whc_filter::scrub(@gotten);
print Client @response;
print Client @gotten,"</PRE>";
if ($gotten[0] =~ /^POST/) {
($dsay .= $&."<BR>") while ($data =~ s/^(.{1,80})//);
print Client "<H4>Posted Data:</H4><CODE>\n";
print Client $dsay,"</CODE>";
}
print Client "</BODY></HTML>\n";
close Client;
$#gotten = -1;
undef %param;
}

__END__
HTTP/1.0 200 OK
Date: Sun, 18 Apr 1999 13:59:18 GMT
Server: Well House Consultants / PAN 1.0
Content-type: text/html

<HTML>
<BODY bgcolor=white text=black>
You have contacted the Server Echo Service that tells you what
your browser sent in its request!<P>
<H2>The Data provided to the server was:</H2>
<BR><PRE>

and whc_filter.pm

Code:

package whc_filter;

# HTML filters.

require 5.003;
use strict;

# scrub encodes & < and > in an array into ISO characters
# Also replaces hard new line characters with <P> and <BR>

sub scrub (\@) {
my ($svar,$k);
$svar = \@_;
for ($k=0;$k<=$#{$svar};$k++) {
$_=$svar->[$k];
s/&/&/g;
s/</</g;
s/>/>/g;
s/\n\r/\n/g; # dos to standard
s/\r/\n/g; # mac to standard
s/\n{2,}/<P>/g;
s/\n/<BR>/g;
$svar->[$k] = $_;
}
}

1;

Posted by John_Moylan (jfp), 18 February 2003

Never did get to the bottom of this!
Ended up doing a 'wget -S' and regexing the response.

This was being done as a quick mockup in Perl, the real thing needs to be written in Java.
Is there anything similar to useragent or wget that would give the server response in Java?

Thanks

jfp

Should I be cross posting this to the Java Forum?

Posted by John_Moylan (jfp), 18 February 2003

Sorry, sorry sorry.

I'm breaking my own rules by posting without looking for myself.

A quick look at javadoc points me towards:
HttpURLConnection which has a method called getResponseCode()

Erm, that'll teach me, the answers always out there...and its normally in the docs, regardless of the language you use.

jfp

This page is a thread posted to the opentalk forum at www.opentalk.org.uk and archived here for reference. To jump to the archive index please follow this link.