Like a real navigator ?
Posted by Franck131313 (Franck131313), 24 March 2005Hi,
Does someone know how to ask Perl to open a web page and to wait for an auto validated form to redirect?
This is my problem:
I have to send data in POST to a script (easy with LWP::UserAgent)
The second script does the same, etc...;
I can do it with LWP::UserAgent and HTML::Form but I would prefer just to post data to the first script and just wait to redirections to be done (Like a navigator do).
Someone can help me?
Posted by admin (Graham Ellis), 25 March 2005If you're looking to write your own client program (browser replacement) directly, you'll find a very simple example to get you started at http://earth/resources/ex.php4?item=p405/web_fetch.pm.
a) Data to be added as form parameters via the GET method should be URLencoded (?xxx=yyy&zzz=aaa type stuff) after $page in the GET command.
b) If you cannot use HTTP 1.0 (which is the easier protocol), HTTP 1.1 requires (the host name to be supplie on a separate line I believe.
c) If you're using the POST method, that data too needs to be supplied, encoded, on a separate line.
While you're researching and testing your client, you might find it useful to run an "echo server" to see what you're sending; we've got an example at http://earth/resources/ex.php4?item=p405/pwebserver that you're welcome to start from.
You might find a book such as the HTTP developer's handbook useful if you're not already famililar with the protocols you'll be using.
A word of caution. If you're writing an automated client (i.e. a robot), please read and comply with the robots exclusion standard. In a nutshell, this means that you should read the file robots.txt on any site you visit, and comply with any requests they make to avoid automatically reading certain pages. You should also limit your traffic so as not to perform a denial of service attack by eating up too much bandwidth all at once.
This page is a thread posted to the opentalk forum at www.opentalk.org.uk and archived here for reference. To jump to the archive index please follow this link.
PH: 01144 1225 708225 • FAX: 01144 1225 793803 • EMAIL: email@example.com • WEB: http://www.wellho.net • SKYPE: wellho