Training, Open Source Computer Languages

Search for:
Print friendly page
Home Accessibility Courses Diary The Mouth Forum Resources Site Map About Us Contact
Automated Browsing in Perl

I'm reminded on today's Perl course just how powerful some of the modules are, and how much you can do in so little code.

LWP::UserAgent turns your Perl into an automated browser .. the following four lines reading the robots.txt off my web site.

use LWP::UserAgent;
$connex = new LWP::UserAgent("agent" => "007");
$response = $connex->get("http://www.wellho.net/robots.txt");
print $response->decoded_content;
(written 2009-09-11)

 
Associated topics are indexed under
P608 - Perl - Robots, Crawlers and Spiders
  [2229] Do not re-invent the wheel - use a Perl module - (2009-06-11)
  [2045] Does robots.txt actually work? - (2009-02-16)
  [1031] robots.txt - a clue to hidden pages? - (2007-01-13)

P224 - Perl - Intersystem Communications
  [2970] Perl - doing several things at the same time - (2010-09-25)
  [2876] Different perl examples - some corners I rarely explore - (2010-07-18)
  [2695] TCP v UDP / Client v Server - Python examples - (2010-03-25)
  [1918] Perl Socket Programming Examples - (2008-12-02)
  [1073] Heartbeat script in Perl - (2007-02-09)
  [604] Perl - multiprocess applications - (2006-02-13)

P405 - Perl - Web Service - Our Own Client
P408 - Perl - Standard Web Modules
  [3485] Perl - retrieving and caching web resources - (2011-10-18)
  [2416] Automating access to a page obscured behind a holding page - (2009-09-23)
  [975] Answering ALL the delegate's Perl questions - (2006-12-09)


Back to
Back Tomorrow
 Previous and next
or
Horse's mouth home		 Forward to
Hotel Booking Scam / Cost of calls to 070 numbers
Some other Articles
Pound Sign in Python Program
But I am reading from a file - no need to prompt (Perl)
Low Sun - Autumn is here
Hotel Booking Scam / Cost of calls to 070 numbers
Automated Browsing in Perl
Back Tomorrow
Are you wanting to learn PHP?
Firefighting with Perl
Websitemediasolution and a goldfish called Carl Johnson
Signwriting is dead. Long live the sign.
3603 posts, page by page
Link to page ... 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73 at 50 posts per page


This is a page archived from The Horse's Mouth at http://www.wellho.net/horse/ - the diary and writings of Graham Ellis. Every attempt was made to provide current information at the time the page was written, but things do move forward in our business - new software releases, price changes, new techniques. Please check back via our main site for current courses, prices, versions, etc - any mention of a price in "The Horse's Mouth" cannot be taken as an offer to supply at that price.

Link to Ezine home page (for reading).
Link to Blogging home page (to add comments).
You can Add a comment or ranking to this page
Public Training Courses
Running regularly at our UK training Centre.
[Schedule] - [About] - [Book]

Like this? ??

Related short articles
Perl - calls to methods that use => - what do they mean?
How to do multidimensional arrays (or rather lists and hashes) in Perl
Perl functions such as chop change their input parameters
Using Perl to generate multiple reports from a HUGE file, efficiently
The difference between dot (a.k.a. full stop, period) and comma in Perl
Finding all matches to a pattern in Perl regular expressions
What order are operations performed in, in a Perl expression?
December courses - PHP, Python, Perl - and a weekend Lua course
Why would you want to use a Perl hash?
Awk v Perl
Perl and CGI - simple form, and monitoring script.
Take the dog on a lead - do not carry her. Perl references.
Research is exciting. But should routine be automated?
Sigils - the characters on the start of variable names in Perl, Ruby and Fortran
Searching through all the files in or below a directory - Ruby, Tcl, Perl
Handling binary data in Perl is easy!
Single and double quotes strings in Perl - what is the difference?
When variables behave differently - Tie in Perl
Perl - a quick reminder and revision. Test yourself!
Not multidimentional arrays - but lists of lists. Much more flexible. Perl!
$ is atomic and % and @ are molecular - Perl
From fish, loaves and apples to money, plastic cards and BACS (Perl references explained)
Perl - making best use of the flexibility, but also using good coding standards
What do I mean when I add things in Perl?
How to interact with a Perl program while it is processing data
Speeding up your Perl code
DNA to Amino Acid - a sample Perl script
Perl from basics
Finding your big files in Perl - design considerations beyond the course environment
How much has Perl (and other languages) changed?
Reading the nth line from a file (Perl and Tcl examples)
Do I need to initialise variables - programming in C, C++, Perl, PHP, Python, Ruby or Java.
How a for loop works Java, Perl and other languages
Rekeying a table - comparison in #Ruby #Perl and #Python
Perl, Python, PHP, Lua, Linux, and more - and business hotel too. Menu for 2011
Perl - database access - DBD, DBI and DBIx modules
Learning Object Orientation in Perl through bananas and perhaps Moose
Making Perl class definitions more conventional and shorter
Perl 6 - significantly nearer, and Rakudo looks very good
Comments in and on Perl - a case for extreme OO programming
Object Orientation in an hour and other Perl Lectures
What is a universal superclass? Java / Perl / Python / Other OO languages
Least Common Ancestor - what is it, and a Least Common Ancestor algorithm implemented in Perl
Exception handling in Perl - using eval
Expect in Perl - a short explanation and a practical example
Lots of ways of doing it in Perl - printing out answers
Increment operators for counting - Perl, PHP, C and others
Copying - duplicating data, or just adding a name? Perl and Python compared
Why do I need brackets in Ruby ... or Perl, Python, C or Java
Some more advanced Perl examples from a recent course
What does blessing a variable in Perl mean?
Multiway branches in Perl - the given and when syntax
Removing the new line with chop or chomp in Perl - what is the difference?
Teaching Lua to a Perl advocate
Alternative loops and conditionals in Ruby and Perl
Further more advanced Perl examples
Unpacking a Perl string into a list
Perl - the duplicate key problem explained, and solutions offered
Teaching examples in Perl - third and final part
Fresh Perl Teaching Examples - part 2 of 3
Are you learning Perl? Some more examples for you!
Ruby - training for automated testing users
Perl course - is it tailored to Linux, or Microsoft Windows?
Chancellor George Osborne inspires Perl Program
File open and read in Perl - modernisation
Setting a safety net or fallback value in Perl
Intelligent Matching in Perl
switch and case, or given and when in Perl
Iterating over a Perl list and changing all items
What is Perl?
After the Perl course in Nurnberg
Binary data handling with unpack in Perl
Perl - skip the classics and use regular expressions
The Perl Survey
Perl Course FAQ
For loop - checked once, or evety time? Ruby v Perl comparison and contrast
Want to do a big batch edit? Nothing beats Perl!
Moving the product forward - ours, and MySQL, Perl, PHP and Python too
Perl and the Common Gateway Interface - out of fashion but still very useful?
Faster network, but not faster browsing until ...
© WELL HOUSE CONSULTANTS LTD., 2012: Well House Manor • 48 Spa Road • Melksham, Wiltshire • United Kingdom • SN12 7NY
PH: 01144 1225 708225 • FAX: 01144 1225 899360 • EMAIL: info@wellho.net • WEB: http://www.wellho.net • SKYPE: wellho

PAGE: http://www.wellho.net/mouth/2402_Aut ... -Perl.html • PAGE BUILT: Fri Feb 3 14:16:04 2012 • BUILD SYSTEM: wizard