Home Accessibility Courses Twitter The Mouth Facebook Resources Site Map About Us Contact
 
For 2021 - online Python 3 training - see ((here)).

Our plans were to retire in summer 2020 and see the world, but Coronavirus has lead us into a lot of lockdown programming in Python 3 and PHP 7.
We can now offer tailored online training - small groups, real tutors - works really well for groups of 4 to 14 delegates. Anywhere in the world; course language English.

Please ask about private 'maintenance' training for Python 2, Tcl, Perl, PHP, Lua, etc.
Firefighting with Perl

If the building is blazing, you'll go right ahead and put the fire out with the extinguisher. Hopefully you'll have been trained to use the right sort of extinguisher on the right fire, but you won't leave the fire to blaze just because the "date last serviced" record has become smudged and you don't know if the extinuisher is technically overdue for overhaul - you can soon chuck it to one side if it fails!

At times, Perl can be a bit of a fire extinuguisher - a language that you'll use to plug the emergency gaps, get a quick answer to a one-off question that's needed in response to the MD's call out from a meeting, etc. Questions like "why is our web server running slowly - are there particular pages draining the c.p.u." and "how many people have visited our newsletter page since we sent out the mailing last week?"

Such code need not be pretty; it can be hacked together and provided that it works, comments / readability / maintainability / design can - temporatily - go out of the window. But even when you're writing firefighting code, ALWAYS check that your results are correct

Finding the slow server records

Both of the questions that I raised above have been asked of me ... the "slow server" one at 3 a.m. when responses were dreadful ... and I wanted to find all access records in the current log file that stepped back in time six seconds or more (records are logged in the order of completion, but the time stamps are the access start time!)

The code remains in our server's live log directory to this day. It is "sort of" documented by being names "slow.pl" which is a bit of a clue, and I have inset my loops. A print statement that I used in early debugging remains commented out, and there is no validation to check that the log file really is present. Here's the code in its current state - intentionally left available in case the particular file re-ignites:

open (FH,"access_log");
$osid = 0;
while (<FH>) {
  $l++;
  ($h, $m, $s) = /(\d\d):(\d\d):(\d\d)/;
  $sid =$s + 60*$m + 3600 * $h;
  if ($osid) {
    $moved = $sid - $osid;
    if ($moved < -6) {
      print "$moved $l ";
      print;
    }
  }
  # print "$moved\n";
  $osid = $sid;
}


It's a cool autumn morning, all is quiet and the logs were tidied up a couple of hours ago by our night porter, affecionally known as 'cron' ... so there have been absolutely no issues yet today. A typical 'Unix Utility' working on the 'no news is good news' paradigm:

-bash-3.2$ perl slow.pl
-bash-3.2$


Counting unique visitors to a particular area

Let's look at the other question - "how many people have come to our newsletter". This is really one-off code at the moment:

#!/usr/bin/perl -na
m!GET /newsletter/! and $counter{$F[0]}++;
m!GET /newsletter/! and $tc++;
 
END {
$th = keys(%counter);
foreach $ho(sort(keys(%counter))) {
  print "$ho $counter{$ho}\n"; }
print "Total hits $tc ... hosts $th\n"; }


How does that work?

I run it with the input files named on the command line, and the -n switch reads each line of each file it turn into $_, which is then matched against the regular expression(s) and counted if they match - actually counted twice, once in a table (hash) by the visitor's IP address - the first field on the line which has been autosplit into @F by the -a switch.

Once all the lines have been read, the END block is run once, listing all the IP addresses in order, together with counts per IP, then the final (desired) results - total accesses and total DIFFERENT hosts accessing - are printed out:

-bash-3.2$ ./pgnl ac_2009082* ac_2009083* ac_200909*
snip
89.124.xxx.xxx 4
92.237.xxx.xxx 104
92.239.xxx.xxx 4
Total hits 2823 ... hosts 131
-bash-3.2$


Even there, the numbers need to be checked somewhat and read with care; each 'newsletter' hit from a 'real' visitor is four accesses - the main newsletter and 3 graphics in the same directory, and the total of 131 hosts includes our own test machines, and also crawlers such as Google. The fulllist / IP numbers gave me further clues, and I was able to say that I estimated that beween 110 and 120 people had clicked through. Expressed as a percentage of the number of emails, and considering that's people who oepned teh email and took a further step as well, I have to say I'm quite pleased.

Back to the Perl code

Would I be proud to supply this code as part of a contract? No. But am I happy with what it allowed me to do very quickly. Absolutely. I used to say that "what took me a week in C now takes me a day in Perl". That's changed, somewhat: "What took me a week now takes me a morning in Perl".
(written 2009-09-07)

 
Associated topics are indexed as below, or enter http://melksh.am/nnnn for individual articles
P604 - Using Perl for Web Site Management
  [314] What language is this written in? - (2005-05-17)
  [2532] Analysing Google arrivals by country of origin - (2009-12-10)
  [3445] Perl and CGI - simple form, and monitoring script. - (2011-09-17)

P222 - Perl - Programming Efficiency and Style
  [743] How to debug a Perl program - (2006-06-04)
  [1181] Good Programming practise - where to initialise variables - (2007-05-09)
  [2657] Want to do a big batch edit? Nothing beats Perl! - (2010-03-01)
  [2688] Security considerations in programming - what do we teach? - (2010-03-22)
  [4611] Hungarian, Camel, Snake and Kebab - variable naming conventions - (2016-01-03)


Back to
Websitemediasolution and a goldfish called Carl Johnson
Previous and next
or
Horse's mouth home
Forward to
Are you wanting to learn PHP?
Some other Articles
Hotel Booking Scam / Cost of calls to 070 numbers
Automated Browsing in Perl
Back Tomorrow
Are you wanting to learn PHP?
Firefighting with Perl
Websitemediasolution and a goldfish called Carl Johnson
Signwriting is dead. Long live the sign.
Easing off in our 50s?
From Lymington by train - last of the slammers
Two days of demonstration scripts in Python
4759 posts, page by page
Link to page ... 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96 at 50 posts per page


This is a page archived from The Horse's Mouth at http://www.wellho.net/horse/ - the diary and writings of Graham Ellis. Every attempt was made to provide current information at the time the page was written, but things do move forward in our business - new software releases, price changes, new techniques. Please check back via our main site for current courses, prices, versions, etc - any mention of a price in "The Horse's Mouth" cannot be taken as an offer to supply at that price.

Link to Ezine home page (for reading).
Link to Blogging home page (to add comments).

You can Add a comment or ranking to this page

© WELL HOUSE CONSULTANTS LTD., 2021: 48 Spa Road • Melksham, Wiltshire • United Kingdom • SN12 7NY
PH: 01144 1225 708225 • EMAIL: info@wellho.net • WEB: http://www.wellho.net • SKYPE: wellho

PAGE: http://www.wellho.net/mouth/2399_Fir ... -Perl.html • PAGE BUILT: Sun Oct 11 16:07:41 2020 • BUILD SYSTEM: JelliaJamb