Sample - extracting statistics from an access log file

For 2023 (and 2024 ...) - we are now fully retired from IT training.
We have made many, many friends over 25 years of teaching about Python, Tcl, Perl, PHP, Lua, Java, C and C++ - and MySQL, Linux and Solaris/SunOS too. Our training notes are now very much out of date, but due to upward compatability most of our examples remain operational and even relevant ad you are welcome to make us if them "as seen" and at your own risk.

Lisa and I (Graham) now live in what was our training centre in Melksham - happy to meet with former delegates here - but do check ahead before coming round. We are far from inactive - rather, enjoying the times that we are retired but still healthy enough in mind and body to be active!

I am also active in many other area and still look after a lot of web sites - you can find an index ((here))

Hashes example from a Well House Consultants training course
More on Hashes [link]

Source code: account Module: P211

=head1 Extracting statistics from a web access log

This is a sample answer to a question set on our
Perl Programming course, asking the delegate to analyse
an apache web access log file in common format and report
on the number of times each host visited the server.

I have extended the sample answer to include an additional
report on the number of times each http status was returned,
and to show a few formatting facilities - sample output
is at the end of the program.

=cut

open (FH,"<","/Users/grahamellis/Downloads/access_log.xyz") or die;

no warnings;
# Supress the "used only once! message about $pedant
$pedant = <FH>;
use warnings;

while (<FH>) {
        ($hostname,$status) = (split)[0,-2];

        $counter{$hostname}++;
        $retcode{$status}++;

        }

sub byac {
        -($counter{$a} <=> $counter{$b})
        }

@inorder = sort byac keys (%counter);

foreach $host (@inorder[0..9]) {
        $int = sprintf ("%20s",$host);
        $int =~ s/ /!/g;

        $rv = sprintf ("host %s accessed us %4d times\n",$int,$counter{$host});
        print $rv;

        }
foreach $host (@inorder[10..$#inorder]) {
        printf ("%15s",$host);
        print ("\n") if (++$n%5==0 or $host eq $inorder[-1]);
        }

# ------------------------

foreach $code (keys (%retcode)) {
        print "$code $retcode{$code}\n";
        }

__END__

Sample Output:

Dorothy:csr1 grahamellis$ perl account
Name "main::n" used only once: possible typo at account line 34.
host !!!!!!!!!!!!!!!skate accessed us 1995 times
host !!!!!!!!!!!!!!!whale accessed us 1044 times
host !!!!!!!!!!!!!o_whelk accessed us 458 times
host !!!!!!!!!!!!!!!perch accessed us 391 times
host !!!!!!!!!!!!!sardine accessed us 348 times
host !!!!!!!!!!!!!sealion accessed us 324 times
host !!!!!!!!!!!!!flipper accessed us 322 times
host !!!!!!!!!!!!aviemore accessed us 318 times
host !!!!!!!!!!!!!!!!seal accessed us 313 times
host !!!!!!!!!!!!!!magnet accessed us 312 times
         walrus trout mussel plaice tuna
           clam lecht dogfish dolphin catfish
        seaweed whiting localhost manatee dab
       pilchard whelk
403 23
304 1061
400 7
200 5076
501 2
302 218
500 143
404 381
Dorothy:csr1 grahamellis$

Learn about this subject

This module and example are covered on the following public courses:
* Perl Programming
* Perl bootcamp
* Learning to program in Perl
* Perl Programming
Also available on on site courses for larger groups

Books covering this topic

Yes. We have over 700 books in our library. Books covering Perl are listed here and when you've selected a relevant book we'll link you on to Amazon to order.

Other Examples

This example comes from our "Hashes" training module. You'll find a description of the topic and some other closely related examples on the "Hashes" module index page.

Full description of the source code

You can learn more about this example on the training courses listed on this page, on which you'll be given a full set of training notes.

Many other training modules are available for download (for limited use) from our download centre under an Open Training Notes License.

Other resources

• Our Solutions centre provides a number of longer technical articles.
• Our Opentalk forum archive provides a question and answer centre.
• The Horse's mouth provides a daily tip or thought.
• Further resources are available via the resources centre.
• All of these resources can be searched through through our search engine
• And there's a global index here.

Web site author

This web site is written and maintained by Well House Consultants.

Purpose of this website

This is a sample program, class demonstration or answer from a training course. It's main purpose is to provide an after-course service to customers who have attended our public private or on site courses, but the examples are made generally available under conditions described below.

Conditions of use

Past attendees on our training courses are welcome to use individual examples in the course of their programming, but must check the examples they use to ensure that they are suitable for their job. Remember that some of our examples show you how not to do things - check in your notes. Well House Consultants take no responsibility for the suitability of these example programs to customer's needs.

This program is copyright Well House Consultants Ltd. You are forbidden from using it for running your own training courses without our prior written permission. See our page on courseware provision for more details.

Any of our images within this code may NOT be reused on a public URL without our prior permission. For Bona Fide personal use, we will often grant you permission provided that you provide a link back. Commercial use on a website will incur a license fee for each image used - details on request.

You can Add a comment or ranking to this page

Further Examples

acc_0 - Loading from a module in another file

access_log.xyz - Log file used in many Well House exercises

account - Sample - extracting statistics from an access log file

acl - Sample answer - web_c2, page 152

aco_1 - Access log - listing hosts via a hash

bankout - initialising a hash

blx.pl - Use of hashes in Perl / CGI

cbdemo - How sort callbacks work

channel - Ferries to France - data file

env2 - ENV module to import variables as globals