Log file analysis (3) - results cached for run time efficiency

Web2 and caching example from a Well House Consultants training course
More on Web2 and caching [link]

This example is described in the following article(s):
• New Example - cacheing results in PHP for faster loading - [link]

If you're searching for a page where you can try this code, select here

This example references the following resources:
http://www.wellho.net/resources/ex.php4?item=j714/Strtok.java

Source code: a_papershow.php Module: H307


<?php
# All the PHP at the top ....................

# If necessary, read the log file and analyse it
# Supress warnings if cache does not (yet) exist

if (filemtime("ac_20090522") > @filemtime("cache_0.txt")) {

# ------------- Code to set up cache --------------

$ip_counter = array();
$hour_counter = array();
$dir_counter = array();
$fh = fopen ("ac_20090522","r");
while ($line = fgets($fh,4096)){
        $lines++;
        # do the work we need for every report
        $shrapnel = explode(" ",$line);
        $ip_counter[$shrapnel[0]]++;

        $flakes = explode(":",$shrapnel[3]);
        $hour_counter[$flakes[1]]++;

        if (preg_match("@^/[-a-zA-Z0-9_]+/@",$shrapnel[6],$grabbed)) {
                $dir_counter[$grabbed[0]]++;
        } else {
                $dir_counter["[top level]"]++;
        }
}

# Generate the result string here

$result1 = "";
$result2 = "";
$result3 = "";

$result1 .= "IP addresses of clients who visited us the most<br>";
arsort($ip_counter);
$cutoff = 20;
foreach (array_keys($ip_counter) as $current_ip) {
        if (++$shown > $cutoff) break;
        $result1 .= "$shown: $current_ip ... $ip_counter[$current_ip]<br>\n";
        }

$result2 .= "Traffic levels through the day (and night!)<br>";
foreach (array_keys($hour_counter) as $thishour) {
        $result2 .= "$thishour ... $hour_counter[$thishour]<br>\n";
        }

$result3 .= "How was traffic divided, directory by directory<br>";
ksort($dir_counter);
foreach (array_keys($dir_counter) as $thisdir) {
        $result3 .= "$thisdir ... $dir_counter[$thisdir]<br>\n";
        }

$fho = fopen("cache_0.txt","w");
fputs($fho,"$lines<%%>$result1<%%>$result2<%%>$result3\n");
fclose($fho);
}

# ------------- End of Code to set up cache --------------

# Read results in from cache

$parts = explode("<%%>",file_get_contents("cache_0.txt"));

if ($_REQUEST[report] > 0) {
        $result = $parts[$_REQUEST[report]];
} else {
        $result = "results come here<br>";
        }

$headline = "Analysed a total of $parts[0] records<br>";

# We can read from cache!

# ***********************************************************
# Now the HTML with placeholders for PHP variables to display
?>
<html>
<head>
<title>Log File Analyis - Faster</title>
</head>
<body>
<h1>Looking at yesterday's log file</h1>
This is the third demonstration in a series, and we'll use caching to
avoid the need for repeated analysis of the incoming files.
<br><br>
This is a demonstration of the analysis of a text file and selection of
data from it for display on a web page.  The sample file we have used as
our data source is a web access log file of some 30 Mbytes, with over
100,000 records each of which looks something like:
<br><br>
<font color=navy>208.102.246.149 - - [22/May/2009:03:30:08 +0100]
"GET /bing/images/WHClogoBW.jpg HTTP/1.1" 200 12344
"http://www.wellho.net/resources/ex.php4?item=j714/Strtok.java"
"Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.10)
Gecko/2009042316 Firefox/3.0.10"</font>
<br><br>
<?= $headline ?>
<hr>
What Report would you like next?<br>
<a href=?report=1>Top 20 IP Address by number of visits</a><br>
<a href=?report=2>Hour by Hour</a><br>
<a href=?report=3>By Directory</a><br>
<br>
Results:<br>
<br>
<?= $result ?>
<br><br>
Copyright ... <?= date("Y") ?>
</body>
</html>

Learn about this subject

This module and example are covered on our public PHP Techniques course. If you have a group of three or more trainees who need to learn the subject, we can also arrange a private or on site course for you.

Books covering this topic

Yes. We have over 700 books in our library. Books covering PHP are listed here and when you've selected a relevant book we'll link you on to Amazon to order.

Other Examples

This example comes from our "Web2 and caching" training module. You'll find a description of the topic and some other closely related examples on the "Web2 and caching" module index page.

Full description of the source code

You can learn more about this example on the training courses listed on this page, on which you'll be given a full set of training notes.

Many other training modules are available for download (for limited use) from our download centre under an Open Training Notes License.

Other resources

• Our Solutions centre provides a number of longer technical articles.
• Our Opentalk forum archive provides a question and answer centre.
• The Horse's mouth provides a daily tip or thought.
• Further resources are available via the resources centre.
• All of these resources can be searched through through our search engine
• And there's a global index here.

Web site author

This web site is written and maintained by Well House Consultants.

Purpose of this website

This is a sample program, class demonstration or answer from a training course. It's main purpose is to provide an after-course service to customers who have attended our public private or on site courses, but the examples are made generally available under conditions described below.

Conditions of use

Past attendees on our training courses are welcome to use individual examples in the course of their programming, but must check the examples they use to ensure that they are suitable for their job. Remember that some of our examples show you how not to do things - check in your notes. Well House Consultants take no responsibility for the suitability of these example programs to customer's needs.

This program is copyright Well House Consultants Ltd. You are forbidden from using it for running your own training courses without our prior written permission. See our page on courseware provision for more details.

Any of our images within this code may NOT be reused on a public URL without our prior permission. For Bona Fide personal use, we will often grant you permission provided that you provide a link back. Commercial use on a website will incur a license fee for each image used - details on request.