The other day, we suffered a gap in service from one of our dedicated hosts and after the event I wanted to go through the server logs to find how long the gap was, and whether there was any unusual activity prior to the outage. Problem was - I was looking in a 60 Mbyte log file with ovr 200,000 records covering some 60 hours of server use.

A quick Python script highlighted the length of the gap and the line numbers in the file, which I could then look at through an editor.

  counter = 0
  oldsid = 0
  oldstamp = ""
  
  for line in open("access_log"):
    counter += 1
    stamp = (line.split(" "))[4]
    day,h,m,s = (stamp.split(":"))
    sid = int(h) * 3600 + int(m) * 60 + int(s)
    if oldsid + 300 < sid:
      print oldstamp,stamp,counter
    oldsid = sid
    oldstamp = stamp
  
  print counter

Gave me the results:

  wizard:feb13 graham$ python gap
   [24/Feb/2013:04:02:15 1
  [25/Feb/2013:00:00:00 [24/Feb/2013:23:59:58 54148
  [26/Feb/2013:19:31:58 [26/Feb/2013:20:09:34 199300
  221290
  wizard:feb13 graham$

Which told me the start time of the outage, the end time, and to look around line 199300 of the file. But the code is obfurscated, difficult to follow, and very much a hack solution that's not easily re-usable. Which is fine as a quick fix in a crisis. However, by switching to an object oriented approach, that same issue can be solved with neater and re-usable code - very much more followable.

Taking my original as a "spike solution", I "refactored" as follows:

  counter = 0
  oldsid = 0
  oldlatest = None
  
  for line in open("access_log"):
    counter += 1
  
    latest = access(line)
    sid = latest.getseconds()
  
    if oldsid + 300 < sid:
      if oldlatest:
        print oldlatest
        print latest,counter
    oldsid = sid
    oldlatest = latest
  
  print counter,"oo version 2"

Which ran as follows:

  wizard:feb13 graham$ python oogap2
  At [25/Feb/2013:00:00:00 to firstgreatwestern.info from 66.249.76.149
  At [24/Feb/2013:23:59:58 to firstgreatwestern.info from 180.76.5.99 54148
  At [26/Feb/2013:19:31:58 to firstgreatwestern.info from 65.55.213.220
  At [26/Feb/2013:20:09:34 to firstgreatwestern.info from 132.185.160.100 199300
  221290 oo version 2
  wizard:feb13 graham$

Not only is that set of results more informative and cleaner, but also the code is easier to follow and to re-use. Alas, that code's no longer the full story as I've defined a class of access objects:

  class access:
    def __init__(this,line):
      linebits = line.split(" ")
      this.stamp = logtime(linebits[4])
      this.sitevisited = linebits[0]
      this.remoteip = linebits[1]
    def getseconds(this):
      return this.stamp.getseconds()
    def __str__(this):
      return "At " + str(this.stamp) + " to " + this.sitevisited + \
        " from " + this.remoteip

Which in turn have made use of a class of logtime objects:

  class logtime:
    def __init__(this,timestring):
      day,h,m,s = (timestring.split(":"))
      this.seconds = int(h) * 3600 + int(m) * 60 + int(s)
      this.printable = timestring
    def getseconds(this):
      return this.seconds
    def __str__(this):
      return this.printable

The end result is probably a more sophisticated approach than I would normally have taken (whether I needed to have logtime as a separate object is a long discussion we could have) - but the whole example provided a neat, real-life xample for the Python Programming Course which I have been running this week.

Original source code [here]. First (intermediate) OO example using a single class [here]. Final example as shown above - [here].
(written 2013-02-28, updated 2013-03-01)

 Y106 - Object Oriented Python
  [477] Class, static and unbound variables - (2005-10-25)
  [834] Python makes University Challenge - (2006-08-15)
  [900] Python - function v method - (2006-10-20)
  [1306] Python class rattling around - (2007-08-16)
  [1348] Screw it or Glue it? Access to Object variables - a warning - (2007-09-12)
  [1925] Introduction to Object Oriented Programming - (2008-12-06)
  [2017] Python - a truly dynamic language - (2009-01-30)
  [2169] When should I use OO techniques? - (2009-05-11)
  [2604] Tips for writing a test program (Ruby / Python / Java) - (2010-01-29)
  [3085] Object Oriented Programming for Structured Programmers - conversion training - (2010-12-14)
  [3399] From fish, loaves and apples to money, plastic cards and BACS (Perl references explained) - (2011-08-20)
  [3436] Moving from scripting to Object Orientation in Python - (2011-09-13)
  [3673] Object oriented or structured - a comparison in Python. Also writing clean regular expressions - (2012-03-26)
  [3878] From Structured to Object Oriented Programming. - (2012-10-02)
  [3947] this or self - what are they, and what is the difference? (Python) - (2012-12-08)
  [4028] Really Simple Class and Inheritance example in Python - (2013-03-04)
  [4129] Simple OO demonstration in C++, comparison to Python - (2013-07-01)
  [4448] What is the difference between a function and a method? - (2015-03-04)
  [4591] From single block to structure and object oriented programming - (2015-12-02)
  [4650] Why populate object with values as you construct them? - (2016-02-18)
  [4721] When to check an object type - Python isinstance example - (2016-11-03)

Q908 - Object Orientation and General technical topics - Object Orientation: Design Patterns
  [485] North, Norther and Northest - PHP 5 Objects - (2005-11-04)
  [1224] Object Relation Mapping (ORM) - (2007-06-09)
  [2322] Looking for a practical standards course - (2009-08-05)
  [2741] What is a factory? - (2010-04-26)
  [2977] What is a factory method and why use one? - Example in Ruby - (2010-09-30)
  [3608] Design Patterns - what are they? Why use them? - (2012-02-12)
  [3716] Learning C++ - a design pattern for your first class - (2012-05-02)
  [3810] Reading files, and using factories to create vectors of objects from the data in C++ - (2012-07-21)
  [3843] Caching Design Patterns - (2012-08-20)
  [4096] Perl design patterns example - (2013-05-20)
  [4098] Using object orientation for non-physical objects - (2013-05-22)
  [4325] Learning to program - what are algorithms and design patterns? - (2014-11-22)
  [4330] Java - factory method, encapsulation, hashmap example - (2014-11-27)
  [4356] Object factories in C++, Python, PHP and Perl - (2014-12-19)
  [4359] How to avoid too many recalculations within an object - (2014-12-21)
  [4377] Designing a base class and subclasses, and their extension, in C++ - (2015-01-01)
  [4396] Java Utility class - flexible replacement for array. Also cacheing in objects and multiple catch clauses example. - (2015-01-16)
  [4421] How healthy are the stars of stage and screen? - (2015-02-09)
  [4581] Thin application, thick objects - keep you main code simple. Example in Ruby - (2015-11-21)
  [4626] Singleton design pattern - examples and uses - (2016-01-20)
  [4663] Easy data to object mapping (csv and Python) - (2016-03-24)
  [4673] Separating detailed data code from the main application - Ruby example - (2016-05-16)

Q906 - Object Orientation and General technical topics - Object Orientation: Individual Objects
  [227] Bellringing and Programming and Objects and Perl - (2005-02-25)
  [507] Introduction to Object Oriented Programming - (2005-11-27)
  [1543] Learning Object Oriented Principles (and perhaps Java) - (2008-02-17)
  [1864] Object Oriented Perl - First Steps - (2008-11-01)
  [2171] Cleaning up redundant objects - (2009-05-11)
  [2173] Basic OO principles - (2009-05-11)
  [2393] A first demonstration of OO, including polymorphism - (2009-09-04)
  [2651] Calculation within objects - early, last minute, or cached? - (2010-02-26)
  [3721] Naming blocks of code, structures and Object Orientation - efficient coding in manageable chunks - (2012-05-06)

Back to Melksham Community - Annual Report / MCAP

Previous and next or Horse's mouth home

Forward to A request for all of Melksham transport groups to work integrated

Backups, Codebase, Strategy and more - dealing with forum incidents
SQL databases from Python - an SQLite example
Melksham to Chippenham (Station) bus services - times and routes
A request for all of Melksham transport groups to work integrated
Spike solution, refactored and reusable, Python - Example
Melksham Community - Annual Report / MCAP
Coffee Shop in Melksham
Afternoon Teas and Breakfasts at Well House Manor
Acceptable User Policy / vexatious interacter
Red, yellow, green or pink dog? Yellow Dog Project