Home Accessibility Courses Twitter The Mouth Facebook Resources Site Map About Us Contact
 
Python and Tcl - public course schedule [here]
Private courses on your site - see [here]
Please ask about maintenance training for Perl, PHP, Lua, etc
 
Spike solution, refactored and reusable, Python - Example

The other day, we suffered a gap in service from one of our dedicated hosts and after the event I wanted to go through the server logs to find how long the gap was, and whether there was any unusual activity prior to the outage. Problem was - I was looking in a 60 Mbyte log file with ovr 200,000 records covering some 60 hours of server use.

A quick Python script highlighted the length of the gap and the line numbers in the file, which I could then look at through an editor.

  counter = 0
  oldsid = 0
  oldstamp = ""
  
  for line in open("access_log"):
    counter += 1
    stamp = (line.split(" "))[4]
    day,h,m,s = (stamp.split(":"))
    sid = int(h) * 3600 + int(m) * 60 + int(s)
    if oldsid + 300 < sid:
      print oldstamp,stamp,counter
    oldsid = sid
    oldstamp = stamp
  
  print counter


Gave me the results:

  wizard:feb13 graham$ python gap
   [24/Feb/2013:04:02:15 1
  [25/Feb/2013:00:00:00 [24/Feb/2013:23:59:58 54148
  [26/Feb/2013:19:31:58 [26/Feb/2013:20:09:34 199300
  221290
  wizard:feb13 graham$


Which told me the start time of the outage, the end time, and to look around line 199300 of the file. But the code is obfurscated, difficult to follow, and very much a hack solution that's not easily re-usable. Which is fine as a quick fix in a crisis. However, by switching to an object oriented approach, that same issue can be solved with neater and re-usable code - very much more followable.

Taking my original as a "spike solution", I "refactored" as follows:

  counter = 0
  oldsid = 0
  oldlatest = None
  
  for line in open("access_log"):
    counter += 1
  
    latest = access(line)
    sid = latest.getseconds()
  
    if oldsid + 300 < sid:
      if oldlatest:
        print oldlatest
        print latest,counter
    oldsid = sid
    oldlatest = latest
  
  print counter,"oo version 2"


Which ran as follows:

  wizard:feb13 graham$ python oogap2
  At [25/Feb/2013:00:00:00 to firstgreatwestern.info from 66.249.76.149
  At [24/Feb/2013:23:59:58 to firstgreatwestern.info from 180.76.5.99 54148
  At [26/Feb/2013:19:31:58 to firstgreatwestern.info from 65.55.213.220
  At [26/Feb/2013:20:09:34 to firstgreatwestern.info from 132.185.160.100 199300
  221290 oo version 2
  wizard:feb13 graham$


Not only is that set of results more informative and cleaner, but also the code is easier to follow and to re-use. Alas, that code's no longer the full story as I've defined a class of access objects:

  class access:
    def __init__(this,line):
      linebits = line.split(" ")
      this.stamp = logtime(linebits[4])
      this.sitevisited = linebits[0]
      this.remoteip = linebits[1]
    def getseconds(this):
      return this.stamp.getseconds()
    def __str__(this):
      return "At " + str(this.stamp) + " to " + this.sitevisited + \
        " from " + this.remoteip


Which in turn have made use of a class of logtime objects:

  class logtime:
    def __init__(this,timestring):
      day,h,m,s = (timestring.split(":"))
      this.seconds = int(h) * 3600 + int(m) * 60 + int(s)
      this.printable = timestring
    def getseconds(this):
      return this.seconds
    def __str__(this):
      return this.printable


The end result is probably a more sophisticated approach than I would normally have taken (whether I needed to have logtime as a separate object is a long discussion we could have) - but the whole example provided a neat, real-life xample for the Python Programming Course which I have been running this week.

Original source code [here]. First (intermediate) OO example using a single class [here]. Final example as shown above - [here].
(written 2013-02-28, updated 2013-03-01)

 
Associated topics are indexed as below, or enter http://melksh.am/nnnn for individual articles
Q906 - Object Orientation and General technical topics - Object Orientation: Individual Objects
  [4650] Why populate object with values as you construct them? - (2016-02-18)
  [4591] From single block to structure and object oriented programming - (2015-12-02)
  [4448] What is the difference between a function and a method? - (2015-03-04)
  [3721] Naming blocks of code, structures and Object Orientation - efficient coding in manageable chunks - (2012-05-06)
  [3436] Moving from scripting to Object Orientation in Python - (2011-09-13)
  [2651] Calculation within objects - early, last minute, or cached? - (2010-02-26)
  [2393] A first demonstration of OO, including polymorphism - (2009-09-04)
  [2173] Basic OO principles - (2009-05-11)
  [2171] Cleaning up redundant objects - (2009-05-11)
  [1925] Introduction to Object Oriented Programming - (2008-12-06)
  [1864] Object Oriented Perl - First Steps - (2008-11-01)
  [1543] Learning Object Oriented Principles (and perhaps Java) - (2008-02-17)
  [507] Introduction to Object Oriented Programming - (2005-11-27)
  [227] Bellringing and Programming and Objects and Perl - (2005-02-25)

Q908 - Object Orientation and General technical topics - Object Orientation: Design Patterns
  [4673] Separating detailed data code from the main application - Ruby example - (2016-05-16)
  [4663] Easy data to object mapping (csv and Python) - (2016-03-24)
  [4626] Singleton design pattern - examples and uses - (2016-01-20)
  [4581] Thin application, thick objects - keep you main code simple. Example in Ruby - (2015-11-21)
  [4421] How healthy are the stars of stage and screen? - (2015-02-09)
  [4396] Java Utility class - flexible replacement for array. Also cacheing in objects and multiple catch clauses example. - (2015-01-16)
  [4377] Designing a base class and subclasses, and their extension, in C++ - (2015-01-01)
  [4359] How to avoid too many recalculations within an object - (2014-12-21)
  [4356] Object factories in C++, Python, PHP and Perl - (2014-12-19)
  [4330] Java - factory method, encapsulation, hashmap example - (2014-11-27)
  [4325] Learning to program - what are algorithms and design patterns? - (2014-11-22)
  [4098] Using object orientation for non-physical objects - (2013-05-22)
  [4096] Perl design patterns example - (2013-05-20)
  [3843] Caching Design Patterns - (2012-08-20)
  [3810] Reading files, and using factories to create vectors of objects from the data in C++ - (2012-07-21)
  [3716] Learning C++ - a design pattern for your first class - (2012-05-02)
  [3608] Design Patterns - what are they? Why use them? - (2012-02-12)
  [2977] What is a factory method and why use one? - Example in Ruby - (2010-09-30)
  [2741] What is a factory? - (2010-04-26)
  [2322] Looking for a practical standards course - (2009-08-05)
  [1224] Object Relation Mapping (ORM) - (2007-06-09)
  [485] North, Norther and Northest - PHP 5 Objects - (2005-11-04)

Y106 - Object Oriented Python
  [4721] When to check an object type - Python isinstance example - (2016-11-03)
  [4129] Simple OO demonstration in C++, comparison to Python - (2013-07-01)
  [4028] Really Simple Class and Inheritance example in Python - (2013-03-04)
  [3947] this or self - what are they, and what is the difference? (Python) - (2012-12-08)
  [3878] From Structured to Object Oriented Programming. - (2012-10-02)
  [3673] Object oriented or structured - a comparison in Python. Also writing clean regular expressions - (2012-03-26)
  [3399] From fish, loaves and apples to money, plastic cards and BACS (Perl references explained) - (2011-08-20)
  [3085] Object Oriented Programming for Structured Programmers - conversion training - (2010-12-14)
  [2604] Tips for writing a test program (Ruby / Python / Java) - (2010-01-29)
  [2169] When should I use OO techniques? - (2009-05-11)
  [2017] Python - a truly dynamic language - (2009-01-30)
  [1348] Screw it or Glue it? Access to Object variables - a warning - (2007-09-12)
  [1306] Python class rattling around - (2007-08-16)
  [900] Python - function v method - (2006-10-20)
  [834] Python makes University Challenge - (2006-08-15)
  [477] Class, static and unbound variables - (2005-10-25)


Back to
Melksham Community - Annual Report / MCAP
Previous and next
or
Horse's mouth home
Forward to
A request for all of Melksham transport groups to work integrated
Some other Articles
Backups, Codebase, Strategy and more - dealing with forum incidents
SQL databases from Python - an SQLite example
Melksham to Chippenham (Station) bus services - times and routes
A request for all of Melksham transport groups to work integrated
Spike solution, refactored and reusable, Python - Example
Melksham Community - Annual Report / MCAP
Coffee Shop in Melksham
Afternoon Teas and Breakfasts at Well House Manor
Acceptable User Policy / vexatious interacter
Red, yellow, green or pink dog? Yellow Dog Project
4759 posts, page by page
Link to page ... 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96 at 50 posts per page


This is a page archived from The Horse's Mouth at http://www.wellho.net/horse/ - the diary and writings of Graham Ellis. Every attempt was made to provide current information at the time the page was written, but things do move forward in our business - new software releases, price changes, new techniques. Please check back via our main site for current courses, prices, versions, etc - any mention of a price in "The Horse's Mouth" cannot be taken as an offer to supply at that price.

Link to Ezine home page (for reading).
Link to Blogging home page (to add comments).

You can Add a comment or ranking to this page

© WELL HOUSE CONSULTANTS LTD., 2019: 404 The Spa • Melksham, Wiltshire • United Kingdom • SN12 6QL
PH: 01225 708225 • EMAIL: info@wellho.net • WEB: http://www.wellho.net • SKYPE: wellho

PAGE: http://www.wellho.net/mouth/4021_Spi ... ample.html • PAGE BUILT: Sat May 27 16:49:10 2017 • BUILD SYSTEM: WomanWithCat