Home Accessibility Courses Twitter The Mouth Facebook Resources Site Map About Us Contact
Identifying the first and last records in a sequence

When programming to analyse event or log files, you'll often find it fairly easy to identify the initial or opening record of a linked series, but harder to spot the closing one until your program has gone well past it. Take this data, for example:

  16:15:47 +00 INF: ===================Move Location===================
  16:15:47 +00 INF: Location: Set Initial Position: 6033998 encoder counts
  16:15:47 +00 INF: Location: Current Position: 6817024
  16:15:48 +00 INF: Location: Current Position: 6155537
  16:15:48 +00 INF: Location: Current Position: 6033996
  16:15:48 +00 INF: Location: Current Position: 6033993
  16:16:01 +00 INF: ===================Move Location===================
  16:16:01 +00 INF: Location: Set Initial Position: 6033998 encoder counts
  16:16:01 +00 INF: Location: Current Position: 6033996
  16:16:01 +00 INF: Location: Current Position: 6034021
  16:16:08 +00 INF: ===================Move Location===================
  16:16:09 +00 INF: Location: Set Initial Position: -5966001 encoder counts
  16:16:09 +00 INF: Location: Current Position: -5949212
  16:16:09 +00 INF: Location: Current Position: -5239635


You can spot the start point (liens with "Initial Position") straight away as you parser the file, but the end points (the last "Current Position" before a new "Initial Position") can only be confirmed as end points once you'v emoved beyond and found either a different record to confirm that the earlier sequence is done, or have reached the end of file. Programming-wise, this means making a note of each potentially final record and replacing it or finalaising a sequence at each subsequent read, with an additional "finalising" check after the file read has been completed. All of which sounds very complex, and if you don't get the code exactly right is prone to error!

There is an alternative - which is to keep note of each intermediate record as you read then in, overwriting them if they turn out not to be the final record. I have illustarted this using a movement object, in Python (version 3):

  starter = re.compile('Initial Position: (-?\d+)')
  increment = re.compile('Current Position: (-?\d+)')
  
  moves = []
  for record in open("steps.txt"):
    start = starter.findall(record)
    if start:
      moves.append(movement(start))
    else:
      inc = increment.findall(record)
      if inc:
        moves[-1].setStepPoint(inc)


And once I've parsed all the data, I can simply print out my objects:

  for move in moves:
    print(move)


Here's the class defintion:

  class movement:
    def __init__(self,startpoint):
      self.startpoint = int(startpoint[0])
    def setStepPoint(self,inc):
      self.nowAt = int(inc[0])
      self.soFar = self.startpoint - self.nowAt
    def __str__(self):
      return "From {} to {} movement is {}".format(
        self.startpoint, self.nowAt,self.soFar)


Complete source code [here]

Sample output:

  WomanWithCat:feb16 grahamellis$ python3 mlog
  From 6033998 to 6033993 movement is 5
  From 6033998 to 6034021 movement is -23
  From -5966001 to -5239635 movement is -726366
  WomanWithCat:feb16 grahamellis$

(written 2016-02-26)

 
Associated topics are indexed as below, or enter http://melksh.am/nnnn for individual articles
Y050 - Python - General
  [4712] A reminder of the key issues to consider in moving from Python 2 to Python 3 - (2016-10-30)
  [4558] Well House Consultants - Python courses / what's special. - (2015-10-28)
  [4434] Public training courses - upcoming dates - (2015-02-21)
  [4408] Additional Python courses added to our schedule - (2015-01-29)
  [4295] A longer Python ... training course - (2014-09-16)
  [4236] Using Python to analyse last years forum logs. Good coding practise discussion. - (2014-01-01)
  [3935] Whether you have programmed before or not, we can teach you Python - (2012-11-25)
  [3911] How well do you know Perl and / or Python? - (2012-11-04)
  [3903] Python Programming class for delegates who have already self-taught the basics - (2012-10-25)
  [3902] Shell - Grep - Sed - Awk - Perl - Python - which to use when? - (2012-10-22)
  [3816] Want to escape the Olympics? Learn to program in the countryside! - (2012-07-23)
  [3798] When you should use Object Orientation even in a short program - Python example - (2012-07-06)
  [3519] Python - current versions and implementations (CPython, Jython, IronPython etc) - (2011-11-13)
  [3489] Python courses and Private courses - gently updating our product to keep it ahead of the game - (2011-10-20)
  [3463] Busy weekend of contrasts. - (2011-10-03)
  [3076] Python through the Snow - (2010-12-01)
  [2822] Python training courses for use with ESRI ArcMap software - (2010-06-23)
  [2778] Learning to program in Python 2 ... and / or in Python 3 - (2010-05-24)
  [2504] Learning to program in ... - (2009-11-15)
  [2394] Two days of demonstration scripts in Python - (2009-09-05)
  [2367] Learning to program - how to jump the first hurdles - (2009-08-20)
  [2285] Great new diagrams for our notes ... Python releases - (2009-07-13)
  [2227] Learning PHP, Ruby, Lua and Python - upcoming courses - (2009-06-11)
  [2020] Learning Python - many new example programs - (2009-01-31)
  [2017] Python - a truly dynamic language - (2009-01-30)
  [16] Python training - (2004-08-16)

Q110 - Object Orientation and General technical topics - Programming Algorithms
  [4707] Some gems from an introduction to Python - (2016-10-29)
  [4652] Testing new algorithms in PHP - (2016-02-20)
  [4410] A good example of recursion - a real use in Python - (2015-02-01)
  [4402] Finding sum, minimum, maximum and average in Python (and Ruby) - (2015-01-19)
  [4401] Selecting RECENT and POPULAR news and trends for your web site users - (2015-01-19)
  [4325] Learning to program - what are algorithms and design patterns? - (2014-11-22)
  [3662] Finding all the unique lines in a file, using Python or Perl - (2012-03-20)
  [3620] Finding the total, average, minimum and maximum in a program - (2012-02-22)
  [3451] Why would you want to use a Perl hash? - (2011-09-20)
  [3102] AND and OR operators - what is the difference between logical and bitwise varieties? - (2010-12-24)
  [3093] How many toilet rolls - hotel inventory and useage - (2010-12-18)
  [3072] Finding elements common to many lists / arrays - (2010-11-26)
  [3042] Least Common Ancestor - what is it, and a Least Common Ancestor algorithm implemented in Perl - (2010-11-11)
  [2993] Arrays v Lists - what is the difference, why use one or the other - (2010-10-10)
  [2951] Lots of way of converting 3 letter month abbreviations to numbers - (2010-09-10)
  [2894] Sorting people by their names - (2010-07-29)
  [2617] Comparing floating point numbers - a word of caution and a solution - (2010-02-01)
  [2586] And and Or illustrated by locks - (2010-01-17)
  [2509] A life lesson from the accuracy of numbers in Excel and Lua - (2009-11-21)
  [2259] Grouping rows for a summary report - MySQL and PHP - (2009-06-27)
  [2189] Matching disparate referencing systems (MediaWiki, PHP, also Tcl) - (2009-05-19)
  [1949] Nuclear Physics comes to our web site - (2008-12-17)
  [1840] Validating Credit Card Numbers - (2008-10-14)
  [1391] Ordnance Survey Grid Reference to Latitude / Longitude - (2007-10-14)
  [1187] Updating a page strictly every minute (PHP, Perl) - (2007-05-14)
  [1157] Speed Networking - a great evening and how we arranged it - (2007-04-21)
  [642] How similar are two words - (2006-03-11)
  [227] Bellringing and Programming and Objects and Perl - (2005-02-25)
  [202] Searching for numbers - (2005-02-04)


Back to
Image indexer / thumbnail display scripts in PHP
Previous and next
or
Horse's mouth home
Forward to
Rumours of bus changes by First in Wiltshire - what we know and suspect
Some other Articles
What is happening on the 231 bus? What are you going to do about it?
Prining a pound sign from Python AND running from the command line at the same time
The end of competition on a bus route - the effects from then end of the 234
Rumours of bus changes by First in Wiltshire - what we know and suspect
Identifying the first and last records in a sequence
Image indexer / thumbnail display scripts in PHP
Getting to the Royal United Hospital - the Hopper and the alternatives
Coats of arms - towns and authorities in Wiltshire
Pressure selling in the fire safety business
4734 posts, page by page
Link to page ... 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95 at 50 posts per page


This is a page archived from The Horse's Mouth at http://www.wellho.net/horse/ - the diary and writings of Graham Ellis. Every attempt was made to provide current information at the time the page was written, but things do move forward in our business - new software releases, price changes, new techniques. Please check back via our main site for current courses, prices, versions, etc - any mention of a price in "The Horse's Mouth" cannot be taken as an offer to supply at that price.

Link to Ezine home page (for reading).
Link to Blogging home page (to add comments).

You can Add a comment or ranking to this page

© WELL HOUSE CONSULTANTS LTD., 2017: 404 The Spa • Melksham, Wiltshire • United Kingdom • SN12 6QL
PH: 01144 1225 708225 • EMAIL: info@wellho.net • WEB: http://www.wellho.net • SKYPE: wellho

PAGE: http://www.wellho.net/mouth/4656_Ide ... uence.html • PAGE BUILT: Sat May 27 16:49:10 2017 • BUILD SYSTEM: WomanWithCat