| |||||||||||
| |||||||||||
Class of records - forum log file analysis
Object Oriented Python example from a Well House Consultants training course
More on Object Oriented Python [link]
Source code: forum.py Module: Y106
"""Python data analysis example.
We have a tab separated fields file - seven years of monthly statistics off the First Great Western Coffee Shop forum which is hosted and administered by Well House Consultants - and we want to come up with annual totals for some of the columns (column 2, new topics and column 3, new posts). The final tqo columns are member applications and most online at a time, and are not significant due to the numbers of applications from people looking to "spam" the forum, or automata reading the forum. Rather than write the program in one chunk, I have separated out the internal detail of how to handle the individual month records into a separate class called monthRecord, and this way I can keep all the very specific format stuff in an area of its own in the program, and have my main program be much clearer and easier to follow, as well as providing good separation between the data format and how I want to handle the data. The effect of this is to let me re-use the same data detail code for othere analyses, and also to allow me to reuse the main program section again even if the data is supplied to me in a different format. NOTE - you may download the data file from http://www.wellho.net/data/fgws.tsv """ class monthRecord(object): def __init__(self,m,y,topics,posts): self.m = m self.y = y self.topics = topics self.posts = posts # This method breaks down an incoming record into component parts and # then calls the constructor to turn it into an object @staticmethod def factory(record): month,newTopics,newPosts,newMembers,mostOnline = record.strip().split('\t') mon,yr = month.split(' ') return monthRecord(mon,int(yr),int(newTopics),int(newPosts)) def getPosts(self): return self.posts def getTopics(self): return self.topics def getYear(self): return self.y # Following code to run only if this program is run directly. # This allows me to pull this file into another program to re-use the code above if __name__ == "__main__": # read in the records from file series = [] for monthLine in open('fgws.tsv'): series.append(monthRecord.factory(monthLine)) # sum the fields in batchs of 12 and come up with a list of lines to print report = [] for yearNo in range(0,len(series),12): totalPosts = 0 totalTopics = 0 for monthNo in range(12): current = series[monthNo+yearNo] totalPosts += current.getPosts() totalTopics += current.getTopics() year = current.getYear() report.append("In year {0:4d} there were {1:6d} new posts in {2:6d} new topics".format (year,totalPosts,totalTopics)) # We want to print the list line first, so reverse the report list. This works in situ, so # that we cannot add the .reverse() onto the end of the for loop line. report.reverse() for line in report: print(line) """ Output: WomanWithCat:y106 grahamellis$ python3 forum.py In year 2007 there were 7330 new posts in 1320 new topics In year 2008 there were 25211 new posts in 2747 new topics In year 2009 there were 25925 new posts in 1892 new topics In year 2010 there were 24293 new posts in 2111 new topics In year 2011 there were 21836 new posts in 1807 new topics In year 2012 there were 18682 new posts in 1657 new topics In year 2013 there were 22285 new posts in 1533 new topics In year 2014 there were 22869 new posts in 1705 new topics In year 2015 there were 20711 new posts in 1421 new topics In year 2016 there were 17400 new posts in 1138 new topics WomanWithCat:y106 grahamellis$ python2 forum.py In year 2007 there were 7330 new posts in 1320 new topics In year 2008 there were 25211 new posts in 2747 new topics In year 2009 there were 25925 new posts in 1892 new topics In year 2010 there were 24293 new posts in 2111 new topics In year 2011 there were 21836 new posts in 1807 new topics In year 2012 there were 18682 new posts in 1657 new topics In year 2013 there were 22285 new posts in 1533 new topics In year 2014 there were 22869 new posts in 1705 new topics In year 2015 there were 20711 new posts in 1421 new topics In year 2016 there were 17400 new posts in 1138 new topics WomanWithCat:y106 grahamellis$ """ Learn about this subject
This module and example are covered on the following public courses:
* Learning to program in Python * Python Programming * Intermediate Python Also available on on site courses for larger groups Books covering this topic
Yes. We have over 700 books in our library. Books
covering Python are listed here and when you've selected a
relevant book we'll link you on to Amazon to order.
Other Examples
This example comes from our "Object Oriented Python" training module. You'll find a description of the topic and some
other closely related examples on the "Object Oriented Python" module index page.
Full description of the source code
You can learn more about this example on the training courses listed on this page,
on which you'll be given a full set of training notes.
Many other training modules are available for download (for limited use) from our download centre under an Open Training Notes License. Other resources
• Our Solutions centre provides a number of longer technical articles.
• Our Opentalk forum archive provides a question and answer centre. • The Horse's mouth provides a daily tip or thought. • Further resources are available via the resources centre. • All of these resources can be searched through through our search engine • And there's a global index here. Purpose of this website
This is a sample program, class demonstration or answer from a
training course. It's main purpose
is to provide an after-course service to customers who have attended our
public private or
on site courses, but the examples are made
generally available under conditions described below.
Web site author
Conditions of use
Past attendees on our training courses are welcome to use individual
examples in the course of their programming, but must check
the examples they use to ensure that they are suitable for their
job. Remember that some of our examples show you how not to do
things - check in your notes. Well House Consultants take no responsibility
for the suitability of these example programs to customer's needs.
This program is copyright Well House Consultants Ltd. You are forbidden from using it for running your own training courses without our prior written permission. See our page on courseware provision for more details. Any of our images within this code may NOT be reused on a public URL without our prior permission. For Bona Fide personal use, we will often grant you permission provided that you provide a link back. Commercial use on a website will incur a license fee for each image used - details on request. |
| ||||||||||
PH: 01144 1225 708225 • EMAIL: info@wellho.net • WEB: http://www.wellho.net • SKYPE: wellho PAGE: http://www.wellho.net/resources/ex.php • PAGE BUILT: Sun Oct 11 14:50:09 2020 • BUILD SYSTEM: JelliaJamb |