Home Accessibility Courses Diary The Mouth Forum Resources Site Map About Us Contact
Python threads - a first example

If you have a process that you want to do several things at the same time, threads may be the answer for you. They let you set up a series of processes (or sub-processes) each of which can be run independently, but which can be brought back together later and/or co-ordinated as they run.

For many applications, threads are overkill but on some occasions they can be useful.

A PYTHON APPLICATION WHERE THREADS WOULD HELP

Let's say that you want to check the availability of many computers on a network ... you'll use ping. But there's a problem - if you "ping" a host that's not running it takes a while to timeout, so that when you check through a whole lot of systems that aren't responding - the very time a quick response is probably needed - it can take an age.

Here's a Python program that "ping"s 10 hosts in sequence.

import os
import re
import time
import sys

lifeline = re.compile(r"(\d) received")
report = ("No response","Partial Response","Alive")

print time.ctime()

for host in range(60,70):
   ip = "192.168.200."+str(host)
   pingaling = os.popen("ping -q -c2 "+ip,"r")
   print "Testing ",ip,
   sys.stdout.flush()
   while 1:
      line = pingaling.readline()
      if not line: break
      igot = re.findall(lifeline,line)
      if igot:
           print report[int(igot[0])]

print time.ctime()

Running that program, it works but it's a bit slow:

[trainee@buttercup trainee]$ python alive
Mon May 9 05:22:51 2005
Testing 192.168.200.60 No response
Testing 192.168.200.61 No response
Testing 192.168.200.62 No response
Testing 192.168.200.63 No response
Testing 192.168.200.64 No response
Testing 192.168.200.65 No response
Testing 192.168.200.66 Alive
Testing 192.168.200.67 No response
Testing 192.168.200.68 No response
Testing 192.168.200.69 No response
Mon May 9 05:23:19 2005
[trainee@buttercup trainee]$

That was 28 seconds - in other words, an extra 3 seconds per unavailable host.

THE SAME APPLICATION, WRITTEN USING PYTHON THREADS

I'll write the application and test it first ... then add a few notes at the bottom.

import os
import re
import time
import sys
from threading import Thread

class testit(Thread):
   def __init__ (self,ip):
      Thread.__init__(self)
      self.ip = ip
      self.status = -1
   def run(self):
      pingaling = os.popen("ping -q -c2 "+self.ip,"r")
      while 1:
        line = pingaling.readline()
        if not line: break
        igot = re.findall(testit.lifeline,line)
        if igot:
           self.status = int(igot[0])

testit.lifeline = re.compile(r"(\d) received")
report = ("No response","Partial Response","Alive")

print time.ctime()

pinglist = []

for host in range(60,70):
   ip = "192.168.200."+str(host)
   current = testit(ip)
   pinglist.append(current)
   current.start()

for pingle in pinglist:
   pingle.join()
   print "Status from ",pingle.ip,"is",report[pingle.status]

print time.ctime()

And running:

[trainee@buttercup trainee]$ python kicking
Mon May 9 05:23:36 2005
Status from 192.168.200.60 is No response
Status from 192.168.200.61 is No response
Status from 192.168.200.62 is No response
Status from 192.168.200.63 is No response
Status from 192.168.200.64 is No response
Status from 192.168.200.65 is No response
Status from 192.168.200.66 is Alive
Status from 192.168.200.67 is No response
Status from 192.168.200.68 is No response
Status from 192.168.200.69 is No response
Mon May 9 05:23:39 2005
[trainee@buttercup trainee]$

3 seconds - much more acceptable than the 28 seconds that we got when we pinged the hosts one by one and waited on each.

HOW DOES IT WORK?

We're going to run code concurrently to ping each host computer and (being Python), we create an object for each of the concurrent tests (threads) we wish to run. Each of these objects inherits from the Thread class so that we can use all of the logic already written in Python to provide our parallelism.

Although the constructor builds the thread, it does not start it; rather, it leaves the thread based object at the starting gate. The start method on the testit object actually triggers it off - internally, the start method triggers the run method of the testit class and alse returns to the calling code. In other words, this is the point at which the parallel processing actually starts, and the run method is called indirectly (via a callback, in a similar way to sort and map make callbacks).

Once parallel processes have started, you'll want some way to bring the responses back together at the end and in this first simple example, I've used a join. Note that this is NOT the same join method that you have in the string class ;-)

A join waits for a run method to terminate. So, having spawned a series of 10 pings in our example, our code waits for them to finish .... and it waits in the order that they were started. Some will be queuing up and completed long before our loop of joins gets to them, but that doesn't matter!

LOOKING AHEAD - MORE ON THREADS

If you're going to be writing a threaded application, there are (broadly) two main approaches you can take. The example that I've shown above uses a separate thread to take each request through from beginning to end and all the threads have the same structure. An alternative strategy is to write a series of "worker" threads each of which performs a step in a multistep process, and have them passing data on to one another. It's the difference between having an employee to walk each order through your factory and having employees at a production line passing each job on.

Where you're running threads, you have to be very much aware of the effect they can have on one another. As soon as you have two workers, their work may interfere with one another and you get involved in synchronisation (locking) of objects to ensure that this doesn't happen.

Locking / synchronisation brings its own further complications in that you have to involve "deadlocks" where two threads both require two resources ... each grabs the first, then waits for the second which (because the other has it) never becomes available.

FOOTNOTES

We've used the operating system's ping process in this example program. Ping responses vary between operating systems and you may need to alter the ping command and regular expression to match the response. The example above has been tested on Fedora Core Linux.

Threading makes heavy use of Operating System capabilities and is NOT as portable (no matter what language you're programming in) than most code


See also Learning Python

Please note that articles in this section of our web site were current and correct to the best of our ability when published, but by the nature of our business may go out of date quite quickly. The quoting of a price, contract term or any other information in this area of our website is NOT an offer to supply now on those terms - please check back via our main web site

Related Material

Additional Python Facilities
  [4298] Python - an interesting application - (2014-09-18)
  [4211] Handling JSON in Python (and a csv, marshall and pickle comparison) - (2013-11-16)
  [4085] JSON from Python - first principles, easy example - (2013-05-13)
  [3469] Teaching dilemma - old tricks and techniques, or recent enhancements? - (2011-10-08)
  [3442] A demonstration of how many Python facilities work together - (2011-09-16)
  [3089] Python regular expressions - repeating, splitting, lookahead and lookbehind - (2010-12-17)
  [2790] Joining a MySQL table from within a Python program - (2010-06-02)
  [2786] Factory methods and SqLite in use in a Python teaching example - (2010-05-29)
  [2765] Running operating system commands from your Python program - (2010-05-14)
  [2764] Python decorators - your own, staticmethod and classmethod - (2010-05-14)
  [2746] Model - View - Controller demo, Sqlite - Python 3 - Qt4 - (2010-04-29)
  [2745] Connecting Python to sqlite and MySQL databases - (2010-04-28)
  [2721] Regular Expressions in Python - (2010-04-14)
  [2655] Python - what is going on around me? - (2010-02-28)
  [2462] Python - how it saves on compile time - (2009-10-20)
  [2435] Serialization - storing and reloading objects - (2009-10-04)
  [2407] Testing code in Python - doctest, unittest and others - (2009-09-16)
  [1876] Python Regular Expressions - (2008-11-08)
  [1337] A series of tyre damages - (2007-09-08)
  [1336] Ignore case in Regular Expression - (2007-09-08)
  [1305] Regular expressions made easy - building from components - (2007-08-16)
  [1149] Turning objects into something you can store - Pickling (Python) - (2007-04-15)
  [1136] Buffering output - why it is done and issues raised in Tcl, Perl, Python and PHP - (2007-04-06)
  [1043] Sending an email from Python - (2007-01-18)
  [901] Python - listing out the contents of all variables - (2006-10-21)
  [753] Python 3000 - the next generation - (2006-06-09)
  [672] Keeping your regular expressions simple - (2006-04-05)
  [663] Python to MySQL - (2006-03-31)
  [463] Splitting the difference - (2005-10-13)
  [239] What and why for the epoch - (2005-03-08)
  [208] Examples - Gadfly, NI Number, and Tcl to C interface - (2005-02-10)
  [183] The elegance of Python - (2005-01-19)

Python - Threads - What why and how
  [2489] Parallel Pinging, using Python Threads or Expect spawn lists - (2009-11-02)
  [903] Pieces of Python - (2006-10-23)

resource index - Python
Solutions centre home page

You'll find shorter technical items at The Horse's Mouth and delegate's questions answered at the Opentalk forum.

At Well House Consultants, we provide training courses on subjects such as Ruby, Lua, Perl, Python, Linux, C, C++, Tcl/Tk, Tomcat, PHP and MySQL. We're asked (and answer) many questions, and answers to those which are of general interest are published in this area of our site.

You can Add a comment or ranking to this page

© WELL HOUSE CONSULTANTS LTD., 2014: Well House Manor • 48 Spa Road • Melksham, Wiltshire • United Kingdom • SN12 7NY
PH: 01144 1225 708225 • FAX: 01144 1225 899360 • EMAIL: info@wellho.net • WEB: http://www.wellho.net • SKYPE: wellho

PAGE: http://www.wellho.net/solutions/python-p ... ample.html • PAGE BUILT: Wed Mar 28 07:47:11 2012 • BUILD SYSTEM: wizard