When you write a program, you should split your code down into a series of named blocks, each of which performs a single logical action. Blocks will be called up in different ways in different languages - and they'll be know as functions, subs, methods, procs (and perhaps other names too), but the principle is very much the same in each case.

Named blocks are usually performed one after another as a sequence of statements from within another block.

In this example, the pow2 function is performed completely before the for loop is run - the returned list being used as input to feed the variable called vv:

def pow2(upto):

   powers = []

   startat = 1

   startpower = 1

   while startpower <= upto:

      powers.append(startat)

      startat *= 2

      startpower += 1

   return powers



for vv in pow2(10):

   print vv

It works, and works well. But the list called "powers" is constructed completely and stored in memory while the for loop is run. That's not a problem in this case with a list of ten values, but it could be a problem if the list was to contain a billion values. Writing things in this style is rather like building up a whole supply of water in a reservoir, then letting it back out as it's required. If the reservoir isn't big enough, you'll get serious flooding and damage to nearby properties.

Python provides an alternative in Generator Functions. A generator function is one that does NOT run to completion when it's first called - instead, it only runs until it has a value available to return, at which point it yields that value back and suspends operation until called again to resume. Here's the program I provided above, modified to use a generator.

def pow2(upto):

   startat = 1

   startpower = 1

   while startpower <= upto:

      yield startat

      startat *= 2

      startpower += 1





for vv in pow2(10):

   print vv

Operation is exactly the same as the example above ... but there is NOT an intermediate list produced - i.e. no reservoir, and no risk of flooding or damage if the reservoir spills. A better model in this case is to think of a water pipe with a tap (faucet) on the end, which can be turned on and off at will as the next element - drop of water - is required.

Generators provide a very neat way of providing data as required on applications that potentially use huge intermediate lists. If you've every wondered what the difference is between range and xrange, or between readlines and xreadlines, it's that the "x" version uses a generator internally. I recall being given a customer problem handling a huge (10 Gb) file, and fixing his problem straight away just by adding an "x" in front of "readlines". That is the power of a generator.

Further example
Further articles on Python functions
(written 2006-01-11, updated 2006-06-05)

 Y105 - Python - Functions, Modules and Packages
  [3474] Python Packages - groupings of modules. An introduction - (2011-10-11)
  [3472] Static variables in functions - and better ways using objects - (2011-10-10)
  [3464] Passing optional and named parameters to python methods - (2011-10-04)
  [3459] Catching the fishes first? - (2011-09-27)
  [3280] Passing parameters to Python functions - the options you have - (2011-05-07)
  [3159] Returning multiple values from a function call in various languages - a comparison - (2011-02-06)
  [2998] Using an exception to initialise a static variable in a Python function / method - (2010-10-13)
  [2994] Python - some common questions answered in code examples - (2010-10-10)
  [2929] Passing a variable number of parameters in to a function / method - (2010-08-20)
  [2878] Program for reliability and efficiency - do not duplicate, but rather share and re-use - (2010-07-19)
  [2766] Optional and named parameters to Python functions/methods - (2010-05-15)
  [2718] Python - access to variables in the outer scope - (2010-04-12)
  [2520] Global and Enable - two misused words! - (2009-11-30)
  [2506] Good example of recursion in Python - analyse an RSS feed - (2009-11-18)
  [2481] Sample code with errors in it on our web site - (2009-10-29)
  [2440] Optional parameters to Python functions - (2009-10-07)
  [2439] Multiple returns from a function in Python - (2009-10-06)
  [2011] Conversion of OSI grid references to Eastings and Northings - (2009-01-28)
  [1879] Dynamic code - Python - (2008-11-11)
  [1871] Optional and named parameters in Python - (2008-11-05)
  [1870] What to do with a huge crop of apples - (2008-11-04)
  [1869] Anonymous functions (lambdas) and map in Python - (2008-11-04)
  [1790] Sharing variables with functions, but keeping them local too - Python - (2008-09-09)
  [1784] Global - Tcl, PHP, Python - (2008-09-03)
  [1464] Python Script - easy examples of lots of basics - (2007-12-08)
  [1202] Returning multiple values from a function (Perl, PHP, Python) - (2007-05-24)
  [1163] A better alternative to cutting and pasting code - (2007-04-26)
  [1134] Function / method parameters with * and ** in Python - (2007-04-04)
  [959] It's the 1st, not the 1nd 1rd or 1th. - (2006-12-01)
  [949] Sludge off the mountain, and Python and PHP - (2006-11-27)
  [913] Python - A list of methods - (2006-11-03)
  [912] Recursion in Python - (2006-11-02)
  [900] Python - function v method - (2006-10-20)
  [821] Dynamic functions and names - Python - (2006-08-03)
  [775] Do not duplicate your code - (2006-06-23)
  [749] Cottage industry or production line data handling methods - (2006-06-07)
  [745] Python modules. The distribution, The Cheese Shop and the Vaults of Parnassus. - (2006-06-05)
  [668] Python - block insets help with documentation - (2006-04-04)
  [418] Difference between import and from in Python - (2005-08-18)
  [386] What is a callback? - (2005-07-22)
  [340] Code and code maintainance efficiency - (2005-06-08)
  [308] Call by name v call by value - (2005-05-11)
  [303] Lambdas in Python - (2005-05-06)
  [294] Python generator functions, lambdas, and iterators - (2005-04-28)
  [105] Distance Learning - (2004-10-31)
  [96] Variable Scope - (2004-10-22)

Back to The fencepost problem

Previous and next or Horse's mouth home

Forward to Smoke and mirrors

Using PHP to output images, XML, Style sheets, etc
Open Source training from Well House Consultants
Merging pictures using PHP and GD
Smoke and mirrors
Python's Generator functions
The fencepost problem
''I don't know'' is sometimes a good answer
Converting between acres and hectares
A new sign
Colour doesn't have to mean colourful