Home Accessibility Courses Twitter The Mouth Facebook Resources Site Map About Us Contact
Python's Generator functions

When you write a program, you should split your code down into a series of named blocks, each of which performs a single logical action. Blocks will be called up in different ways in different languages - and they'll be know as functions, subs, methods, procs (and perhaps other names too), but the principle is very much the same in each case.

Named blocks are usually performed one after another as a sequence of statements from within another block.

In this example, the pow2 function is performed completely before the for loop is run - the returned list being used as input to feed the variable called vv:

def pow2(upto):
powers = []
startat = 1
startpower = 1
while startpower <= upto:
powers.append(startat)
startat *= 2
startpower += 1
return powers

for vv in pow2(10):
print vv


It works, and works well. But the list called "powers" is constructed completely and stored in memory while the for loop is run. That's not a problem in this case with a list of ten values, but it could be a problem if the list was to contain a billion values. Writing things in this style is rather like building up a whole supply of water in a reservoir, then letting it back out as it's required. If the reservoir isn't big enough, you'll get serious flooding and damage to nearby properties.

Python provides an alternative in Generator Functions. A generator function is one that does NOT run to completion when it's first called - instead, it only runs until it has a value available to return, at which point it yields that value back and suspends operation until called again to resume. Here's the program I provided above, modified to use a generator.

def pow2(upto):
startat = 1
startpower = 1
while startpower <= upto:
yield startat
startat *= 2
startpower += 1


for vv in pow2(10):
print vv


Operation is exactly the same as the example above ... but there is NOT an intermediate list produced - i.e. no reservoir, and no risk of flooding or damage if the reservoir spills. A better model in this case is to think of a water pipe with a tap (faucet) on the end, which can be turned on and off at will as the next element - drop of water - is required.

Generators provide a very neat way of providing data as required on applications that potentially use huge intermediate lists. If you've every wondered what the difference is between range and xrange, or between readlines and xreadlines, it's that the "x" version uses a generator internally. I recall being given a customer problem handling a huge (10 Gb) file, and fixing his problem straight away just by adding an "x" in front of "readlines". That is the power of a generator.

Further example
Further articles on Python functions
(written 2006-01-11, updated 2006-06-05)

 
Associated topics are indexed as below, or enter http://melksh.am/nnnn for individual articles
Y105 - Python - Functions, Modules and Packages
  [4361] Multiple yields and no loops in a Python generator? - (2014-12-22)
  [4212] Python functions - an introduction to how they work - (2013-11-16)
  [4161] Python varables - checking existance, and call by name or by value? - (2013-08-27)
  [4029] Exception, Lambda, Generator, Slice, Dict - examples in one Python program - (2013-03-04)
  [3945] vargs in Python - how to call a method with unknown number of parameters - (2012-12-06)
  [3931] Optional positional and named parameters in Python - (2012-11-23)
  [3885] Default local - a good choice by the author of Python - (2012-10-08)
  [3852] Static variables in Python? - (2012-08-29)
  [3766] Python timing - when to use a list, and when to use a generator - (2012-06-16)
  [3695] Functions are first class variables in Lua and Python - (2012-04-13)
  [3662] Finding all the unique lines in a file, using Python or Perl - (2012-03-20)
  [3474] Python Packages - groupings of modules. An introduction - (2011-10-11)
  [3472] Static variables in functions - and better ways using objects - (2011-10-10)
  [3464] Passing optional and named parameters to python methods - (2011-10-04)
  [3459] Catching the fishes first? - (2011-09-27)
  [3280] Passing parameters to Python functions - the options you have - (2011-05-07)
  [3159] Returning multiple values from a function call in various languages - a comparison - (2011-02-06)
  [2998] Using an exception to initialise a static variable in a Python function / method - (2010-10-13)
  [2994] Python - some common questions answered in code examples - (2010-10-10)
  [2929] Passing a variable number of parameters in to a function / method - (2010-08-20)
  [2878] Program for reliability and efficiency - do not duplicate, but rather share and re-use - (2010-07-19)
  [2766] Optional and named parameters to Python functions/methods - (2010-05-15)
  [2718] Python - access to variables in the outer scope - (2010-04-12)
  [2520] Global and Enable - two misused words! - (2009-11-30)
  [2506] Good example of recursion in Python - analyse an RSS feed - (2009-11-18)
  [2481] Sample code with errors in it on our web site - (2009-10-29)
  [2440] Optional parameters to Python functions - (2009-10-07)
  [2439] Multiple returns from a function in Python - (2009-10-06)
  [2011] Conversion of OSI grid references to Eastings and Northings - (2009-01-28)
  [1879] Dynamic code - Python - (2008-11-11)
  [1871] Optional and named parameters in Python - (2008-11-05)
  [1870] What to do with a huge crop of apples - (2008-11-04)
  [1869] Anonymous functions (lambdas) and map in Python - (2008-11-04)
  [1790] Sharing variables with functions, but keeping them local too - Python - (2008-09-09)
  [1784] Global - Tcl, PHP, Python - (2008-09-03)
  [1464] Python Script - easy examples of lots of basics - (2007-12-08)
  [1202] Returning multiple values from a function (Perl, PHP, Python) - (2007-05-24)
  [1163] A better alternative to cutting and pasting code - (2007-04-26)
  [1134] Function / method parameters with * and ** in Python - (2007-04-04)
  [959] It's the 1st, not the 1nd 1rd or 1th. - (2006-12-01)
  [949] Sludge off the mountain, and Python and PHP - (2006-11-27)
  [913] Python - A list of methods - (2006-11-03)
  [912] Recursion in Python - (2006-11-02)
  [900] Python - function v method - (2006-10-20)
  [821] Dynamic functions and names - Python - (2006-08-03)
  [775] Do not duplicate your code - (2006-06-23)
  [749] Cottage industry or production line data handling methods - (2006-06-07)
  [745] Python modules. The distribution, The Cheese Shop and the Vaults of Parnassus. - (2006-06-05)
  [668] Python - block insets help with documentation - (2006-04-04)
  [418] Difference between import and from in Python - (2005-08-18)
  [386] What is a callback? - (2005-07-22)
  [340] Code and code maintainance efficiency - (2005-06-08)
  [308] Call by name v call by value - (2005-05-11)
  [303] Lambdas in Python - (2005-05-06)
  [294] Python generator functions, lambdas, and iterators - (2005-04-28)
  [105] Distance Learning - (2004-10-31)
  [96] Variable Scope - (2004-10-22)


Back to
The fencepost problem
Previous and next
or
Horse's mouth home
Forward to
Smoke and mirrors
Some other Articles
Using PHP to output images, XML, Style sheets, etc
Open Source training from Well House Consultants
Merging pictures using PHP and GD
Smoke and mirrors
Python's Generator functions
The fencepost problem
''I don't know'' is sometimes a good answer
Converting between acres and hectares
A new sign
Colour doesn't have to mean colourful
4350 posts, page by page
Link to page ... 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87 at 50 posts per page


This is a page archived from The Horse's Mouth at http://www.wellho.net/horse/ - the diary and writings of Graham Ellis. Every attempt was made to provide current information at the time the page was written, but things do move forward in our business - new software releases, price changes, new techniques. Please check back via our main site for current courses, prices, versions, etc - any mention of a price in "The Horse's Mouth" cannot be taken as an offer to supply at that price.

Link to Ezine home page (for reading).
Link to Blogging home page (to add comments).

You can Add a comment or ranking to this page

© WELL HOUSE CONSULTANTS LTD., 2014: Well House Manor • 48 Spa Road • Melksham, Wiltshire • United Kingdom • SN12 7NY
PH: 01144 1225 708225 • FAX: 01144 1225 899360 • EMAIL: info@wellho.net • WEB: http://www.wellho.net • SKYPE: wellho

PAGE: http://www.wellho.net/mouth/561_Pyth ... tions.html • PAGE BUILT: Thu Sep 18 15:30:25 2014 • BUILD SYSTEM: WomanWithCat