When you write a program, you should split your code down into a series of named blocks, each of which performs a single logical action. Blocks will be called up in different ways in different languages - and they'll be know as functions, subs, methods, procs (and perhaps other names too), but the principle is very much the same in each case.
Named blocks are usually performed one after another as a sequence of statements from within another block.
In this example, the pow2 function is performed completely before the for loop is run - the returned list being used as input to feed the variable called vv:
def pow2(upto):
powers = []
startat = 1
startpower = 1
while startpower <= upto:
powers.append(startat)
startat *= 2
startpower += 1
return powers
for vv in pow2(10):
print vv
It works, and works well.
But the list called "powers" is constructed completely and stored in memory while the for loop is run. That's not a problem in this case with a list of ten values, but it could be a problem if the list was to contain a billion values. Writing things in this style is rather like building up a whole supply of water in a reservoir, then letting it back out as it's required. If the reservoir isn't big enough, you'll get serious flooding and damage to nearby properties.
Python provides an alternative in
Generator Functions. A generator function is one that does NOT run to completion when it's first called - instead, it only runs until it has a value available to return, at which point it yields that value back and suspends operation until called again to resume. Here's the program I provided above, modified to use a generator.
def pow2(upto):
startat = 1
startpower = 1
while startpower <= upto:
yield startat
startat *= 2
startpower += 1
for vv in pow2(10):
print vv
Operation is exactly the same as the example above ... but there is NOT an intermediate list produced - i.e. no reservoir, and no risk of flooding or damage if the reservoir spills. A better model in this case is to think of a water pipe with a tap (faucet) on the end, which can be turned on and off at will as the next element - drop of water - is required.
Generators provide a very neat way of providing data as required on applications that potentially use huge intermediate lists. If you've every wondered what the difference is between range and xrange, or between readlines and xreadlines, it's that the "x" version uses a generator internally. I recall being given a customer problem handling a huge (10 Gb) file, and fixing his problem straight away just by adding an "x" in front of "readlines".
That is the power of a generator.
Further example
Further articles on Python functions (written 2006-01-11, updated 2006-06-05)
Associated topics are indexed as below, or enter http://melksh.am/nnnn for individual articles
Y105 - Python - Functions, Modules and Packages [96] Variable Scope - (2004-10-22)
[105] Distance Learning - (2004-10-31)
[294] Python generator functions, lambdas, and iterators - (2005-04-28)
[303] Lambdas in Python - (2005-05-06)
[308] Call by name v call by value - (2005-05-11)
[340] Code and code maintainance efficiency - (2005-06-08)
[386] What is a callback? - (2005-07-22)
[418] Difference between import and from in Python - (2005-08-18)
[668] Python - block insets help with documentation - (2006-04-04)
[745] Python modules. The distribution, The Cheese Shop and the Vaults of Parnassus. - (2006-06-05)
[749] Cottage industry or production line data handling methods - (2006-06-07)
[775] Do not duplicate your code - (2006-06-23)
[821] Dynamic functions and names - Python - (2006-08-03)
[900] Python - function v method - (2006-10-20)
[912] Recursion in Python - (2006-11-02)
[913] Python - A list of methods - (2006-11-03)
[949] Sludge off the mountain, and Python and PHP - (2006-11-27)
[959] It's the 1st, not the 1nd 1rd or 1th. - (2006-12-01)
[1134] Function / method parameters with * and ** in Python - (2007-04-04)
[1163] A better alternative to cutting and pasting code - (2007-04-26)
[1202] Returning multiple values from a function (Perl, PHP, Python) - (2007-05-24)
[1464] Python Script - easy examples of lots of basics - (2007-12-08)
[1784] Global - Tcl, PHP, Python - (2008-09-03)
[1790] Sharing variables with functions, but keeping them local too - Python - (2008-09-09)
[1869] Anonymous functions (lambdas) and map in Python - (2008-11-04)
[1870] What to do with a huge crop of apples - (2008-11-04)
[1871] Optional and named parameters in Python - (2008-11-05)
[1879] Dynamic code - Python - (2008-11-11)
[2011] Conversion of OSI grid references to Eastings and Northings - (2009-01-28)
[2439] Multiple returns from a function in Python - (2009-10-06)
[2440] Optional parameters to Python functions - (2009-10-07)
[2481] Sample code with errors in it on our web site - (2009-10-29)
[2506] Good example of recursion in Python - analyse an RSS feed - (2009-11-18)
[2520] Global and Enable - two misused words! - (2009-11-30)
[2718] Python - access to variables in the outer scope - (2010-04-12)
[2766] Optional and named parameters to Python functions/methods - (2010-05-15)
[2878] Program for reliability and efficiency - do not duplicate, but rather share and re-use - (2010-07-19)
[2929] Passing a variable number of parameters in to a function / method - (2010-08-20)
[2994] Python - some common questions answered in code examples - (2010-10-10)
[2998] Using an exception to initialise a static variable in a Python function / method - (2010-10-13)
[3159] Returning multiple values from a function call in various languages - a comparison - (2011-02-06)
[3280] Passing parameters to Python functions - the options you have - (2011-05-07)
[3459] Catching the fishes first? - (2011-09-27)
[3464] Passing optional and named parameters to python methods - (2011-10-04)
[3472] Static variables in functions - and better ways using objects - (2011-10-10)
[3474] Python Packages - groupings of modules. An introduction - (2011-10-11)
[3662] Finding all the unique lines in a file, using Python or Perl - (2012-03-20)
[3695] Functions are first class variables in Lua and Python - (2012-04-13)
[3766] Python timing - when to use a list, and when to use a generator - (2012-06-16)
[3852] Static variables in Python? - (2012-08-29)
[3885] Default local - a good choice by the author of Python - (2012-10-08)
[3931] Optional positional and named parameters in Python - (2012-11-23)
[3945] vargs in Python - how to call a method with unknown number of parameters - (2012-12-06)
[4029] Exception, Lambda, Generator, Slice, Dict - examples in one Python program - (2013-03-04)
[4161] Python varables - checking existance, and call by name or by value? - (2013-08-27)
[4212] Python functions - an introduction to how they work - (2013-11-16)
[4361] Multiple yields and no loops in a Python generator? - (2014-12-22)
[4407] Python - even named code blocks are objects - (2015-01-28)
[4410] A good example of recursion - a real use in Python - (2015-02-01)
[4441] Reading command line parameters in Python - (2015-02-23)
[4448] What is the difference between a function and a method? - (2015-03-04)
[4645] What are callbacks? Why use them? An example in Python - (2016-02-11)
[4662] Recursion in Python - the classic example - (2016-03-07)
[4719] Nesting decorators - (2016-11-02)
[4722] Embedding more complex code into a named block - (2016-11-04)
[4724] From and Import in Python - where is the module loaded from? - (2016-11-06)
Some other Articles
Using PHP to output images, XML, Style sheets, etcOpen Source training from Well House ConsultantsMerging pictures using PHP and GDSmoke and mirrorsPython's Generator functionsThe fencepost problem''I don't know'' is sometimes a good answerConverting between acres and hectaresA new signColour doesn't have to mean colourful