When you call a function / method in your program, you expect it to run to completion before returning control to you, passing back a result. For example, you might call a method to read a series of objects and store them into an array ... and then you could take that array of objects and, iterating through it, analyse each object in turn.

Such an approach works well if you've not got very many objects, but it's not clever if you have a lot. You're going to spend a lot of time - and fill up a lot of memory - storing the objects; perhaps it will lead to out-of-memory conditions, or "thrashing" of swap space as your application pages data on and off the disc. When you think about it, reading the data in is like feeding water into a reservoir, and if you keep doing it the dam holding the water overflows or bursts.

A better approach is to read in the information object by object, then manipulate them one at a time. Such code can be easily enough written, but the code's all going to be in a main program loop (or at least it will contain a series of calls at the application level to atomic routines that process one record at a time - a poor structure with things that should be internal to the data handlers being visible at the application level.

The solution offered in Ruby - and it's a neat one - it to use an iterator function / method - a method which yields a result as it runs, making a temporary return and carrying on when resumed. This is one aspect of what's sometimes referred to as parallelism.

I have added three examples to our web site in the last couple of days. A "shows you how" mechanism is [here]. There's an example that drip feeds data as if from a tap [here] (and that comparative analogy is a good one), and there's a final example [here] in which two routines are paralleled - the first feeding the data from the tap drip by drip, and the second drinking it one drop at a time.


The concept of iterators is also applied in Lua where they're known as coroutines, and in Python where they're generator functions.

Illustration - Public domain / Dam at Kaw Lake. [source]
(written 2010-10-01)

 

Associated topics are indexed as below, or enter http://melksh.am/nnnn for individual articles

R112 - Ruby - Parallelism
  [3156] Splitting data reading code from data processing code - Ruby - (2011-02-04)
  [4680] Processing data line by line - iterator in Ruby with yield - (2016-05-19)

Back to Christmas 2010 - Well House Manor, Melksham, Hotel

Previous and next or Horse's mouth home

Forward to Ruby - examples of regular expressions, inheritance and polymorphism

Strong arm tactics in the bus industry? Poor result for the customer!
Travel, Transport and Tourism - A vision for Melksham, 2026
How to set up short and meaningfull alternative URLs
Ruby - examples of regular expressions, inheritance and polymorphism
Ruby - yield; parallel routines
Christmas 2010 - Well House Manor, Melksham, Hotel
What is a factory method and why use one? - Example in Ruby
Creating, extending, traversing and combining Ruby arrays
Why do I need brackets in Ruby ... or Perl, Python, C or Java
Formatting your output - options available in Ruby