Multiple processes (forking) in Python

What is forking

If you want a process to head off in two different directions, you can fork it. That results in identical copies of the process being created - the original (parent) and the copy (child), each with their own complete set of variables, and program counter, etc.

The clever thing is that the parent can tell it's the parent because the fork function return the process id of the child to it, and the child can tell its the child because it gets a false return from the fork. Thus you can test the code and have parent and child go off in different logic:

gerald = fork()

if not gerald:

  # child - doesn't know parent's PID

else:

  # parent - gets process id of child

Full Sample source

Why would you fork a process?

If you're writing something like a print driver, then it can be a good idea to fork a process off to print, while you continue with the main parent code.

We've got an example of this technique in use to "ping" to a whole series of remote systems ... forking off a new child each time that an IP address is entered, with potentially lots of children waiting for slow / inaccessible hosts in parallel

Full Sample Source

What is you want to correlate responses from the children?

The "trick" here is to open a pipe before you fork; forking connects the read handle in the child to the write handle in the parent, and vice versa, so that the child can remain in touch with the child.

Full Sample source

Note that that parent process will now have to wait for the child - so you're not getting the same parallel benefits. You can use signals, and methods that check whether data's available in the buffers, to add greater flexibility if you need to do so.

Alternatives

Python threads are a lighter weight way of parallel processing - where execution remains within a single process. I've got an article about that [here] in the solution centre.

You can also start off two completely independent processes - on the same computer or on different computers - and have them talk through a network port - TCP or UDP. That's a subject for a further post! Now made [here]

(written 2010-03-25) 5f34

Associated topics are indexed under

Y303 - Python Network Programming
  [4087] Python network programming - new FTP and socket level examples - (2013-05-14)
  [2765] Running operating system commands from your Python program - (2010-05-14)
  [2695] TCP v UDP / Client v Server - Python examples - (2010-03-25)
  [2368] Python - fresh examples of all the fundamentals - (2009-08-20)
  [2365] Counting Words in Python via the web - (2009-08-18)

P223 - Perl - Interprocess Communication
  [3940] Run other processes from within your Perl program - (2012-12-03)
  [3412] Handling binary data in Perl is easy! - (2011-08-30)
  [3011] What are .pid files? - (2010-10-23)
  [3010] Children, zombies, and reaping processes - (2010-10-23)
  [2970] Perl - doing several things at the same time - (2010-09-25)
  [1918] Perl Socket Programming Examples - (2008-12-02)
  [604] Perl - multiprocess applications - (2006-02-13)

Back to
Methods that run on classes (static methods) in Python

Previous and next
or
Horse's mouth home

Forward to
TCP v UDP / Client v Server - Python examples

Some other Articles

Ruth Davis, 1916 - 2010
Email metrics and filtering
Garlic bread without garlic
Multiple processes (forking) in Python
Methods that run on classes (static methods) in Python
Flexible search and replace in Python
New brochures for the Melksham area
The World Company Register - is it another scam?
Can my dog eat potatoes? Doggie Dietary Research, and political sleaze!