For 2023 (and 2024 ...) - we are now fully retired from IT training.
We have made many, many friends over 25 years of teaching about Python, Tcl, Perl, PHP, Lua, Java, C and C++ - and MySQL, Linux and Solaris/SunOS too. Our training notes are now very much out of date, but due to upward compatability most of our examples remain operational and even relevant ad you are welcome to make us if them "as seen" and at your own risk.

Lisa and I (Graham) now live in what was our training centre in Melksham - happy to meet with former delegates here - but do check ahead before coming round. We are far from inactive - rather, enjoying the times that we are retired but still healthy enough in mind and body to be active!

I am also active in many other area and still look after a lot of web sites - you can find an index ((here))
Controlling multiple asyncronous processes in Perl

Running processes in parallel

Perl is a great glueware language which includes a number of facilities that allow you to run other processes from yout per script - for example, you can open a file using a | character in the file handle, and you can run another process by calling it up in backquotes.

Examples

open (PRHI,"du -sk * |"); # Report disc use to me
open (PRHO,"| mail ... "); # mail the output on PRHO
$weblet = `wget http://www.wellho.net/index.html`;
           # Read a web page

In all these cases, though, you start off writing syncronised code in which you need to be aware of when to write to, or read from, the other process. You're also running just running your single Perl process so you've only got a single flow.

FORKING

The fork() function allows you to divide your Perl process into two - each identical except for the value that's returned by the fork call itself. Once you've forked, your (now) two processes can each go off down their own route in the logic.

When you fork, one of the processes retains the process id that the original pre-fork process used, and the fork() function returns to it the new process id that's associated with the other process. This process is known as the parent.

The other process, know as the child process, gets a new process id assigned to it and a zero return value from the fork. If it need to know its parent's process if for later on, that id should be saved PRIOR TO the fork. Thus:

$parent = $$;
unless ($pid = fork()) {
                # Child process goes here
  # $parent is parent and $$ is child
                exit();
                }
# Parent process goes here
# $pid is child, $$ is parent

Forking is only half of the story, though. Once you've got two or more processes running, under most circumstances they'll need to be able to communicate with oen another. Typically a child process that's running a task will need to communicate with a parent to return results, or to let the parent know that the child task has completed. And you need to select your mechanism for this communication very carefully - you certainly don't want to put any of the processes into a tight checking loop that would run to the severe detriment of other processes running on the same c.p.u,

COMMUNICATING VIA PIPES

If you open a PIPE before you fork your process, you're setting up a read and a write file handle with the write linked to the read. When you fork, the write file handle of the parent is attached to the read file handle of the child, and the read file handle of the parent to the write file handle of the child. In other words, the parent can write a message to the pipe that the child will be able to read, and the child can write a message to the pipe that the parent will be able to read.

For a parent forking just a single child, simple file handles can be used and teh scheme is easy to work. For a parents that's forking multiple children, you really need the parent to have a list of file handles and unfortuantley this isn't possible - you have to use a "typeglob".

Here's an example of a parent process that's forking a whole list of children:

@waitlist = (10,20,16,12,8,22);
foreach $item (@waitlist) {
        pipe *{$item},RETHAND;
        unless ($pid = fork()) {
                # Child process
                sleep $item;
                print RETHAND "Completed $item\n";
                exit();
                }
        }

Each of the children is going to run a sleep command - that's an example of the processing that you would do in there - and then prints its results back to the parent.

Using the code above, each of the children will be asleep at the same time, for anywhere between 8 and 22 seconds.

Let's then wait for the children to wake up:

foreach $item (@waitlist) {
        $response = <$item>;
        $now = localtime();
        print "$now - $response";
        }

That works well enough - after the longest sleep time of 22 seconds, all the child processes will have completed and the parent process will then exit.

fire:~/feb06 grahamellis$ perl pingalong
Started Sun Feb 26 15:52:10 2006
Forked by Sun Feb 26 15:52:10 2006
Sun Feb 26 15:52:20 2006 - Completed 10
Sun Feb 26 15:52:30 2006 - Completed 20
Sun Feb 26 15:52:30 2006 - Completed 16
Sun Feb 26 15:52:30 2006 - Completed 12
Sun Feb 26 15:52:30 2006 - Completed 8
Sun Feb 26 15:52:32 2006 - Completed 22
Completed Sun Feb 26 15:52:32 2006
fire:~/feb06 grahamellis$

Complete Source of this example
That's OK, but not brilliant. If you look at the timing of each of the completed processes, you'll see that the parent waited for the children in the order they were forked - so the parent is waiting for some of the slower children to complete while the faster but younger ones are waiting for attention.

ALERTING A PROCESS VIA A SIGNAL

If a process wants to communicate with another asynchronously (i.e. unexpectedly), it can do so via a signal.

The receiving process must first be set up to allow it to receive signals; this is done by setting an approapriate member of the %SIG hash. There are around 30 different signals available, but most of them already have other uses and we suggest you use the elements called USR1 or USR2. Thus:

$SIG{USR1} = "doneit";

sub doneit {
        $gotone = 1;
        }

in the source code of the parent / prior to the fork. This means that any process receiving a USR1 signal will simply set the $gotone variable to 1 so that the code in that receiving process can handle it in due course. It is important to limit the code in the signal handler function to almost nothing, since the signal can be received at any time and more complex code can lead to very nasty syncronisation and locking problems.

Also in the parent code, then, you'll check for the signal being received; since our parent isn't doing any processing in its own right, we'll put it into a slow sleep loop:

while (! $gotone) {
                sleep 1;
                }

Now all the child need to do is to use a kill command (yes, killing is not fatal to either child or parent) to send the signal:

kill "USR1",$parent;

and the parent will be alerted.

How does the parent know which child process has contacted it, though, if all it receives is a non-descript tug on the sleeve? It has to look and find out.

The first way you do this, which is somewhat clunky and inefficicient if there's a lot of signalling going on, is through a temporary file. This mechnaism, though, is perfactly adequate for a series of processes where there isn't much communication - for example, a setup where a Perl script is monitoring 20 or 30 long-running tests.

The child need to write a message to the parent BEFORE it signals:

open (FH,">zz$item.tmp");
print FH "Completed $item\n";
close FH;
kill "USR1",$parent;

and the parent then looks for, reads, deletes any signal files:

@temps = <zz*.tmp>;
foreach $rfile (@temps) {
        open (RF,$rfile);
        $response = <RF>;
        close RF;
        unlink $rfile;
}

If you run our previous example, with a whole series of faster and slower children spawned, under this new scheme, there's great news. The children that only sleep for a few seconds alert the parent and are dealt with while the others continue to slumber - even if the short-sleep children were started after the slower ones:

fire:~/feb06 grahamellis$ perl pinga2
Started Sun Feb 26 16:22:39 2006
Forked by Sun Feb 26 16:22:39 2006
Sun Feb 26 16:22:47 2006 - Completed 8
Sun Feb 26 16:22:49 2006 - Completed 10
Sun Feb 26 16:22:51 2006 - Completed 12
Sun Feb 26 16:22:55 2006 - Completed 16
Sun Feb 26 16:22:59 2006 - Completed 20
Sun Feb 26 16:23:01 2006 - Completed 22
Completed Sun Feb 26 16:23:01 2006
fire:~/feb06 grahamellis$

Complete Source of this example
READING BACK ASYNCRONOUSLY VIA SIGNALS AND PIPES

Temporary files aren't a great way of signalling on a busy system - it's much easier to write information to pipes in the children and have the parent check the pipe. However, any attempt to read from the pipe withour appropriate caustion would result in the reader waiting until data is available - the very problem we had in the first example and managed to alleviate in the second.

If we use a combination of signals and pipes, and Perl's select command, we can have each child process write back on a much more efficient pipe, and then signal the parent to say that it's done so. The parent can then look around with the select and see where the message is from.

Here's the code to reap all the sleeping children from that last example:

while ($kids > 0) {
        while (! $gotone) {
                sleep 1;
                }
        $gotone = 0;
        foreach $item (@waitlist) {
                $rin = $win = "";
                vec($rin, fileno(*{$item}), 1) = 1;
                $ein = $rin ;
                if (select($rin,$win,$ein,0)) {
                        sysread($item,$response,40);
                        $now = localtime();
                        print "$now - $response";
                        close $item;
                        $kids--;
                        }
                }
        }

A little more complex - and note the use of sysread rather than <> or read to overcome any buffering issues. It runs quickly, though, and all the children are dealt with within a second of them finishing, even if the finish out of any forecastable order.

fire:~/feb06 grahamellis$ perl pinga3
Started Sun Feb 26 16:31:01 2006
Forked by Sun Feb 26 16:31:01 2006
Sun Feb 26 16:31:09 2006 - Completed 8
Sun Feb 26 16:31:11 2006 - Completed 10
Sun Feb 26 16:31:13 2006 - Completed 12
Sun Feb 26 16:31:17 2006 - Completed 16
Sun Feb 26 16:31:21 2006 - Completed 20
Sun Feb 26 16:31:23 2006 - Completed 22
Completed Sun Feb 26 16:31:23 2006
fire:~/feb06 grahamellis$

Complete Source of this example


See also Perl for Larger Projects course

Please note that articles in this section of our web site were current and correct to the best of our ability when published, but by the nature of our business may go out of date quite quickly. The quoting of a price, contract term or any other information in this area of our website is NOT an offer to supply now on those terms - please check back via our main web site
Related Material

Perl - Interprocess Communication
resource index - Perl
Solutions centre home page

You'll find shorter technical items at The Horse's Mouth and delegate's questions answered at the Opentalk forum.

At Well House Consultants, we provide training courses on subjects such as Ruby, Perl, Python, Linux, C, C++, Tcl/Tk, Tomcat, PHP and MySQL. We're asked (and answer) many questions, and answers to those which are of general interest are published in this area of our site.
Training, Open Source
computer languages
Search our site for:
!!logtop!!
PerlPHPPythonMySQLApache / TomcatTclRubyJavaC and C++LinuxCSS 

!!booktop!!

!!bookstatus!!
!!bookform!!
Course Booking

Other Perl solutions
Go! Handling Cookies through CGI.pm
Go! "Perl - I didn't know you could do that"
Go! File Locking
Go! Graphic User Interfaces (GUIs)
Go! Data Monging
Go! Analysing incoming data lines
Go! Context - List, scalar and double quotes
Go! Controlling multiple asynchronous processes in Perl
Go! Nasty Characters in Web Applications
Go! Using LWP to write Web Clients
Go! Overview of what we're expecting in Perl 6
Go! Parrot - Perl's new Virtual Machine
Go! Making all your .html files run a Perl script
Go! Is Perl truly an OO language?
Go! POD (Plain Ole Documentation) - how to check it
Go! New to programming. Portable code. Perl or Java?
Go! Pattern Matching - a primer on regular Expressions
Go! Writing to and reading from files
Go! Interfacing applications to a MySQL database engine
Go! Object Orientation in Perl - First Steps
Go! What makes a good variable name?
Go! Accessing a MySQL database via a browser, Perl and CGI
Go! The wonders of Perl
Go! Using Perl to read Microsoft Word documents
Go! Solution Centre - all article listing
HOME ACCESSIBILITY RESOURCES SEARCH FORUM FAQ STAFF ACCESS CONTACT it's a white rule!

WELL HOUSE CONSULTANTS LTD.: Well House Manor • 48 Spa Road • Melksham, Wiltshire • United Kingdom • SN12 7NY
+44 PH: (0) 1225 708 225 • FAX: (0) 1225 707 126 • EMAIL: info@wellho.net • WEB: http://www.wellho.net • SKYPE: wellho