Programming in Expect can be very different (and more challenging) than conventional programming. Rather than write code (structured, if you like) through a series of events, you'll do better to start off with a co-operation diagram of the process that you want to automate [[For the newcomer, Expect is a language / system that lets you automate the running of command line based programs that require user interaction, with Expect acting rather like a puppeteer in pulling the strings and watching the responses ofthe target program.]]

As you'll want to store, analyse and make decisions based on the outputs from your manipulation of the target program, Expect is usually built in top of the Tcl language which provides a rich array of facilities ... and the three most fundamental additional commands are:

spawn to start a controlled process
send to send text to that controlled process
expect to wait for a response from that controlled process.

There are many other commands available too - and each of the three I mentioned has an interesting array of options associated with it.

The question arose yesterday - "What do I need to know to look at / debug / maintain Expect scripts that have been written by others and I maintain / what are the common issues and pitfalls". Now THERE is an interesting question to help me clear my mind. What did I come up with?

1. Know the fundamentals of Tcl - that it's a command based language and what thet implies, and understand the difference between "...", (...), [....] and {....}. Know when to use a $ character in front of a variable name, and when NOT to.

2. Understand the significance of spaces, new lines and blocking with { and }.

Then for debug / specifics on Expect:

a) be aware of the "--" option on many commands (the option to end all other options) - what it is and what it means.

b) be aware of different line teminators \r and \n for carriage return and new line / line feed.

c) remember that you much "expect" to receive an echo of what you send

d) When expecting a glob or regular expression, look for a clear end value and try to avoid wild cards on the end - they are a recipe for complexity as you don't know where your inputs will get trimmed at each expect

e) Beware cardinal values. Someone is going to have computers called "ping" and "pong" and if you're automating ping that could give some issues.

f) Remember to wait for trailing spaces on prompts if they're issued by the program you're automating

g) If the program that you're automating is upgraded, you MUST retest your expect script and you can "expect" it to need updating if the user interface has changed AT ALL. One change from a lower case to a capital letter in a prompt might be all that it takes.

h) You may need to wait on turn-around for some programs (expect can send very quickly once it recieves a trigger - far faster than a human user), and you have "-s" and "-h" - slow and human - optiond for the send command if you need them.

i) If your program stutters (goes one protocol exchange every 10 seconds), it's probably got out of sync and the expected strings are not being received - so it's timing out.

j) Remember the log_user command which lets you add a debugging trace onto the screen (other options too)

k) You can control multiple spawned processes if you manage the spawn_id, and you can wait on multiple processes using the -i option to expect.

And finally, buy a hard hat to protet yourself against brickbats that I EXPECT your colleagues will throw at you when they hear all the dreadful puns you unintentially drop.

We run Tcl courses that include Expect - there's a public course several times a year, and private courses can be arranged too. In practise, the Expect element is either vital (to around 30% of delegates) or of little interest (around 70%) and we tailor the courses as need be for the particular groups, running extra sessions on the public courses as appropriate. We're also happy (please email) to provide help / assistance / support / consultancy days on Expect.
(written 2007-05-02, updated 2007-05-03)

 T242 - Tcl/Tk - More on Expect
  [435] Expect for Windows - (2005-09-04)
  [1411] Buffering of inputs to expect, and match order - (2007-10-27)
  [1475] Tcl/Tk - updating your display while tasks are running - (2007-12-16)
  [1531] Expecting a item from a list of possibles - (2008-02-04)
  [2475] Quick easy and dangerous - automated logins via Tcl / Expect - (2009-10-24)
  [3009] Expect in Perl - a short explanation and a practical example - (2010-10-22)
  [3448] Checking all the systems on a subnet, using Expect and Tk - (2011-09-18)

T212 - Tcl/Tk - Expect Processes
  [287] Checking that all our servers are up and accessible - (2005-04-22)
  [675] Adding PHP tags to an old cgi program - (2006-04-08)
  [1785] What is running on your network? (tcl and expect) - (2008-09-04)
  [2474] Using Tcl and Expect to automate repetitive jobs - (2009-10-24)
  [2489] Parallel Pinging, using Python Threads or Expect spawn lists - (2009-11-02)
  [4678] Expect with Ruby - a training example to get you started - (2016-05-18)

T211 - Tcl/Tk - What is Expect? Why use it?
  [286] Automating regular manual procedures - (2005-04-21)
  [1174] Installing Tcl and Expect on Solaris 10 - a checklist - (2007-05-02)
  [1409] What is Expect? - (2007-10-26)
  [1469] Curley brackets v double quotes - Tcl, Tk, Expect - (2007-12-12)
  [1602] Automating processes through Expect - (2008-04-05)
  [3286] Should we cover expect and/or Tk on our public Tcl courses? - (2011-05-11)
  [3572] Adding Expect on top of Tcl - what is it and where can I get a training course to learn about it? - (2012-01-08)
  [4405] Backup procedures - via backup server - (2015-01-24)

Back to May day 2007 in pictures

Previous and next or Horse's mouth home

Forward to Installing Tcl and Expect on Solaris 10 - a checklist

Sorting out for a site map
A pu that got me into trouble
Local elections - choosing who to vote for
Cheat Sheet / Check list for Expect maintainers
May day 2007 in pictures
Traffic light control for horses
Smoking, or no? The law insists we spell out the obvious
Emailing as HTML (Web Page) - PHP example
Moving out some of the web site bloat