If you've got a program that runs for a long time, your users will wish to be kept informed of progress and how much longer there is to go. Now that's not always easy to predict (and I'm sure that most of you have made fun of such forecasts in the past) but its's much much much better than sitting staring at a blank screen.

There are a couple of "gotcha"s the first time you try to write a neat status line that overwrites itself repeatedly during the running of your program - it's not JUST a case of switching a \n (new line) to a \r (return) character. Three things that can be gotchas are:

1. Buffering. When your program outputs, you see the results on the screen but NOT right away - all programming languages buffer their output and write it out in a fewer, larger blocks or "buffers", typically writing the buffer out when a new line is thrown. It makes sense really - you wouldn't dream of having the rubbish man come around to collect the wrapper every time you finished a packet of crisps as that would be inefficient. Better to wait for his visit each Tuesday. In coding terms, though, if you don't *have* a \n character going out, you have to tell your program to flush the buffers under a different regime.

2. Output channel. If you output to the screen, that's great. Except that if you output to the screen in the same way that you output your results, they can appear mixed up and if the program's user runs via a redirect or pipe, the progress report AND real results both get written to the file. Result? No progress bar seen on the screen and, frankly, a mess in the file. Better to output to STDERR which is usually available to you and was originally intended for error message output.

3. Droppings. If you're overwriting each report line with another, then you have to be careful that report lines don't get shorter; if they were indeed to get shorter, then spurious characters can get left at the right from older lines and give the user a confusing report. A report line that concludes with directorys remaining - xxx where xxx is 110, then 105, then 92, then 84 ... will confusedly report 110, 105, 925, 845 .... Better to use a specific with format such as with Tcl's format command, printf in Perl, or the % operator in Python.

I've put together a source code example in Perl showing these three issues resolved in a simple program and there's a much fuller example that adds a real time completion estimate and an interrupt and intermediate report capability too.

A quick aside of forecasting time-to-go. You may have to start off with a very rough guess as forecasting a long event based on a very short-to-date run is notoriously hard, so do come back during your run from time to time and re-evaluate your total time estimate!
(written 2006-03-09, updated 2006-06-05)

 P667 - Perl - Handling Huge Data
  [762] Huge data files - what happened earlier? - (2006-06-15)
  [975] Answering ALL the delegate's Perl questions - (2006-12-09)
  [1397] Perl - progress bar, supressing ^C and coping with huge data flows - (2007-10-20)
  [1920] Progress Bar Techniques - Perl - (2008-12-03)
  [1924] Preventing ^C stopping / killing a program - Perl - (2008-12-05)
  [2376] Long job - progress bar techniques (Perl) - (2009-08-26)
  [2805] How are you getting on? - (2010-06-13)
  [2806] Macho matching - do not do it! - (2010-06-13)
  [2834] Teaching examples in Perl - third and final part - (2010-06-27)
  [3374] Speeding up your Perl code - (2011-07-30)
  [3375] How to interact with a Perl program while it is processing data - (2011-07-31)

P210 - Perl - Topicalization and Special Variables
  [493] Running a Perl script within a PHP page - (2005-11-12)
  [969] Perl - $_ and @_ - (2006-12-07)
  [1136] Buffering output - why it is done and issues raised in Tcl, Perl, Python and PHP - (2007-04-06)
  [1221] Bathtubs and pecking birds - (2007-06-07)
  [1232] Bathtub example - (2007-06-14)
  [1289] Pure Perl - (2007-08-03)
  [1444] Using English can slow you right down! - (2007-11-25)
  [1508] How not to write Perl? - (2008-01-15)
  [1704] Finding operating system settings in Perl - (2008-07-10)
  [1705] Environment variables in Perl / use Env - (2008-07-11)
  [1728] A short Perl example - (2008-07-30)
  [1829] Dont bother to write a Perl program - (2008-10-10)
  [1860] Seven new intermediate Perl examples - (2008-10-30)
  [1922] Flurinci knows Raby Lae PHP and Jeve - (2008-12-04)
  [2833] Fresh Perl Teaching Examples - part 2 of 3 - (2010-06-27)
  [2876] Different perl examples - some corners I rarely explore - (2010-07-18)
  [2972] Some more advanced Perl examples from a recent course - (2010-09-27)
  [3449] Apache Internal Dummy Connection - what is it and what should I do with it? - (2011-09-19)
  [4301] Perl - still a very effective language indeed for extracting and reporting - (2014-09-20)
  [4395] Preparing data through a little bit of Perl - (2015-01-15)
  [4682] One line scripts - Awk, Perl and Ruby - (2016-05-20)
  [4700] Obfurscated code - it might work, but is it maintainable? - (2016-07-02)

Back to A pile of sand? Where do we stand?

Previous and next or Horse's mouth home

Forward to Training Centre Pictures

If it's Sunday, it must be Bedwyn
How similar are two words
Simple but rugged form handling demo
Training Centre Pictures
Progress bars and other dynamic reports
A pile of sand? Where do we stand?
Carnival
What is your business latency and potential?
Odd one out.
Real chance to make a difference