Home Accessibility Courses Twitter The Mouth Facebook Resources Site Map About Us Contact
 
Python and Tcl - public course schedule [here]
Private courses on your site - see [here]
Please ask about maintenance training for Perl, PHP, Lua, etc
 
An introduction to file handling in programs - buffering, standard in and out, and file handles

Stdout and Stderr

Programmers typically don't write code to output to the screen / current window, as to to so would be to provide an inflexible system - instead, they output to what's known as "standard out", also known as stdout. Usually stdout defaults to the screen / current window so there's no difference as far as the user is concerned, but ...
• It can be redirected to a file in the command line / shell so that the user can run a program, saving the output for further processing / emailing
• It can be piped into another command through the shell - so that a whole series of programs can be bolted together to process data through a series of stages
• It can be sent by email to the instigator of batch or timed jobs (set up by at or cron / crontab for example) rather than the instigator having to be up in the middle of the night when a bookkeeping job runs
• It can be routed to a user's browser via mechanisms such as the Common Gateway Interface (CGI) through the Apache httpd web server - thus producing results via a web page

In addition to stdout, there's another output called "Standard error" or stderr - and that's where the programmer should send warnings and errors. The idea of stderr is that error messages come up on the screen where the user's running a program from, even if the user has chosen to send the main output from the program to a file by redirecting stdout. The user wants to know if a program has had problems without having to delve deep into the output reports!

File handles and buffering

Output from programs is "buffered" ... setting up a route from the processor to the screen / file / browser for every single character would be inefficient, so there's an intermediate element that's called a "file handle". When a programmer outputs something, it's actually buffered by the file handle and then gets sent in batches - typically around 4k - to wherever the output is routed. It's efficient to do it this way.

On input (from a file, or from "Standard In" a.k.a. stdin) there is also a file handle buffer; again, this allows a whole sector from the disc to be efficiently transferred into memory, whereas the program will typically call it up line by line.

There are times that buffering would be inconvenient, and it's common (but not universal) for output buffers to be flushed at times other than when they're full:
• When a user input is about to be requested (it's nice to be able to read the prompt before you have to answer a question)
• When the file is closed (so that the complete data is stored)
• When a new line character is added to the buffer on a file handle that's outputting to the screen (so that slower running processes will be seen to be making progress)
• and at other times under program control (to allow the programmer to output a series of dots as a progress bar and have them appear to indicate progress rather than in a big splurge when the job's complete!)

Opening and closing file handles

In most of the languages we teach, stdin, stdout and stderr are open and available to you in all of your programs, without the need to exceptionally open them. There's an difference in PHP - which is first and foremost a web page language, where you'll need to open stdin as follows:
  $standard_in = fopen ("php://stdin","r");
before you can read from it as follows:
  $line = fgets($standard_in);

But if you want to access a file through a file handle (the usual way to do it!), you'll need to open the file with some sort of open or fopen statement first. That sets up the file handle buffers, etc, for you for the particular file you're going to be accessing - it would take an impractical amount of time and space to have every file on your disc automatically opened and available to you!

When you open a file handle for output, you've usually got a choice as to whether you open it for write (which means that any file that already exists with the same name in the same folder / directory will be completely overwritten), or for appens (which means that anything you write to the file will be added onto the end of any pre-existing data).

File Handles in Perl

In Perl 5 (and Perl 4 if you can remember it!), file handles are special variables which are expressed as bare words - there's no $ @ & % or * in front of the variable name. It's conventional to write them all in CAPITALS; that way, programmers coming to look at the code later know straight away that they're looking at a file handle and not some other sort of structure such as a predefined function or sub.

You have two standard output file handles available to you:
  STDOUT
  STDERR

You have two standard input file handles available to you:
  STDIN
  DATA
(DATA can read anything you've stored after a __END__ line in your program)

There are two forms of the open statement - one with two parameters, and one with three. In the latter, three parameter case, the first parameter is the file handle name, the second indicates if it's read, write, or append and the third is the file name. In the two parameter case, the mode (read, write append) and name are combined; that's the older way, but it's less elegant and I recommend you always use the more modern, but slightly longer, version for any new code you write.

When you're outputting to a file Perl (as always) has a staggering variety, but I recommend print or printf be used in most cases. With both of those built in functions, you specify the file handle directly after the function name and before a list of the values to be output. By default, print and printf send output to STDOUT, but there's no reason why you can't explicitly code this if you want to.

Input to a Perl program uses the < ... > operator, with the "..." being replaced by the file handle - that's STDIN or a named file handle. This operator defaults to reading from a file named on the command line (or STDIN if none is given) if you leave out the "..." completely. If you save the result of a read operator into a list, you'll read all data from the source up to the end of file and save each line of data into the next element of a list. Very useful - but not recommended if you've potentially got a huge file what may lead to memory overflow (we have customers with 10Gb files!), nor if you're reading from the keyboard, as few users will apprecite / understand that they need to type in ^D on Unix / Linux systems, or ^Z n/l on Windows systems to indicate "end of file".


See [here] for a Perl example including output to file, STDOUT implicity, STDOUT explicity, and STDERR.
(written 2010-09-21)

 
Associated topics are indexed as below, or enter http://melksh.am/nnnn for individual articles
H109 - PHP - Input / Output
  [4483] Moving from mysql to mysqli - simple worked example - (2015-05-03)
  [3424] Divide 10000 by 17. Do you get 588.235294117647, 588.24 or 588? - Ruby and PHP - (2011-09-08)
  [3159] Returning multiple values from a function call in various languages - a comparison - (2011-02-06)
  [3029] PHP data sources - other web servers, large data flows, and the client (browser) - (2010-11-04)
  [1780] Server overloading - turns out to be feof in PHP - (2008-09-01)
  [1442] Reading a file multiple times - file pointers - (2007-11-23)
  [1113] File and URL reading in PHP - (2007-03-20)
  [1096] Sample script - FTP to get a file from within PHP - (2007-03-01)
  [1094] PHP fread - truncated data - (2007-02-27)
  [997] Most recent file in a directory - PHP - (2006-12-18)
  [709] Handling huge data files in PHP - (2006-05-04)
  [653] Easy feed! - (2006-03-21)
  [616] printf - a flawed but useful function - (2006-02-22)
  [114] Relative or absolute milkman - (2004-11-10)

P215 - Perl - More about Files
  [3839] Spraying data from one incoming to series of outgoing files in Perl - (2012-08-15)
  [3412] Handling binary data in Perl is easy! - (2011-08-30)
  [3320] Reading the nth line from a file (Perl and Tcl examples) - (2011-06-09)
  [2405] But I am reading from a file - no need to prompt (Perl) - (2009-09-14)
  [1832] Processing all files in a directory - Perl - (2008-10-11)
  [1709] There is more that one way - Perl - (2008-07-14)
  [1225] Perl - functions for directory handling - (2007-06-09)

Q101 - Object Orientation and General technical topics - Programming Principles
  [4645] What are callbacks? Why use them? An example in Python - (2016-02-11)
  [4632] Remember to ask the question before you listen for the answer - (2016-01-26)
  [4611] Hungarian, Camel, Snake and Kebab - variable naming conventions - (2016-01-03)
  [4325] Learning to program - what are algorithms and design patterns? - (2014-11-22)
  [4206] Writing the perfect program in Tcl? - (2013-11-13)
  [4153] Rooms available tonight - how to code an algorithm from first principles - (2013-08-19)
  [4118] We not only teach PHP and Python - we teach good PHP and Python Practice! - (2013-06-18)
  [4090] Test Driven Development in Python - Customer Comes First - (2013-05-16)
  [4061] Seamless, integrated IT - we have a long way to go! - (2013-04-11)
  [4003] Web and console - same principle, same code - Ruby example - (2013-02-14)
  [3954] Lesson 1 in programing - write clean, reuseable and maintainable tidy code - (2012-12-16)
  [3928] Storing your intermediate data - what format should you you choose? - (2012-11-20)
  [3878] From Structured to Object Oriented Programming. - (2012-10-02)
  [3673] Object oriented or structured - a comparison in Python. Also writing clean regular expressions - (2012-03-26)
  [3551] Some terms used in programming (Biased towards Python) - (2011-12-12)
  [3548] Dark mornings, dog update, and Python and Lua courses before Christmas - (2011-12-10)
  [3542] What order are operations performed in, in a Perl expression? - (2011-12-07)
  [3456] Stepping stones - early coding, and writing re-usable code quickly - (2011-09-24)
  [3026] Coding efficiency - do not repeat yourself! - (2010-11-02)
  [2915] Looking up a value by key - associative arrays / Hashes / Dictionaries - (2010-08-11)
  [2878] Program for reliability and efficiency - do not duplicate, but rather share and re-use - (2010-07-19)
  [2769] Easy - but for whom? - (2010-05-18)
  [2737] Improving your function calls (APIs) - General and PHP - (2010-04-24)
  [2586] And and Or illustrated by locks - (2010-01-17)
  [2550] Do not copy and paste code - there are much better ways - (2009-12-26)
  [2510] The music of the stock market - (2009-11-22)
  [2415] Variable names like i and j - why? - (2009-09-22)
  [2327] Planning! - (2009-08-08)
  [2310] Learning to write high quality code in Lua - (2009-07-30)
  [2228] Where do I start when writing a program? - (2009-06-11)
  [2022] Pre and post increment - the ++ operator - (2009-02-03)
  [2001] I have not programmed before, and need to learn - (2009-01-19)


Back to
Removing the new line with chop or chomp in Perl - what is the difference?
Previous and next
or
Horse's mouth home
Forward to
Testimonials - Well House Consultants Open Source courses
Some other Articles
Well House Consultants - a potted history
Multiway branches in Perl - the given and when syntax
Cheap Country Hotel in Melksham, Wiltshire?
Testimonials - Well House Consultants Open Source courses
An introduction to file handling in programs - buffering, standard in and out, and file handles
Removing the new line with chop or chomp in Perl - what is the difference?
Well House Manor - the history of the hotel
Initial handling of phone calls and walk in visitors
The Well House team - September 2010
A Melksham news roundup
4759 posts, page by page
Link to page ... 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96 at 50 posts per page


This is a page archived from The Horse's Mouth at http://www.wellho.net/horse/ - the diary and writings of Graham Ellis. Every attempt was made to provide current information at the time the page was written, but things do move forward in our business - new software releases, price changes, new techniques. Please check back via our main site for current courses, prices, versions, etc - any mention of a price in "The Horse's Mouth" cannot be taken as an offer to supply at that price.

Link to Ezine home page (for reading).
Link to Blogging home page (to add comments).

You can Add a comment or ranking to this page

© WELL HOUSE CONSULTANTS LTD., 2019: 404 The Spa • Melksham, Wiltshire • United Kingdom • SN12 6QL
PH: 01225 708225 • EMAIL: info@wellho.net • WEB: http://www.wellho.net • SKYPE: wellho

PAGE: http://www.wellho.net/mouth/2964_An- ... ndles.html • PAGE BUILT: Sat May 27 16:49:10 2017 • BUILD SYSTEM: WomanWithCat