Home Accessibility Courses Twitter The Mouth Facebook Resources Site Map About Us Contact
For 2021 - online Python 3 training - see ((here)).

Our plans were to retire in summer 2020 and see the world, but Coronavirus has lead us into a lot of lockdown programming in Python 3 and PHP 7.
We can now offer tailored online training - small groups, real tutors - works really well for groups of 4 to 14 delegates. Anywhere in the world; course language English.

Please ask about private 'maintenance' training for Python 2, Tcl, Perl, PHP, Lua, etc.
An introduction to file handling in programs - buffering, standard in and out, and file handles

Stdout and Stderr

Programmers typically don't write code to output to the screen / current window, as to to so would be to provide an inflexible system - instead, they output to what's known as "standard out", also known as stdout. Usually stdout defaults to the screen / current window so there's no difference as far as the user is concerned, but ...
• It can be redirected to a file in the command line / shell so that the user can run a program, saving the output for further processing / emailing
• It can be piped into another command through the shell - so that a whole series of programs can be bolted together to process data through a series of stages
• It can be sent by email to the instigator of batch or timed jobs (set up by at or cron / crontab for example) rather than the instigator having to be up in the middle of the night when a bookkeeping job runs
• It can be routed to a user's browser via mechanisms such as the Common Gateway Interface (CGI) through the Apache httpd web server - thus producing results via a web page

In addition to stdout, there's another output called "Standard error" or stderr - and that's where the programmer should send warnings and errors. The idea of stderr is that error messages come up on the screen where the user's running a program from, even if the user has chosen to send the main output from the program to a file by redirecting stdout. The user wants to know if a program has had problems without having to delve deep into the output reports!

File handles and buffering

Output from programs is "buffered" ... setting up a route from the processor to the screen / file / browser for every single character would be inefficient, so there's an intermediate element that's called a "file handle". When a programmer outputs something, it's actually buffered by the file handle and then gets sent in batches - typically around 4k - to wherever the output is routed. It's efficient to do it this way.

On input (from a file, or from "Standard In" a.k.a. stdin) there is also a file handle buffer; again, this allows a whole sector from the disc to be efficiently transferred into memory, whereas the program will typically call it up line by line.

There are times that buffering would be inconvenient, and it's common (but not universal) for output buffers to be flushed at times other than when they're full:
• When a user input is about to be requested (it's nice to be able to read the prompt before you have to answer a question)
• When the file is closed (so that the complete data is stored)
• When a new line character is added to the buffer on a file handle that's outputting to the screen (so that slower running processes will be seen to be making progress)
• and at other times under program control (to allow the programmer to output a series of dots as a progress bar and have them appear to indicate progress rather than in a big splurge when the job's complete!)

Opening and closing file handles

In most of the languages we teach, stdin, stdout and stderr are open and available to you in all of your programs, without the need to exceptionally open them. There's an difference in PHP - which is first and foremost a web page language, where you'll need to open stdin as follows:
  $standard_in = fopen ("php://stdin","r");
before you can read from it as follows:
  $line = fgets($standard_in);

But if you want to access a file through a file handle (the usual way to do it!), you'll need to open the file with some sort of open or fopen statement first. That sets up the file handle buffers, etc, for you for the particular file you're going to be accessing - it would take an impractical amount of time and space to have every file on your disc automatically opened and available to you!

When you open a file handle for output, you've usually got a choice as to whether you open it for write (which means that any file that already exists with the same name in the same folder / directory will be completely overwritten), or for appens (which means that anything you write to the file will be added onto the end of any pre-existing data).

File Handles in Perl

In Perl 5 (and Perl 4 if you can remember it!), file handles are special variables which are expressed as bare words - there's no $ @ & % or * in front of the variable name. It's conventional to write them all in CAPITALS; that way, programmers coming to look at the code later know straight away that they're looking at a file handle and not some other sort of structure such as a predefined function or sub.

You have two standard output file handles available to you:

You have two standard input file handles available to you:
(DATA can read anything you've stored after a __END__ line in your program)

There are two forms of the open statement - one with two parameters, and one with three. In the latter, three parameter case, the first parameter is the file handle name, the second indicates if it's read, write, or append and the third is the file name. In the two parameter case, the mode (read, write append) and name are combined; that's the older way, but it's less elegant and I recommend you always use the more modern, but slightly longer, version for any new code you write.

When you're outputting to a file Perl (as always) has a staggering variety, but I recommend print or printf be used in most cases. With both of those built in functions, you specify the file handle directly after the function name and before a list of the values to be output. By default, print and printf send output to STDOUT, but there's no reason why you can't explicitly code this if you want to.

Input to a Perl program uses the < ... > operator, with the "..." being replaced by the file handle - that's STDIN or a named file handle. This operator defaults to reading from a file named on the command line (or STDIN if none is given) if you leave out the "..." completely. If you save the result of a read operator into a list, you'll read all data from the source up to the end of file and save each line of data into the next element of a list. Very useful - but not recommended if you've potentially got a huge file what may lead to memory overflow (we have customers with 10Gb files!), nor if you're reading from the keyboard, as few users will apprecite / understand that they need to type in ^D on Unix / Linux systems, or ^Z n/l on Windows systems to indicate "end of file".

See [here] for a Perl example including output to file, STDOUT implicity, STDOUT explicity, and STDERR.
(written 2010-09-21)

Associated topics are indexed as below, or enter http://melksh.am/nnnn for individual articles
Q101 - Object Orientation and General technical topics - Programming Principles
  [2001] I have not programmed before, and need to learn - (2009-01-19)
  [2022] Pre and post increment - the ++ operator - (2009-02-03)
  [2228] Where do I start when writing a program? - (2009-06-11)
  [2310] Learning to write high quality code in Lua - (2009-07-30)
  [2327] Planning! - (2009-08-08)
  [2415] Variable names like i and j - why? - (2009-09-22)
  [2510] The music of the stock market - (2009-11-22)
  [2550] Do not copy and paste code - there are much better ways - (2009-12-26)
  [2586] And and Or illustrated by locks - (2010-01-17)
  [2737] Improving your function calls (APIs) - General and PHP - (2010-04-24)
  [2769] Easy - but for whom? - (2010-05-18)
  [2878] Program for reliability and efficiency - do not duplicate, but rather share and re-use - (2010-07-19)
  [2915] Looking up a value by key - associative arrays / Hashes / Dictionaries - (2010-08-11)
  [3026] Coding efficiency - do not repeat yourself! - (2010-11-02)
  [3456] Stepping stones - early coding, and writing re-usable code quickly - (2011-09-24)
  [3542] What order are operations performed in, in a Perl expression? - (2011-12-07)
  [3548] Dark mornings, dog update, and Python and Lua courses before Christmas - (2011-12-10)
  [3551] Some terms used in programming (Biased towards Python) - (2011-12-12)
  [3673] Object oriented or structured - a comparison in Python. Also writing clean regular expressions - (2012-03-26)
  [3878] From Structured to Object Oriented Programming. - (2012-10-02)
  [3928] Storing your intermediate data - what format should you you choose? - (2012-11-20)
  [3954] Lesson 1 in programing - write clean, reuseable and maintainable tidy code - (2012-12-16)
  [4003] Web and console - same principle, same code - Ruby example - (2013-02-14)
  [4061] Seamless, integrated IT - we have a long way to go! - (2013-04-11)
  [4090] Test Driven Development in Python - Customer Comes First - (2013-05-16)
  [4118] We not only teach PHP and Python - we teach good PHP and Python Practice! - (2013-06-18)
  [4153] Rooms available tonight - how to code an algorithm from first principles - (2013-08-19)
  [4206] Writing the perfect program in Tcl? - (2013-11-13)
  [4325] Learning to program - what are algorithms and design patterns? - (2014-11-22)
  [4611] Hungarian, Camel, Snake and Kebab - variable naming conventions - (2016-01-03)
  [4632] Remember to ask the question before you listen for the answer - (2016-01-26)
  [4645] What are callbacks? Why use them? An example in Python - (2016-02-11)

P215 - Perl - More about Files
  [1225] Perl - functions for directory handling - (2007-06-09)
  [1709] There is more that one way - Perl - (2008-07-14)
  [1832] Processing all files in a directory - Perl - (2008-10-11)
  [2405] But I am reading from a file - no need to prompt (Perl) - (2009-09-14)
  [3320] Reading the nth line from a file (Perl and Tcl examples) - (2011-06-09)
  [3412] Handling binary data in Perl is easy! - (2011-08-30)
  [3839] Spraying data from one incoming to series of outgoing files in Perl - (2012-08-15)

H109 - PHP - Input / Output
  [114] Relative or absolute milkman - (2004-11-10)
  [616] printf - a flawed but useful function - (2006-02-22)
  [653] Easy feed! - (2006-03-21)
  [709] Handling huge data files in PHP - (2006-05-04)
  [997] Most recent file in a directory - PHP - (2006-12-18)
  [1094] PHP fread - truncated data - (2007-02-27)
  [1096] Sample script - FTP to get a file from within PHP - (2007-03-01)
  [1113] File and URL reading in PHP - (2007-03-20)
  [1442] Reading a file multiple times - file pointers - (2007-11-23)
  [1780] Server overloading - turns out to be feof in PHP - (2008-09-01)
  [3029] PHP data sources - other web servers, large data flows, and the client (browser) - (2010-11-04)
  [3159] Returning multiple values from a function call in various languages - a comparison - (2011-02-06)
  [3424] Divide 10000 by 17. Do you get 588.235294117647, 588.24 or 588? - Ruby and PHP - (2011-09-08)
  [4483] Moving from mysql to mysqli - simple worked example - (2015-05-03)

Back to
Removing the new line with chop or chomp in Perl - what is the difference?
Previous and next
Horse's mouth home
Forward to
Testimonials - Well House Consultants Open Source courses
Some other Articles
Well House Consultants - a potted history
Multiway branches in Perl - the given and when syntax
Cheap Country Hotel in Melksham, Wiltshire?
Testimonials - Well House Consultants Open Source courses
An introduction to file handling in programs - buffering, standard in and out, and file handles
Removing the new line with chop or chomp in Perl - what is the difference?
Well House Manor - the history of the hotel
Initial handling of phone calls and walk in visitors
The Well House team - September 2010
A Melksham news roundup
4759 posts, page by page
Link to page ... 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96 at 50 posts per page

This is a page archived from The Horse's Mouth at http://www.wellho.net/horse/ - the diary and writings of Graham Ellis. Every attempt was made to provide current information at the time the page was written, but things do move forward in our business - new software releases, price changes, new techniques. Please check back via our main site for current courses, prices, versions, etc - any mention of a price in "The Horse's Mouth" cannot be taken as an offer to supply at that price.

Link to Ezine home page (for reading).
Link to Blogging home page (to add comments).

You can Add a comment or ranking to this page

© WELL HOUSE CONSULTANTS LTD., 2022: 48 Spa Road • Melksham, Wiltshire • United Kingdom • SN12 7NY
PH: 01144 1225 708225 • EMAIL: info@wellho.net • WEB: http://www.wellho.net • SKYPE: wellho

PAGE: http://www.wellho.net/mouth/2964_.html • PAGE BUILT: Sun Oct 11 16:07:41 2020 • BUILD SYSTEM: JelliaJamb