Tcl - Some example of HOW TO in handling data files and formats

For 2023 (and 2024 ...) - we are now fully retired from IT training.
We have made many, many friends over 25 years of teaching about Python, Tcl, Perl, PHP, Lua, Java, C and C++ - and MySQL, Linux and Solaris/SunOS too. Our training notes are now very much out of date, but due to upward compatability most of our examples remain operational and even relevant ad you are welcome to make us if them "as seen" and at your own risk.

Lisa and I (Graham) now live in what was our training centre in Melksham - happy to meet with former delegates here - but do check ahead before coming round. We are far from inactive - rather, enjoying the times that we are retired but still healthy enough in mind and body to be active!

I am also active in many other area and still look after a lot of web sites - you can find an index ((here))

During the Tcl course I was running earlier this week, we got involved in a number of interesting topics such as
• How to clean up input lines into lists of fields (use split)
• How to reformat awkward fields (use regexp and regsub)
• How to combine files that vary in format from year to year (define the format in a list of fields for each year)
• How to trap incomplete records (use info vars, and / or catch)
• How to save data based on a non-numeric key (use arrays)
• How to sort an array (You can't - sort a list of the keys)
• How to sort a list of keys based on an array element (use the -command option to lsort)

I had with me data files for passenger journeys booked to and from stations in Great Britain, year by year, from 2004 to 2010, and a further file with various extra data about each station, and during the course I prototyped / demonstrated the way - in Tcl - that I would manipulate this data though all the criteria above to produce a combined file. After the course, I completed the example and added comments too - and it's now [here]. There are good solutions within the code to each of the questions above ... and some other nice demos too like a use of uplevel to run a command in the variable namespace of the calling proc rather than in your own namespace.

You'll see from the code that Tcl remains an excellent language for applications of this type, even though it's very different indeed to languages like PHP and Python, which I code in more frequently. The same exercise could, indeed, have been done in Ruby or Perl, PHP, Lua or Python ... but it was a pleasure to do it in Tcl for a change. See [here] for details of our Tcl courses.

If you should come to this page through "rail interest", I've uploaded all the data files - including the output - in our rail resource at http://www.wellho.net/resources/Z501.html. The output file (which is a correlation which researchers may find useful) is at http://www.wellho.net/resources/ex.php4?item=z501/railstats.txt.

(written 2011-03-04)

Associated topics are indexed as below, or enter http://melksh.am/nnnn for individual articles

T209 - Tcl/Tk - File and Directory Handling
  [779] The fragility of pancakes - and better structures - (2006-06-26)
  [785] Running external processes in Tcl and Tcl/Tk - (2006-06-29)
  [1407] Reading from another process in Tcl (pipes and sockets) - (2007-10-26)
  [1426] Buffering up in Tcl - the empty coke can comparison - (2007-11-10)
  [1467] stdout v stderr (Tcl, Perl, Shell) - (2007-12-10)
  [2467] Tcl - catching an error before your program crashes - (2009-10-22)
  [3320] Reading the nth line from a file (Perl and Tcl examples) - (2011-06-09)
  [3429] Searching through all the files in or below a directory - Ruby, Tcl, Perl - (2011-09-09)
  [3617] The fileutil package and a list of file system commands in Tcl - (2012-02-18)
  [4461] Reading from a URL, and reading Json, from your Tcl script - (2015-03-12)
  [4523] Catching failed commands and not crashing the program in Tcl - (2015-10-10)
  [4524] Tcl - a new example for data reformatting - (2015-10-10)

T208 - Tcl/Tk - Arrays and dicts
  [122] Passing arrays to procs in Tcl - (2004-11-18)
  [1282] Stringing together Tcl scripts - (2007-07-29)
  [1283] Generating traffic for network testing - (2007-07-29)
  [1405] Sorting in Tcl - lists and arrays - (2007-10-24)
  [1427] Arrays in Tcl - a demonstration - (2007-11-10)
  [1614] When an array is not an array - (2008-04-17)
  [2466] Tcl - passing arrays and strings in and back out of procs - (2009-10-22)
  [3415] User defined sorting and other uses of callbacks in Tcl and Tk - (2011-09-02)
  [3582] Tcl collections - lists, dicts and array - (2012-01-16)
  [3614] Tcl - dicts - a tutorial and examples - (2012-02-14)
  [3638] Sorting dicts and arrays in Tcl - (2012-03-04)

T207 - Tcl/Tk - Procedures and Variable Scope
  [96] Variable Scope - (2004-10-22)
  [308] Call by name v call by value - (2005-05-11)
  [409] Functions and commands with dangerous names - (2005-08-11)
  [775] Do not duplicate your code - (2006-06-23)
  [1163] A better alternative to cutting and pasting code - (2007-04-26)
  [1404] Tcl - global, upvar and uplevel. - (2007-10-24)
  [1782] Calling procs in Tcl and how it compares to Perl - (2008-09-02)
  [2476] Tcl - uplevel to run code at calling level - (2009-10-24)
  [2520] Global and Enable - two misused words! - (2009-11-30)
  [2929] Passing a variable number of parameters in to a function / method - (2010-08-20)
  [3414] Passing back multiple results in Tcl - upvar and uplevel - (2011-09-01)
  [3629] Sharing lots of values in Tcl without having lots of global declarations - (2012-02-28)
  [4398] Accessing variables across subroutine boundaries - Perl, Python, Java and Tcl - (2015-01-18)

T205 - Tcl/Tk - String Handling in Tcl
  [404] How to check that a string contains a number in Tcl - (2005-08-06)
  [943] Matching within multiline strings, and ignoring case in regular expressions - (2006-11-25)
  [1403] Square Bracket protection in Tcl - (2007-10-23)
  [1410] Tcl / regsub - changing a string and using interesting bits - (2007-10-27)
  [1601] Replacing the last comma with an and - (2008-04-04)
  [2472] split and join in tcl and expect - (2009-10-23)
  [3285] Extracting data from a string / line from file - Tcl - (2011-05-10)
  [3576] Tcl - apparently odd behaviour of string trimleft - (2012-01-13)
  [4205] Regular Expression Substitution - Tcl - (2013-11-12)
  [4454] Everything is a string - even a list - (2015-03-11)
  [4455] Working out distance between places, using OS grid references and a program in Tcl - (2015-03-11)

T050 - Tcl/Tk - Tcl, Tcl/Tk and Expect - General
  [2429] Tcl scripts / processes on a web server via CGI - (2009-09-27)
  [2474] Using Tcl and Expect to automate repetitive jobs - (2009-10-24)
  [2504] Learning to program in ... - (2009-11-15)
  [2681] Tcl - a great engineering language - (2010-03-17)
  [3286] Should we cover expect and/or Tk on our public Tcl courses? - (2011-05-11)
  [4206] Writing the perfect program in Tcl? - (2013-11-13)
  [4434] Public training courses - upcoming dates - (2015-02-21)
  [4460] Using Object Oriented Tcl and the Tk toolkit together - real life example - (2015-03-12)
  [4616] Still teaching Tcl in 2016? - (2016-01-06)

Back to
Date of the Census (which month and day?)

Previous and next
or
Horse's mouth home

Forward to
Melksham Campus - what, why, questions, concerns and how you can input

Some other Articles

No news is good news.
Sunny day - Clanger and Picket Woods, Shearwater, and Westbury
Buses - what they cost and their future direction in the SW and in Wiltshire
Melksham Campus - what, why, questions, concerns and how you can input
Tcl - Some example of HOW TO in handling data files and formats
Date of the Census (which month and day?)
What do the following web sites have in common?
Tcl - the danger of square brackets in a while command
What proportion of the British Workforce still works a Monday to Friday, 9 to 5, week?
Past and Present - Melksham Station

4759 posts, page by page

Link to page ... 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96 at 50 posts per page

This is a page archived from The Horse's Mouth at http://www.wellho.net/horse/ - the diary and writings of Graham Ellis. Every attempt was made to provide current information at the time the page was written, but things do move forward in our business - new software releases, price changes, new techniques. Please check back via our main site for current courses, prices, versions, etc - any mention of a price in "The Horse's Mouth" cannot be taken as an offer to supply at that price.

Link to Ezine home page (for reading).
Link to Blogging home page (to add comments).

You can Add a comment or ranking to this page

Like this? ??