awk - a powerful data extraction and manipulation tool

For 2023 (and 2024 ...) - we are now fully retired from IT training.
We have made many, many friends over 25 years of teaching about Python, Tcl, Perl, PHP, Lua, Java, C and C++ - and MySQL, Linux and Solaris/SunOS too. Our training notes are now very much out of date, but due to upward compatability most of our examples remain operational and even relevant ad you are welcome to make us if them "as seen" and at your own risk.

Lisa and I (Graham) now live in what was our training centre in Melksham - happy to meet with former delegates here - but do check ahead before coming round. We are far from inactive - rather, enjoying the times that we are retired but still healthy enough in mind and body to be active!

I am also active in many other area and still look after a lot of web sites - you can find an index ((here))

Unix and Linux come with a whole toolkit of utility programs for filtering, sorting and manipulating data. As a "one off" job, you can use them from the command line, and for repeated jobs you can save a series of instructions for these tools in a shell script. Yesterday, I ran a private shell scripting bootcamp course, and found myself enjoying working with

grep - the filtering tool
sort - the tool that re-orders data
awk - for pattern matching and data manipulation
sed - the stream editor
tr - for transforming characters

Awk is especially worthy of mention. With Awk, you can select rows of data based on what they contain, or what is in certain fields, and manipulate those lines. Instructions may be given to allow different instructions depending on how they're elected, and awk also includes integer and floating point arithmetic capabilities, and the ability to undertake actions before and after the processing of any of the data, in matching clauses called BEGIN and END.

I have added sample awk scripts here and here to our web site, and a slightly longer article on awk here. And, yes, I can deliver training on awk and shell scripting ;-) ... Linux/Unix Basics will get you started, and we can tailor to meet your needs from there!
(written 2007-09-25)

Associated topics are indexed as below, or enter http://melksh.am/nnnn for individual articles

A166 - Web Application Deployment - Linux Utilities
  [63] Almost like old times - (2004-09-26)
  [71] Comparators in Linux and Unix - (2004-10-03)
  [1361] Korn shell course - (2007-09-22)
  [1690] Conversion of c/r line ends to l/f line ends - (2008-06-28)
  [2145] Using the internet to remotely check for power failure at home (PHP) - (2009-04-29)
  [2320] Helping new arrivals find out about source code examples - (2009-08-03)
  [2484] Finding text and what surrounds it - contextual grep - (2009-10-30)
  [2638] Finding what has changed - Linux / Unix - (2010-02-17)
  [3446] Awk v Perl - (2011-09-18)
  [3764] Shell, Awk, Perl of Python? - (2012-06-14)
  [3902] Shell - Grep - Sed - Awk - Perl - Python - which to use when? - (2012-10-22)
  [4586] Extending your bash shell with aliases, functions and extra commands - (2015-11-28)
  [4682] One line scripts - Awk, Perl and Ruby - (2016-05-20)

A101 - Web Application Deployment - Linux -An Introduction For Users
  [73] vi - full circle - (2004-10-04)
  [74] pushd and popd - (2004-10-05)
  [152] Aladdin, or careful what you wish. - (2004-12-15)
  [249] An easy way out - (2005-03-17)
  [430] Linux commands - some basics - (2005-08-31)
  [431] File permissions of Linux and Unix systems - (2005-08-31)
  [593] Finding where the disc space has gone - (2006-02-06)
  [659] Web Application Components - (2006-03-28)
  [679] More or less on the edge of the page - (2006-04-11)
  [703] Copying files and preserving ownership - (2006-04-28)
  [710] Linux training Glasgow, Python programming course Dundee - (2006-05-05)
  [711] THE home directory or MY home directory - (2006-05-06)
  [749] Cottage industry or production line data handling methods - (2006-06-07)
  [984] Cardinal numbers and magic numbers - (2006-12-14)
  [1012] Moving files between Windows / DOS and Linux / Unix - (2006-12-30)
  [1013] Copy multiple files - confusing error message from cp - (2006-12-30)
  [1068] ls -l report, Linux / Unix - types and permssions - (2007-02-06)
  [1259] Where am I and how did I get here? - (2007-07-05)
  [1287] Work and play at Well House Manor - Football and Shell Shortcuts - (2007-08-02)
  [1288] Linux run states, shell special commands, and directory structures - (2007-08-03)
  [1408] Wireless hotel tips - FTP and Skype connections failing - (2007-10-26)
  [1438] Copy and paste / cut and paste and other vi techniques - (2007-11-20)
  [1527] Selecting file names in a shell - one word or another - (2008-02-02)
  [1651] ls command - favourite options - (2008-05-23)
  [1764] Yank and Push - copy and move in vi - (2008-08-21)
  [1803] FTP passive mode - a sometimes cure for upload hangs - (2008-09-20)
  [1893] Some Linux and Unix tips - (2008-11-18)
  [1897] Keeping on an even keel - (2008-11-21)
  [1902] sstrwxrwxrwx - Unix and Linux file permissions - (2008-11-23)
  [1904] Ruby, Perl, Linux, MySQL - some training notes - (2008-11-23)
  [2201] Running straight from the jar, but not from a tar - (2009-05-26)
  [2203] Always use su with minus. And where do programs come from? - (2009-05-27)
  [2299] How much space does my directory take - Linux - (2009-07-20)
  [2300] What does x on a linux directory mean? - (2009-07-21)
  [2479] Accidentally typed ci rather than vi? - (2009-10-27)
  [2494] Making Linux Politically correct - (2009-11-06)
  [2636] Linux - useful tips including history and file name completion - (2010-02-15)
  [2831] Recording (a macro) in vi - (2010-06-27)
  [3179] Oops - I typed ci not vi, and have lost my file ... - (2011-02-21)
  [3256] Displaying a directory or file system tree - Linux - (2011-04-22)
  [3791] The Kernel, Shells and Daemons. Greek Gods in computing - (2012-07-01)
  [3819] Packing a tar, jar or war file - best practise - (2012-07-26)

Back to
Korn Shell scripts on the web

Previous and next
or
Horse's mouth home

Forward to
korn tips - some useful korn shell techniques

Some other Articles

Apache Tomcat Performance Tuning
One business, four different angles!
Getting the local voice heard
korn tips - some useful korn shell techniques
awk - a powerful data extraction and manipulation tool
Korn Shell scripts on the web
Korn shell course - resources
One computer per delegate, or pair programming?
No Thank You

4759 posts, page by page

Link to page ... 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96 at 50 posts per page

This is a page archived from The Horse's Mouth at http://www.wellho.net/horse/ - the diary and writings of Graham Ellis. Every attempt was made to provide current information at the time the page was written, but things do move forward in our business - new software releases, price changes, new techniques. Please check back via our main site for current courses, prices, versions, etc - any mention of a price in "The Horse's Mouth" cannot be taken as an offer to supply at that price.

Link to Ezine home page (for reading).
Link to Blogging home page (to add comments).

You can Add a comment or ranking to this page

Like this? ??