awk - a powerful data extraction and manipulation tool

Unix and Linux come with a whole toolkit of utility programs for filtering, sorting and manipulating data. As a "one off" job, you can use them from the command line, and for repeated jobs you can save a series of instructions for these tools in a shell script. Yesterday, I ran a private shell scripting bootcamp course, and found myself enjoying working with

grep - the filtering tool
sort - the tool that re-orders data
awk - for pattern matching and data manipulation
sed - the stream editor
tr - for transforming characters

Awk is especially worthy of mention. With Awk, you can select rows of data based on what they contain, or what is in certain fields, and manipulate those lines. Instructions may be given to allow different instructions depending on how they're elected, and awk also includes integer and floating point arithmetic capabilities, and the ability to undertake actions before and after the processing of any of the data, in matching clauses called BEGIN and END.

I have added sample awk scripts here and here to our web site, and a slightly longer article on awk here. And, yes, I can deliver training on awk and shell scripting ;-) ... Linux/Unix Basics will get you started, and we can tailor to meet your needs from there!
(written 2007-09-25 11:53:31)

Associated topics are indexed under

A101 - Web Application Deployment - Linux -An Introduction For Users
  [2636] Linux - useful tips including history and file name completion - (2010-02-15)
  [2494] Making Linux Politically correct - (2009-11-06)
  [2479] Accidentally typed ci rather than vi? - (2009-10-27)
  [2300] What does x on a linux directory mean? - (2009-07-21)
  [2299] How much space does my directory take - Linux - (2009-07-20)
  [2203] Always use su with minus. And where do programs come from? - (2009-05-27)
  [2201] Running straight from the jar, but not from a tar - (2009-05-26)
  [1904] Ruby, Perl, Linux, MySQL - some training notes - (2008-11-23)
  [1902] sstrwxrwxrwx - Unix and Linux file permissions - (2008-11-23)
  [1897] Keeping on an even keel - (2008-11-21)
  [1893] Some Linux and Unix tips - (2008-11-18)
  [1803] FTP passive mode - a sometimes cure for upload hangs - (2008-09-20)
  [1764] Yank and Push - copy and move in vi - (2008-08-21)
  [1651] ls command - favourite options - (2008-05-23)
  [1527] Selecting file names in a shell - one word or another - (2008-02-02)
  [1438] Copy and paste / cut and paste and other vi techniques - (2007-11-20)
  [1408] Wireless hotel tips - FTP and Skype connections failing - (2007-10-26)
  [1288] Linux run states, shell special commands, and directory structures - (2007-08-03)
  [1287] Work and play at Well House Manor - Football and Shell Shortcuts - (2007-08-02)
  [1259] Where am I and how did I get here? - (2007-07-05)
  [1068] ls -l report, Linux / Unix - types and permssions - (2007-02-06)
  [1013] Copy multiple files - confusing error message from cp - (2006-12-30)
  [1012] Moving files between Windows / DOS and Linux / Unix - (2006-12-30)
  [984] Cardinal numbers and magic numbers - (2006-12-14)
  [749] Cottage industry or production line data handling methods - (2006-06-07)
  [711] THE home directory or MY home directory - (2006-05-06)
  [710] Linux training Glasgow, Python programming course Dundee - (2006-05-05)
  [703] Copying files and preserving ownership - (2006-04-28)
  [679] More or less on the edge of the page - (2006-04-11)
  [659] Web Application Components - (2006-03-28)
  [593] Finding where the disc space has gone - (2006-02-06)
  [431] File permissions of Linux and Unix systems - (2005-08-31)
  [430] Linux commands - some basics - (2005-08-31)
  [249] An easy way out - (2005-03-17)
  [152] Aladdin, or careful what you wish. - (2004-12-15)
  [74] pushd and popd - (2004-10-05)
  [73] vi - full circle - (2004-10-04)

A166 - Web Application Deployment - Linux Utilities
  [2638] Finding what has changed - Linux / Unix - (2010-02-17)
  [2484] Finding text and what surrounds it - contextual grep - (2009-10-30)
  [2320] Helping new arrivals find out about source code examples - (2009-08-03)
  [2145] Using the internet to remotely check for power failure at home (PHP) - (2009-04-29)
  [1690] Conversion of c/r line ends to l/f line ends - (2008-06-28)
  [1361] Korn shell course - (2007-09-22)
  [71] Comparators in Linux and Unix - (2004-10-03)
  [63] Almost like old times - (2004-09-26)

Back to
Korn Shell scripts on the web

Previous and next
or
Horse's mouth home

Forward to
korn tips - some useful korn shell techniques

Some other Articles

Apache Tomcat Performance Tuning
One business, four different angles!
Getting the local voice heard
korn tips - some useful korn shell techniques
awk - a powerful data extraction and manipulation tool
Korn Shell scripts on the web
Korn shell course - resources
One computer per delegate, or pair programming?
No Thank You

2675 posts, page by page

Link to page ... 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54 at 50 posts per page

This is a page archived from The Horse's Mouth at http://www.wellho.net/horse/ - the diary and writings of Graham Ellis. Every attempt was made to provide current information at the time the page was written, but things do move forward in our business - new software releases, price changes, new techniques. Please check back via our main site for current courses, prices, versions, etc - any mention of a price in "The Horse's Mouth" cannot be taken as an offer to supply at that price.

Link to Ezine home page (for reading).
Link to Blogging home page (to add comments).

You can Add a comment or ranking to this page

Public Training Courses

Running regularly at our UK training Centre.
[Schedule] - [About] - [Book]