Exercises, examples and other material relating to training module A166. This topic is presented on public courses Deploying LAMP - Linux, Apache, MySQL, Perl/PHP
, Linux Basics
, Deploying Java Applications on Linux / Unix
Related technical and longer articlesawk - pattern scanning and formattingKorn Shell - BasicsKorn Shell - History, storing to file, commandsKorn Shell Scripting - an introduction
|Articles and tips on this subject||updated|
|3902||Shell - Grep - Sed - Awk - Perl - Python - which to use when?|
Last week, I found myself teaching a Multi-vendor Advanced Unix Data Tools and Techniques course as a guest presenter. The tools that the course 'majored' on are grep, sed, awk and Perl ... being an advanced course, some knowledge was assumed ahead of time, and so reference was made to other utilities ...
|3764||Shell, Awk, Perl of Python?|
Very frequently, IT system users find they've got a data file in one format and they need to filter it and transform it. On Linux and Unix systems, utilities such as grep and cut were the tradtions, perhaps with some awk.
Perl provided a single-program replacement for shell scripts that bolted together ...
|3446||Awk v Perl|
"The AWK utility is a data extraction and reporting tool" - Wikipedia
"Perl was developed as a language to make report processing easier" - Wikipedia
So what's the difference?
Looking through all the lines of a file, report the 6th and 1st fields on each line that contains the string "pix" and report ...
|2638||Finding what has changed - Linux / Unix|
If you want to find out what has changed in a file, use the diff command. And if you want to find out what has changed across a whole directory structure, use diff -r
If I want to compare a backup with my current system - to see what has been altered (perhaps when something has gone wrong and I want ...
|2484||Finding text and what surrounds it - contextual grep|
grep is a very useful tool for finding all the lines in a file (or series of files) that contain a particular string or match some other pattern / criteria. For example, I have a file that contains data for 52 staff members and I want to find who knows Lua ...
[trainee@melksham Download]$ grep Lua ...
|2320||Helping new arrivals find out about source code examples|
On 30th July, nearly 5300 people arrived at source code examples on our web site from Google searches, from 4000 unique IP addresses.
How do I know that? From analysing our log files:
-bash-3.2$ grep 'GET \/resources\/ex\.php.*\.google\.' ac_20090730 | grep -v Googlebot | wc
5382 129055 1832168
|2145||Using the internet to remotely check for power failure at home (PHP)|
Do you have freezers and other electrical equipment at home and worry about whether they'll be all right while you're away on holiday? Do you have a broadband connection with a rarely changing IP address, and a web site that you host with a web space provider? Then you'll be able to set up a "heartbeat" ...
|1690||Conversion of c/r line ends to l/f line ends|
Situation - Excel saved file that I want to read into a Lua program I'm writing on Linux. End of line characters on the Excel file are [cr], and Linux used [lf]. Quick solution - sed
sed 's/\x0d/\x0a/g' stationusage.txt > st2.txt
Literally "substitute globally every hex 0D with a hex 0A" ...
|1366||awk - a powerful data extraction and manipulation tool|
Unix and Linux come with a whole toolkit of utility programs for filtering, sorting and manipulating data. As a "one off" job, you can use them from the command line, and for repeated jobs you can save a series of instructions for these tools in a shell script. Yesterday, I ran a private shell scripting ...
|1361||Korn shell course|
It's been quite a while since I presented a course that majored on the Korn Shell - I know the subject right enough (having written a complete 2 day course that got rave if niche reviews during a previous lifetime), and I jumped at a chance to brush up on my old skills to day, to recall the differences ...
|71||Comparators in Linux and Unix|
I always say that it'll be a sad day when I don't learn anything ... I'm always delighted to pick up nuggets and reminders from
trainees on courses whilst appreciating that the major knowledge flow has to be the other way.
So, in the spirit of always learning, may I (re)present you with the sdiff command ...
Examples from our training material
|as1|| awk - BEGIN, END and patterns |
|as2|| awk - match in whole line or on specific field|
|as3|| Using awk to select and format data, and calculate|
|aspers.txt|| grep example output|
|demo.awk|| awk script to analyse a data file |
|demo.pl|| Sample Perl script to analyse data file |
|gents.dat|| Input data for utilities |
|ladies.dat|| input data for utilities|
|requests.txt|| input data for utilities |
Some modules are available for download
as a sample of our material or under an Open Training Notes License
for free download from http://www.training-notes.co.uk
Topics covered in this module
Piping and redirection of utilities.
Options and related commands.
Grep, sort, awk, sed cut and Perl.
Find, man and info.
If you are looking for a complete course and not just a information on a single subject, visit our Listing and schedule
Well House Consultants specialise in training courses in
Linux and LAMP
. We run
throughout the UK (and beyond for longer courses), and
at our training centre in Melksham, Wiltshire, England.
It's surprisingly cost effective to come on our public courses -
even if you live in a different
country or continent to us
We have a technical library of over 700 books on the subjects on which we teach.
These books are available for reference at our training centre. Also
available is the Opentalk
Forum for discussion of technical questions.