|
| Home | Accessibility | Courses | Diary | The Mouth | Forum | Resources | Site Map | About Us | Contact |
awk - pattern scanning and formatting
AWK - AHO KERNIGHAN AND WEINBERGER PATTERN SCANNING LANGUAGE
awk (and its relatives gawk and nawk) is a pattern scanning and processing language that goes one step beyond grep; not only does it match, but it processes too. This means its a really powerful tool (and quick and easy too) for reformmating and reporting on data files.
Let's see an example. I would like to find all of my people who have Python skills in our sample data file:
[trainee@pasanda shell]$ awk /Python/ requests.txt
hazel PHP Python Perl Ruby ASP
leane PHP Python ASP Perl Java
olivia MySQL Python ASP PHP
adam Tcl/Tk Perl Python MySQL
barry Python XML Java Perl PHP
harry PHP Python Java
ken Tcl/Tk Python Java Perl
nigel PHP Python Java Perl
rupert Java Python MySQL
[trainee@pasanda shell]$
hazel PHP Python Perl Ruby ASP
leane PHP Python ASP Perl Java
olivia MySQL Python ASP PHP
adam Tcl/Tk Perl Python MySQL
barry Python XML Java Perl PHP
harry PHP Python Java
ken Tcl/Tk Python Java Perl
nigel PHP Python Java Perl
rupert Java Python MySQL
[trainee@pasanda shell]$
That's done just what a grep would do, but I can add a specification that I want to manipulate my output "just printing the names" I can do so:
[trainee@pasanda shell]$ awk '/Python/{print $1}' requests.txt
hazel
leane
olivia
adam
barry
harry
ken
nigel
rupert
[trainee@pasanda shell]$
hazel
leane
olivia
adam
barry
harry
ken
nigel
rupert
[trainee@pasanda shell]$
So that's "match a regular expression, and if it matches output as follows". Awk takes a series of pattern and operation statements and you'll usually wrap them in single quotes to protect them from the shell. Let's do something a bit more interesting:
[trainee@pasanda shell]$ awk 'BEGIN{print "Python Skills"; count=0};
/Python/{count++; printf "%3d %-10s mainly %s\n",count,$1,$2;}'
requests.txt
Python Skills
1 hazel mainly PHP
2 leane mainly PHP
3 olivia mainly MySQL
4 adam mainly Tcl/Tk
5 barry mainly Python
6 harry mainly PHP
7 ken mainly Tcl/Tk
8 nigel mainly PHP
9 rupert mainly Java
[trainee@pasanda shell]$
/Python/{count++; printf "%3d %-10s mainly %s\n",count,$1,$2;}'
requests.txt
Python Skills
1 hazel mainly PHP
2 leane mainly PHP
3 olivia mainly MySQL
4 adam mainly Tcl/Tk
5 barry mainly Python
6 harry mainly PHP
7 ken mainly Tcl/Tk
8 nigel mainly PHP
9 rupert mainly Java
[trainee@pasanda shell]$
Here, the block marked BEGIN is run before any lines of input are read (a block marked END could also have been provided) and then each line of the incoming data is checked against the pattern and a block is performed if it matches. You can specify multiple pattern and match blocks if you wish; awk is a complete language. Rather than write long and complex instructions, you can save your awk program into a file:
[trainee@pasanda shell]$ awk -f demo.awk requests.txt
Python Skills
1 hazel mainly PHP
2 leane mainly PHP
3 olivia mainly MySQL
4 adam mainly Tcl/Tk
5 barry mainly Python
6 harry mainly PHP
7 ken mainly Tcl/Tk
8 nigel mainly PHP
9 rupert mainly Java
Do not consider: antonia barbara cherry delia ethel florence
gloria iris jenny kerry margaret nina petra queenie rita
sally tina uva venus wendy xena yollanda zoe charles david
ed fred graham ivan john len morris orpheus peter quentin
steve tommy ulsyees victor william xavier yuri zachary
9 staff out of 52 have Python skills
[trainee@pasanda shell]$
Python Skills
1 hazel mainly PHP
2 leane mainly PHP
3 olivia mainly MySQL
4 adam mainly Tcl/Tk
5 barry mainly Python
6 harry mainly PHP
7 ken mainly Tcl/Tk
8 nigel mainly PHP
9 rupert mainly Java
Do not consider: antonia barbara cherry delia ethel florence
gloria iris jenny kerry margaret nina petra queenie rita
sally tina uva venus wendy xena yollanda zoe charles david
ed fred graham ivan john len morris orpheus peter quentin
steve tommy ulsyees victor william xavier yuri zachary
9 staff out of 52 have Python skills
[trainee@pasanda shell]$
Here's the complete awk script:
BEGIN{print "Python Skills"; yes=0; no=0; pass=""};
END{print "Do not consider:",pass;}
END{print yes,"staff out of",yes+no,"have Python skills"};
{
if (/Python/) {
yes++;
printf "%3d %-10s mainly %s\n",yes,$1,$2;
} else {
no++;
pass = sprintf("%s %s",pass,$1);
}
}
END{print "Do not consider:",pass;}
END{print yes,"staff out of",yes+no,"have Python skills"};
{
if (/Python/) {
yes++;
printf "%3d %-10s mainly %s\n",yes,$1,$2;
} else {
no++;
pass = sprintf("%s %s",pass,$1);
}
}
See also Linux Basics Course
Please note that articles in this section of our
web site were current and correct to the best of our ability when published,
but by the nature of our business may go out of date quite quickly. The
quoting of a price, contract term or any other information in this area of
our website is NOT an offer to supply now on those terms - please check
back via our main web site
Related Material
Web Application Deployment - Linux Utilities
Web Application Deployment - Linux -An Introduction For Users
resource index - Deployment
Solutions centre home page
You'll find shorter technical items at The Horse's Mouth and delegate's questions answered at the Opentalk forum.
At Well House Consultants, we provide training courses on subjects such as Ruby, Perl, Python, Linux, C, C++, Tcl/Tk, Tomcat, PHP and MySQL. We're asked (and answer) many questions, and answers to those which are of general interest are published in this area of our site.
PH: 01144 1225 708225 • FAX: 01144 1225 707126 • EMAIL: info@wellho.net • WEB: http://www.wellho.net • SKYPE: wellho