Korn Shell - Basics

What is a Shell Program / Script?

A Shell is a program which reads user inputs and controls program output - a command line interpretter. Instructions typed in are split down into component parts (tokenized) and acted on accordingly - typically with the computer's underlting operating system running one or more programs or commands, and returning the output of those programs as described within the instruction.

The term "Shell" is typically used in the Unix and Linux world - a DOS command line interpretter is the near equivalent for Windows.

Where you have a requirement to rerun the same series of commands on a Unix / Linux system, you can save your commands / instructions into a file and then perform the file as a whole - and that is commonly known as a "Shell Script" or "Shell Program".

All mainstream programming languages include conditional constructs (to allow the programmer to specify code which may or may not be performed depending on the result of a test), loops (to allow a specific piece of code to be repeatedly executed a number of times without it having to be repeated in the program), variables (to allow information to be stored within the program from one operation through to following ones) and functions (to allow a piece of code that's required multiple times to be called up by a single easy name whenever it's needed). Shells are no exception - they include conditionals, loops, variables, and functions.

Why are there several different shells available?

The original shell - knows as the Bourne Shell - is found in a file called "sh". Dating back many years now, to the days of printing terminals and batch computing, it was (and is) very strong on programming facilities compared to facilities that are really needed for good command line interaction. The Bourne shell remains the bedrock of Unix and Linux systems, with the operating system's internal scripts that are run at startup and shutdown time programmed in "sh".

In order to provide better interaction, The C Shell was developed and this provided far greater interaction facilities - for example, the ability to rerun a previous command which is something that is exceedingly useful on the command line, but pretty much unwanted in batch files. The C Shell was adopted as the standard for many users, especially on Unix varients such as SunOS / Solaris.

In order to make the best of both worlds, the Korn Shell (ksh) was developed. Compatible with the Bourne Shell, but adding to it the intercative capabiities (all be it with a different syntax) of the C shell, it was / is intended to provide the best of both worlds.

And there are other shells you'll come across too - ranging from "bash" - the bourne again shell - which is very popular on Linux systems, through tcsh (an open source varient of the C shell) right up the alphabet to zsh. You may also see reference to the pdksh - the public domain Korn shell - which is in practise the flavour used in Linux.

Some exceptions to the "just another shell" rule - note that rsh and ssh are remote shell commands / protocols - not really shells in their own rights, but mechanisms through which shells can be accessed via a network. Rsh allows you to run a command on a remote server, and the ssh program allows you to conatct one machine from a nother on a network, and run commands remotely with the traffic between the two hosts being encrypted to stop intermediate snooping.

Korn Shell Basics

-* Each instruction starts with the name of the command (which may be something that's built in to the shell, or a separate operating system command).

-* Instructions are followed, optionally, by a number of parameters which are separated from one another by white space

-* Instructions are terminated by a new line character.

Example:

$ grep -c public Temperatures.java WellHouseInput.java
Temperatures.java:2
WellHouseInput.java:6
$

The instruction is a seperate command called "grep" in this case, and it has four parameters. The first is "-c, the second is "public", the third is "Temperatures.java" and the fourth is "WellHouseInput.java". When run on the machine these notes were written on, you'll see that we typed the command against as shell prompt "$" character, and it produced two lines of output.

Metacharacters

As well as tokenising and passing through commands, shells support a number of metacharacters which effect how they do that parsing and what they do as they run commands.

PIPING AND REDIRECTION

> redirects the output from a command (on what is known as STDOUT or the regular job output) to a file named thereafter. If the file already exists, the prior contents are cleared out first.

>> redirects the output from a command (on what is known as STDOUT or the regular job output) to a file named thereafter. If the file already exists, new contents are added on to the end

< redirects the input to a command (know as STDIN or the regular input) from the named file rather than from the keyboard.

| diverts the STDOUT of one command in to the STDIN channel of the next one, thus making a production line of commands.

$ ls .. > first
$ date > second
$ ls .. >> second
$ cat -n second | more
     1 Sat Sep 22 07:11:39 BST 2007
     2 ABC
     3 abc.html
     4 ac_20060330.xyz
     5 ac_20060331.xyz
     6 ac_20070123.xyz
(etc)
$ cat -n < first
     1 ABC
     2 abc.html
     3 ac_20060330.xyz
     4 ac_20060331.xyz
     5 ac_20070123.xyz

As well as STDOUT, commands have a second output channel known as STDERR which is used for error messages. The idea of STDERR being separate is that you can direct the regular output to a file for later use, but still see the error messages on the screen. In the Korn shell, you can divert STDERR too using 2>.

$ ls sfdsdf
ls: sfdsdf: No such file or directory
$ ls sdfsdf > demo
ls: sdfsdf: No such file or directory
$ ls hjsdfhj 2> demo
$ cat demo
ls: hjsdfhj: No such file or directory
$ ls / wobble > head 2> body
$ cat head
/:
bin
boot
dev

etc

$ cat body
ls: wobble: No such file or directory
$

You can also combine STDOUT and STDERR to the same file using 2>&1. Doing two separate diverts to the same file will NOT work, as one output overrides the other.

$ ls / wobble > hh 2>&1
$ ls / wobble > ii 2> ii
$ diff ii hh
0a1
> ls: wobble: No such file or directory
$

Other combined uses ... you'll often pipe, pipe, pipe, pipe, redirect.

Example:

$ gzip -dc /usr/share/man/man1/ksh.1.gz | groff -T html > ksh.html
$ ls -l
total 168
-rw-r--r-- 1 trainee users 38 2007-09-22 07:21 body
-rw-r--r-- 1 trainee users 39 2007-09-22 07:18 demo
-rw-r--r-- 1 trainee users 2643 2007-09-22 07:11 first
-rw-r--r-- 1 trainee users 101 2007-09-22 07:21 head
-rw-r--r-- 1 trainee users 139 2007-09-22 07:22 hh
-rw-r--r-- 1 trainee users 101 2007-09-22 07:24 ii
-rw-r--r-- 1 trainee users 136357 2007-09-22 07:36 ksh.html
-rw-r--r-- 1 trainee users 2672 2007-09-22 07:11 second

WILD CARDS

If you write *.txt to match all the file names that end in .txt, it's actually the shell that's doing the work and providing the extra parameters. You can see this in the Korn shell by doing a set -x, then using a wildcard:

$ set -x
$ ls -l *.*
+ ls -l ksh.html ksh.txt
-rw-r--r-- 1 trainee users 1150 2007-09-22 07:47 ksh.html
-rw-r--r-- 1 trainee users 127555 2007-09-22 07:45 ksh.txt
$

This means that you can use the following wildcards to match EXISTING FILE NAMES ONLY (different to Windows Wild cards)

* - match 0 or more characters
? - match any one character
[xyz] - match one character from the list
~ - shorthand for your home directory
~fred - shorthand for fred's home directory.

Note that a range [a-z] is allowed, as is a ! for not - [!aeiou]

Other Korn shell "Wild cards" are:

@(...|...) - match one occurrence of any option
+(...|...) - match one or more occurrences of any option
*(...|...) - match zero or more occurrences of any option
?(...|...) - match zero or one occurrence of any option
!(...|...) - match anything that does NOT include any option

Examples:

+ ls
body demo first head hh ii ksh.html ksh.txt second
$ ls +(ks|he)*
+ ls head ksh.html ksh.txt
head ksh.html ksh.txt
$ ls !(hh|ii)
+ ls body demo first head ksh.html ksh.txt second
body demo first head ksh.html ksh.txt second
$

Although not REALLY wildcards, don't forget . and .. for "current directory" and "parent directory" respectively.

As with all Unix and Linux shells, you must NOT try to use these wild cards to create new file names.

QUOTING

If you enclose part of your command line in ' characters, you're saying that you wantit taken as a single parameter. Using " characters instead is similar, but allows any shell variable mentioned within to be expanded. And a \ character removes the special significance from just the following character.

$ cat peter john glover
cat: peter: No such file or directory
cat: john: No such file or directory
cat: glover: No such file or directory
$ cat "peter john glover"
cat: peter john glover: No such file or directory
$ cat 'peter john glover'
cat: peter john glover: No such file or directory
$ cat peter\ john glover
cat: peter john: No such file or directory
cat: glover: No such file or directory
$

If you use BACKQUOTES, you're doing the opposite of protecting a part of your command line - you're isntructing the shell to perform the contents before it performs the rest of the command, and substitute the information in place. Thus

$ date +%A
Saturday
$ ls > `date +%A`.txt
$ ls -l Sat*
-rw-r--r-- 1 trainee users 64 2007-09-22 09:26 Saturday.txt
$

CONTINUATION AND SPLITTING A LINE

You may specify several commands on one line, separating them with a ; characted, and you may use a \ to continue on form one line to another.

$ ls \
> -l
total 168
-rw-r--r-- 1 trainee users 38 2007-09-22 07:21 body
-rw-r--r-- 1 trainee users 39 2007-09-22 07:18 demo
-rw-r--r-- 1 trainee users 2643 2007-09-22 07:11 first
-rw-r--r-- 1 trainee users 101 2007-09-22 07:21 head
-rw-r--r-- 1 trainee users 139 2007-09-22 07:22 hh
-rw-r--r-- 1 trainee users 101 2007-09-22 07:24 ii
-rw-r--r-- 1 trainee users 1150 2007-09-22 07:47 ksh.html
-rw-r--r-- 1 trainee users 127555 2007-09-22 07:45 ksh.txt
-rw-r--r-- 1 trainee users 64 2007-09-22 09:26 Saturday.txt
-rw-r--r-- 1 trainee users 2672 2007-09-22 07:11 second
$ pwd ; ls
/home/trainee/korn
body demo first head hh ii ksh.html ksh.txt Saturday.txt second
$

ENVIRONMENT VARIABLES

When you start a new shell, you're provided with a number of "shell variables" - named memory locations with values relating to the current shell set in them.

You may use them within your commands by preceeding them with a $.

You may set them using an "=" assignment - NO DOLLAR is this case, please! (You never use a dollar when setting a variable in Shell - only when using it in a readonly context)

The default environment variables that you're provided with effect the operation of your shell or commands called up within it in some way or other, and if you change the values they contain you'll effect the subsequent behaviour of the shell and its children.

$ echo $WINDOWMANAGER
/usr/X11R6/bin/kde
$
$ PS1='Hello: '
Hello: pwd
/home/trainee/korn
Hello: PS1='$ '
$

COMMENTS

In the Korn shell, any TOKEN that starts with a # character indicates that the rest of that command is to be treated as a comment.

$ echo "The # chacater starts a comment"
The # chacater starts a comment
$ echo If I type a # in a line
If I type a
$ # This is a comment
$

You are strongly encouraged to comment your scripts, using

- Blank lines to separate logical code blocks
- Major comments on line starting with # characters
- Specific notes about individual commands on the line ends

THE HISTORY MECHANISM

The Korn shell remembers the commands that you have run, and makes them available to you through the history mechanism. The actual command used is called fc, but it's aliased in several ways to other names.

$ history
29 PS1='Hello: '
30 pwd
31 PS1='$ '
32 echo The # character starts a comment
33 echo The # character oh - \
34 echo "The # chacater starts a comment"
35 echo If I type a # in a line
36 # This is a comment
37 alias
38 alias 'll=ls -l'
39 ll
40 ll /
41 FCEDIT=vi
42 export FCEDIT=vi
43
44 alias
$ r 35
echo If I type a # in a line
If I type a
$

Also note that $_ is an environent variable that means "last parameter, previous line".

You can also set up a command line editor so that you can interactively use your history list:

$ set -o vi
$

Then - ESC to enter command mode, KJHL for navigation, i, x, a and more.

See also Linux Basics Course