Horse's Mouth

« September 2007 | Main | November 2007 »

October 31, 2007

What software version do we teach?

To go with the very latest - even if the customers don't have it in common use yet, or to stay with older and well loved versions even though there is a new model out? A thorny question and one I'm addressing at the moment for the 2008 brochure. I write ...

"""With a few notable exceptions, Open Source programming languages and operating systems change very little from one release to the next in terms of how the programmer uses them for fundamental tasks. And when you think of it, that's a very good feature as it helps to extend the life cycle of code. It also means that few users are in a rush to upgrade to the very latest release when it come out, and indeed many people take a sensible decision to freeze with a software version for quite long periods.

In general, it's our philosophy to install a recent stable version that's in common use for our delegates to use on each delegate system. Training notes are updated to include information on significant changes in the very latest versions, and in some cases on published plans for enhancements in upcoming releases. Additional demonstration systems/versions are provided where there is a significant new release that isn't yet in common or stable use, or where there is an older version that remains common and may be used by the delegates in their day to day work.

• Perl
Intro - Advanced - Web

Perl version 5 has been around for many years, and forms the core of our Perl training. We have already been talking with delegates about Perl 6, helping them to be prepared to take best advantage of it when it comes into common use - but we don't see that changeover happening at all quickly. As and when stable versions of Perl 6 become available during the life of this brochure, we will provide firstly demonstrations and then both versions on delegate's systems.

Exceptionally, the change from Perl 5 to native Perl 6 will be a big one, and when the time is appropriate we may offer separate courses and "Perl 6 for Perl 5 programmers" days too. Please watch our web site for details

• Apache httpd and Tomcat web servers.
httpd - Tomcat - Linux with httpd - Linux with both

We are mostly training on httpd version 2.2 (to include the new proxy balancer where appropriate) but with some systems on 2.0 and comment back to 1.3; On Tomcat, current version is 6, but with a great deal of 5.5 in use and much reference back to legacy versions. Of all the products on which we provide training, companies are slowest to upgrade their web servers

• Linux / Unix
Basics - Admin - Web Server - Java Deployment

Our Linux and Unix systems run a variety of distributions including Red Hat, SuSE, Ubuntu, Solaris and OSX.

• PHP
Background - Intermediate - Object Orientation

We use version 5 of PHP for our courses, but with a legacy PHP 4 server available (in practice, PHP 4 code usually runs easily on PHP 5). With PHP, there are server configuration issues which effect how the code runs - you may hear people talk about "register globals", "magic quotes" and "short tags" for example. We configure our servers to be forward-looking in how we set them up, bearing in mind that we'll probably be using PHP 6 before the next brochure is printed and so will many of our clients. And we already talk about PHP 6."""

For 2008, you'll also here talk of - and in some cases see production versions of - Python 3000, Tcl 9, MySQL 6, Java, Ruby ....

Posted by gje at 08:12 AM | Comments (0)

Related topics: via article database
More about Graham Ellis of Well House Consultants

October 30, 2007

Good, steady, simple example - Perl file handling

I write a lot of demonstration pieces, and there's always a need for a balance.

Spike solutions / proof of principle and concept work well in front of a class. But then they are subject to major criticism due to a lack of robustness, and for security reasons, if they are published without comment (or even if published WITH comment, as many people seem to ignore comments!)

Fully "salemanproofed" (if you're in sales support) or "Squaddieproofed" (if you're doing military systems) examples are great - except that they contain so much extra testing and checking that you can't see the wood for the trees when explaining them to a newcomer programmer.

Catch 22? To some extent yes, but at times I try to steer the happy medium and it works. For example, I wrote a Perl example in front of the class yesterday ... checking files, opening, reading, writing them in Perl ... it shows many mechanisms and whilst it's not "A Rembrandt" it's a reasonable job.

And I have added comments and POD (Plain Ole Documentation) so that it's useful for other too. See here (Separate window will open).

A couple of extra comments - yes, I KNOW that
• I should not hard code the file names and paths
• The error messages should be improved
• There should be a check on command line params / a usage line
• Maybe I should use the 3 parameter form of open
• And that you could come up with other issues too.
But it makes a great example to teach the principles!

Posted by gje at 11:07 PM | Comments (0)

Related topics: via article database

Useful link: Perl training

October 29, 2007

Better technology makes for longer commutes?

5% more people now have a commute that takes over an hour ago than was the case in 1996. So says the radio this morning. Why should that be in a country when we're improving technology and supposedly can move things quicker and more efficiently generation by generation?

The radio highlighted traffic and congestion as a cause, and that's where the article lead - indeed, for business journeys where I have to drive I had been reviewing my 2008 strategy and I'll now be staying away in a lot more areas, and staying an extra night when I present (for example) a course in Cambridge. It's foolish to drive when tired, and I know there are times I'm getting close to my limit.

But there are other causes of longer commutes too - changing demographics and the cost of housing in the stockbroker belt have pushed some commuters further out - people traveling in from Chippenham and Hungerford and Andover - a handful in 1996 but a significant flow in 2006. And then there's local issues which I have dealt with here - a longer post with pictures!

Over the past month, I've watched how The Coffeeshop has grown in popularity - in posts and viewers who are starting to take more interest in their daily time with the railway - after all, "an hour each way" is 8% of their time which is pretty darned significant! I expect that site will be even busier come the December changes!

On the positive "side", perhaps this is an excellent time for our Business Hotel. As I write, we've only been running for a year but most rooms were sold last night and we've a busy week ahead. We provide the businessman with what he or she needs to have a good night's sleep, facilities to prepare personally and businesswise for the day, and to do a really good job.

And on that positive note, I should get ready for my Perl Course

Posted by gje at 09:12 AM | Comments (0)

Related topics: via article database

October 28, 2007

What we teach - expained for the non-technical

"Explain each of the subjects that we teach in a short paragraph written for mangers and booking / administrative people who aren't into the technology themselves" ... a challenge from Lisa which I have spend some time on today. And as I'm giving a Perl course tomorrow, what better way to get back into the Perl mood by using a piece of Perl to extract all those new descriptions from the markup that I've included them in!

Technology for PHP
PHP binds together HTML (used for web page design), databases (such as MySQL) which are used for data storage, management and analysis, and programming. The combined facilities allows the PHP-knowledgeable web site author to provide sites which combine all the technologies into complete web based applications.

PHP Programming
PHP is a programming language that was developed and designed for use on the web, and it's now so popular that you'll find it available on more domains than any other language. That's not because it's free - it's because it's a very flexible and efficient tool. PHP can be used for applications as easy as color and font control to handle accessibility issues, through to complex on line ordering and stock and order tracking systems.

Object Oriented programming with PHP
Object Oriented programming techniques enable larger applications and systems to be written and maintained in a controlled manner without ongoing programming work getting out of hand. System life cycles can be extended, with increased code re-usability, maintainability and robustness.

The MySQL Relational database
MySQL is a relational database engine. It lets you enter, store and search tables of information. Those tables can "relate" to each other, allowing a complex data set to be stored, and complex searches made. The database engine does all the hard work of storing and formatting the data for you - all you have to do is program the engine using the SQL language.

Deploying Apache and Tomcat
Apache httpd and Apache Tomcat are two web servers which, when used together, provide a flexible, stable and configurable platform for organisations to deploy web based applications written in Java. You'll find the two servers in use together on many large scale services - commercial, educational, voluntary sector, research and governmental.

Python Programming
Python is powerful, portable, object-oriented open source programming language for writing stand alone programs, quick scripts, and prototypes for large applications.

Ruby Programming
Ruby is an object oriented scripting language that has rapidly grown in use and favor across a wide variety of applications in the past two years. Perhaps THE biggest use or Ruby is within the Rails framework, where it is used to provide the programming power behind web based applications that can be quickly and efficiently written.

Learning to program in Perl / Perl Programming
Perl is a very widely used programming language indeed, but it's name isn't well know outside the programming community as it's an open source language that lack the marketing "hype" - it's just a superb technical tool in a very wide variety of applications. It's name - PERL - stands for the "Practical Extraction and Reporting language" which is a pretty good sound-bite to tell you what it does.

A new release of Perl - Perl 6 - has been undergoing development for a considerable time. Our course covers Perl 5, and will continue to do so while it is dominant, but also includes a short look ahead to Perl 6 so that new code developers will be able to take advantage of it once it's available to them.

Using Perl on the Web
When internet applications took off, Perl was especially suitable as the language in which they should be written. It interfaces well to web applications and servers, has a quick development cycle, and it's open source so that are no complex licensing issues. Many web applications are written in Perl to this day, although languages such as PHP (which was designed specifically for the web use) and Ruby are strong and often used alternatives.

Perl for Larger Projects
Perl is a scripting language which allows knowledgeable staff to achieve a very great deal in areas as diverse as data analysis, system management, and web interaction very quickly indeed. For larger and more complex requirements, Perl includes an object oriented interface which is the subject of this course, which also covers other topics associated with handling significantly large amounts of data, such as code efficiency and data that overflows the space available during processing.

Tcl Programming
Tcl is a programming language that's used in a wide variety of niche applications - ranging from Numerical Control Machines to Integrated Cirsuit design. It's also very strong as an Engineer's language in system testing, where the with the additional Expect extension provides a tool that would be hard to better.

Tcl - the Tk Toolkit
The TK Gui (Graphic User Interface) provides an extension to the Tcl language which allows the programmer to construct an application that the user can easily control through buttons, sliders and other graphic elements. Programmers need a thorough understanding of Tcl before they learn the extras of Tk.

Regular Expressions
Regular Expressions provide a powerful tool for matching strings - for example, if you wanted to look at some text in your program and work out whether it was in the correct format for a postcode, you would use a regular expression. But ... regular expressions are so terse and yet flexible that they're very hard for the newcomer to self-teach, which is why we offer this one day course.

Deploying Java applications on Linux and Unix
For each Web Server you run, you'll have hundreds of thousands of visitors browsing your site. The software that these visitors browse through is kept simple - but that's at the cost of a complex array of technologies on the Web Server. If you are responsible for installing or looking after such a web server - running Linux, Apache httpd, Apache Tomcat and perhaps MySQL or another database to handle applications written as Java Servlets or JSPs, using HTML, CSS and perhaps CGI too, this is the course for you - it takes you through the steps of deploying and hen looking after the various technologies.

Deploying LAMP - Linux, Apache, MySQL Perl / PHP / Python
More Internet domains are run using LAMP (Linux, Apache, MySQL, PHP) servers that any other technology. And if you have staff who are responsible for looking after these servers, or helping configure them, then this course provides an ideal knowledge level without turning them into knowledgeable programmers. It's ideal if they're going to be installing software written by others - for example using some of the wide range of Open Source applications which are available.

Linux Basics
Linux is the operating system that's now used on the majority of web servers. Even if your organisation uses Microsoft Windows at the Desktop, you may find Linux behind the scenes.

Linux Administration
The Linux operating system is commonly used on Web servers - and many other server too - to provide the background services on that computer. We offer a Linux basics day for staff who are going to be involved in uploading and configuring web applications, a Linux Admin day for staff who are going to take some responsibility for server configuration and maintainence too, and a selection of courses on installation and administration of the web server software. This course covers the administration necessary to look after a web server within an existing network, but does not cover general Linux administration in great depth - please ask us if you need training which includes subject such as name services, network configuration, firewall setup, mail servers, etc.

Linux Web Server
The Apache httpd web server, installed on Linux, is perhaps the most often used web server / operating system combination in the world. This course covers the installation, configuration and maintainance of an Apache httpd web server within the Linux environment. Regular tasks include analyzing server usage, taking backups, configuring extra domains, and dealing with security issues and management reports.

Java Bootcamp
The Java language is now firmly established in a wide variety of applications, with an incredible range of facilities and options available to go with it. One of the most common uses of Java programs is on a web server, where "Servelets" or "JSP"s provide the logic behind web based applications from banking to weather forecasting, and from government sites through scientific research.


Programming in C
The C language is the bedrock of modern computing - it underlies your operating system, and it's the language in which other languages are written. However, only a small proportion of programmers actually need to develop and maintain code in C these days, as for the majority of applications the higher level languages that are themselves written in C are going to be more efficient.

C++ for C Programmers
The C++ programming language adds syntax to the C language on which it is based, allowing C to be easily and efficiently expanded into much larger applications. Because C++ maintains C compatibility, the resultant language has some quite complex features, making C++ courses especially valuable in getting staff new to the language up to speed.

C and C++ Programming
C++ is a programming language used for major systems, based on the fundamental C language but with additional features such as Object Orientation which make it very suitable for bigger uses. To some extent, C++ has been displaced by other more modern languages, but there's still a number of application areas where it's the ideal choice, and many others where heritage code needs to be maintained.

Posted by gje at 09:19 PM | Comments (0)

Related topics: via article database

October 27, 2007

The little gestures that can really count

I admired the responsiveness of the canteen in the Cambridge office I was in this week just gone. I was running two courses thought the week, and the second group seems to be heavier drinkers (of coffee!) than the first - indeed, I popped into the canteen on Thursday morning and asked for a topup of our supplies.

Good on them - I note we have an extra urn of coffee on Friday morning - but no extra hot water as that did not run out. That's a quiet responsiveness that I admire, and know that we try to emulate. And I know we have an excellent team that does emulate it too.

If you've been reading this Blog you'll have seen some trials and tribulation I've found in Cambridge - compounded last night by a dreadful session on the London Orbital Car Park (a.k.a. M25). But it's little things like that canteen, and the twenty-plus lovely people I trained this week, that "Make business a pleasure." Now where have I heard that tagline before.

Posted by gje at 07:15 AM | Comments (0)

Related topics: via article database

Sparse and Greedy matching - Tcl 8.4

Problem Analysis Once you have done a sparse or greedy count, all your following counts will also be sparse or greedy - this is a well documented bug in Tcl 8.4!

Reminder ...
.* is a greedy match - any character and AS MANY AS POSSIBLE
.*? is a sparse match - any character but AS FEW AS POSSIBLE

set demo abc@def@ghi@jkl
#
# Correctly working greedy matches
regexp (.*)@(.*) $demo all first second
puts "$first and $second"
#
# Correctly working sparse matches
regexp (.*?)@(.*?) $demo all first second
puts "$first and $second"
#
# Both matches will be SPARSE here
regexp (.*?)@(.*) $demo all first second
puts "$first and $second"
#
# Both matches will be GREEDY here
regexp (.*)@(.*?) $demo all first second
puts "$first and $second"

Running that, we get:

earth-wind-and-fire:~/oct07/camb grahamellis$ tclsh spag
abc@def@ghi and jkl
abc and
abc and
abc@def@ghi and jkl
earth-wind-and-fire:~/oct07/camb grahamellis$

Suggested Work-around

You can emulate a sparse match by excluding the delimiter character from he group that you would be sparse matching, and using a greedy match.

Thus
regexp (.*?)@(.*) $demo all first second
can be rewritten as
regexp {([^@]*)@(.*)} $demo all first second
if you really needed sparse followed by greedy!

result:
abc and def@ghi@jkl

Posted by gje at 07:09 AM | Comments (0)

Related topics: via article database

Useful link: Tcl training

Buffering of inputs to expect, and match order

If you're using the expect command to wait for one of a series of inputs within your expect program, information will be checked in the program's internal buffers inthe following order:
a) Any string that matches expect_before will be found
b) Matches to each of the possible patterns in expect in turn
c) Matches to expect_after.

Note that you can't be certain how much information will have been written to the internal buffer up to any point, so you need to exercise care when specifying the possible sequences.

Once expect_before expect_after or expect matches as string, the section up to that string is transferred to the expect_out(buffer) variable that you can reference in your continuing Tcl, and that section is removed from the internal bugger. The following expect command will start looking from where the previous successful match was found.

Note that the -notransfer option to expect will prevent the matched section from being removed from the internal buffer.

The expect_background command allows you to specifiy an expect command that runs, in effect, in an asyncronous background loop. You can use it to receive inputs from processes that may be responding from time to time while you're doing other processing in the foreground, without having to go into a wait loop until one of those background processes responds.

Posted by gje at 07:07 AM | Comments (0)

Related topics: via article database

Tcl / regsub - changing a string and using interesting bits

Regexp matches a string to a regular expression, and regsub goes one further in that it replaces the found string with something else, saving the transformed output into a new variable. But what if I want the output pattern to include part of the string that was matched? I can refer to the "interesting bits" using \1, \2, etc, if I wish.

Example:

# In Tcl,
# regub needle haystack newneedle saveithere
#
# Refer to matches in the newneedle via \1
# Example - to surround dogs, cats and rabbits with !s
#
regsub -all (cat|dog|rabbit) "The cat and dog and budgie" \
  {!\1!} result
puts $result

Results:

earth-wind-and-fire:~/oct07/camb grahamellis$ tclsh rsd
The !cat! and !dog! and budgie
earth-wind-and-fire:~/oct07/camb grahamellis$

Posted by gje at 07:05 AM | Comments (0)

Related topics: via article database

Useful link: Tcl training

October 26, 2007

What is Expect?

Expect is an extension to the Tcl language which provides three key new commands - spawn which lets you start up another process, send which lets you send information to that process and expect through which you can receive back responses from that process.

For the majority of users of Tcl, Expect is not relevant in their day to day use of the language. But for a significant minority, it is vital and indeed the whole reason that they're using Tcl in the first place.

Here's an example of an Expect session which connects to a remote host computer via ssh, grabs some data from that remote system, extracts vital reporting information from that system and gives a qucik, clear and easy report.

puts "This is a bit o' Tickle"
log_user 0
 
spawn ssh -l trainee 192.168.200.67
 
expect "sword: "
send "abc123\r"
expect {$ }
send "df -k\r"
expect {$ }
set igot $expect_out(buffer)
send "exit\r"
 
regexp {(\d+)%} $igot all numbah
 
puts "First reported disk slice is $numbah/100 full"
 
puts "That were easy, weren't it?"

Result is:

earth-wind-and-fire:~/oct07/camb grahamellis$ expect se1
This is a bit o' Tickle
First reported disk slice is 97/100 full
That were easy, weren't it?
earth-wind-and-fire:~/oct07/camb grahamellis$

Expect is used in test environments, in integrated cirsuit design, in general systems admin tasks ... and many more places. Anywhere that you want to automate command line actions. And you do ned a good understanding of Tcl before you can use it - we cover both Tcl and the handful of extra Expect commands on our public Tcl Programming course, and on private courses we can tailor the presentation to inlcude much more expect coverage.

We use Expect on our own web site ... have a look at our Server Status page and you'll get a report from various web servers where we host domains - it's genarated by an expect scrips which uses the ping utility to test all the hosts in parallel.

Note - the example above is a training example - a spike solution. A live piece of code would include various other error checking operations, handling situations where the remote connection failed, for example.

Posted by gje at 11:42 PM | Comments (0)

Related topics: via article database

Wireless hotel tips - FTP and Skype connections failing

I'm on line this week through a Spectrum interactive account, who's web site tells me that they have won a contract to supply BAA at Heathrow Terminal 5 and that they have an exclusive 7 year deal at Travelodge, where I have been staying. It's not all been a bed of roses - quite apart from a major loss of service which I've briefly comment on elsewhere, I've had a couple of other issues that I thought I would comment on as clearly this is a supplier who intends to be around in a big way, and doesn't appear to be providing a transparent connection.

a) FTP. Upload connections are being lost at the start of transfers; the solutions I have been using is to use scp instead. Thus

[within ftp]
puts xz.tar

has been replaced by
[at command line]
scp xz.tar whc@www.wellho.net:/home/whc/httpdocs/pix/xz.tar

b) Skype - the Skype port appears to be blocked. Perhaps SpectrumInteractive have an arrangement with one of the telecoms companies that provides phone service to some Travelodge locations to do this - but here at Swavesey there are no phones in the rooms and even the public phone has been removed. I guess most of us have a mobile these days ....

c) I have two laptops here with me. On the good side, I'm able to log either of them in as it appears that my signup hasn't been restricted to a single specific mac address as I find with many other Wifi service suppliers. Slightly less positive (but to be expected, I suppose, for security reasons) I can't use the hotel's wireless hub to communicate between my 2 computers even though they're both on the same subnet. Solution - software that lets me computer to computer network, in effect setting up one of the machines as a wireless base station itself!

Posted by gje at 06:18 AM | Comments (0)

Related topics: via article database

Reading from another process in Tcl (pipes and sockets)

If you want to read from another process on the same system that you're running your code on in Tcl can be done using a pipe - you can use open with the command name instead of the file name, preceeded by a pipe character (|) - thus:

set said [open |df r]
 
while {[gets $said this] >= 0} {
  puts $this
}

You can also talk to a process on another system if it's running as a server using sockets. In Tcl all you need to do in the simplest case is to use a socket command instead of an open, and you can use puts and gets on the resource that's returned. Here's an example of a Tcl script that connects to a web server and downloads a page of html:

set talkto [socket 192.168.200.67 80]
 
puts $talkto "GET /index.html HTTP/1.0\n"
flush $talkto
set lines 0
 
while {[gets $talkto stuffing] > -1} {
  incr lines
  puts $stuffing
}
puts $lines

Posted by gje at 05:48 AM | Comments (0)

Related topics: via article database

Useful link: Tcl training

October 25, 2007

Away or home - which do I prefer?

The hotel is spartan and the staff unfriendly. The Wifi didn't work and the support company was informative about the problems, but passing the buck on to others and not able to actually fix the issues for many hours. The restaurant had unset / unwiped tables all over, with food smeared on the menus. To reach the service area I can see from my room is a two mile drive down and back up the dual carriageway. And when you get there and order their advertised and pictured breakfast special, the cup of coffee that comes with the product is only a third of the size of the one photographed. The rude woman tells you she's served you right - that you should read the word "regular" and ignore the picture.

At lunch today, one of my delegates asked me if I prefer training in our own centre, or travelling the world and seeing places.

Actually, I enjoy both. Some away trips don't have these trials and tribulations ... and traffic jams and emergency fire drills and directions to the wrong building ... that I have also had this week. Where there's an opportunity to see somewhere interesting in the evening, enjoy some well presented services, and take in a different culture it can clearly be fun for a few days. And it can also be instructive to us as to what we should be doing on the hotel side of the business.

Am I enjoying this trip? "Quiet amusement" I think I have to say. Yes - I enjoy seeing how things should not be. I can take a certain smug delight in realising that if we're even just mediocre, we win hands down over so many others. But I can also learn a series of warnings and look carefully to make sure that we're clear on those points. And at the end of the day (I hate that phrase's overuse but it's the right one here!) I've got a bed to sleep in that's going to let me get ready for the following day's Tcl course, and at a price that's not so high that I resent the poor value.

Posted by gje at 06:46 PM | Comments (0)

Related topics: via article database

October 24, 2007

Sorting in Tcl - lists and arrays

Tcl's lsort command lets you sort a list - and that can be a list of the keys of an array. You can't sort the array, but once you have the list of keys you can sort that and use it to iterate through the array in any order that you like.

As everything (except an array) is a string in Tcl, when you sort a list the language defaults to an asciibetic sort - which you can vary with the -integer or -real options to lsort. And if you want to specify another criterion, you can do that too, using the -command option to specify the name of a proc that returns -1, 0 or +1 to indicate the ordering of two input records.

There's examples of all three following - I'm sorting a list of keys from an array asciibetically, numerically, and by the value that the key points to:

proc report {order say} {
  puts $say
  global stop
  foreach pl $order {
    puts "$pl: $stop($pl)"
  }
}
#
proc byp {a b} {
  global stop
  return [string compare $stop($a) $stop($b)]
}
#
set stop(1) Bath
set stop(2) Bristol
set stop(6) Keynsham
set stop(3) Easterton
set stop(4) unused
set stop(5) unused
set stop(10) London
set stop(11) "The West"
#
set bystop [lsort [array names stop]]
set properbystop [lsort -integer [array names stop]]
set byplace [lsort -command byp [array names stop]]
#
report $bystop "By Ascii stop"
report $properbystop "By numeric stop"
report $byplace "By place name"

Here are the results:

earth-wind-and-fire:~/oct07/camb grahamellis$ tclsh allsorts
By Ascii stop
1: Bath
10: London
11: The West
2: Bristol
3: Easterton
4: unused
5: unused
6: Keynsham
By numeric stop
1: Bath
2: Bristol
3: Easterton
4: unused
5: unused
6: Keynsham
10: London
11: The West
By place name
1: Bath
2: Bristol
3: Easterton
6: Keynsham
10: London
11: The West
4: unused
5: unused
earth-wind-and-fire:~/oct07/camb grahamellis$

Posted by gje at 08:05 PM | Comments (0)

Related topics: via article database

Useful link: Tcl training

Tcl - global, upvar and uplevel.

In Tcl, variables are "scoped" to the procedure in which they're used. That's a sensible default, since you want to be able to write main applications without the risk of interfering with variables within your procs.

A global command in a proc allows you to refer to a variable in the main code from within a proc. (No need to declare it global at the top level too)

An upvar command allows you to reference a variable in your calling code from within your proc, and is commonly used for "pass by name" requirements where you're writing a proc that must alter an incoming variable.

The uplevel command allows you to run a piece of code as if it was in the namespace of your calling code and not in your proc - and you can even specify a number of levels of code up or an absolute level number.

Suggestion. Use upvar whenever you need to call by name. Use global sparingly. And think very carefully before you ever use uplevel!

Example:

proc demo {what where} {
global counter
upvar $where hereitis
uplevel incr dc
append hereitis ", UK"
incr counter
puts "Demo run"
return "$what $hereitis"
}
#
set dc 0
set counter 0
set place Melksham
set rz [demo "eat in" place]
puts "function returns: $rz"
puts "place will have UK on the end: $place"
puts "counter is number of calls to demo: $counter"
puts "variable dc has become: $dc"

Runs as

Dorothy:~/oct07 grahamellis$ tclsh uld
Demo run
function returns: eat in Melksham, UK
place will have UK on the end: Melksham, UK
counter is number of calls to demo: 1
variable dc has become: 1
Dorothy:~/oct07 grahamellis$

Posted by gje at 08:16 AM | Comments (0)

Related topics: via article database

Useful link: Tcl training

October 23, 2007

Square Bracket protection in Tcl

I was writing some string match examples in Tcl yesterday - and had the need to explicitly match square brackets within my string. But I couldn't just write the square brackets into the match string ...

• The square bracket first needed protection from the Tcl parser before it even got to the string match command.

• Then it needed protection within the string match command to prevent it being interpreted as the start of a range.

In both cases, a \ character can be used to provide protection - that's 2 \ characters that I need - but then the \ that protects the [ in the string match itself needs protecting from the command line interpreter, making a total of three protective characters needed. Thus:

  if {[string match *\\\[* $saying]} {
    puts WOW!
  }

"Does it have to be that hard?" you ask. No - you can also use { and } to defer the string match pattern, so that it's not interpreted by the command line, thus reducing the number of backslashes needed back down to just the one:

  if {[string match {*\[*} $saying]} {
    puts WOW!
  }

Here's a longer sample program also showing the use of { and ] in a string match to indicate any one character from a range:

foreach saying {"The [world] is round"
    "pie in the sky" "Horse hooves"} {
  puts $saying
  if {[string match *\[ys\] $saying]} {
    puts Y
  }
  if {[string match {*[ys]} $saying]} {
    puts Y
  }
  if {[string match *\\\[* $saying]} {
    puts WOW!
  }
  if {[string match {*\[*} $saying]} {
    puts WOW!
  }
  }

and here is how that runs:

Dorothy:~/csr1 grahamellis$ tclsh ll2
The [world] is round
WOW!
WOW!
pie in the sky
Y
Y
Horse hooves
Y
Y
Dorothy:~/csr1 grahamellis$

Posted by gje at 06:23 AM | Comments (0)

Related topics: via article database

Useful link: Tcl training

Tcl - append v lappend v concat

Should you use append, lappend or even concat to add to variable in Tcl?

append puts one string directly on the end of another, without adding any extra characters beyond those in the incoming variables.

lappend treats the target string as a list, and will usually add an extra space on the end of it prior to appending the new item so that the result is also a list with ONE new item added ... the added item being protected using additional characters such as { and } to turn it into a list element if necessary.

concat (and note the different syntax!) takes two LISTS and joins them together - it differs from lappend in that lappend takes the second element as a single item and not as a list. Concat is something that many occasional Tcl users seem to forget about ... and then waste time writing a loop of lappends!

Example - source code

set people "John Joan Jane"
set gatecrashers "Bill Ben Boris"
#
# append adds to string, lappend adds to list
puts $people
append people t
lappend people Julian
puts $people
#
# concat connects two lists, lappend adds one item to a list
set everyone [concat $people $gatecrashers]
puts $everyone
lappend people $gatecrashers
puts $people

In operation:

Dorothy:~/csr1 grahamellis$ tclsh alc
John Joan Jane
((append will add "t" to Jane, lappend adds new item Julian))
John Joan Janet Julian
((concat adds a whole new list))
John Joan Janet Julian Bill Ben Boris
((but lappend adds a single item to the list))
John Joan Janet Julian {Bill Ben Boris}
Dorothy:~/csr1 grahamellis$

Posted by gje at 06:03 AM | Comments (0)

Related topics: via article database

Useful link: Tcl training

Tcl - using [] or {} for conditions in an if (and while)

In Tcl, the use of square brackets tells the interpreter to "do this first" whereas curly braces are a request to defer execution of a block - perhaps suppressing it completely as in the action on an if statement.

So the if statement's action will NEVER be written in square brackets - [ ] - as that would cause it to be performed irrespective of the actual condition and would defeat the whole purpose of the condition! But it's a different matter with the condition itself. Have a look at this code:

set cost [gets stdin]
 
if [expr $cost > 20] {
    set gpt [expr $cost * 0.9]
    puts "Group ticket at $gpt"
 }
if {$cost > 20} {
    set gpt [expr $cost * 0.9]
    puts "Group ticket at $gpt"
 }
if {[expr $cost > 20]} {
    set gpt [expr $cost * 0.9]
    puts "Group ticket at $gpt"
 }

All three statements produce the same end result, but they do it differently.

The first - using [ ] - evaluates the expression before it ever runs the if comment, to which is passed just a true or false value and not the condition itself. This works well for an if but should never be used in a while, since the condition would never be rechecked and you would have an infinite loop.

The second - using { } - passes the string containing into if which assumes an expr command is to be used to evaluate it. You would usually want it to be expr-ed, so most of the time thats a good and fair assumption.

Using { } and the [ ] with it gives you the deferred block that you'll need when giving a condition for the while, but then also gives you the ability to embed any command - not JUST an expr - with it. And this is the structure you'll need to use for more complex while loops.

Posted by gje at 05:52 AM | Comments (0)

Related topics: via article database

Useful link: Tcl training

October 22, 2007

Dark Dawn

It's that time of year once again when an early start means scraping ice off the car, and a start in the dark and perhaps the fog. But living at 51 degrees north gives us the variety of seasons and the dampness that causes the fog also causes the whole country to be so green, as you have seen in yesterday's post - and I love it.

This entry is being "pre-written" or "postdated" as I'm not really inclined to write before I leave before 05:00 for my drive up to Cambridge to present a Tcl Course there - or rather two courses of 2.5 days each. And I know that by the time I get to the Travel Lodge at the end of the day, I'll be cream crackered and in no state to write.

Posted by gje at 04:16 AM | Comments (0)

Related topics: via article database

October 21, 2007

Pictures Framed

A comment the other day reminded me just how powerful picture frames are - as shown in these pictures of Wiltshire, all offset by the photographic composition. Select individual images to enlarge them!

Posted by gje at 01:11 PM | Comments (0)

Related topics: via article database

Someone else's wedding

I haven't a clue who the happy couple are ... but we came across the wedding party when out yesterday - at the Old Mill, Harnham, Salisbury.

A beautiful sunny day, and it was one of those occasions where a couple who are, perhaps, the most private of individuals normally are more than happy to be photographed by all and sundry. The setting was idyllic, and I hope that they - whoever they are - have a long and happy time together.

Further pictures here

Posted by gje at 11:26 AM | Comments (0)

Related topics: via article database

October 20, 2007

Perl - progress bar, supressing ^C and coping with huge data flows

If you're handling a huge amount of data (gigabytes!) in a Perl program, memory won't allow you to slurp it all into a list and you'll traverse the data with a loop from file or from database. And because of the sheer volume of data, it may take a while to process. During such proessing, you may wish to:
• See a progress bar on the screen
• Be able to ask for intermediate results
• Disable ^C (Control-C) interrupts to avoid accidental termination

A progress bar can be printed out as text, using \r rather than \n at the end of each line. If you do that, though, you need to turn autoflush mode on so that yo actually see the reports rather than having them buffered, and you should use a formatted print that always generates the same length line to avoid "droppings" being left on the end when a short line overwrites a long line. If you're likely to send your output to a file, the progress bar should go to STDERR.

You can replace your ^C handler by changing the signal handler in $SIG{INT} to call a sub of your choice. As such a sub can be called at any stage in your loop it's good practise to simply note that ^C has been entered by setting a flag, and then checking at the end of each loop iteration whether or not there's a need to report. And don't forget to turn off that need to report once you have done the output - otherwise you'll report at EVERY iteratin thereafter!

Having used ^C to generate a report rather than kill your process, how can you actually kill it? There are other ways via the OS, but better to allow the user to enter ^C twice in succession, and on the second entry in (say) 3 seconds give an interactive "Quit?" option.

Net result of applying these techniques? A program that you can really control, and a great deal of time saved as you no longer have to wait for it to run to completion every time. Psychologically much better too - you know how it's getting on rather than having to wait and see.

On yesterdays's Perl for Larger Projects course, I wrote a complete example showing almost all of the above in a single program that analyses a massive web log file. I've commented it, so if you're already something of a Perl expert or have access to our Handling Huge Data course notes it wll already make sense to you. Source code Link. If this is something you need to learn, we would welcome you on the course

Here is some sample output to complete the example.

[trainee@campion huge]$ perl hugehunter
we have done 160000 lines ( 10.60%)
^C
Intermediate report
6 crawl-66-249-73-81.googlebot.com
1 crawler4067.ask.com
2 host86-148-48-188.range86-148.btcentralplus.com
1 livebot-65-55-209-97.search.live.com
1 lj511086.crawl.yahoo.net
1 lj511737.crawl.yahoo.net
2 lj511971.crawl.yahoo.net
1 lj512128.crawl.yahoo.net
1 mail.bezanilla.cl
[etc]
Quit? y
[trainee@campion huge]$

Note - example used logs to STDOUT not STDERR. You'll need to select STDERR before you set autoflush if you switch it over as recommended.

Posted by gje at 08:08 AM | Comments (0)

Related topics: via article database

Useful link: Perl training

October 19, 2007

Using PHP to upload images / Store on MySQL database - security questions

Yesterday, the page on our website that shows you how to upload an image from a browser in a PHP script, store the image in a database, and later on retreive and redisplay the image, was visited 250 times. Not bad for such a specific subjec t on what I'll admit is something of an obscure web site.

One of our 250 visitors wrote to me - an excellent question - concerning security issues with such uploaded images - "How do you secure such uploads? I mean, i changed the extension of a .exe file as .png and uploaded without any issues. I do realize that the image will not be displayed. But in theory the file is still located in the filesystem. Wouldn't this mean someone can upload a trojan or infected file and mess up the entire server?"

He's 100% right to think carefully about the validity / checking of what can be uploaded through scripts such as this. Yes, they can be used for ANY "blob" of data - any bit patterns at all, including all sorts of nasties. That's not just limited to trojans and infected files as you suggest, but also includes copyrighted images, pornographic pictures, and images of text which makes illegal statements / claims. And I'm sure you could find other things too.

But if the data is held in the database AND ONLY RECALLED / USED AS AN IMAGE, then it is not messing up the whole server - it's not being run in any way. If the .jpg standard allowed for the embedded execution of code that it would be a problem as you suggest, but to my knowledge that's not going to be the case. If my server script that recalled the file said "if it looks like an executable, go run it" that would be a problem ... but it does not. All it does is sends out the image, labelled as a .jpg, to a browser.

The uploading of images that should not be uploaded (copyright, etc,) is a more difficult matter to answer. And it's really the same question that anyone who has a web presence onto which public data uploads is accepted has to be aware of and think through. In the case of our demonstration, we have many people trying out the scripts and so any image which really should not be there quickly sinks into the obscure depths and is no longer accessible, so the system is in effect self-cleansing. For a live web site where the purpose is publication, you need to look at other measures such as having moderator presence, an acceptable user policy, a way of reporting abuse, perhaps an automated spam trap, and more.

More links:
Try out the upload script
View recent uploads
Source code listing

Posted by gje at 08:03 AM | Comments (0)

Related topics: via article database

Useful links: PHP training, MySQL training

October 18, 2007

Dont just convert to Perl - re-engineer!

There's a difference between converting a program from one language to another and re-engineering a program to make best use of a new language - and I had a couple of clear examples of that on our Perl for Larger Projects course today.

The following is "converted" code ...

if (! open (FH,"../Desktop/access_log.xyz")) {
print STDERR "We have a weeny problem there!\n";
exit (1);
}

Or "re-engineered" you can write ...

open (FH,"../Desktop/access_log.xyz")
   or die "We have a weeny problem there!\n";

The following is "converted" code again ...

while ($line = <FH>) {
   if (index($line,"cliad") > -1) {
      print "$line";
   }
}

which is much better written as

@info = <FH>;
@matched = grep(/cliad/,@info);
print @matched;

or perhaps even

print (grep(/cliad/,<FH>));

Why re-engineer?
• Performance
• Maintainability
• Readability
• Extendability

Posted by gje at 11:37 PM | Comments (0)

Related topics: via article database

Useful link: Perl training

October 17, 2007

Business to Business Networking - North and West Wilts / Melksham

We held our third Speed Networking event of the year at Well House Manor yesterday evening ... and our third great success. 20 businesses books ... with everyone to spend five minutes with each other business attending, with a break for a buffet half way through an exhilarating evening. Everyone left saying what a great time they had had, what good contacts had been made, and asking us to let them know when we start taking bookings for 12th March 2008 - our next event.

On the Well House side, the evening was organised and run by Christine and Martin, Chris and Lisa ... and run so well that I was able to sit back, confident that all was as it should be in the networking room, the dining room, and below stairs too.

Posted by gje at 07:14 PM | Comments (0)

Related topics: via article database

October 16, 2007

First Alternative / what has happened there?

There are certain life-changing events that happen, and they stick in the memory for years afterwards. And one such is when I parted company with First Alternative, for whom I had been exclusively training for the best part of 10 years - initially as an employee and then as a contractor. This was an arrangement which suited them because they could cancel the courses at short notice and not have to pay me, and it suited me because it allowed me to develop Well House Consultants materials and sell the places on courses at First Alternative which (earlier) were sold through their catalogue exclusively. I have not written about this in the past; far more important to get on than look back ... but an email that asked me about it has lead me to write as follows.

I presented a week long Perl course in that August [2000], and Lisa joined me for an evening meeting with Mick, where we were to discuss the program for 2001. We had prepared a steady growth plan, and were fully prepared to present our initial proposal under which both Mick Hosegood / First Alternative AND ourselves would benefit dramatically, taking advantage of all the building up work we had been doing on the Perl and Java courses.

Lisa and I were taken aback - shocked - when Mick asked us about one of our own contacts and suggested that we had accepted a direct order from that contact as a result of stealing the sales lead from him. We were accused of taking a copy of his database too. Pointing out that we had taken an order from a different company to the one that the sales lead in question referred to, based in a totally different city, didn't wash. Pointing out that people shop around and they had found us independently didn't wash. "I don't want to hear anything. You have just completed the last course you will ever give for me. Please give me your key and leave now" ....

I would say that we denied stealing a copy of the database (for that is in effect what we were accused of), but we didn't. We weren't given a chance to say anything - we had been tried and found guilty before we even walked in to that meeting. "Kangaroo Court" we felt!

But they do say that revenge is sweet, don't they? I'll admit to taking pleasure in hearing, a year or two later, of a customer who turned up for a course one Monday morning shortly thereafter, having traveled a long distance the night before ... to be met by one startled and embarrassed secretary who was the sole occupant on the training centre. The course wasn't running - no tutor - but so certain was the proprietor that I had stolen the database that he hadn't bothered to contact people and cancel bookings!

It turned out that First Alternative had done us a huge favour in the medium term. We had asked ourselves what we would do if something went belly-up there and had some vague ideas ... Lisa and I drove off to find a quiet table nearby and discuss how we would develop "Plan B". Residual nastynesses (such as a struggle to get payment for courses that I had already presented) served as an early reminder of how much better we were going to be on our own, and we had sufficient resources to handle the initial cash flow issues with setting up our own training centre for those issues to be more of an irritant than anything else.

And look where we are now. We've grown and blossomed - from a single training room that was part of our home, up to our own training centre with luxury hotel accommodation as well as our offices / HQ. And course have grown from Perl and a Java course to include Linux, MySQL, PHP, Python, Tcl/Tk, Ruby and more. Start here to browse the full details.

From a business viewpoint, I do understand that Mick was worried that a growing percentage of his business could be lost at the whim of a single contractor. Also that he would do better to control when the breakup happened if he felt it was likely, and do it in such a way that it would be a quick break. I'm told that business went very quiet there, and that declared his staff redundant just before Christmas the following year, closing the centre completely; he now hires a room nearby if he wants to run a course.

We have tracked First Alternative over the years since we parted, more on an emotional basis than anything else - with their "First Alternative" (alternative to Sun Microsystems) Solaris and FrameMaker training courses, and a host of other tryouts, they were never in the same market as we were.

Posted by gje at 11:22 AM | Comments (0)

Related topics: via article database

October 15, 2007

Autumnal lighting on a London trip

I was in London all day - hence this late-posted entry, and a brief one at that. Sharing a trio of photos taken during the day.

Posted by gje at 11:23 PM | Comments (0)

Related topics: via article database

October 14, 2007

Ordnance Survey Grid Reference to Latitude / Longitude

I've already written about conversion of postal addresses to latitude and longitude using Google's interface - but what is I want to convert the Ordinance Survey's grid references into latitude and longitude? Google's interface doesn't support the OS units - and in any case it's "just" a a calculation. I have put quotes around the word "just" because it's actually a complex calculation that I had no desire to re-code from learned papers.

Of course ... it's been done already. Perl's Geography::NationalGrid module which you can download from the CPAN. Now I wanted to use that in a PHP page to demonstrate to you, so I've written a quick Perl command line program that my PHP calls up when it needs to and .... bob's your uncle.

Try it out here (examples ST906637 is in Melksham ... SJ347904 is in Liverpool) :

Enter a grid reference:

The source code is also available for you:
The Perl command line program
The PHP which calls up that Perl

External links -
Descritpion of coordinate system and conversion algorithms (.pdf)
Perl Cpan module

Update = August 2008 I've also added code for the OPPOSITE mapping - Latitude and Longitude to OS Grid reference.

• Demonstration - here
• Source code of PHP page - here
• Source code of Perl program - here

Posted by gje at 07:39 AM | Comments (0)

Related topics: via article database

October 13, 2007

Converting from postal address to latitude / longitude

Do you sometimes find odd gaps in new procedures you follow - "I can see everything I need to do to make X happen, except" ?

I hit one of those yesterday. When learning how to interface to Google maps ... all the information was there, except there was no way that I could spot to click on a map and say "what latitude / longitude is this?". Of course, I may well have missed an option.

Why the problem? Because I wanted to hard code certain data - such as our location - into the demo I was writing.

Alternative approach - to type in a place name and have the system tell me the latitude / longitude which I can do .. not via any standard page I could find but vis Google's own API.

Source code - here

Working demo - here

Posted by gje at 07:36 AM | Comments (0)

Related topics: via article database

Controlling and labelling Google maps via PHP

"Can you use PHP to control what you show on a Google Map?" An innocent question that lead me off on an interesting exploration when I should (!) have been doing other things - but a very worthwhile couple of hours, and great fun!

How do you do it?

1. Apply for your Google Maps key (you'll need to set up a Google account first if you don't have one)

2. Load their sample Javascript into the head of your web page, and modify it to relect your scale and location.

3. Add in a <div id="map" ... element to your web page

Probably a good idea to test at this point!

4. Add in a form for the controls you want, a handler for form inputs, and modify the Javascript to drop in the contents of PHP variables where necessary.

And Bob's your uncle!

I have an example in use here.
And the source code is here.

Is there much else I could do? Absolutely! I've just pushed the door open a crack and I see a blinding light shining out.

Posted by gje at 12:23 AM | Comments (0)

Related topics: via article database

Useful link: PHP training

October 12, 2007

Well House Manor - a year on

It's exactly a year since we welcomed the first delegate on a training course at Well House Manor ... and we've come a long way since. As well as courses this week, we've had a hotel lively with guests, and we've hosted a number of business events too.

We couldn't have done it without "the team" - Leah and Martin, Christine and Nigel, joined recently by Chris who's done a lot in a little time. And I couldn't have done it without Lisa who has put in some incredible love and work. But most of all ... we would be nowhere without you - our friends and customers.

Posted by gje at 11:28 AM | Comments (0)

Related topics: via article database

October 11, 2007

Error logging to file not browser in PHP

When you first install PHP on your web server, any errors and warnings in you code are flagged up in the browser window. And that's ideal for development purposes - any problems are quickly indicated to you, with pointers to where the issue lies.

But when you go live, there's a different story, Of course, there shouldn't be any problems remaining ... but ... if there are, then you'll not want your user to be alerted to the detail, including the line of code in which it occurred.

You can change the error behaviour of PHP through configuration switches in your php.ini file .... turning the display of errors off, and turning logging to file on. These two lines:

display_errors = On
log_errors = Off

being replaced by these three:

display_errors = Off
log_errors = On
error_log = /home/octoweb/oops.txt

Posted by gje at 11:43 PM | Comments (0)

Related topics: via article database

Useful link: PHP training

October 10, 2007

New software product for warmblooded programmers

Have you come across the latest software product known as "Girlfriend"?

Girlfriend is a piece of software that was designed to meet requirements of both the leisure and utility market, providing mobile facilities, facilities for home entertainment, and also undertaking a management role under which it takes care of repetitive and monotonous tasks that you want to avoid doing yourself.

A sneak preview of Girlfriend showed off the mobile facilities well. A lovely looking piece of software, well dressed for presentation, with some exciting hints of what might be found inside the package later. Attempts at this early stage to access the home entertainment parts were met with "permission denied" messages.

The test release of Girlfriend performed well in terms of mobile facilities, and the home entertainments module was now operational too. Users found that the basic facilities were quick and easy to use, but that the more unusual and advanced facilities needed some prior research and were sometimes quick tricky to get working. Resource sharing facilities especially tended to be awkward, with some versions of Girlfriend shutting down completely from time to time with no apparent reason. The test release also supported a few of the repetitive tasks, which were done efficiently and well.

Girlfriend 1.0 was the first version which you had to install on your own system and pay a fee for rather than running from a CD / binary download for free. Although extra repetitive management and repetitive tasks were supported, they tended to be less cleanly performed than the few that you saw in the test release. It was also noted that Girlfriend 1.0 installed a whole lot of other pieces of software that were needed to help it run, such as Toothbrush and SinkTidy.

In a piece of software that you'll be using long term, such as Girlfriend 1.0, there are other things that needed to change too. Some of the quick and easy home use facilities became unusable at time when there was a heavy loading on the utility tasks, and some of the mobile facilities starting looking very much less pretty. Some of the advanced features settle down well, but other which had worked before ceased to be usable. And the cost of ownership rose - the space taken grew, as did the resources needed to host the software, and unexpected additional items had to be purchased from time to time. But, overall, Girlfriend 1.0 was an excellent piece of software - far more going "for" it than "against".

For release 2.0, an upgrade was required to provide extra storage facilities for the Daughter package that now came with Girlfriend. Many of the resources of Girlfriend were taken up with servicing the Daughter package, with the mobile and home entertainment modules being severly limited in their capacity. However, the daughter package was a delight even though it was very different indeed to the inital preview that had been seen just a couple of years previously.

A new version of the software is, I understand, to be released in the near future. It will require a full, non-transferable license to run it, it will be limited to a single processor. The marketing folks have decided to rename it "Wife".

The above article is base on an idea from last week's delegates ...

Posted by gje at 12:33 PM | Comments (0)

Related topics: via article database

October 09, 2007

Delegates of all shapes and sizes

It's not normal for me to talk about my course delegates ... and it's never done without permission. But I'm breaking from normality (but not going to NEVERland) today to introduce you to Gary.

Gary checked in on Sunday. And went out for a run before the course on Monday; went out for another run yesterday evening. Eats carefully. Good to see someone who looks after himself. Gary was telling me that he's hoping to run the London Marathon next year, ballot for places allowing.

And then I learned that Gary is only half the man he used to be. Literally.

Gary's web site - telling you all about it (and, yes, offering to sell you his story ;-)) is at www.garypesticcio.com ... and here's his picture now.

Can you tell which one he is?

Posted by gje at 04:56 PM | Comments (0)

Related topics: via article database

October 08, 2007

An email update for past guests and delegates

If you attended a course with us in 2002, or stayed at Well House Manor in the early weeks that out hotel was open, there's a good chance that you'll have fond memories, but that we won't be a part of your every day contact group. How to keep in touch? Well ... email, although much aligned for spam, is a good route and I've spent much of today emailing old friends and contacts - those who filled in reviews and gave us an indication that they welcome such information this way, and those that are still "ongoing cases". If you haven't received an email and are feeling left out ... here is some of the general text that I've been in additional to personalised text too

I'm writing to let current and past customers of Well House Consultants know about some of the exciting things that have been going on, to remind you of our help forums should you have any technical questions you feel I can help with, and to tell you about a couple of special offers that we've got running at the moment. I'll just give you brief introductions to most of the subject and then provide you with links to further details so that you can easily get to the parts of interest.

ONE link to start you off, if you like, to ALL of our news:
  http://www.wellho.net/news/index.html

** WELL HOUSE MANOR - HOTEL **

We opened a hotel and conference centre in our home town of Melksham last year. That may sound like a peculiar move for a high tech training company, but we had so many visitors coming to the town for our courses, all looking for rooms, all looking for specific business / computer facilities, that it was a very natural step. "Well House Manor" has been open for - almost - a year for course delegates and has proven very popular.

Link to the hotel's web site:
  http://www.wellhousemanor.co.uk/
For past delegates, we're offering a weekend special - see
  http://www.wellho.net/news/weekend.html
  http://www.wellho.net/news/christmas.html

** TRAINING COURSES **

I remain personally very busy presenting our public and private training courses - the full schedule of public courses is available now until the end of NEXT year, and I have only got a couple of unbooked weeks before this Christmas. There's good space available on some of the public courses until the year end, and we're offering "first night free" for new bookings if you come on a public course and stay with us. Can I tempt you by ...

* MySQL - 15th and 16th October
* Perl for Larger Projects - 17th to 19th October
* Perl Programming - 29th October to 2nd November
* Python Programming - 12th to 14th November
* Deploying Apache httpd and Tomcat - 15th to 16th November
* Technology for PHP - 26th November
* PHP Programming - 27th to 30th November

There's links to all of these from:
  http://www.wellho.net/news/public.html
and our main / general site will take you to all or our other courses (if you're looking for Tcl or C or Ruby or Linux ...) from
  http://www.wellho.net

** TECHNICAL SUPPORT **

I want to remind you of some of the services that we don't charge for as well ;-)

Our Opentalk forum allows you to ask technical questions that you can't find obviously answered in your course notes and elsewhere.
  http://www.wellho.net/cgi-bin/opentalk/YaBB.pl

Almost every day, we add a technical tip to our web site - you can read them on "The Horses's Mouth" at
  http://www.wellho.net/horse/
and they're categorized every week by subject so that you can find all the resources on a particular topic by 2 or 3 clicks from our resource index:
  http://www.wellho.net/resources/modules.html

Finally ... we've nearly 700 books on our shelves here at "Well House Manor" for you to read during your course, or at other times you're in the area. Most of the books are listed online too:
  http://www.wellho.net/library/index.html

** IN SUMMARY **

We have a lot of news - a lot going on at Well House Consultants. Please continue to use the resources we've described on this page.

If you fancy a weekend away, or you need some further training - please do get in touch.

We've even got a new freephone number - 0800 043 8225 which will reach us at any time (yes, I mean that!) and as ever the team of Lisa, Graham the rest can be reached via email - info@wellho.net, lisa@wellho.net, graham@wellho.net and so on.

Graham

Graham Ellis
Well House Consultants, 404 The Spa, Melksham, Wilts
http://www.wellho.net graham@wellho.net
0800 043 8225 (freephone) or +44 (0) 1225 708225 (phone)
+44 (0) 1225 707126 (fax)

Posted by gje at 11:21 PM | Comments (0)

Related topics: via article database

October 07, 2007

Monitoring mod_jk and how it is load balancing

Apache httpd with mod_jk has the ability to balance the load forwarded to a number of instances of a Java application running in different Tomcat containers, probably on different host c.p.u.s in most circumstances. I covered this a few days ago - see here for details.

But there's a follow up question "how is the loading working out", and the status monitor in mod_jk provides you with a web front end to the stats. There's a sample configuration file here; the captured display looks something like:

Posted by gje at 06:02 PM | Comments (0)

Related topics: via article database

First Great Western - information for customers

With some of the oldest trains in regular public use on the railway network, and a very high premium to pay for their franchise to the government, the First Group doesn't have an easy task with their "FGW" or "First Great Western" franchise under which they operate nearly all the trains in a wedge out from London's paddington station to the South West, the Bristol area, South Wales, and the Cotswolds.

With what is in effect a tax to pay to the government, shareholders, staff, train leasing, rail access charges and many more drains on their finances, there are times that First Great Western (FGW's) primary concern appears to be on its financial results or on keeping going at all, rather than the reason that it's there in the first place - which (in my view) is to serve the traveling public.

Amongst the problems seriously effecting the passenger - the customer - you'll find ... timetable changes which destroy the usage pattern of a line ... shortening of trains that leads to severe overcrowding ... train cancellations that lead to inconvenience journeys ... a complex and at times expensive fare system ... refitted trains that some find less comfortable ... and organisation issues that can make problems hard to resolve. See an introduction to the issues

Yet get to talk quietly over breakfast with some of the directors of FGW (as I did in August), and you'll find that they are very much aware of the issues - in my particular case with the TransWilts service from Swindon to Salisbury via Melksham - and are looking for ways in which to make improvements. And indeed some improvements have been made, others are promised and in the pipeline, and others have been proposed, worked on, and then passed over (that's a kind way to say they didn't happen).

Why have I headed this piece "First Great Western - Information for Customers"? Because I want to tell you about the First Great Western Coffeeshop which is a forum that's run by customers of First Great Western, for customers of First Great Western. Over 100 new topics were posted there last month, with over 400 follow up messages added by nearly 100 members. And in the first 6 days of October, we have already had half the number of follow ups that we had in the whole of last month.

The First Great Western Coffeeshop is a valuable source of knowledged information too, and not just a voice for criticism - indeed, reasoned argument and discussion is encouraged but "flaming" and name-calling will lead to a gentle reprimand; the board operators want to work WITH FGW to improve the issues that they and we all know need to be dealt with, but at the same time we all acknowledge they've a difficult job to do and - more often than not - the train does turn up within a few minutes of due time, there is enough room for most of the passengers who want to squeeze on to do so, and they'll get you there in one piece.

Posted by gje at 08:46 AM | Comments (0)

Related topics: via article database

October 06, 2007

Using a MySQL database to control mod_rewrite via PHP

Question: How can I set up Apache httpd / mod_proxy to use a program rather that a list of URL patterns to control my rewrites?

Specify a rewrite map type prg in your httpd.conf or .htaccess file. For example:

RewriteEngine On
RewriteMap tryme prg:/home/trainee/website/andy
RewriteRule (.*\.htm) ${tryme:$1}

Question: How can I write that program in PHP

1. Start the script with a line such as
#!/usr/local/bin/php
or a pointer to whereever the cli version of PHP is located.

2. Ensure you have a
set_time_limit(0);
at the top of that code to ensure it won't time out

3. Open your STDIN for interactive read within the PHP:
$keyboard = fopen("php://stdin","r");

4. Write a daemonic loop to read from keyboard, translate as appropriate, and printg out the mapped URL.

When you restart you httpd with the newly configured httpd.conf, your mapping should be in place. Well - it works for me ;-) but you do need to watch things like buffering!

Question: Can that program access a MySQL database?

Yes - just add in a mysql_connect or mysqli_connect at the top of your code, and appropriate database queries further down, and that's all there is too it. Remember you may need to "bounce" your web server to restart the rewrite daemon.

Putting all the answers together?

There's source code showing most of this ... a fully working example .. here

Posted by gje at 12:08 AM | Comments (0)

Related topics: via article database

Useful links: PHP training, MySQL training

October 05, 2007

Static variables in PHP

When you call a function, you typically want a nice clean start with no "leftovers" from the previous call. And that's what you get by default in PHP. However, there can be exceptions where you want data to be retained and you can do that using:

EITHER a global variable which is shared with the main PHP script and retained throughout its run
OR a static variable which is retained by the function while your script is running, but is NOT shared with the main code.

The typical use of a static variable is in what we call an iterator - a function which steps through a series of results, generating the next value in the sequence each time it's called. Here is a code example:

<?php
$walrus = 77;
function nextmeal () {
   static $walrus = 0;
   $munch = array("fish","seagull","penguin","ice");
   $walrus++;
   $walrus %= count($munch);
   return $munch[$walrus];
   }
for ($k=0; $k<10; $k++) {
   print "What's for dinner - ".nextmeal()."?<br />";
   }
print $walrus; # Just to show you this is a different walrus!
?>

And here are the results:

What's for dinner - seagull?
What's for dinner - penguin?
What's for dinner - ice?
What's for dinner - fish?
What's for dinner - seagull?
What's for dinner - penguin?
What's for dinner - ice?
What's for dinner - fish?
What's for dinner - seagull?
What's for dinner - penguin?
77

You'll note that our Walrus gets a different meal each day, because the static variable $walrus within the nextmeal function is retained. But (just as a demonstration) the $walrus variable in the main code is set to 77 and remains at that value.

Full source code here and run it here. Learn about it on a course run here.

Posted by gje at 07:46 AM | Comments (0)

Related topics: via article database

Useful link: PHP training

October 04, 2007

Simple page password protection - PHP

So many demonstrations are too clever - the wood gets lost for the trees and the newcomer is left not knowing what the important bits are. So a request overnight for a REAL simple way of password protecting some content based on a single password that can be changed [in the code] from time to time provided me with an opportunity to provide a new training example and to fill in the gap in really easy examples.

There's a script here and when you first call it up, you're asked for a password. Enter "abc123" and the same URL will give you back the protected content. If you want to see the source, look here.

Is it a thorough script? NO! You need to log in each and every time. Single password only. No facility for changing password without altering the page. No way to register or get password reminders .... but you don't always want a thorough script with all these bells and whistles, do you?

Posted by gje at 08:16 AM | Comments (0)

Related topics: via article database

Useful link: PHP training

October 03, 2007

Etag in http headers - what is it?

If you've ever looked at the headers you receive back from your web server, you may have noticed an Etag. What is it?

An Etag is an "entity tag" and it provides a unique identifier for the resource being supplied. Browser requests can use an etag and an if-match header to check whether a resource has been modified since it was served, providing an efficient caching facility that relies on content rather than on timeouts.

Posted by gje at 08:30 AM | Comments (0)

Related topics: via article database

October 02, 2007

Load Balancing with Apache mod_jk (httpd/Tomcat)

A single instance of Apache httpd can handle a very large number of simple requests for files to be served ... but a single instance of Apache Tomcat can handle far fewer requests for applications to be run. After all, there's a big difference between just handing someone a piece of data and having to work out something complicated to answer a question.

And so with users wishing to have a single point of contact for a web site, it makes sense to run an instance of httpd which serves out all the simple files, but passes requests on to multiple instances of Tomcat when it gets something that needs a bit of work doing. There are no fewer that three ways of doing this - mod_proxy_balancer, mod_rewrite and mod_jk. And I've got examples of the configuration files for mod_jk here today.

Here's an example of the extra configuartion needed in the Apache httpd configuration file - (/usr/local/apache2/conf/httpd.conf)

LoadModule jk_module modules/mod_jk.so
JkWorkersFile /usr/local/apache2/conf/jkworkers.properties
JkMount /latmjdemo* catkin

And here's the jkworkers.properties file:

worker.list=catkin
worker.oak.port=8009
worker.oak.host=192.168.200.1
worker.oak.lbfactor=5
worker.elm.port=8009
worker.elm.host=192.168.200.158
worker.elm.lbfactor=15
worker.catkin.type=lb
worker.catkin.balanced_workers=oak,ash
worker.catkin.sticky_session=1

Traffic is forwarded to a Tomcat server called "Oak" on 192.168.200.1, or a Tomcat server called "Elm" on 192.168.200.158, with that latter getting 3 forwards for every one passed to Oak.

The "sticky_session" is worth comment. Rather than randomly forwarding tarffic to either server, httpd will forward users who already have sessions established to the same system right through their session. That way, a multiple page process (such as an on line ordering system) can easily be implemented without the need for a lot of extra code to share work-in-progress data between the various Tomcat server.

In order for sticky sessions to work, you need to configure your jvmRoute in Tomcat to reflect the server name; all of this covered on our Deploying Java Applications on Linux course.

Posted by gje at 09:33 PM | Comments (0)

Related topics: via article database

Choosing between mod_proxy and mod_rewrite

If you're connecting an Apache httpd server to an Apache Tomcat server, you can do so via proxied http requests - i.e. have your customer facing http server relay the request on, perhaps having modified it, to Tomcat and then passing the response back. Two different Apache modules give you the facility - mod_proxy and mod_rewrite.

With mod_proxy, You specify the start of the URL and how it is to be replaced in the proxy request, and it forwards on that basis:

ProxyRequests Off
<Proxy *>
        Order deny,allow
        Allow from all
</Proxy>
ProxyPass /jd http://192.168.200.1:8882/latmjdemo
ProxyPassReverse /jd http://192.168.200.1:8882/latmjdemo

With mod_rewrite, you use a regular expression instead, and you must use a [P] modifier to trigger the request to be proxied onward rather that written forward within the same server, for example:

RewriteEngine On
RewriteRule je/(.*) http://192.168.200.1:8882/latmjdemo/$1 [P,L]

Which is best? mod_proxy is simpler, and you can (in Apache http2 2.2) also use mod_proxy_balancer to share the forwarding to different proxy servers. mod_rewrite has more flexibility and can be used if you want to share servers, but forward all visits that form part of the same session to the same server in the pool to provide you with practical load balancing.

We cover this subject briefly on our Deploying Apache httpd and Tomcat course and can cover it in more depth on private courses. (The sample configurations above happen to come from a private course I was giving today!)

See also:
Load balancing with mod_jk
A comparison between http, ajp and https protocols

Posted by gje at 09:11 PM | Comments (0)

Related topics: via article database

Python v Ruby

Ruby v Python ... I've been asked for a comparison!

• Both are object oriented scripting languages - or rather claim to be scripting languages, but are really compile-and-run via their own virtual machines, in just the same way as Perl and Java are.

• Both are modern, Open Source languages with a wide range of standard classes and methods provided as standard, and lots of other libraries available too.

• Python is a clean, minimalist language which is not cluttered with lots of alternative syntax and special characters @ and $ and the like used in conjunction with variable names, whereas Ruby uses a number of those specials to indicate variables, scope, etc. From the programmer's viewpoint, this means that Python code may be a little longer, but will be easier to read and maintain by a programmer who doesn't spend large amounts of time coding in the language.

• Ruby also provides capabilities such as topicalisation which allow the knowledgeable programmer to shorten code to the confusion of the novice maintainer. No bad thing if you have a small team of Ruby pogrammers who will be responsible for the whole life of the application, but an issue if you use other people to maintain.

• Python's typical applications can be regarded as being very wide ranging. We have trained user on all sort of applications, some (but a minority) of which are web based. Ruby is, to date, very much used in "Ruby on Rails" - a Web framework - so it tends to be very much a language that's associated with web content applications.

Which would I choose for a new project? Given the artificial choice of just these two languages, I would probably go Ruby if I was looking at a web-based content system / shopping cart / on line application, and I would go Python otherwise. But with existing code in either language, or a support team who knew one language but not the other, I would be very comfortable with either language for either type of application.

Oh - and I would be happy to run a Ruby Course or a Python Course for you!

Posted by gje at 08:03 PM | Comments (0)

Related topics: via article database

Useful links: Python training, Ruby training

Maidenhead Coffee Challenge

What do you get for a 70 pound hotel room? I'm in the fair town of Maidenhead tonight and I'm going to share with you the game of "Coffee Challenge" which is played to get a cup of hot coffee at the end of a long day.

Firstly, fill the kettle ...

... and I'll give you a clue. There's a cup you drink out of later that can be used as a ladle.

Then place the kettle on its base and plug it in ...

... and I'll give you a clue. Although there's a place for it on the coffee stand, the base isn't actually attached so it can be positioned beside the rubbish bin on the floor and plugged in from there.

The hotel IS generous with their sachets of coffee, so I can sit at the computer and drink all night if I want to ....

... or rather I could until my knees give out. And then ...

I can sit in bed and watch the TV ... I think I have a pair of binoculars in the car, and I AM going to take great care not to impale myself on those bare metal poles sticking out at the bottom of the bed.

And in the morning, what better to wake up to that a nice, refreshing, cleansing shower ...

But let's be fair - at just £70.00 per night, what should I really have expected. Wait - WE charge £70.50 per night, and include breakfast, for delegates on our courses at our own hotel, and I think we do much better!

Posted by gje at 12:42 AM | Comments (0)

Related topics: via article database

October 01, 2007

New Month, New Quarter, New Laws

1st October 2007.

The minimum wage has risen for adults to 5.52 per hour
Check - we're well generous compared to that!

Statutory holidays have risen for full time employees to 24 days per year
Check - as that includes bank and public hoidays, well in there too. And will still be OK at April 2009 when it rises to 28 days

Sale of cigarettes is to 18 and overs only now, not 16 and overs.
Check - These are not products we sell

Fuel tax rises by 2p per litre
Check - Will put up our travel costs - need to review milage rate from December - note that any on site courses booked by 31st December will be at current rate.

"Steady as she goes" All good practise changes, I think.

Posted by gje at 07:09 PM | Comments (0)

Related topics: via article database