Horse's Mouth

(Note use of command line options for autosplit mode, and for looping through all lines of an incoming file. Also note use of sprintf to do the awkward bit of the formatting, while leaving print to do the easy stuff.)

Posted by gje at 03:42 PM | Comments (0)

Bluegrass comes to Melksham

Well - what did you expect on a Friday afternoon - a sensible technical post, or something about some music?

Posted by gje at 02:52 PM | Comments (0)

Public Transport from London to Melksham, Wiltshire

In answer to a delegate who will be attending a course with us, I have put the follwowing updated information together about how to travel by train or coach from London to Melksham, Wiltshire in the early evening, and in the morning. Republished here to help others coming on courses, to stay at our hotel, or simply looking for Melksham public transport links.

Dates used are samples - should also apply on other weekdays

Public transport ... London to Melksham, evening of Friday, 10th July

1. By train from London (Paddington) at 17:45, change at Swindon, arrive 19:10
Current best single fare on offer 54.50 / buy on day 64.00

2. By train from London (Waterloo) at 16:50, change at Salisbury and Trowbridge, arrive 19:47
Current best single fare on offer 22.90 / buy on day 32.80
(Buy a Waterloo to Trowbridge via Salisbury, and a Trowbridge to Melksham)

3. By National Express coach - 18.00 from Victoria, arrive 21:05
No change necessary; also calls at Heathrow
Single fare 17.70

There are no earlier nor later services on these routes - in each case they are the only service that evening. As an alternative, you could catch a train from Paddington direct to Chippenham; from Chippenham station, there's an evening bus service about every 90 minutes to Melksham (route 234).

Public transport ... London to Melksham, morning of 13th July.

There are no trains connections from London to Melksham in the morning, nor is there a National Express coach service.

The following nearest equivalent options are available:

a) Direct train from London (Paddington) to Chippenham. During the day,the 234 bus to Melksham leaves every hour from Chippenham BUS station, which is on the other side of the town from the railway station.

b) Train from London (Waterloo) to Trowbridge via Salisbury; the 234 bus during the day calls near Trowbridge station, for Melksham.

Other Options

You could also travel from London to Bath and change there onto the 271, 272, 273 or X72 buses. Most of the day there are 2 services per hour, but the two run just a few minutes apart. Also note that the journey from London to Melksham via Bath is a 'dogleg' - longer distance, longer time, likely to cost more.

Taxis are usually available at Chippenham, Trowbridge and Westbury stations (but if you're going to be very late, best book ahead);

Quite frankly, we're ashamed of the paucity of train services calling at Melksham (a town or 20,000), and the lack of proper integrated connections into the buses. That's even though the town isn't responsible for running the services, and we HAVE campaigned hard for an improvement.

If you are a customer of ours, get in touch as you plan your travel; we very often pick up guests and delegates from Westbury, Trowbridge or Chippenham with prior arrangement.

Posted by gje at 08:06 AM | Comments (0)

Getting home from Melksham

I have written so many blogs and notes about how to reach us in Melksham, but very few on getting home afterwards. "Isn't it just a mirror of getting to you" you may well ask, and of course in many ways it is ... but there are a few extra notes I can add.

There are often specialist subjects at the end of a course that aren't applicable to every delegate ... and for delegates who have a long way to travel, or an early flight to catch, we can usually jiggle the presentation of these last few items around so that they can leave a little early. And indeed, our delegate groups are often happy for us to start a little earlier some days, or run a little later, so that we can conclude promptly.

For delegates who CAN'T (or don't want) to get home after the course, we can offer a further night's accommodation. And we can then drop them off for the 07:17 train, or arrange an early taxi to the airport. Some regular customers who fly to the USA leave before it's light, even in summer, but we can still provide breakfast (I was here at 04:00 this morning - early and unusual even for me, but it can and will be done!)

Posted by gje at 07:51 AM | Comments (0)

June 18, 2009

A day in the life of a hotelier

It's been an exhausting day ...

... and it isn't even 3 p.m. yet.

Breakfast for five and all checked out. Meeting for 3. Coffee service and meeting for 8. Lunch for 7. ...

... still to come - conclusion of the meeting, and "prepping" for breakfast for 20, and a further meeting and lunch for eight tomorrow.

But, seriously, it's rewarding. Meeting new people, listening and learning life's experiences. I'm stopping by for a mid-afternoon internet 'fix' ... meeting in the Wilts, and I can hear help in the kitchen clearing down. And I'll be raring to go again in an hour.

Posted by gje at 02:16 PM | Comments (0)

What difference does using the XHTML standard really make?

I enclose my HTML attributes in quotes - or at least when I remember, I do!

I always provide a matching close tag to go with each opening tag - except when I forget to do so, or am in a hurry

I make sure I provide an alt attribute with each image - although sometimes I really don't know what I would say in such a tag, so I skip it

If I need an & character, I encode it as & - mind you, when I cut and paste I may miss some!

What difference does it really make to the people viewing my pages if I've got of few of these transgressions?

It was pointed out to me the other day that one of the sites I looked after didn't conform to the XHTML transitional standard in one area of the page, and there was also a specific user problem being reported. Nad it's very nice to have a "we confirm" logo there. So I took the opportunity to add a poll to the site, for regular users, asking if the changes I had made (which cleared 96 errors and 1 warning!) had made any difference to users.

I'm going to conclude that the exercise was worth doing, as it gives me a nice warm feeling. It will make things better for the occasional person. It's suggested that it will now render faster (not sure if I believe that). It stops people telling me that can't see my page because the HTML is broken (even if the reason they can't see it is something else - like our site is blocked at their work place because it's too good and they spend too long with us). And it probably help our search engine ranking - the cleaner the code, the easier for the search engine to understand it, and the more professional it looks.

Posted by gje at 01:19 PM | Comments (0)

June 17, 2009

Client side (Applet) and Server side (Servlet) Graphics in Java

What is the current release of Java? Is it Java 1.6, Java 2 (1.6) or Java 6? Yes it is - they're all different names for the same thing!

Java started out as Java 1.0 ... and since code that would run on that Virtual Machine / written in that language will still run on today's version, it's still technically "Java 1". But would you seriously 'buy' a product that's not had a major revision in more than a dozen years? You would think twice ... and so it was that the marketing gurus switched to the "Java 2" name, and then to "Java 5" for 1.5 and "Java 6" for 1.6. And in some ways that makes huge sense beyond the marketing side too, because Java 6 is a very different beast to Java 1.0

Java 1.0's main use was in Applets - code running within a user's browser, sandboxed into an internal frame within a browser window and with carefully limited access beyond that frame. A very close parallel to Flash these days. But things have moved on ... Java has moved from browser / client side through to server side, where it's running complete applications (perhaps under Tomcat through the Servlet / JSP interface, tag libraries, Struts, Spring and the rest!)

From its early days, Java supported graphics through the AWT - Abstract Windowing Toolkit - a useful but low level interface. You call methods to paint with a Graphic object on a Frame object - methods with name like setcolor and drawLine and drawString. If you're working at this level, you'll be using pixel addressing so you'll find that you're providing your own utility methods with names like toXPixel and toYPixel - mapping world to screen coordinates; there's an example of such an applet here.

So is such graphics useful now that most Java runs on the server? It turns out that yes, it is!. As well as working on a Frame within a browser window, Java can work on a psuedoFrame in memory ... and then that frame can be stored as - for example - a .jpg file. We have an example of such a Servlet (for that's what it is) here. Technically, it uses a BufferedImage object, on which you run a createGraphics - it took me a while to work out the first time, but the example I'm pointing you at (link on "here") has all the hard word done for you and produces a graphic of data that the server extracts from MySQL database table. A truly 'put it all together' application!

Posted by gje at 08:56 AM | Comments (0)

June 16, 2009

What should a web site cost you?

I don't know, but I was asked my opinion on website costs for a government department (the Department for Transport) which have been revealed in a Freedom of Information request here. And I have compared the figures given to the figures for our main web site (www.wellho.net) and for the volunteer "First Great Western Coffee Shop" forum.

Quoting the FOI pre-amble:

A copy of the information requested following an agreed reduction in scope is enclosed. A. Unique visitor data and separate staff costs for adding and updating content on the Department for Transport corporate website are not available for the financial year 2002-03 but have been provided for subsequent years.

In keeping with the spirit and effect of the Freedom of Information Act, all information is assumed to be releasable to the public unless exempt. The Department will, therefore, be simultaneously releasing to the public the information you requested, together with any related information that will provide a key to its wider context.

The three most recent years are as follows:

So how reasonable are those figures?

Looking at our own website (http://www.wellho.net) the last year's figures are as follows:

And looking at the "First Great Western Coffee Shop", I have six months of data as follows:

Realistically, the number of unique visitors to "The Coffee Shop" in a year would be between 50,000 and 60,000 ... (educated estimate) as it's a web site where people come back time and time again. The number of individual requests to the server in that six month period was 4,785,623 indicating 118 requests per visitor. That would extrapolate to around 9,200,000 requests in a year

The figures for www.wellho.net are slightly depressed (circa 5%) by a server move last July. The number of individual requests was 44,918,001 - indicating 20 requests per visitor.

These two servers are extreme cases; a typical figure may be perhaps 50 accesses per unique visitor, giving 173,947,600 accesses to the DfT website. Let's come up with a grand comparison:

My analysis of my own systems was done using Perl (what else! - a heavy data handling task, one off programs that won't have a long maintained shelf life ... the language fits).

Here's the program for counting unique visitors in a day:

#!/usr/bin/perl -na
$c{$F[0]}++;
END{ print ("Unique visitors",0+keys(%c),"\n"); }

And here's the program for going through a year of log files (they happen to all be in one directory, start with ac_, and there are no other files starting that way in the same directory!

#!/usr/bin/perl
foreach $fl (glob("ac_*")) {
  print "$fl\n";
  $nf ++;
  open (FH,$fl) ;
  while (<FH>) {
    @F=split;
    $c{$F[0]}++;
    $lns++;
    }
  }
print ("Unique visitors in $nf days - ",0+keys(%c)," / accesses $lns\n");

Posted by gje at 06:52 PM | Comments (0)

Changing a variable behaviour in Perl - tieing

What can you do with a scalar variable? When you reduce it to lowest level programming principles, not a lot ... you can create it, destroy it, save a value into it, and read a value back from it. That's about it, when you think of it!

In Perl, the low level memory accessors within the language go through just a handful of routines (it's really an object interface), which you can override for individual variables if you like. What does this mean? It means you can change the characteristics of a variable if you like. Look at this program:

use propercase;
tie $yourname,"propercase";
print "Enter your name ";
$yourname = <STDIN>;
print "Welcome, $yourname";

Good - but because of my propercase class it will force the name to be properly cased:

earth-wind-and-fire:~/jun09 grahamellis$ perl ppc
Enter your name graHam
Welcome, Graham
earth-wind-and-fire:~/jun09 grahamellis$

Clever, isn't it? ... Here is the module that defines "propercase"

package propercase;
sub TIESCALAR {
  my ($class,$value) = @_;
  bless \$value,$class; }
sub STORE {
  my ($inst,$value) = @_;
  $$inst = ucfirst(lc($value)); }
sub FETCH {
  my ($inst) = @_;
  return $$inst; }
1;

Other examples of a tied scalar are for a variable which automatically has spaces trimmed off the start and end, a stack (where each time to save a value it is actually pushed onto a list, and each time you call it back, the next value is returned) and a variable that persists between runs of your program, as tieing links tha variable to a file on the disc.

Tieing is also possible to lists and hashes - in the case of a hash it can provide a very useful interface indeed to a database table - you can insert / replace / select / update database records with simple assignment statements. The methods you need to implement to use a tied hash are TIEHASH (the constructor), STORE, FETCH, EXISTS, DELETE, CLEAR, FIRSTKEY and NEXTKEY. There are examples in our Tieing in Perl resource module, which is covered on our Perl for Larger Projects course.

Posted by gje at 07:41 AM | Comments (0)

June 15, 2009

So what is this thing called Perl that I keep harping on about?

"An overview of the Perl Language"

It almost sound like one of those dreaded school essays that I had to write in my several attempts to get an English "O" level (they were the things that came before GCSEs) ... but these days I enjoy writing. No - this little article is one of a decreasing number that I need to write to have written at least something on every single one of the training topics and associated subjects listed on our web site. So - how would I overview Perl in a short article?

Perl stands for the "Practical Extraction and Reporting Language", and it's an excellent acronym for once!

Perl is - above all else - PRACTICAL if you want to write a quick piece of code to perform a task, especially if that task involves EXTRACTING information from a gob of data and REPORTING on the results. It's so practical that I can throw together in a morning a piece of code that would take me a week in C (but, mind you, I've had to spend a long time getting to know it well enough to do that). It's almost too practical at times - I can throw a piece of code together that's so quick to write and terse that I can't understand it when I come back to it a week later, which makes the maintainance of work done by others in a hurry pretty darned hard. Which is why every Perl Course I give starts with a chapter which includes telling you how to comment (for the person who has to maintain your code) and document (for the user!).

Uses

Perl was written - or the first version was - in 1988, by Larry Wall. He was looking for a scripting language that was secure, network aware, and easy to use and as he couldn't find exactly what he needed, he wrote one. Being a US government employee at the time, he couldn't sell copies ("that would complete with private industry") but he could give copies away ... and he did, but he retained copyright and he does to this day, at the end of a very long and slack licensing system which allows it to be used for almost anything. So what is it used for?

It's used for all sort of things. I used to say 1/3 web based, 1/3 system admin and 1/3 heavy data handling, but there's much more to it that that. The safe_mysqld script is written in Perl, so is the miniserv which I can use behind the scenes on this site ... and so is the software I used for "The Horse's Mouth". When I had a crisis of access on the web server, I wrote a few lines of Perl to go back through my log files and look for certain feature ("Practical extraction and reporting") and so it goes on.

Characteristics

1. Never use 2 characters where one will do
2. If something looks like it could be useful, include it, even if what it does can already be achieved another way
3. Make it as 'natural' as possible - don't force strange syntaxes. "DWIM" - 'Do what I mean"!
4. Embrace technology; don't shy away from new things, but embrace them.
5. Language assumed programmer knows what (s)he is doing.

Syntax

1. "C" based - operator and operand type language, semicolon statement separator, curly braces around blocks.
2. No need to declare variables (but uses sigils - $ for scalar, @ for list, % for hash & for a piece of code)
3. Automatic memory allocation; automatic creation of implicit variables (autovivification)
4. Eclectic - lots of ways of doing the same thing!

Expandability

1. The basics are built in
2. Perl is distributed with lots of extra modules you can load when you need them
3. The CPAN (Comprehensive Perl Archive Network) provides many more modules - exotic ones, and those likely to be upgraded asynchronously from Perl itself
4. You can write your own shareable modules in Perl
5. You can write extra C code to build in too
6. You can used Perl as a web module
7. You can talk to a Perl program via files, sockets, shared memory, databases from any other language that supports them.

OK - you've guessed it, I'm a fan. I have a huge respect for Larry Wall, who I have hand the honour of meeting on a couple of occasions, and I am honoured to have been able to build up Well House Consultants based on Open Source Languages such as Perl.

Posted by gje at 07:07 PM | Comments (0)

Perl references - $$var and \$var notations

In Perl, if I write:
$stuff = "Porridge";
then I'm setting up a variable to contain the value "Porridge".

If I write:
$stuffat = \$stuff;
then I'm setting up a variable to contain the ADDRESS OF the variable $stuff.

So if $stuff was in memory at (hex)fe80 ... then $stuffat will be assigned that value 0xfe80. And we can look use it by using an extra $ character :
print $$stuffat;
which literally means "the contents of the contents of stuffat".

What use is this?

Lots ... not only for scalars, but for the addresses of lists too. If I write
$holder = \@biglist;
I have assigned a single scalar variable to point to what is potentially a huge dataset and (for starters) I can easily pass it around to various subs without the need to replicate the data.

Other languages have this concept - "pointer" - too. See & and * in C, and the use of & in PHP.

Posted by gje at 02:00 PM | Comments (0)

How do I query a database (MySQL)?

From my mailbox ... "What are the different ways of queries in databases? Could u pls explain each way in more detail? thanks"

The term "query" means something a little bit different in the term "SQL" to it does in regular English usage - the dictionary definition is to ask for information / pose a question, but as a part of SQL ("structured query language"), it means and request of a database and that can include:
• Setting up and modifying table structures
• Inserting and modifying data held in tables
• retrieving information about the structure of tables and how the database is running
• retrieving information from the tables of a database - the actual data.
Only the last of those is a "query" in terms of normal English usage.

All SQL database queries [exception - Microsoft Access] are made via a service or daemon - a process running on a server computer. That process is designed to provide the one and only route to the data, so that multiple concurrent users will be able to work together on the same information with no risk of one adversely effecting the other through lack of syncronisation. And the requests passed to that service (Microsoft word) or daemon (unix / Linux word) will be in "SQL".

SQL queries all start with a keyword such as CREATE (to create a table), INSERT (to insert data) or SELECT (to retrieve the actual information held). And they are the ONLY way to access the database. However, they are not very easy (they are not designed to be easy!) for the human user - they are much more easily formed within another program, with the human user calling that other program as his / her interface to the data. For example, if you were to book a hotel room or training course with us, a number of database 'queries' would be run - SELECTs mostly with a few INSERTs - in the process, which would be completely hidden from you. Indeed, you've probably done some SELECTs without realising it to read our web site already, as almost every page has some database access.

I can explain SQL, and SELECTs, in much more detail - but there are whole books on the subject - it's massive. So that explanation would usually be a part of one of our courses, in which we would cover some elements of database design, as well as how to access the data. Such a course would be useful if you'll be accessing the data directly in your own programs (in languages such as Perl, PHP, or Python), but it's really not necessary for you to learn SQL just to use a web site that makes use of it internally. If everyone who stayed at our hotel was required to know SQL before they booked, it would be a strange world.

Some further links for you:
  http://www.wellho.net/course/mqfull.html
(next course is on 13th and 14th August and we currently have places available)
  http://www.wellho.net/solutions/mysql-mysql-select-join...group-etc.html
(that's an example of a SELECT showing each of the elements in the command)
  http://www.wellho.net/mouth/515_MySQL-an-FAQ.html
(A link to our MySQL FAQ - many common questions answered)

Posted by gje at 12:28 PM | Comments (0)

June 14, 2009

Cornerstone Cafe, Melksham

There's a natural concern when a business changes hands. And a deepening concern when virtually all the previous staff disappear. But sometimes, a change is a breath of fresh air and a new bright and different beginning. And so it is with "Kim's Seaside Shack" - or should I say The Cornerstone Cafe in Melksham.

Prior to Kim taking over the Cafe, the Cornerstone was run by a local church; a friendly place to pop in for a cup of tea and a sandwich at lunchtime, with all welcomed irrespective of creed. And yet ... the bible quotes and biblical pictures meant that some felt uncomfortable there, and I had to be very careful which customers I took there at lunchtime; even some of our Melksham friends who had a Christian style upbringing found it a little too strong.

But I have to hand it to Kim ... she's got it right. In a fiercely competitive market in Melksham, her cafe is a bustling place at lunchtime whereas there are some others which are quieter, and one or two depressingly empty. And it's not because Kim's food is "Cordon Bleu", nor because she's selling it cheap (sorry Kim - the cakes look lovely; the stuff I eat is very good, but not "Gordon Ramsay"!). It's because of the atmposphere, and because she's caring for her customers and their needs. She makes The Cornerstone into a destination you want to go to.

What's all this about Mackerel Fishing Trips, and going to see the lighthouse? Where are the donkey rides in Melksham? I'm sorry to disappoint, but we're 40 miles from the seaside at Severn Beach. No - this is part of Kim's decoration of the cafe for those customers of hers who won't be making it to the seaside this summer ... she's brought the seaside to Melksham for them.

The cafe is open, Monday to Saturday, until late afternoon. It's on the corner of the Market Square in Melksham, next to the Castle News Agents and across from the Town Hall. There's a range of baked potatoes, sandwiches, an all day breakfast, salads - oh - and home made cakes, cream teas, scones and a wide range of teas, coffees and cold soft drinks too.

I expect this article will hang around a while, so by the time you read it the Seaside theme may be long since gone. But that wasn't Kim's first theme and I don't suppose it will be her last one either ... why not pop in there sometime; you'll be assured of a warm welcome, whoever you are. The "Christian" may have gone, but the respect and looking after everyone which is supposed to be an integral part of that faith is very much alive - indeed far more so than it ever was in the past - at "The Cornerstone".

Posted by gje at 12:01 PM | Comments (0)

Handling nasty characters - Perl, PHP, Python, Tcl, Lua

Are your writing or maintaining a web based application that uses forms? If so, you have better be aware of some of the nasty characters that are around!

The < character, when echoed back from a users's input 'unchallenged', may form the start of a tag. So that in a relatively benign case, a user who enters <em> at the start of his name will have his name emphasised back to him ... and to anyone else to whom that data is echoed unless your application cleans up.

The " character too can cause problems when echoed - if it gets written into a tag that's already got an attribute that's quoted, you can get some odd results. A user who enters 44" type="password into an unchallenged box that's echoed may be able to make the next form come up with the field he is entering using blobs rather than the actual characters typed in the box.

The ' character can be a snare too - if your application stores the entry uncleaned in a database, then with appropriate following code after the quote (I am not giving an example here!) can do severe damage.

And those are just three examples of special characters that can cause problems if they are not carefully considered; others include ` . + \ & % and even the humble space. And if you are unwise enough to treat a user's input as a regular expression, you're opening the way for the user to start performing all sorts of nasties with other characters too such as * ? [ ] | ( and ) (and this list is not - and is not intended to be - complete!)

Have I frightened you so much that you never want to provide a user input box again? I hope not, because there are robust and easy solutions!

I find it helpful to draw diagrams to show how the variables flow through my code and are processed, labelling each of the legs with the function / code necessary to clean up and close loopholes. The variable conditions ("from web", "in memory", "as part of XML string", "in database" and "sent back to web") will be the same no matter what language you're using. The labels on the flow lines will vary, depending on the functions in the language and how much work the web / database interfaces in the language do for you, and how much is left up to you.

Here is the diagram for PHP; you'll typically use "stripslashes" to bring a string into memory, with most of the rest of the work done by PHP. "addslashes" or "mysqlrealescape" converts the data for database storage, and "htmlspecialchars" gets it read for sending back to the web.

For Perl, you can use a module like CGI.pm, or you can roll your own. Personally, I have a sub that I call collectform that turns up via a use in most of my apps, and another called webify that cleans for output. They need to hand things like hex codes (%2B) and + characters which PHP handles silently for you (one of the differences between the ethos of the languages - Perl being general purpose, whereas PHP is written by a web programmer, for web programmers).

With Python, the cgi module provides methods such as cgi.Fieldstoragecgi.escape which add, in single calls, the necessary converters to the language. There's an example in our source code library here (and further examples linked from that page too!.

If you're using Tcl as your server side scripting language, we have sample of source code that tidies up nasty characters here. And if you're a Lua Programmer, then we have an example here.

Posted by gje at 11:18 AM | Comments (0)

Taking a pride in the community

How do you encourage young people to take a pride in their community, rather than engaging is antisocial behaviour? By encouraging their inclusiveness and by providing them with facilities.

Lisa and I took a short train trip yesterday, and found ourselves at the station with eye-catching silhouettes of - well, you can see the young people's names - in an area that used to be notorious for its graffiti. I understand that the work is that of the younger kids, and their older brothers and sisters then take a sibling pride and "woe betide" anyone who tries to tamper with or deface them. Of course, allowing the station to be painted up in this way is also eye-catching, distinctive, and attractive for us more mature members of society.

Looking around further, a BMX park, plenty of fishing facilities, and we were in a town that offers an example to others. Like all towns, though, I'm sure that there are other things / areas they could learn from.

Where am I talking about? "Not telling" says I ... unless you follow this link!

Posted by gje at 08:40 AM | Comments (0)

June 13, 2009

Alumni - revisiting and supporting the old University

I have just completed a survey for my old university - I won't name them here, but researchers will be able to find out who they are easily enough, as I made a comment and recommended one of their courses - which actually competes to some extent with what we do - in a comparison chart here recently, and my "vanity" web site includes my CV.

I have some fond memories of "X.X.X." as we knew it in those days, but have long since lost realistic touch. A sprinkling of school re-unions in the distant past, looking through lists of past graduates and what they're doing in the university's newsletter, and my school's one too, has brought few if any re-contacts with people who's company I enjoyed, and as I was living at home / commuting to both of these establishments, with a handful (school) and none (university) of my fellow students living in my own town, no firm "outside learning" links that have endured were established. Indeed, I can recall returning - just once - to my prior school. I was well received and have an endearing memory of realising that Mr Wood, the Latin teacher who had been a real tartar, was a truly lovely gentleman when I saw him in a different light. However, the overall feeling I've got from such events from the staff who taught me is "nice to see you - very briefly - now let us get on with our current job and I can't say that I can blame them for this attitude.

So my University survey was filled in looking fairly negative - "no, I don't use these facilities", "no, I don't read your bumph very much", "no, I don't attend your alumni events" and "no, I'm not going to give you shed loads of money". Yet at the same time, I have said that I am happy to advise current students and talk to groups of them / lecture, etc.

Times move on. Had I been resident at the University or School, had I been on a course / with a group who all get involved in reunions themselves, then I would be far more interested in reunions and even arranging them. But I'm not going to spend time and money attending events that past experience has not brought me to enjoy. I will add that Lisa and I have attended a number of her school reunions - and we host her class site on our servers and she does the updates. They are fun - I get involved through the osmosis, and really it's a post-school club where the school itself is not involved in it as a form of self-promotion.

Times move on. And I no longer live anywhere near the University or schools involved; beyond the very occasional involvement, I can make better and more focused use of my resources with current people and issues, and in my current place - what I can do, though, is go give heartfelt thanks to the people and institutions who helped get me there, and I'm happy to do so in public too.

Times move on. And I can find more local and focused ways to make gifts (financial or otherwise) that subsidising / promoting educational establishments which were paid for in one way or another at the time. My parents put in a very great deal financially - and we did the same for our children and would do it again. Others may wish to be philanthropic and bestow scholarships or pay for rebuilds of places they went to long ago - but isn't that just a case of putting in my money where others (such as the government / local authorities) have taken a look and decided it's not the best way to spend? It's notable that there's a new school being built 400 yards from where I live, and it's not funded by voluntary donations.

And yet ... for all the negativity I'm expressing, I would be more than happy to share appropriate life's experiences with current students. Via email, on the phone, in person, singly or in groups. But the key is in 'appropriate' sharing and (from an unscientifically small sample), I've felt that past attempts to help in this way - at the request of X.X.X. - have been so far off target as to provide little of use. I'm not requiring that every request for help should result in a life-revealing meeting, and indeed "success" could be regarded as having just one in ten such meetings make a difference (rather like selling a house - 10 people view and 9 do not make an offer, but it's a successful sale)

The final page of the University's survey asks for additional information, and I don't know if my logic / reasoning above will be of any interest to them - but I will send them a link to this text (the form box gives me 4 lines, which suggest that only a short comment is expected, so I'll keep it short). I'll admit to thinking that I won't hear further ... as the emails to fill in the survey all come from Paulo Gxxxx, who is "Head of Alumni Relations", but he doesn't invite me to contact him in any way - he tells me to contact Cxxxx Exxxxxx-Hxxxxxxx, who has a much modest role. I expect Paulo would want to talk with me if I was offering him shed loads of money ...

Learning from the above ... since Well House Consultants is also into the training business and we have a lot of 'Alumni', many of whom read this Blog.

1. I love the subjects I teach, and I love to hear how people are getting on with them. There is a DIRECT FORM back to me here to comment on the blog, and here to follow up on technical questions or for any other comment. I might end up answering with a suggestion of a course if I think it's appropriate - yes, that would bring us revenue - but I will not ask for a financial donation to give us extra for your course which will already have been paid for when you took it.

2. Our past delegates are our ambassadors, and we want to encourage you to remember us, and to be in touch. And we enjoy it when you do. "Come as a Student, leave as a friend" we say and - if you happen to be passing - stop by for a coffee. Gratis. If you want to stop over on your way to the South West, we'll provide a room at delegate rate at the hotel and I'll put aside that extra time to catch up with you if I possibly can.

There are other differences too, of course. More recent delegates v experiences than ended over 30 years ago. A tiny organisation where you WILL know the proprietor - he taught you - versus a huge organisation. And an owner operated business v one with purely employees.

Keeping up with past customers is such a pleasure when your heart is personally in it - and mine is. And that will endure while I remain

Posted by gje at 10:11 AM | Comments (0)

June 12, 2009

Sending awkward characters by email in Perl

Carrying on from the email work I described in a recent article, I wrote an example today in Perl where I used the mailx program to send an email, piping to it in a system call via an echo.

For a one-liner, without special characters in the text, this is easy ... but to get echo to pass through all characters including the " delimiter and new lines took a bit more effort - but we did manage in the end with an absurdly easy looking piece of code - see here. There's an extended example, with the emial being routed to different addressed depending on its content, here.

Final - and off topic - example for today. Formatting in Perl with (s)printf. Did you know you can use "b" for binary? Example - and more formatted printing - here

Posted by gje at 04:41 PM | Comments (0)

Loading external code into Perl from a nonstandard directory

Here's a piece of Perl code that loads in an external module.

push(@INC, "/var/opt/OV/bin/OpC/monitor");
eval { require "monitor_utils.pm"; };
if ($@) {
  print STDERR "could not load monitor_utils.pm: $@\n";
  exit 1; }

It comes from a delegate's existing application, and is worthy of further comment:

• The @INC list is the places from which Perl loads modules, and the code adds an extra place to that list.

• require rather than use pulls the module in, since use happens at compile time, and so would happen BEFORE the extension to @INC.

• The require is within an eval so that any error condition can be trapped, rather than the code crashing, if it failed.

• $@ is the error report from the eval. If it's empty, that's good news.

• the application exits with a '1' error code if the require fails - at OS level, 1 is an error and 0 is a success.

Posted by gje at 04:35 PM | Comments (0)

Transforming data in Perl using lists of lists and hashes of hashes

In Perl, you don't have conventional two dimensional arrays - you have lists which are single dimensional only. However, you can have a list that contains a reference to another list, in effect a list of lists. Very powerful stuff as it leads to a very flexible way of handling data structures - we cover it briefly on our Perl Programming Course and in much more depth on Perl for Larger Projects.

As you write code using lists of lists (and hashes of lists and the other alternatives too), you are very strongly advised to draw a diagram of what you are trying to achieve. It will do wonders for you. That way you can see the difference between a list which is a concatenation of other lists, and a list of lists. Say, for example, I had three options for breakfast, 4 for lunch and 4 for dinner, a concatanated list would be 11 elements long, but a list of lists would contain three elements. There's a source code example here from today's course. The diagram alongside shows how this code works.

One of the big uses of hashes of lists (as an example of a collection of collections) is in inverting data. Let's say I have a file in which each line starts with a person's name, and the person's skills are then listed along the line. But I want to turn that around so that I can see a list of skills, with the people holding each skill alongside the skill. It turns out to be a handful of lines of code - see here. What works for people and skills is equally good in transforming a web access log file, listing by visiting client, into a report on each of the web pages on your website and telling you how many people have visited each of them!

Three further examples: reading the people / skills file into a list of hashes and the data inversion with a web (CGI) interface. Finally, the data inversion with a search facility.

Posted by gje at 04:25 PM | Comments (0)

Why sendmail one way, and pop3 the other?

When you send emails, you'll use a "sendmail" protocol / service, whereas when you receive them, you'll typically use a "pop3" or similar protocol / service. Why not use just one protocol for all the stages of email transmission?

Well in sendmail protocol, the client program pushes the email to a server - in other words the server is the receiver, but in pop3 protocol, the client program pulls email from the server. So if you were to use sendmail for both, it would mean that you would have to run a server on your local machine, awaiting connections from clients ... in other words, your ISP's system would have to keep looking for your machine every so often and passing emails to you if it found your server running. This might sound messy - and it is; but it's actually what Demon Internet used to do in the very early days, when we were using MS-DOS rather than windows. It worked excellently, but it was very odd to get online and then to have to wait up to 300 seconds each time an email check was wanted, rather than being able to easily trigger the check locally as you can do with a local client.

As part of this week's Perl course, I have been reviewing some more mature modules and I have just posted up two extra source code examples - using sendmail and pop3 - into our emailing to and from Perl module which is part of our Perl on the Web course. We also cover emailing from web pages on our PHP Programming course. And I always remind people when teaching these modules that they should use automated emails with great care - they're required to confirm registrations or orders, but can easily be abused to send spam. And remember that people who'll sign up for a newsletter will forget that they have done so if you don't update them regularly, and may unjustifiably accuse you of being a spammer!

Posted by gje at 09:17 AM | Comments (0)

What is CGI.pm / A dozen new examples

Perl's CGI.pm module allows both a structured and Object Oriented approach to generating and handling web pages via the Common Gateway Interface. It was written (a long time ago now) to provide a library of routines to help Perl programmers write web applications without having to (re)write or roll their own Common Gateway Interface code, and to provide a logically robust tool to ensure that tags balanced, via functions which generate both open and close elements writing the same single call.

CGI.pm - with so much to do - is a very large module, and there are a lot of different function calls to learn if you're going to use all the parts of it. So it is arguable as to how much you gain by using it - in other words, whether the positives outweigh the negatives. Personally, I make use of some elements such as CGI::Cookie, which provides Perl code to handle the unique date stamp format that cookies, use, then I 'roll my own' for the rest. But then I am a bit of a geek who knows the protocols, and I'm always looking for lean demos ...

We have a major training module on Using the CGI.pm modules, which we present briefly during our Perl on the Web course. There are over a dozen source code examples there ... and yesterday, I notes that they weren't available on our web site for delegates to access after the course ... well - the are now - follow this link!

Posted by gje at 08:53 AM | Comments (0)

June 11, 2009

Running a piece of code is like drinking a pint of beer

Q: What is the effect when I drink a pint of beer?

A: I get slightly tipsy.

But that's too simplistic!

A: The brewery has some more money
A: There's a glass to wash up
A: I need the loo!

Running a piece of code is like drinking a pint of beer - as well as a headline result, you get extra variables / side effects ... and the normal assignment statement (the = sign in all the languages we teach except Tcl) has a single target.

In Perl, though, side effect variables can be set too - and that's done especially with a regular expression match. The 'headline' return value is a true or false return, telling whether or not the string matched a patter, but there's more available to you if you want to use it:

$1, $2 etc ... for 'interesting bits' that matched parts of the pattern
$`, $& and $' for the bits before and after the matched part and the match itself

And if you use a match in a list context, you'll get back a whole list of all the interesting bits ... matched many times over if you have used the "g" modifier on the match.

From today's course ... source code that shows lots of these extra variables

Posted by gje at 07:04 PM | Comments (0)

Do not re-invent the wheel - use a Perl module

"If you think 'surely someone has done this before', you're probably right ... and in Perl, you'll find the resource you need available as a module on your system, or if it's not quite to common, on the CPAN". I was reminded of this advise today, when I got involved with web site checking ... and rather than writing my own robotic browser in Perl, I used the LWP module ("Library for Web Processes" in case you wondered!)

What can I do with LWP? Well - I have several new examples to show you.

Reporting all the internal and external links from a page - this uses LWP::Simple, standard on my Perl and easy to use

A short example that grabs a page and echos its content and status, using a minimal series of calls to the more complete LWP module

A script that grabs a web page, then checks all the links from it - a prototype example which needs some more work, but it's already found a broken link to an external site from one of our pages - and such things are very time-consuming to monitor by hand!

Here's an example of the sort of outputs you can get from that last program:

Dorothy-2:perl grahamellis$ perl goodlinks http://www.wellhousemanor.co.uk/
Status from http://www.wellhousemanor.co.uk/whm.css is 200
Status from https://lightning.he.net/~wellho/hotel/reservation.php is 500
Status from http://www.wellhousemanor.co.uk/rooms.html is 200
Status from http://www.wellho.net/happens/rooms.php is 200
Status from http://www.wellhousemanor.co.uk/amenities.html is 200
Status from http://www.wellhousemanor.co.uk/events.html is 200
Status from http://www.wellhousemanor.co.uk/contact.html is 200
Status from http://www.westwiltshire.gov.uk/index/env/env-health-service
/food-hygiene/scores-on-doors.htm is 404
Status from http://www.wellho.net is 200
Status from http://www.wiltshirebusinessoftheyear.co.uk/ is 200
Status from http://www.aguafabrics.com/default.asp is 200
Status from http://www.hoteldesigns.net/industrynews/news_2745.html is 200
Status from http://www.macformat.co.uk is 200
Status from http://www.wellhousemanor.co.uk/art.html is 200
Status from http://www.tripadvisor.co.uk/ is 200
Status from http://www.tripadvisor.co.uk/Hotel_Review-g528775-d645951-
Reviews-Well_House_Manor-Melksham_Wiltshire_England.html is 200
Status from http://www.freeindex.co.uk/profile(Well-House-Consultants-Ltd)
_44477.htm is 200
Status from http://validator.w3.org/check is 200
Dorothy-2:perl grahamellis$

Posted by gje at 06:44 PM | Comments (0)

Where do I start when writing a program?

Q: Where do I start when writing a program?

A: Start off by thinking about what you want to achieve, and what inputs you have available to you. In other words, look at what the customer provided and what the customer wants to get (yes, you may be your own customer!). If it's too much to think through in one step, split it into a series of stories, and write each stage in turn, frequently refactoring as you go.

As well as the inputs and outputs, think about the major structures you'll use internally within your program - your collection variables (arrays, lists, hashes, dictionaries, tuples ... whatever is available in the language of your choice), and think what is likely to be common code which will allow you to (re)use you own code later, or to call in someone else's stuff from day 1.

When you've given this some thought and you're moving to a coding stage, write your comments first - they will in essence form a textual flowchart, help you identify pitfalls early, and avoid you coding up too many blind alleys.

On today's Perl Course, I went through these principles, in order to start answering the question "what operating system(s) do people who use my website run?" .... you can see my initial (all comments!) design here. I then went on to add some code - a first test or - 'spike' solution - see here. The results of running that program on a recent log file from our own site:

Dorothy-2:perl grahamellis$ perl whatos
There are 80520 good records and 34 mucked up ones
unknown ... 49716
Googlebot ... 9574
Windows NT 5\.1 ... 21230
Dorothy-2:perl grahamellis$

which is very short .. but an excellent demonstration of principle.

Although I am talking about Perl in this note, you'll find that what I've suggested is equally suitable for use in Python, PHP, Ruby, Lua ... indeed, in all the other languages we teach. If what I've written is of interest to you, have a look at our "learning to program in ...." courses, where an extra day on the front of the main course helps newcomers to programming. But if it's stuff you're happy with, start on our ".... programming" courses which assume prior knowledge of things like this

Posted by gje at 06:22 PM | Comments (0)

Learning PHP, Ruby, Lua and Python - upcoming courses

The following programming courses are all booking well, but still have good availability:
PHP - starting 6th July
Ruby - starting 14th July
Lua - starting 10th August
Python - starting 17th August
Also Deploying LAMP (Linux, Apache httpd, MySQL, PHP/Perl) from 20th July. Can be taken as separate Linux Basics, introduction to Linux Admin and Linux Web Server courses.

Other courses - Apache Tomcat, Java Programming, Perl Programming, C and C++ programming, MySQL and Tcl are also scheduled within the next 3 months - see schedule from where there are links to every course.

The programming courses listed are for delegates with at least a little prior programming experience. If you are new to computer programming, we offer an additional day immediately prior to each of the programming courses, in which we cover some of the initial programming principles in order to help you get started, and how they apply to the particular language you're going to be learning. That way, you're prepared for the 'main course' and can benefit from interacting with delegates who have experience in other languages, and be able to gain a very great deal from the course without holding others back.

With an extra day, we name our courses for programming newcomers "Learning to program in xxxx", at an extra cost of £250.00 per delegate. All of our course have a maximum of eight delegates so you'll get plenty of help / attention to your requirements from the tutor, and we guarantee that the course will run even if there's just one delegate booked. Which means that a "Learning to program ..." course is often a one-on-one introduction on the first day, followed by a group course of 3 to 5 people - a perfect learning environment!

Posted by gje at 08:52 AM | Comments (0)

June 10, 2009

Revision / Summary of lists - Perl

We've started the third day of this week's Perl course with a revision of some features of lists ... posting here as there are some useful reminders (and I came back later to add more comments!)

# Revision / Summary of lists - Perl
 
$abc = 16; # single value
@abc = (16,14,12,11,10,8,7,2); # multiple values - a list
 
print "Single value is $abc\n";
print "Single value from \@abc $abc[2] (third element)\n";
print "Whole list \@abc: @abc \n";
 
# ------ Processing lists as a whole via functions;
# ------ grep to FILTER
# ------ map to transform every element
 
print "process whole list ...";
# Regulr expression - a 1 follwed by another character NOT a 0 1 or 2
@teens = grep(/^1[^012]/,@abc);
print "Teenagers are @teens\n";
 
@nextyear = map($_+1,@abc); # Add 1 to each element
push @nextyear,1;
print "Next year ages will be @nextyear\n";
 
# ------ $#listname tells us top index number
 
print "You now have ",$#abc," Children\n";
 
# -------- Handling each element of a list
# ---- use "for" if I need to know the key no.
# ---- use "foreach" if I just want to process content
 
for ($k=0; $k<=$#nextyear; $k++) { # Start ; end condition; step syntax
  print "Child $k will be $nextyear[$k] next year\n";
  }
 
foreach $kiddo (@nextyear) {
  print "We will have a $kiddo y.o. child\n";
  }

We run public Perl Programming courses 4 to 6 times a year, and various advanced Perl courses regularly but a little less often - see here for details.

Posted by gje at 09:54 AM | Comments (0)

June 09, 2009

How important is a front page ranking on a search engine?

At the risk of researching and reaching an obvious conclusion (along the lines of "passengers will get upset if their train is delayed and they are not told why", which a half million pound report concluded the other week!), I've spent a few minutes taking a snapshot of old log files to see what proportion of search engine referrals were from links on the second and subsequent pages - on the basis that everyone says (assumes?) that appearing on the first page is critical, but no-one says how critical.

Drum roll ...

Of 14,000 search arrivals analysed, just over 500 were on the second or a subsequent page of results that someone pulled up, leaving 13,500 on the page they initially arrived at. So that's a 27:1 ratio.

Second result page: 283
Third result page: 73
Fourth result page: 46
Fifth result page: 20
Sixth result page: 11
Seventh result page: 7
Eighth result page: 3
And no more that 3 on any subsequent page

Very interestingly, 58 of the 500 or so 'subsequent pages' pointed to the first page, which (from a brief analysis) looks like people clicking back to the first page having visited a later one - the "I wonder what there is further down ... I've found nothing useful after all ... let's go back to that result I found on the first page ..." scenario perhaps?

This is just a snapshot - not scientific enough to be provide any general conclusions. Indeed, further analysis of data over a much longer period showed a 15:1 rather that a 28:1 ratio; an educated guess suggests to me that weekend visitors are more likely to look the second and subsequent pages than weekday ones. But I think there's enough data to say that - for the site which I analysed - front page ranking is indeed very important. Now I just wish I had an easy way of analysing the position on the pgae - what proportion of hits are 'above the fold'!

((Image above is in the public domain - from www.freeclipartnow.com))

Posted by gje at 05:51 AM | Comments (0)

June 08, 2009

Trowbridge - a missed opportunity? Melksham - into the breach?

Trowbridge is the county town of Wiltshire, and it a long and illustrious history, some magnificent Georgian buildings, the River Biss and a canal which brings tourists through the area already.

However, I've had to ask Lisa (who is a professional graphic artist) to 'Photoshop' my headline picture because - try as I might - I couldn't get a picture of the Biss Aquaduct without this ugly (IMHO) pipe in the foreground that was blocking the view.

I understand from a Wiltshire Time article that the developers of the Waterside Complex in Trowbridge have gone into administration, and there's considerable doubt now with regards to that "Transforming Trowbridge" project - something of a shame as there are parts of the town where such a transformation is desperately needed!

Interestingly, there *is* a market / place for an entertainment area / complex in West Wiltshire, and there was a suggestion made the other week that the area between Bowerhill and the A350 Semington bypass would be a sensible place for it - as indeed it would. Good access / the ability to provide plenty of parking (rather that being hemmed in within a town), and facilities to serve Chippenham and Trowbridge as well as Melksham - a catchment of around 80,000 people within a few miles. Ah - I'm probably doing no more that flying a kite here ...

Further pictures of Biss Aquaduct and Widbrook here

Posted by gje at 08:49 AM | Comments (0)

CSS Style Diagrams - working out where attributes come from

I have just been helping a customer who's got a problem with white text on a white background ... and hasn't been able to easily identify where the problem lies. Some of his code looks like this:

<div id="Layer6">
<div align="center" class="style11 style13">
<div align="left" class="style17">
<span class="style12">[Phone no]</span>
</div>
</div>
</div>

And I found myself struggling as well! The solution is to draw up a diagram showing the nesting and where each of the relevant attributes is sourced from ... filling in a table that looks something like this:

That will then give him (or me or you) a firm foundation on which to edit the source and experiment to confirm the analysis, which it turn will lead on towards a solution. I suspect the problem will become quickly apparent!

I made one further little suggestion - is there any way he could rename some of his styles? Using words like "style17" doesn't make them very self-descriptive, and so it is very easy for logic errors to slip in. In fact, I suspect that the customer's problem relates to the style called "style11 style13" which may be a typo. We tend to use words like "bodytext" and "headline" for our styles, and it makes the analysis much easier.

... "style11 style13" means that they are applying both style11 AND style13 to the div and it is correct syntax for combining classes.

That is one serious case of div-itis. Any way they could use HTML elements rather than so many divs? That too would help them to better understand their layout and probably cut down on the use of some many styles.

Posted by gje at 07:11 AM | Comments (0)

June 07, 2009

A (biased?) comparison of PHP courses in the UK

In answer to an email asking about alternative PHP courses ...

I'm the tutor of our PHP courses, and I also keep a keen eye open on the options. Occasionally, I do a 'competitive survey' and compare what we have to offer in terms of prices and facilities to other people offering training in the subject; it has been quite a while since I last did this, and your email triggered me to have a quick look around online at what's currently on offer. I came up with the following - looking at our 4 day course (which from your emails so far is the most likely one to be right for you) and finding in each other supplier's case the closest alternative.

Well House Consultants, Melksham - 4 day, 1100 pounds
iBuildings, London - 4 day, 1295 pounds
ForLinux, Newark - 4 day, 1200 pounds
GB Training, Bradford or London - 4 day, 1375 pounds
HOTT, various venues, - 4 day, 1095 pounds
London School of Computer Education - 30 hours, 750 pounds
City University, 20 hours, 350 pounds
Learning Tree, London - 2 day, 1125 pounds
Bromley College, Bromley, Kent - 1 day, 100 pounds
Pale Purple, Nr Birmingham - 1 day, 250 pounds
Credativ, Rugby - duration / price not quoted

Now ... those are LIST prices, and almost every training outfit will offer you discounts in some circumstances - we'll reduce the cost of the second and subsequent delegate on a course by 100 pounds, for example, and others have much deeper discounts - foe example, Learning Tree will sell you 6 places on 2 or 3 day courses for 4900 pounds, so you're looking at reducing their 1125 pound 2 day course to 816 pounds.

Most training providers reserve the right to cancel courses if they don't have enough confirmed bookings, and others schedule their courses "on demand" to try to alleviate the problem of thin, profitless courses. Two companies above - ourselves and HOTT guarantee that they'll run the course for you to a schedule, at the price quoted and undelayed. Our courses always run in Melksham, theirs in 9 centres from Bristol to Belfast and Edinburgh, and they schedule them on the same date in every place. So you may find you book Bristol, but have to attend in London, or Norwich ...

Courses of 2 days and less worry me. I don't see that there is enough time for practical exercises and in depth coverage of all the topics needed - I fear that too many delegates will be lost too quickly. If you're very bright and used to quick learning, staying overnight near the course venue to brush up, they may work well for you. And in one day, I don't think you'll get more than an introduction unless the pre-requisite list is "must be a genius". To be fare to Pale Purple, they do a whole series of separate and different 1 day courses and can probably advise which ones (plural) you would need to learn the whole subject.

The City University and London School of Computer Education courses are both 10 sessions spread over a number of weeks - evening courses or afternoon courses, which look like a very good idea if you work in London (perhaps you commute?), and The City University looks to be good value.

I can (of course ;-)) recommend our own Well House Consultants courses which in fact are just about the lowest priced of the commercial offerings, especially when you consider travel costs to other venues if you live in the South West. And my own degree comes from City University, who certainly 'did me proud' so I would recommend them.

Books and online training are other alternatives for you - I don't know which (if either) you personally find effective. We have a very large selection indeed of PHP books at our Melksham Training centre and you would be welcome to visit one evening, spend an hour or two discussing your needs, have a coffee, browse the library. And we would make no charge for that evening - I really like to help people work out what they need (do check dates with me - I sometimes have other evening meetings and often train on site)

Posted by gje at 08:38 PM | Comments (0)

June 06, 2009

Adding a newsfeed for your users to a multipage PHP application

As I wrote it, I realised this was turning into rather a long article, but never mind. It shows the major new components added to a "4 layer model" application I was working on the weekend before last to add in a newsfeed for logged in users to which they can contribute, together with a headlines-only display for none logged in members. The code may look longish, but it was written and tested very quickly.

If you are not familiar with the 4 layer model, start here or attend our PHP techniques workshop ... this is a more advanced article

In the particular multi page application I was modifying, page #2 is the main navigation menu, so the "finish processing stage 2" code at the top level will branch the user to many other pages in the system. Page #6 is where I added the news update:

case 2: // Navigation request made
  if ($_REQUEST[studinfo] == 1) $current = 3;
  if ($_REQUEST[studinfo] == 2) { $current = 4;
    $fill[topmsg] = "Please enter new user data"; }
  if ($_REQUEST[studinfo] == 3) $current = 6;
  if ($_REQUEST[studinfo] == 4) $xlevel = 2;
  break;

Adding data to the newsfeed is simply done by appending the data to a file, and I've checked when data is submitted that both a title and a body have been given (you'll see a comment requesting this when we come to the template form further down this example). I have used the string "::-::" as a separator / marker in the first line of each new news item in the hope that users will never add such a thing in their data ... and I'm fully aware that if I was writing something for "Joe Public" to enter data into rather that some benign and small club, this would be a bit of a hole; in such a case, I would store to a MySQL database, and within this code might have logic to delete all but the most recent 100 postings.

case 6: // Add to news feed request
  if (eregi("[a-z]",$_REQUEST[topbits]) and
    eregi("[a-z]",$_REQUEST[bottombits])) {
    $now = time();
    $fho = fopen("newsfeed.txt","a");
    fputs($fho,"$now::-::$_SESSION[username]\n");
    fputs($fho,"$_REQUEST[topbits]\n");
    fputs($fho,"$_REQUEST[bottombits]\n");
    fclose ($fho);
    }
  $current = 2;
  break;

In the 4 layer model, when you've finished handling the data submitted in the form on one page, you prepare for the next. I have added calls to a function I called newsfeed on all pages that I want to display a feed ... here is the example in the logic that prepares for a new news item, which reports a substantial amount of the old news to that the new article author can write in context. You will note that I'm putting the changing data into an array called $fill rather than generating the code directly - this is the element of the 4 layer model which allows me to separate the business logic from the look and feel - exactly what I need to do to allow for maximum future flexibility, and to allow me to pass the maintainance of code to programmers, and of the look and feel to graphic artists!

case 6: // Prepare for adding to news feed
  $fill[title] = "OurClub Adding to news feed";
  $fill[news] = newsfeed(2);
  break;

Here is the actual template ... for ease of understanding for the newcomer (i.e. to avoid ending up with a large number of files for just a demo piece) in this example, we wrote it into a "here string" ... in which we replace %-letters-% with whatever we have in the matching element of the $fill array:

$body_template[6] = <<<CLUB_TEMPLATE
<h1>Add a message to the newsfeed</h1>
Please give a headline and text of article. If you don't want to post
an article after all, just submit a blank page and it won't be added.
<form method=post>
Headline: <input name=topbits><br>
Text of article <textarea name=bottombits rows=20 cols=60></textarea><br>
When you are done ... <input type=submit></form>
<hr><b><u>The latest news feed!</u></b><br><br>%news%<br>
CLUB_TEMPLATE;

Finally, here is the 'business logic' which interprets all of the elements of the newsfeed and returns a string suitable for rendering. You'll note that even at a 'club level', I've added a number of coding / security checks - i.e. all the data the user has entered in the news goes through stripslashes and htmlspcialchars in order to render harmless any attempted injection attacks. What it does NOT do is provide any form of moderation - i.e. checking for news that breaks copyright, is libellous, adult, incites breaking of the law ... that remains a manual process.

function newsfeed($level) {
  foreach (file("newsfeed.txt") as $line) {
    $lp = explode("::-::",$line);
    if (count($lp) == 2) {
      $cur_news = $lp[0];
      $cur_repo = trim($lp[1]);
      $cur_lines = -1;
    } else {
      if ($cur_lines == -1) {
        $art_title[$cur_news] = $line;
        $art_autho[$cur_news] = $cur_repo;
        $cur_lines = 0;
      } else {
        $art_text[$cur_news] .= $line;
        $cur_lines++;
      }
    }
  } // End of feeder reader
  krsort($art_title);
  $response = "";
  $narts = 3;
  foreach (array_keys($art_title) as $tstamp) {
    $tsf = date('H:i \o\n l jS M Y',$tstamp);
    $an++;
    if ($level == 0) {
      $response = $art_title[$tstamp];
      break;
    } elseif ($level == 1) {
      $response .= "<br><b><u>$art_title[$tstamp]</u></b> by ".
        "$art_autho[$tstamp] at $tsf";
      if ($an > $narts) continue;
      $response .= "<br><br>";
      $response .= nl2br(htmlspecialchars(stripslashes("$art_text[$tstamp]")));
      if ($an > $narts * 3) break;
    } elseif ($level == 2) {
      $response .= "<br><b><u>$art_title[$tstamp]</u></b> by ".
        "$art_autho[$tstamp] at $tsf<br><br>";
      $response .= nl2br(htmlspecialchars(stripslashes("$art_text[$tstamp]")));
      if ($an > $narts * 3) break;
    }
  }
  if ($level == 1) {
    $response .= "<br><a href=\"?studinfo=4\">show more</a>";
    }
  return $response;
}

Posted by gje at 11:42 PM | Comments (0)

June 05, 2009

Melksham Traders - where do we go?

Some people tell me how quiet Melksham in these days, and other tell me that it's bustling. The picture above shows that it's somewhere between the two - there's certainly a reasonable number of people about, but we could do with more.

Some shopkeepers / shop owners are concerned at the effect that a new out of town supermarket (Asda) will have on them ... others report that in previous places they've lived, spin-off business from out of town stores has rejuvenated their places of work and the work centres too ... "People no longer go to Northampton" ...

Some businesses tell me how hard hit they were when the road through the town was closed for works. Others wish that the section outside their places could become pedestrian areas.

Melksham has a very wide range of town centre businesses - and a very wide range of views. In my role with the Chamber of Commerce, I have been meeting many of them and I'm scratching my head as to how we can and shoudl work together, yet meld these differing viws. And we have other differing views too - with regards to topics like the re-opening of the Canal / improvement of the river frontage.

Yet scratching my head does seem to work and bring *some* areas of Unity. The town traders / business map that we're working on had support from nearly all. The call for an improved rail service brings opposing sides on the other arguments together.

Melksham has some wonderful stores - some new, some old and old fashioned (like "The Toy Shop" pictured above), where you can go back into what seems like a past age and browse wonderful stacks of toys - and people do, and they buy - and this wide selection gives Melksham a bright future.

Then there are always the characters, and the character signs. Have a look at this picture I took yesterday, and see if you can stop anything odd in the signs

Posted by gje at 04:34 PM | Comments (0)

Configuring httpd, or Tomcat, to run CGI scripts in Perl

Here's a Perl / CGI script - a "Common Gateway Interface" program that I want to run on my web server, and access via my browser. What server to I need to run it on, and how do I set that server up?

#!/usr/bin/perl
 
print ("Content-type: text/html\n\n");
$now = localtime();
 
print "<h1>It really is<h1>$now";
 

You can run CGI (and through it Perl Scripts like the one above) on either the Tomcat OR the httpd servers. It will run on httpd, and it will walk on Tomcat as it's in a Java servlet container ;-) ... but if you've only a bit to do, you choose.

How do we set 'em up?

Tomcat

1. In the conf/web.xml file, enable the cgi servlet and also allow for all URLs that start /cgi-bin to be routed to it.

2. In the server/lib directory, rename servlets-cgi.renametojar to servlets-cgi.jar

3. In the WEB-INF folder in your web application, add a cgi folder and place your CGI script in there.

4. Stop / Restart your Tomcat.

5. Try it out! http://whereverserver:myport/appname/cgi-bin/script.pl (and you can route it via httpd and mod_proxy and balancer too if they're in use!)

httpd

1. In the conf/httpd.conf file, change the ScriptAlias for cgi-bin to point to the directory you'll be using for CGI - usually a sibling to your htdocs directory.

2. In the same file, set the Options for that directory to include ExecCGI

3. Create the directory for the CGI scripts if it doesn't already exist

4. Add a CGI script to the cgi-bin folder, making sure that it's executable!

5. Stop / Restart your httpd

6. Try it out! http://whereverserver/cgi-bin/script.pl

Some Interesting Extras!

1. You can point both configurations to the same cgi directory, so that scripts can be run by either httpd or Tomcat (If you do the same with your web applications, it means that a single web application ans all its support files can be kept in the same place, irrespective of which server serves which parts.)

2. You can have one instance of httpd forwarding to other instances of httpd through mod_proxy_httpd and mod_proxy_balancer, allowing a single domain name and port to be served by multiple machines through a single front end. A great solution when the traffic load is too high for one machine, but too low for an sophisticated solution such as a hardware load balancer.

We cover the setup of Apache http on our Linux Web Server course and the setup of Apache Tomcat on our Apache httpd and Tomcat course.

Posted by gje at 03:32 PM | Comments (0)

Multiple web applications under Tomcat - what are the options?

If you're wanting to run a number of web applications under Apache [Jakarta] Tomcat, do you need multiple versions of Tomcat, or multiple machines? No, you don't ... but it may be helpful for you to do so. Here are diagrams of four options (open source, remember - "never provide just one solution where you could provide four!"):

Starting on the right, you have:

• Multiple Web Applications on a single Tomcat. With this option, each web application will have its own directory name on the same domain, or you could use virtual hosting. If the metrics of the applications (busy periods) are at different times of day, this approach makes useful sharing of facilities, and if there's a lot more running on the computer and not much on the Tomcat, it saves a lot of duplication. However, if you stop / restart tomcat then you're going to effect all the web applications at once.

• Several Instances of Tomcat on a single machine. Here, you're able to stop and start each tomcat independently, but you'll need two different IP addresses or TCP port numbers, and both will stop / restart if you stop / reboot your operating system. But this would be a commonly used approach for an organisation with a quite small / specialised / departmentalised Tomcat setup, with a need for a test / development system and a live system too.

• Virtual Machines. Becoming a favourite at ISPs and where a computer department is forming a company's ISP ... a series of virtual machines each on the same physical host.

• Two separate Computers. Computers are cheap these days ... and this is how I work on our own web site - with the main web server being a totally different machine to the one we use for R&D and testing. You may tell me that I'm old fashioned in not using virtual machines, but the production machine we have is sized up to meet demand and is located on a stinking fast connection in London, and the development / testing machine is a laptop which I can carry on developing on during a train journey or - as I was last week - on a boat between Cairnryan and Larne, where even mobile internet was out of reach. And switching the computer off as we arrives at Carinryan and I drove down to Wigtown and beyond didn't stop our live system! [[Illustration - Wigtown, Book capital of Scotland!]]

(Diagram at top of page taken from the board of the Deploying Apache httpd and Tomcat course I am presenting at present.

Posted by gje at 10:22 AM | Comments (0)

June 04, 2009

Enjoying the summer weather

The long, long evenings and the lovely weather are giving us some great opportunities to walk; here's Gypsy, with 200 yards of our home, in the fields. Having been with us for about six weeks, she's now well attached and can be let off the lead in appropriate places - and on the way up to "The Foresters" via footpaths and Snarlton lane, that's all the way from our door until the last few hundred yards.

The cluster of businesses near the Sandridge Road / Blackmore Road / Snarlton Lane / Church Lane corner has changed (in a little detail) in recent times ... we now have
• The Forester's Arms (Pub - local, but no longer food every lunchtime and evening)
• The Co-op, with ATM outside. A mid-sized co-op, open for a wide range of goods, early to late
• A Petrol Station
• The Forest Fish Bar (formerly "Tricia's Place"). Tricia was a past master (or should that be "passed mistress") at fish and chips, but sold out to a couple under whom quality fell away to the extent that we transferred our loyalty to "The Fisheries" in Union Street, in spite of the parking there being difficult and having to fight through the town. However, a new name and (it appears) a further ownership - or at least managerial - change, and I'm glad to report that "The Fish Bar" has improved again - at least enough for me to welcome the opportunity to pop in there when I'm walking through that part of town.

Posted by gje at 10:35 PM | Comments (0)

June 03, 2009

Past Delegate Offer - Summer Holiday / Weekend Break

Why not get away for the weekend or a short summer break? There's lots of lovely countryside, and plenty to do, in Wiltshire - and it's only 100 miles from London.

Stay with us at Well House Manor in Melksham this summer and explore the area - we're just a few hundred yards from Melksham Town Centre, about 3 miles from the National Trust village of Lacock, 5 miles from Corsham, 6 miles from Devizes, 8 miles from the Westbury White Horse, 10 miles from Bath, 14 miles from Longleat and also from the stone circle at Avebury and Silbury Hill, and less than 20 miles from Stonehenge.

Well House Manor is our Business Hotel and training centre - all bedrooms are en-suite, spacious, with large desks, mini fridges, laptop safes, 50 channel large screen TVs, wired and wireless internet. You can use the hospitality tray in your room (tea, coffee, chocolate) or help yourself to freshly ground coffee and soft drinks in our customer lounge, 24 hours a day, and browse our library of some 800 books. A superior continental breakfast buffer is offered from 7 a.m. (8 a.m. at weekends, earlier on prior request).

All of our rooms are double [superking beds] or twin; the rate of £80.00 per night (single) or £90.00 per night (double) includes all the facilities we've listed above. If you've attended a Well House Consultants Course at any time in the past, or a First Alternative course that I presented when I worked there, we'll extend the delegate rate of £70.00 per room (single or double occupancy) to you for any stay that includes a Friday or Saturday night ... and also for "high summer" stays - from now until 6th September.

Book online here (enter "delegate" in the coupon code if you have attended or booked a course) or call us on 0800 043 8225 (freephone) or 01225 709638, or email info@wellho.net.

If you're a past delegate and you'll be bringing some technical questions with you, please let me know ahead of time ... and I'll do my best to make sure I'm around sometime during your stay.

We have plenty of parking, including secured covered areas for cycles and motorbikes, and we welcome arrivals by public transport - let us know if you're arriving in Melksham on the evening train or coach and we can collect you; the regular Bath buses (272 and 273) stop right outside. You are very welcome to bring your own wine and beer (off-license, 2 doors away, we provide openers) and phone in for Indian, Chinese, Pizza or other food.

Well House Manor
48 Spa Road
Melksham
Wiltshire SN12 7NY

http://www.wellhousemanor.co.uk

0800 043 8225 / 01225 709638

info@wellho.net

Posted by gje at 07:59 AM | Comments (0)

June 02, 2009

If nothing, make it nothing.

Here's a peculiar PHP statement I wrote this morning:
  if ($aindex[$chk] == "") $aindex[$chk] = "";

What was I doing? I am actually checking to see whether there's anything in an array element and if there isn't (which could be because it's empty or because it doesn't yet exist), then it should be created if necessary and made to contain an empty string. In PHP, = = tests whether two expressions have an equal values, but = = = tests whether they are of the same type and with equal value.

The important aspect of the assignment that's made above is that it ensures that the array ends up with an element who's key is the contents of $chk, so that later on in my code I can go though all of the items using array_keys even for $chk values for which there is no data.

Posted by gje at 12:38 PM | Comments (0)

June 01, 2009

Global Index to help you find resources

How can you find all blog articles on Ruby? How about everything that I've written about looking after forums, and with all the information about spammers close to it? With 15,000 URLs on the site, these apparently easy questions aren't so easy any more.

However - In an attempt to help (largely to help myself during courses!) I have added in a Global Index Page that lists all of our modules and categories about which I have blogged, and also lists longer examples and source code links too. The idea is that I can quickly scan up and down for what I want ... in a module and in nearby ones ... during a course, rather like the sidebar we have when projecting a .pdf document.

The global index page is a huge one; don't try it on dialup - but as all the links are targeted to a new window, it can remain displayed for hours. There ARE quick links to jump within the page, and if you've been on one of our courses and have a manual, the module number is the number in the top left of the page.

If you wondered why there was a series of obscure technical blogs this morning .. that's because I have started to add blogs about subjects that I train on, but haven't spoken to as of yet. Give it time, and perhaps the whole could turn into a book!

Posted by gje at 07:21 PM | Comments (0)

Keyboard reading in Perl - character by character not line by line

If I'm typing in my age in response to a prompt and I start with a "6" character, will you assume that I'm between 60 and 69 years old? You might ... but then I may backspace (having accidentally struck the "6" key and start with a slightly lower digit. But in my program, I do NOT want to have to write my own code to deal with user inputs / errors of this type, and that is why most programming languages read up to and including a new line (a "commit") after which they return just the "cooked" or cleaned up string to the user.

With Perl, "you can do anything" ... you may have heard that said before ... so it is possible to read character by character, using the underlying C language libraries "ioctl" and "fcntl". Which makes it operating system dependent. We have an example here ... and you'll find other examples of clever uses of Perl's IO on that page too, such as turning off echo for password entry, giving the user just a limited time to reply to a prompt, and checking whether or not an input is available before proceeding, rather than locking in a wait state for the keyboard.

Posted by gje at 09:32 AM | Comments (0)

Dripping taps, Java threads and the single thread model

Have you ever lain awake at night and listened to dripping taps. "drip, drip, drip" ... until you eventually fall asleep? With one tap, it's rhythmic, but with two taps the two rhythms get intertwined, and result in an apparently patternless racket. If only one tap would wait for the other, it would sound so much better!

With Java threads - that's where you have several program streams running within the same program - you have a very powerful facility, but also a need co-ordinate when you get to points where a shared resource needs to be used, and that's where you use Java's synchronized keyword, and you synchronise based on an object.

Threads and synchronisation are used within Tomcat to handle multiple connections at the same time, and they're used very effectively and hidden from the programmer / Java developer. But the Java developer does need to bear in mind that there may be several user all in his code at the same time, and if there's a danger of such users interfering with each other he should consider the single thread model for his Servlet. In a nutshell, this holds each user of his application at the starting gate until the previous user has completed - it reduces the maximum possible throughput, but on a quiet to medium application, it can make the coding and testing very much easier!

Posted by gje at 09:22 AM | Comments (0)

Tcl/Tk - why does the canvas widget use a display list?

"Tcl/Tk uses a display list when you draw on a canvas widget." OK - so what does that really mean?

With most of the languages that we teach, any graphics you draw go straight into a frame buffer of a memory map that represents a frame buffer, so the earlier elements added to your graphic will be destructively overwritten by any later elements which happen to land on the same pixel. You have to be careful with drawing order, and if you need to remove a later element, you need to have a way of repairing the hole / repainting part of the screen when you do so

With Tcl/Tk, though, your elements are added into to a separate list in memory, in which all the graphic components are stored, and from which they can be replayed as and when required. This allows - you - for example to pick up and drag a component of your display without the need for a repaint.

Once again - Tcl and Tcl/Tk may be the oldest scripting language(s) that we teach, but they are very very neat and clever!

Posted by gje at 08:59 AM | Comments (0)

MySQL server replication

Following on from yesterday's article about MySQL tuning ... there may come a time that a single server can't cope. What should you do? Consider replicating your server. This works well if you've got an application which is "read mostly" as you can then have clients reading from a series of machines, but isn't a great solution in a "write mostly" environment, as all write need to be made to the master machine. Good news is that "write mostly" environments are rare, save for logging applications.

Replication also provides additional robustness ...

There's a longer article on our web site here which tells you how to replicate your server.

Posted by gje at 08:47 AM | Comments (0)