« March 2006 | Main | May 2006 »
April 30, 2006
Disc Partitioning
When installing Linux, you're able to choose how to split up (partition or slice) your disc(s) and which part of the file system to put on each. Here's a partitioning algorithm that we came up with for an organisation that has a variety of servers.
WHAT do you want on separate partitions?
/ - duh, yes, must be a partition name
swap - ANY chance of running out of memory, have a swap area!
/var - if mail system, printer controller, etc
/usr - if you're looking to share /usr (not likely these days); also worth
separating if you're doing a dual boot and both OS's need to start before the the 4Gb mark on the disk.
/usr/local - if you'll be adding lots of extra s/w and want to upgrade the OS later on without reloading all your local extras OR if you want to NFS / Samba share your /usr/local
/tmp - if you have lots of meaty applications that write / read huge temporary files
/home - if you have any local users apart from admin's accounts
/boot - if you're acting as a tftp boot server; also perhaps because grub is here
RULE in general - use separate partitions for things that have different backup strategies, and things that you don't want to crash the system when the users get them full.
How big should they be?
$64000 question
/ - Need not be too big - 2Gb should be twice what you need
/usr - Build is typically 2Gb to 7Gb - allow, say, 9Gb.
/tmp - Allow, say 2Gb. If you're running an A0 pixel mapped plotter though, you'll need a *** of a lot more space, or if you're doing fluid dynamics, seismic oil data processing.
/var - Allow 1Gb PLUS space for mail, print queues, etc - see /tmp
/usr/local - How much extra s/w are you likely to add? Allow 2 or 3 Gb - lots of s/w, especially Java to add? Allow an extra 4Gb on top ...
swap - authorities say "twice the size of your memory"
/home - as much space as you have left!
Add together the various amounts if you're not having a separate partition for each ... so
Typical system, 512 Mb memory
/ - 2 + 9 + 2 + 3 = 16 Gb
swap - 1 Gb
/var - 3 Gb
/home - the rest
* Do NOT put swap at the very end!
* Multidisc systems - spread swap between all discs with fastest r.p.m.s
* Tend to have / and swap as primary.
Posted by gje at 08:32 AM | Comments (0)
Related topics: via article database
More about Graham Ellis of Well House Consultants
April 29, 2006
Erin Brent - rest in peace
"Come as a student, leave as a friend" we say on our coffee mugs. And so it was with Erin Brent, who came to us on a Tcl course 3 or 4 years back. Lisa and I stayed subsequently with Erin and Richard in Oxford, and they visited us in Melksham for a weekend. She returned to Australia early last year, where she passed away towards the end of the year. The news came as a shock to me - a chance comment from a delegate on my course. Glad I know but deeply, deeply saddened.
I last saw Erin at the Oxford University IT Staff conference about 6 months before she left the UK. In all the mad hubbub of the coffee break, Erin pressed a book that she had been telling us about into my hand - specially bought and brought in - and disappeared off cheerfully into the crowd to let me talk to a group I had around my exhibition table. My condolences go out to Richard; they understood each other, and I cannot imagine what he must have gone through and how he must feel.
Posted by gje at 05:51 AM | Comments (0)
Related topics: via article database
April 28, 2006
Copying files and preserving ownership
If you're copying a file on a Unix / Linux / OS X operating system, use the cp command. Use cp -r to copy a directory and all its contents - the -r means "recursive". If you use the cp command to copy files that you have read access to, but do not own, then you'll be made the owner of the new copies; for the command to do otherwise would have security implications.
So what if you need to copy whole directory structure and retain the current user and group assignments? First, you'll need to log in as the system administrator (root). Use the su - command and be careful!
Once you're logged in as root, change to the directory that's above the one you wish to copy, and copy via the tar command. For example, to copy everything in /home to /second/home:
cd / mkdir /second tar cpf - home | (cd second; tar xpf -)
The tar utility is more often used to make an archive file containing all of the information in a directory or series of directorys.
• Specified with the c parameter, tar creates an archive and with the x parameter, it extracts from an archive
• With the f - option, tar is instructed to write to stdout or read from stdin.
• The p option preserves ownerships on extraction, to prevent all of the files being changed to the ownership of the person running the command ... and for reasons of inter-user security, this only works for root.
Posted by gje at 08:53 AM | Comments (0)
Related topics: via article database
April 27, 2006
Iterators - expressions tha change each time you call them
If you're programming and you write the same expression into your code twice without changing any of the variables, you're simply writing the same expression twice, right? For example, here's a piece of Perl code that exits if $userval is 0, but adds it in to a total and keep going if it's not zero.
if ($userval == 0) {
print "job done\n";
exit;
}
$total += $userval;
If you're reading from a file, though, things are different. Refer to the same file handle twice, and you'll get the next line read in each time - thus:
if (<FH> == 0) {
print "job done\n";
exit;
}
$total += <FH>;
will read from a file handle TWICE - if the first line read is zero, the program exits. If the first line was NOT zero, a second line is read and that value in that line is added to $total.
Why the difference? Because the read from operator - <> in Perl - is what we call an iterator, and each time we refer to it, it moves on to the next value.
This one often catches new programmers ... the solution is to read the value just once and save it into a variable - thus:
$userval = <FH>;
if ($userval == 0) {
print "job done\n";
exit;
}
$total += $userval;
Posted by gje at 06:21 AM | Comments (0)
Related topics: via article database
Minature Cyclists only - limited headroom
I don't know if any of you have come across Warrington Cycle Campaign's facility of the month showing off some of the thoughtless designs for cycle lanes .... but I came across my own example in Oxford earlier this week.
In our current planning for Well House Manor, we're planning to provide a bedroom with wheelchair access - but what about other provisions for customer's needs while we're at it? An excellent suggestion was made yesterday that we should not provide white switches on a white background for people whose sight isn't the best, that we should similarly provide grabrails in a tasteful colour (something like pink on dark blue was mentioned rather than the high contrast institutional bright yellow), and that the fire alarms shouldn't follow the modern fad of having a recorded voice saying "Fire, please leave the building" in case we have visitors whose English isn't good enough to understand the alert.
Posted by gje at 03:34 AM | Comments (0)
Related topics: via article database
April 26, 2006
As I walked in to Oxford
I walked along the tree-lined Banbury Road into Oxford this morning, wide pavements, walkers, cyclists, blossoms and budding trees, sun shining. A truly beautiful morning. Even the traffic was flowing quite well for Oxford.
A flash, and two small birds darted out playfully from the trees - ah - the courtship of spring. Uncaring in anything but love, they flew straight in front of a car and fell, one dead, one dying to the road.
Posted by gje at 01:54 PM | Comments (0)
Related topics: via article database
April 25, 2006
Extremes costs of getting on line
Where I'm working, they have a setup for visitors to use their wireless network; just put in the network name, browse and you get a login screen. Logged in, gave password, and on line. And a clear statement of the terms and conditions and security issues. And it's part of what they provide for all provide for visitors, without charging them.
Where I'm staying, I can use their connection at 15p per minute.
I haven't investigated where I'm staying any further - how it works - I can get by very well indeed from where I'm working.
Posted by gje at 10:13 AM
Related topics: via article database
April 24, 2006
Catch up weekend
It's been a long working weekend - after last weekend's trip to Halifax, Haworth and Crich
there was a great deal to catch up on, and training on JBoss and LDAP, httpd and Tomcat precluded me from doing so during the week.
I expect you're wondering what the picture is? We often have to look at things from a different angle, and this was an angle I noticed in the Tramway Workshop at Crich. I think there's a further lesson in the workshop, where you can see all sort of skills coming together in different enthusiasts - taking a tram that looks to be beyond hope and restoring it to pristine condition.
The week starts again in half an hour, just as the previous week seems to have finished. I'm off to Oxford to present Perl and Linux courses there. Back on line with another "Horse's Mouth", internet access willing!
Posted by gje at 05:38 AM
Related topics: via article database
April 23, 2006
For any North and West Wilts and Swindon Readers
Can I ask you, please, to write by email or fax to your MP (Anne Snelgrove in Swindon, James Gray in Chippenham, Michael Ancram for Melksham and Andrew Murrison for Trowbridge, Westbury, Dilton March and Warminster) and encourage them to attend the debate in parliament on Tuesday - on the subject of GREAT WESTERN FRANCHISE. This might be the last realistic chance to get changes to the draft timetable that withdraws all through trains on the Swindon to Salisbury line, and replaces them with a minimal service under which line usage would wither. Some 40% of our customers arrive by this train at Melksham
There's a sample here - the letter I have written to my own MP, and more general notes on the proposals here
If you're one of our many customers who doesn't live in the area but has come to Melksham by train in the past, perhaps you could help too be emailing one of the MPs listed above or your own. Even if they reply "not my constituent" to you, at least you have raised the visibility of this issue with them. Thanks!
Posted by gje at 08:44 PM
Related topics: via article database
Is than an uphill or downhill task?
Someone was telling me a joke about the [nationality]ish Bobsleigh team at the Winter Olympics, who insisted that the track was gritted for them before they would take the Bob down it ....
What a tiny difference there is between a gentle upward slope and a gentle downward slope! On a downhill slope, push out the Bob and it gathers speed and it's own momentum but uphill, it's push, push push all the way. On so many things we do, there's a knack of recognising which way the slope lies and choosing those routes that won't call for constant input to keep them rolling.
Posted by gje at 02:56 PM
Related topics: via article database
In the hospitality business
We sometimes describe our training centre as a "daytime hotel" - we're very much in the customer service business here. There will be tea, coffee and cold drinks available when you arrive, the day's newspaper and the TV for you to "chill" if you've arrived early, or plenty of workstations available and connected to the internet through our 10 Meg connection if you want to check you email, link into your company's VPN, or carry on with a practical you had started the previous day.
Throughout the day, drinks and snacks are on hand, and we provide lunch too - sometimes at our training centre, other times at a local pub or restaurant. As all host should, we'll listen carefully to dietary requirements, likes and dislikes and we can tailor what we provide to suit the whole group. With a maximum of just seven delegates on a public course, and the course having exclusive use of the training and lunch facilities, there's always plenty of food to go around. Over lunch, your tutor's perfectly happy to discuss your application and wider topics that aren't on the course - anything from PHP for personal pages / sites on a Tomcat course through to telling you about how we restored the old Spa house and turned it into our training centre.
At the end of the day's training, delegates often stop around to check their email, to chat further, and to enquire about where to eat in Melksham or their best route home. Come the long days of summer, we've been known to join the delegates and walk along the canal to "The Barge" at Seend - such is the friendly atmosphere of almost every course ... and if you happen to be the only one staying in town for a longer course, we'll make a point of inviting you back at least one evening. We love it!
Come the end of the course, we're on hand with directions for the best route to the M4 ... or M5 ... or we'll run you across to Melksham station for the train, or Bristol Airport for your flight. And the service doesn't end there; after your course, you'll know where all the resources are on our web site, and you'll be welcome to post questions on the forum or by email to your tutor. Some of you will enjoy it so much that you'll come back for Geekmas, which we celebrate in November. Ah - I should leave the detail until nearer the time.
So really we're trainers and hoteliers ... it's just that we haven't, just yet, got the bedrooms and breakfast service running.
Posted by gje at 09:36 AM
Related topics: via article database
April 22, 2006
Ant and Make
In order to produce a file of answers in many applications, you need to take several files of inputs and run a program to generate the results. The inputs themselves may be answers produced by running further programs on further inputs, and so on until you have a whole huge tree of inputs that are used to produce a single (or relatively few) file of answers.
What do you do if an input is changed? Well - you have to run all the programs again and produce fresh intermediate answers, from which you can go on and produce new final results. If it's something you do frequently, you can automate the whole business through a shell script or batch file.
But in a big system, you'll be repeating a lot of needless work in your shell script on many occasions. If only one input has been altered, it's unlikely that all the branches of your tree will need to be rebuilt - you'll really only need to rebuild the sections of the tree below where the inputs have changed and to do the whole thing could be a huge waste of resources.
Both Ant and Make are tools that help you do only the work that you need. In both cases, you specify the structure of the build tree though:
• "target"s - the answer files - at all levels - that you'll be writing
• a number of "sources" for each target - input files which, if changed mean the target must be remade
• actions (commands) that need to be run to produce the target from the sources
Then, when you run your ant or make, it compares the timestamps on the source files with those on the current target if it exists, and runs the actions if necessary. Missing targets will always cause an action to be run, and targets that become new sources for further actions will trigger off that further action as well.
If you've defined your tree well in a file called build.xml (ant) or makefile, all you need do to modify your answer file is to make any appropriate changes to the input files and type ant or make. Might take a while to set up in the first place, but once you've got a system running, the efficiency is unbelievable!
Ant is a Java tool and is used extensively in creating Java applications for the web - files such as .jar .war and .ear resources - for deployment through Tomcat, JBoss and other tools. Make is primarily a Unix / Linux tool that's traditionally used for building programs from sources, and installing those programs
Posted by gje at 08:06 AM
Related topics: via article database
April 21, 2006
The on line booking experience
I want two single rooms, en suite, in the centre of Oban, for the nights of 30th April and 1st May. Easy enough? Well, you would think so.
The first hotel site didn't tell me whether of not they had singles. But I didn't bother to take that any further it wasn't easy to find exactly where they were in the town either, and I don't think that Dad and I will want the included five course meal each evening either.
The second hotel site's booking page caught me out by having a default year of 2005, and then when I tried to book for this month insisted I should complete my stay by 30th March 2006.
The third site insisted that we wanted to share a room - kept knocking the "2" I entered back down to 1.
The Palace Hotel looked better, and I clicked through to reserve. "2 Rooms, Palace hotel, Ness Bank, INVERNESS" it came back. Err no - I want Oban. Clicked the back button. Sure enough, the Oban hotel description page links to the booking page for the Inverness hotel of the same name. Ach - what's a hundred miles of so between friends?
I take my hat off (or would if I was wearing one) to the "Falls of Lora Hotel" for a nice website that showed off the hotel, and the rooms, and gave me the details I wanted. No faffing about with on line booking - they've got an enquiry page and as I wanted to get this sown up tonight, I've phoned and booked. The phone answered in one ring, the lady I spoke to went through what she had (and hadn't) got for the nights, and I'm a happily booked camper. Oh - if you know that part of Scotland, you'll know we're not in the centre of Oban. Much better the experience of a caring hotel in a lovely spot that forcing our way through a system to stay in the town.
Posted by gje at 12:54 AM
Related topics: via article database
April 20, 2006
Healthier eating
As part of our new hotel venture, Lisa and I are looking to take out life cover. We both went for medicals the other week, and I've now heard that they would like to rerun some tests on me; a couple of things like a slightly raised blood pressure and cholesterol level. Fair enough - and as the doctor's letter says (and research and the family expert agrees) these things are prone to go up and down and first tests err on the side of caution. But, nevertheless, a bit of a wake up call and a trigger to cause me to look at a few life-style / diet issues.
So ... out go:
Caffine (tea, coffee, coke ...)
Red meats (beef, lamb, pork)
Alcohol (not sure why - I only drink a very little and advice is somewhat contradictory)
Dairy products (eggs, milk, cheese)
Yes, I know - there's thousands of diets and schemes around and I'm very new to this. I see also references to fizzy drinks, salt, sugar and other things not good for one. But then I'm not looking to loose weight which so many of these schemes are encouraging ... a new minefield to learn.
Will keep you posted here from time to time - I'm nearly a week into this now, and apart from a slight headache one morning which was I put down to an abrupt lack of caffine, no downsides to report. Whether or not it'll make any difference for my retests on 4th May, I just don't know; I suspect not much. But then, I'm looking at a long term change here and the life policy is just a trigger. I expect I'll ease up on the regime a little after a few weeks and settle down into what should be a full but healthier pattern than perhaps the one I've been following in recent years.
Posted by gje at 06:32 AM
Related topics: via article database
April 19, 2006
Testing you Perl / PHP / MySQL / Tcl knowledge
There are some aspects of our work - such as training and helping people specify and plan their systems - that have a positive aura about them and are naturally forward looking. Yes, we celebrate those jobs.
Other aspects, such as support and repair work, start off with a natural "down" - we're called in because of a problem, but chances are of a positive outcome in the end once we get the thing fixed or suggest ways forward. Again, great work if a bit of a gamble on the outcome - some things that people want to do just ain't practical or within their knowledge, they're unwilling to put time and perhaps money aside to learn, and they go away disappointed.
When we put up our on line quizzes way back in 2001, we didn't look at them as anything particularly different - a nice way to help potential training delegates find out what level of course was best for them, or if they knew most of it anyway and could save themselves the money. Well - that was the idea.
Alas, if a user gets 12 our of 15 questions on a quiz right, he will often look at the three he got wrong in great detail. Perhaps he'll feel cheated if the question can be mis-read in any way, or if there are any shades of grey at all in the answer. So the quizzes become quite high maintenance with us having to explain and justify questions rather more often than I would like ...
We have seriously discussed "pulling" the quizzes - and it may happen at some stage - but I do see them used by about a dozen people a day. I've upgraded the look and feel of all four quizzes this morning, I've added a reminder at the top of each of them that questions get dated and we would appreciate updates, and I've given them what is - I hope - a new lease of life.
Please try out the quizzes ...
PHP Quiz
Perl Quiz
Tcl Tk and Expect Quiz
MySQL Quiz
and the quiz index and FAQ is here
Please email me - graham@wellho.net if you spot anything that's ambiguous or out of date (or plain wrong!)
Oh - and do YOU have any good questions you think I should add? Please let me know
Posted by gje at 08:03 AM
Related topics: via article database
Useful links: Perl training, PHP training, Tcl training, MySQL training
Adding Virtual Hosts
If you're running your own server and want to look after multiple domains, you can easily do so using Apache httpd's Virtual Hosting. It's as easy as 1-2-3-4 ...
1. Create new host names in your DNS
2. Create home directories and a test index.html
3. Set up virtual hosts in httpd.conf (redhat) or vhosts.xxxx.conf (SuSE) - detail
4. Restart Apache
(Quick summary from yesterday's training ... thought it would be nice for some of the more technical amongst you to see it all in one place. If you.re not yet that technical, but wannabe, have a look at our deploying LAMP course or our associated resources)
Posted by gje at 07:07 AM
Related topics: via article database
April 18, 2006
The call of nature
Can anybody tell me why it is that when a member of the family gets home, they're desperate to get into the house to go to the loo, but when we get home the cat is desperate to get out of the house, probably for the same purpose?
Posted by gje at 10:26 PM
Related topics: via article database
Are you free on Friday evening?
I try not to "mix my interests" too much, (full story on other web site) but many of you will be aware of the fight that's going on in Melksham to help save our train service from reduction from an already 5 trains each way per day to just 2. Not only would such a reduction effect a over a third 
of Well House Consultants trainees but the knock on effect on commuter traffic to Swindon, and journeys across the county such as Chippenham to Salisbury would be severe; the A350 road that's parallel with "our" part of the line is already overcrowded, and ticket sales have risen 35% per year in each of the last five years.
Well - we have an 11th hour meeting this Friday evening - 21st April, arrive from 18:00 at Bridge House, Stallard Street, Trowbridge . The local MP for Trowbridge and Westbury will be there - and also many councillors, representitives of the First group who are making the cuts at the behest of the Department for Transport.
Open meeting - anyone welcome (and I would personally double-welcome your support); subject being cuts on both Southampton -Swindon (our service) and Portsmouth - Bristol and South Wales where there's a cutting back to an hourly service and reduction of train lengths planned - in spite of current overcrowding.
It would take something of a miracle to get the cuts reversed at this stage, but it's worth the try; if we don't, scenes like the illustration - the 17:02 from Melksham heading up to Chippenham and Swindon - will be of historic interest only after this December.
Posted by gje at 06:58 AM
Related topics: via article database
April 17, 2006
Presentation, Business and Persistence layers in Perl and PHP
I've been writing about the Presentation, Business and Persistence tiers (and within each of them the MVC or Model, View, Controller or MVC structure) for a JBoss presentation I'm doing this week - but I scarcely expected to find myself putting them to good if unusual use this evening when sorting out an old page on our web site which had - err - laid neglected in a Perl script since before the last site update. How to fix this without re-writing what's actually a very nice piece of Perl ...
I've designated the Perl to be the business layer, and I've added an "Internal" interface type to it as well as the "post" and "get" interface.
In my new-look PHP page, I call up the Perl script through that internal interface passing in parameters as appropriate from the PHP using a shell_exec call. The internal interface returns a result set which I can then integrate in my presentation layer.
Neat - and what could have been a major rewrite was done in an hour. I've always said that the Java principles are great for the larger applications / sites and I'm glad to have had an opportunity to apply them!
Posted by gje at 09:21 PM
Related topics: via article database
Useful links: Perl training, PHP training
April 16, 2006
Name Services - telling your LDAP from your DNS
If you're just logging in to a single computer then it's sensible to have your user account and password defined on that computer - but what if you're a bit of a bigger organisation and you may need to have the same login at several places? Sun's NIS (Network Information Service) solved this issue many years ago for networks where a single central admin point was available - and it works well to this day, being included in Linux distributions where it does a sterling job.
But what if you've got a larger organisation where you want distributed admin, and where there may be a number of users with the same name in different departments? That's where you'll want to use LDAP - the Lightweight Directory Access Protocol. You arrange your data into a hierarchy, with appropriate servers and administrators at each level and - well - it can work a treat. LDAP is great for a wide range of information that you want to share around within your company; it will securely feed user account data, encoded passwords and much more. A truly broad service.
Users will also want to contact remote systems via their internet connection, and resolve host names down to IP addresses so that information can be routed correctly to external systems. This resolution is achieved using DNS (the domain name service) a.k.a. BIND. Again, data is an hierarchy but it's not secured and traffic is cached and sent worldwide. DNS is very good for resolving host names and mail forwarders over a very wide area - but it's a narrow service, in practise limited to hosts and mailers
It seems to be the season for LDAP and DNS ... I was configuring them on a Linux course last week, and I have something similar this coming week and in Oxford the week after next too.
Posted by gje at 05:10 PM
Related topics: via article database
April 15, 2006
A couple of days away
No "Horse's Mouth" yesterday nor this morning - Lisa and I have been up to visit a potential supplier of everything from bollard lights to tables chairs and reception desks for Well House Manor; delighted with what we saw and more posts to follow on the subject, no doubt. On the way, though, we took chance to get a few hours break and here are a couple of snaps ...
Keithley and Worth Valley Railway
Posted by gje at 08:50 PM
Related topics: via article database
April 13, 2006
Staying in the country
Spring is here at last - and early morning is a great time to look at the countryside. On Wednesday morning, I woke to the sound of the dawn chorus - just the birds, no traffic, And these are the scenes from the window at Chris and Delene's house just a few miles from the city of Bristol.
It's somewhat ironic that worsening traffic in Bristol has lead me to stop over the occasional night with son and daughter-in-law in this quiet spot when I have a longer course to give in the City. But it also gives me a wonderful chance to catch up with their news and to spend some much enjoyed time in their company - Delene, Chris, many thanks for making me so welcome.
Posted by gje at 06:03 AM
Related topics: via article database
Supporting users on Linux and Unix
Q: If you're the administrator on a Linux or Unix system and you want to help a user look after some of the files and directories, or run applications, under their own account, what's the best login to use?
A: You should be logged in as if you were the user him/herself
Q: Why?
A: Because you want the ownerships of any file system objects you create, and processes, and any logs to be assigned back to that user, and you want the environment (PATH etc) to be the one that the user would be using as well.
Q: Does this mean I have to ask the user to give me his / her password then?
A: No. certainly not. Here's what you should do:
a) Log in as normal through YOUR OWN user account
b) Use the command su - to become the administrator ... and, yes, you need the root password to do this
c) Then use the command su - george to become the user george. You will NOT be asked for George's password, since the administrator can "su" to any other account without giving one - he/she has already logged in and gained sufficient authority via the root account.
Q: Is that real a minus sign after the su command? Is it needed?
A: Yes, it is, yes it is. It tells the su command to set up a new environment for you using the settings for the user that you're about to become. Without it, you'll be running with your original users environment, but with the new user's authorities. Occasionally that's convenient if you have a lot of aliases but it can leave some huge security loopholes and it means that you won't be seeing the system in the same way the user will, meaning that you won't (for example) be able to exactly reproduce any problems that he's calling for support on.
Posted by gje at 05:07 AM
Related topics: via article database
Useful link: Linux training
April 12, 2006
Iran has enriched uranium ...
Iran has enriched uranium ... and the USA in the shape of George Bush is very upset about it. Isn't there a degree of hypocrisy there?
Posted by gje at 06:42 AM
Related topics: via article database
Mirroring a dynamic site
We're delighted to welcome crawlers such as Yahoo, Inktomi, Google and MSN spiders through our web site to index content - it's in our mutual interest. Those crawlers are all well written to analyse the pages that they collect, note patterns, and tailor their activities to make the best use of a dynamic site with 16 alternative versions (4 x font sets, 4 x colour sets) of displaying each page.
Robots which simply collect the whole of a web site for local offline browsing, such as HTTrack and (in some guises) Wget can be more problematic; they're simply not well formed to mirror dynamic sites and will try to gather every possible page, skewing web statistics and in bad cases leading to restriction of our resources for others. And it's doubtful whether any realistic use will be made of the data gathered.
I was watching HTTrack struggling to copy our dynamic website to a static mirror yesterday morning - every 4 seconds, another hit; it took 5 minutes just to get the help pages for the adhoc query demo by the time it had them in green on black in a tiny font, blue on yellow in a huge font, and all the intermediate settings. It's a waste of our resources and, frankly, I doubt whether the person making the mirror will find it of any use.
So as a web site owner, should I discourage such mirroring, and if so, how?
My first thought is to modify my robots.txt file to disallow all downloads by wget and httrack - except that I would need to check that they actually respect the standard before I go to the trouble, and that in any case we WELCOME user who sensibly use the utilities to download a few pages for offline viewing.
A second thought is to use my denial of service mechanism to trigger a delay where file access from a single remote host get delayed once they reach a certain threshold in a certain time - except that this would be just as likely to trap the legitimate / welcomed "bots" unless I put in some user agent specific logic that would need to be high maintenance - updated as new agents come along. And I certainly don't want to go down the "ban xxx IP address" road either.
A look at the HTTrack FAQ rather confirms my worries that neither of the above solutions is ideal; although it respects robots.txt, that can be turned off. Advise to users suggests that they use time limits and for large mirrors, ask the webmaster first and try not to download during working hours and do not download too large websites - use filters and I see all four of these pieces of advise NOT followed ... it also advises Are the pages copyrighted and, yes, they are.
But, in reality, it's no great issue to us if one or two users pull huge amounts of files they'll never use off our system.
Posted by gje at 04:56 AM
Related topics: via article database
April 11, 2006
Letter Boxes
Spring is in the air, the days are getting longer and there are some great photos around! On Sunday afternoon, down in Bath, I was snapping away and I've added a few more local area pages to the Wiki - see the Royal Crescent and Farleigh Hungerford Castle for example.
Some scenes just don't fit into a 4 x 3 or 3 x 2 type format ... the Royal Crescent being an especially tricky subject - so I'm back to a 17 x 5 format that I played about with last year ...
The format also works very well for scenes that otherwise turn into lines - here's Bristol taken from the top of the hill on the way to the airport at Lulsage ...
And I couldn't resist this sign - always one for re-use, I may use it on the Well House Manor site to illustrate the ... quiet ... nature of the hotel surrounds ...
Posted by gje at 06:35 AM
Related topics: via article database
More or less on the edge of the page
Have you noticed that whenever you're travelling somewhere new, it's always on the edge of two maps sheets and you have to keep cross-referencing?
The Linux, Unix and OS X utilities more and less are used to view a file one page (sheet) at a time, and pressing the space bar takes you a page forward. Did you know that by using the enter (return) key you can also step forward one line at a time, so that you don't have the sheet edge problem once you find the lines that you're interested in!
Posted by gje at 06:27 AM
Related topics: via article database
April 10, 2006
Why are maps rarely to scale?
Few maps are ever drawn, truly, to scale - if they were, then features such as roads and railways would be hairline-thin and the may would be of little use to anyone!
Image produced from the Ordnance Survey Get-a-map service. Image reproduced with kind permission of Ordnance Survey and Ordnance Survey of Northern Ireland.
Posted by gje at 06:15 AM
Related topics: via article database
April 09, 2006
Sympathetic development
It's all systems go on 48 Spa Road ... we'll be taking over "The Old Manor" on 31st May. Through the summer, we'll be thoroughly updating both inside and out, and we'll be opening in the Autumn as Well House Manor - a hotel specialising in accommodation for business visitors to Melksham.
Well House Manor is set on a three quarters of an acre site, just five minutes walk from the town centre. The closeness to the town, and yet the large space available, make it ideal for business men who will be driving down to Wiltshire, and want to stay in a town with good access to a choice of shops, pubs and restaurants.
The size of the site and location would also have made it ideal for housing development. I've been looking through some of the developer's plans that accompanied a planning application made a year or two back for over 20 dwelling units to be built. Fascinating; the application was never approved, and I can understand that it would have been rather out of place; we happened by a development that's underway by the same developer in Trowbridge yesterday and - well - I can understand why the neighbours felt that something this
would have been out of place in what is a quiet, ample-gardened leafy area of Melksham.
So how did the developers present their case? One of the big issues was access off the B road at the front into the site; at present, the entrance is "blind" and I know that when I visit there I come out with extreme caution. Even on foot, I walk out between the walls currently there on Friday and was nearly run over by a cyclist on the pavement. It turn out that the developers were planning to narrow the road so that they could get better visibility .. except that the plans described it as widening the pavements to make it safer for pedestrians. Isn't it interesting how the same change can be cast in two different lights!
((Footnote - we'll be neither widening the pavement nor narrowing the road, but we WILL be dropping the wall and increasing the visibility splay so that there will no longer be an issue; we're updating the access to an existing building and won't have any great change in traffic level, so that we're not going to over-engineer up to the level of entrance required for a cul-de-sac with 22 houses and a children's playground on it.))
Posted by gje at 07:53 AM
Related topics: via article database
April 08, 2006
Melksham, Wiltshire
Melksham is one of those towns that people have heard of, but they can't quite place. One of those towns that people bypass as they drive by, not knowing the friendly community that's there. It's our home, though, and we love it here.
As you'll see from the stone block (photo, right) in one of the town's buildings, Melksham has seen itself over the years as very much a working town. Woollen, rope, and dairy industries thrived here in the times before heavy engineering with the likes of Spencer's, GEC, and Avon Rubber who are still in the town. There's a great deal of modern business too, set out of town in areas such as Bowerhill.
A mixture of old and new buildings in the town centre provides a patchwork quilt of surprises, with many an architecturally interesting corner hidden away behind a modern building - a great and safe place to explore. And we've a wealth of pubs, restaurants and take-aways in the town too, making it a great base for touring Wiltshire - we're just 4 miles from Lacock, 6 miles from Corsham, 6 miles from Devizes. We're close to Bath and Stonehenge, Avebury and Westbury, Longleat and Bowood, Bradford on Avon and Malmesbury too .. in fact the list is endless!
Would you like to see some more pictures of Melksham?
Posted by gje at 01:05 PM
Related topics: via article database
Adding PHP tags to an old cgi program
Our web site has grown over the years, and we've developed a number of very useful scripts such as our quotation system that's written in Perl and our live server test program in Tcl and Expect. Now we require to run those scripts within the current look and feel, with its printer friendly flexibility, customisation to reflect the country from whom our visitors arrive, and so on, all of which are facilities offered by PHP. This can't simply be done by adding PHP tags into the output of the Perl or Tcl programs - cgi output is NOT parsed by the PHP interpreter - so how have we done it?
* New PHP pages call up the cgi scripts, now using them as application servers, and extract the pertinant information they return.
* Our cgi-bin scripts are little changed, but we have added markers in the form of HTML comments to indicate the start and end of the content generated.
* In the case of CGI scripts which use form inputs, we have also updated the scripts so that they can read parameters from the command line in addition to URL-encoding.
Try out our new server status page and see how it works. You can see the source code here and you can still run it the old way here (Shhh - don't tell the look-and-feel team).
The extra code in the PHP page is just:
$howsit = implode("",file("http://$_SERVER[SERVER_NAME]/cgi-bin/net/livetest.exp"));
preg_match('/<!-- start -->(.*)<!-- end -->/is',$howsit,$gotten);
Posted by gje at 07:43 AM
Related topics: via article database
Useful link: PHP training
April 07, 2006
Wiltshire to get an extra MP
We may be set to loose most of the already limited train service through Melksham, but it looks like we'll be getting a brand new MP at the next General Election.
The Boundary Commission reported - as long ago as 2001 - that an extra constituency should be added to Wiltshire due to the increasing population; I've put up details here. Most of the changes are around the Chippenham area, where a new seat grabs elements from North Wilts, Devizes and Westbury.
I'm predicting that the political organisations and incumbent MPs in Westbury, Devizes and North Wilts will stick with the seats they're already occupying, and we'll see a brand new organisation covering Bradford on Avon, Melksham, Corsham and Chippenham which are the four major towns in the new constituency.
I'm not one to advocate change for changes sake, but the plans might turn out to be rather good for Chippenham and Melksham. The current North Wilts MP is very much a countryside person and the town of Chippenham doesn't always seem to come at the top of his agenda. Likewise, Melksham really doesn't seem to "fit" in with Marlborough and Devizes with their rolling downs, wide open spaces and much higher lifestyle. All three of the existing constituencies have populations of over 80000 for their MPs to cover, and the new is around 65000.
Posted by gje at 06:30 AM
Related topics: via article database
April 06, 2006
Helicopter views and tartans
"Can we have more helicopter views?" asked a client from whom we're running a series of courses - by which they mean overviews that put all the elements of the technologies in place on a diagram - showing how tomcat relates to MySQL and how JBoss and Struts and EJBs fit in to the picture. For sure you can, and I note the term "Helicopter views" to add to my dictionary of metaphors.
"Do you have any Scots in the group?" I ask my class, as I use another Metaphor to explain how MySQL's select command restricts the returned values to just a tiny subset of the data available. If you have a tartan where 20% of the warp threads are green, and where 20% of the weft threads are green, then your cloth has only 4% of its area being full green - green on green. Translate that into a Select command; if you choose just a fifth of the columns in a table, and you then use a where clause where just 20% of the rows are selected, your result set, efficiently, is just 4% of the data held.
Posted by gje at 06:36 PM
Related topics: via article database
April 05, 2006
Keeping your regular expressions simple
There's something macho about programmers when it comes to regular expressions - so often they'll try and do the whole job in a single regular expression when doing a two stage process is much more logical, much quicker in operation, and far easier to code.
Take a requirement that came up this morning, for example, to extract data from a netlist. The data to be handled might look like:
[5560] = !V5*V6*V7*!V8 + [5223]
and the requirement is to return a list elements separated by = or + signs, and then to return a list of nodes or inputs within those elements. In other words, what's wanted is:
['5560']
['!V5', 'V6', 'V7', '!V8']
['5223']
I think I have rather made my point already by how I described the requirement, haven't I? Two regular expressions ...
I'm running a Python course, so the example's in Python ... full source code and more Python regular expression examples available!
Posted by gje at 12:15 PM
Related topics: via article database
Both ends of the animal
If this is "The Horse's Mouth", then should my partner be writing a companion blog called "the hind legs of the donkey"? Lisa was not amused by the suggestion yesterday evening, and declined both on the basis that she doesn't feel that it's a commitment she should be taking on, and also that she doesn't fancy that end of the equine beast; it wasn't a term she had come across before and I was surprised to find how very British it was.
A google search reveals "talking the hind legs off a donkey" in use by telephone companies selling their services to compulsive talkers, a number of blogs by predominantly female writers who are coming out and admitting verbosity, and the inevitable phrase sites. As Lisa talked on one particular tiny subject all the way from home to Chippenham yesterday, it might be more appropriate than she admits ...
>> Horse picture from our Iceland series
Posted by gje at 08:56 AM
Related topics: via article database
April 04, 2006
Architectural Heritage - Devizes
Looking around at the local town of Devizes on Sunday evening ....
 |  |  |  |
 |  |  | |
 |  |  |  |
... all these pictures taken while our Kebab cooked!
Posted by gje at 02:02 PM
Related topics: via article database
The best Open Source library in town
In our library, we've over a hundred books that cover Perl, sixty books that cover PHP, and over 40 books on each of MySQL and Python. It might not sound quite so impressive to report twenty books on Tcl/Tk but that's just about every one that's been published ...
You can search through the descriptions of our books by subject and by author online, and if you're on a course at our training centre, you're more than welcome to borrow books overnight.
Realistically, there's no way as a tutor on various Open Source subjects that I can know everything - but you can be re-assured that with our library and our high speed net connection, answers to all but the most obscure of questions can be found within a minute or two.
If you're not a customer but would like to browse through our books here in Melksham, please email me. I would be more than happy for you to drop by with an appointment for an hour or two one evening and use the library. We'll even offer you a complimentary coffee!
Posted by gje at 07:10 AM
Related topics: via article database
Python - block insets help with documentation
For many programmers, documenting their code is a chore, yet every piece of code that's going to be (re)used needs both instructions for the user, and adequate explanation of techniques for the code maintainer. The need for BOTH of these forms of support documentation is often overlooked by the newcomer, but that's really a subject for another day ...
In some languages such as Perl, you can write code that's crystal clear, easy to follow, and has the artistic beauty of a Rembrandt. And you can also write code that makes a dog's dinner look pretty. It's up to the development team and their managers to set coding styles and standards - see our sample standards document for an example of what's good and bad practise. Then in Perl you can use POD (Plain Old Documentation) to provide your user docs.
Python is particularly clever in that it forces certain good practise on the programmer, yet without imposing the need for extra levels of declaration. And it does this by using simple techniques such as the indented block structure. example.
When I first came to Python (I admit it), I swore and cursed at the need to inset blocks! But I've learn just how clever that structure really is. You loose a rats nest of { and } characters (and there's no "done", "endif" or "fi" either) and you gain a layout where you can very clearly see which code goes with which conditional statement. Try it once, and you'll hate it. Try it three times, and you'll love it and never want to go back to the bad old ways. And it helps make you code self documenting with that documentation remaining up to date with code changes.
On the user documentation side too, Python has some great tools. Documentation strings allow you to provide structured documentation with each module and each defined function or method, and that documentation can be explored with automated tools such as pydoc. Whether you're writing classes to be called by other programmers, or top level applications, you'll be thanked a thousand times over by your users even if they don't actually email you to say it. Oh - and with documentation strings, the instructions remain in the same file as the source so that they won't get lost at the point of distribution!
Posted by gje at 07:02 AM
Related topics: via article database
Useful link: Python training
April 03, 2006
Well House Manor goes ahead
Under English planning law, a "Bed and Breakfast" and a "Guest House" are two totally different classifications - you can run a B&B from the place at which you live with no special permissions even if it's in a residential area, but a Guest House or Hotel is described as a business and the property needs to be classified as such.
You may have read about our "Well House Manor" project - to provide accommodation for delegates on our course in Melksham ourselves in the future, and you may have noticed that I've been slightly quiet (especially for me) on the subject. That's because we've been going through planning hoops and the final go-ahead has been dependent on us getting change of use to "C1" approved. Although we'll be making very little change in practise to the traffic levels / curtilage at "The Old Manor", there were elements of doubt in whether the planning would be accepted. It started with the "history" of planning applications on the site, where the previous owner had been looking to sell to a property developer and have one house knocked down and replaced by 22; as you can imagine, the neighbours had been up in arms and view any new application with a pinch of salt and look at it with a fine toothcomb. Add in the access issue, where the current turning onto the B road outside is blind and can be fixed well (but not perfect to 2.4 x 90m specification for new build), and 8 neighbours with various bones to pick on subjects like trees, and wanting to buy some of the garden themselves, and you'll see that the whole thing wasn't quite a rubber stamping exercise.
Anyway - it's all "GO!" now. Full approval for C1 use, which is described as a "Hotel or Guest House"; a couple of caveats but none that will hold back what we'll be doing. Lisa has been doing a fantastic job already and there's plans, drawings, budgets, finances, professionals all coming together for our takeover at the end of May. We'll then be doing something of a "refurb" - after all, the building was scheduled for demolition if the 22 houses had gone ahead so the decor has become a little timed and some worn components haven't been replaced. And we'll be opening in the Autumn. Very exciting!
Link - Interim information
Link - Ground and usage plans
Link - Places to stay until we open
Link - The embryonic "official" Well House Manor site
Posted by gje at 04:58 AM
Related topics: via article database
April 02, 2006
Database design - get it right from first principles
It's VITAL to get your data(base) design more or less right before you write too much code - otherwise you'll end up wasting a lot of time and effort writing kludge code and - worse - forcing your users into work-arounds for the lifetime of the system at great expense to everyone in time, effort and sanity. But how DO you get the design right?
Start off with some sample data and apply Codd's principles of database normalisation - no table cells with multiple values, no repeated information and no calculated results to be stored ... and split the tables where necessary so that you can link them back together with a join when you present them. Sound complicated? Start off with a sheet of paper (or, better, a whiteboard) and draw up what you "really" want - and you'll soon spot where to normalise!
Then try out some test data and tables - through a very simple client like MySQL, with the SQL commands stored in a text file, since you can use those to seed your test databases later on.
At the end of last week, I ran a MySQL course and came up with an excellent normalisation example with one of the delegates. He has a number of clients (table) each of whom he produces a number of brochures for (table). Each brochure is available in a number of translations (table) and goes through a number of revisions (table) in each language. We also came up with a separate language table.
We then produced three test files of SQL commands (MySQL flavour) - one to flush old tables and create the new structure during experimentation, one to populate the tables with test data, and one to try out our multitable join syntax. NOTE - every table also has its own unique id column and, as a tip to help your sanity, use ids in different number ranges for each of the tables - on the test data, we started at 1, 101, 201, 301, etc. Name the id column "x"id where x is the first letter of the table name, and keep them unique.
You can see the table structure for the demo here
The test data is here
and the test join is here
Will the USER of the system want to see the data via this structure? No WAY - the user wants it formatted rather differently but that's where a programming language such as PHP comes in - to do the presentation layer. What the user wants to see is a report by client, with a matrix for each brochure showing rows for each language and columns for the latest revisions and their status. Have a look at this demo page and it will show you such a table.
We have the PHP source code available if you want to see how it's done (and a simpler example - source and running if you want to start gently).
Our customer has a long way to go with his system - date handling, blog handling for the texts, logins and authorities and many more subjects were discussed - but I'm confident that the basic table layout he has is correct and that the design is a robust one. And that's going to give him a system with a long and trouble free life.
See our MySQL FAQ for further tips and our MySQL resource index for loads more examples. Full MySQL documentation is available on the main MySQL web site
Posted by gje at 06:05 AM
Related topics: via article database
April 01, 2006
PHP Image viewing application
PHP has some marvellous facilities for image handling, and I've incorporated some of them into a new thumbnail-viewing demonstration that takes you around many of the images used on our Wiki pages.
The source code of the thumbnail viewer is also available, and you're very welcome to make use of it ... nothing too fancy (or rather, PHP does all the fancy stuff ;-)
In the grounds of Corsham Court more pictures
Posted by gje at 11:07 AM
Related topics: via article database
Useful link: PHP training
Ruby course - oops - it's not happening
I spotted a 1 day Ruby conversion course - presented by one of the "movers and shakers" behind Ruby and Rails - over Easter. And in Europe rather than the USA or Japan too. Fabulous - I'm always keen to learn from the masters; time and money spent listening to the originators of Perl, PHP, MySQL and the rest has always been a fabulous investment and it lets me in turn give greater depth to my training courses in England.
So I faxed in my booking (no online booking system - curious!), and checked up on my flights - at a pinch it could be done to get back for my Tuesday course. But then, silence; I got a bit worried yesterday - just two weeks to go when it disappeared off their organiser's website. I phoned them up. "Yes, we have received you booking fax. We should really have got back to you, shouldn't we? Sorry that we didn't. Oh - no - the course isn't full - in fact we've cancelled it ..." Grrrrr!
With niche subjects such as Tcl, Ruby - and even with some of the other we're involved with, it's hard to forecast which dates will book well, and which won't. So I have an element of sympathy for the supplier; indeed, it's very common practise to cancel courses - sometimes as late as just a few days before they're due to run. It's not the first time it's happened to me. But that's only ONE element of sympathy that I have, and I've got TWO elements of disgust at the business practise - it leaves the tutor, who's usually working under contract to such outfits, with no income for the days in question at the last minute, and it leaves the delegates who have scheduled time off work, flights, and hotels with a scramble to find an alternative and perhaps with travel expenses they can't recoup.
One of the things that Lisa and I decided that we would do differently when we set up the Well House Consultants training centre and our program of public courses was a policy that we don't cancel courses - your booking's acknowledged the same day that you send it in, and you then know that the course will run. Yes - a tiny minority of customers have found themselves receiving one on one training in the past, and no doubt that will happen again sometime in the future.
Personally, I'm doubtful that you can learn Ruby in a day, even if you're already familiar with another programming language, and we offer a 2 day Ruby course and feel that's a bare minimum. You'll see dates on the course description page and you can book with confidence - use the form on the description page to book, and make your travel plans. And, yes, I can predict that there will be just a handful of delegates ... and that you'll have a fabulous, interactive course.
Posted by gje at 10:14 AM
Related topics: via article database
Useful link: Ruby training