Horse's Mouth

An example of an injection attack using Javascript

Delegates sometimes ask me what an "injection attack" is, and for examples. I came across a really good example this morning on (oops!) one of our own pages - it's now fixed, but I'm documenting here and showing you what was happening so that you can learn from it.

Visiting one of the most popular pages on our web site this morning, all the expected content came up, but then this popup came on top of the page. It had me rathe rworried - the Image upload, store on database, retrieve, send back to browser demonstration is one of the busiest on our web site, and it had suddenly started coming up with this message. I tried it several times, and it was consistent - so it wasn't a genuine "database busy" messahe.

What had happened? We have a "review this page" facility on may of the pages on our web site - it encourages visitors to add comments. The comments don't get directly posted, but come to me for approval - this means that I can eliminate adverts for Viagra. And I had just approved a series of posts. Several of them were technical - included code - and I had overlooked the need to add in a call to htmlspecialchars. One of the visitor comments included:

die ("");

which - alas - caused the script (which was being sent out to the browser) to run the Javascript ... outputting to the screen, then reverting to the previous page if the database was missing.

Cure? Simply adding an htmlspecialchars on each comment, problem fixed. I'm reluctant to call an "injection attack" as I don't think that the commenter was actually malicious - he was helpful, but he just caught me out. But what a great example of the sort of user input you need to be so careful to protect against!

Saturdays out from Melksham - to Oxford, to Didcot or to Swindon

Fancy a day out with the family from Melksham to Oxford, to Swindon or to Didcot? We took a train trip on Saturday to Oxford from Melksham Station ... three adults and one dog - total return fare to Oxford £24.15.

The train left Melksham station (free parking) at 09:18. We bought our ticket on the train ("Groupsave 3 Offpeak day return to Oxford, please") and - changing at Swindon and Didcot - got to Oxford at 10:40. We returned on the 14:37, with the same changes, and got back to Melksham at 15:45. We could also have caught the 20:07 back from Oxford and got back into Melksham at 21:53.

If there had been four adults, with a "Groupsave 4" it would only have cost us £24.40, and children aged 5 to 15 can travel with the group (but not more children than adults) for just a pound each. These are book-on-the-day fares and apply every Saturday - although you should recheck the train times in case there are engineering works.

What is there to do in Oxford? There's lots! There's shopping. There's "The Oxford Experience". There's the bus tour showing you the University City. You can climb the Carfax tower. You can wonder at the old fashioned market. You can go into the colleges and look around. You can choose from a wonderful selection of places to eat lunch. You can browse the books at Blackwells (one of the best bookshops I know out of London). Hire a punt at Magdalen bridge, take a boat trip on the Thames (both seasonal) or visit the botanic garden (open daily thoughout the year). If Oxford United are playing at home, you can watch their match at the Kassam Stadium. Or - if you're like us, with dog in tow, you can see the open spaces and architectural beauty of the city.

We walked up the Oxford Canal past Isis lock, then across into the back streets of the old but fashionable suburb of Jericho, where we sat outside a cafe and had a breakfast bap and a coffee. Across to the Woodstock Road, past the site of the recently-demolished Radcliffe Infirmary and down St Giles to the city centre. Then to the Meadows of Christ Chuch College, leading down to the river Thames (know locally as the "Isis") which are open all year. A walk across Folly bridge and a short section of the Thames Path before it was time to head up Oxpens and back to the station for our return journey, tired but happy, to Melksham.

On Malachite Green

"Malachite" - Copper Carbonate. But know to me as a rather rich, bright, exotic green colour that's so bright and rich it's rarely seen. I moved to the south east of London in 1959, at a very young age - with no car for our day to day family life (and that was no great hardship), we used the southern region of British Railways and the bus (and Shank's Pony) to get around. Malachite was the livery that had been used on the trains a few years earlier - but I remember the darker green, and "Malachite" had a sort of exotic mystery when it was mentioned.

So there I was on the communter trains of the new "Southern" from Balham to Norbury for the latter part of this week - and what is probably a drudge of the commute to most people had a certain old wow factor for me even on that short journey

And then, in Oxford today, we walked up the canal ...

8th Feb 2010 ... Paul Clark writes ...

You mention Southern Railway 'green'. I was told that senior officers met in Ryde, Isle of Wight to discuss the development of Southern Railways. Officers could not agree on a livery colour and the Chairman thought the debate when on far too long. Breaking for lunch the group walked along Ryde high street. Passing a hardware store the Chairman popped in and came out with a length of green string. Cutting the string in half, he turned to the Chief Engineer and said this will be the livery colour. This half for you to match with a paint supplier and this for me to monitor your achievement.

That's an interesting anecdote, Paul ... and it's 'way out' enough to be true. - Graham

Both feet on the same pavement

Wednesday Morning 08:30, February, Moorgate, London.

I'm fascinated by some people who feel that the rules don't apply to them. The guy I just had to walk past, bathed in a cloud of smoke of the almost-enclosed lobby with its clear "it is illegal to smoke here" signs, the young lady using her personal stereo in the quiet carriage of the train on Sunday evening, and the guy in the same carriage who was making a mobile phone call. "I'm on the train" he shouts; actually, we all knew even before he told us, and we could all have told him that he was in the one carriage where calls are strongly discouraged!

I get grouchy at times, I'm afraid; there are elements of me which are definitely "Victor Meldrew" - the grumpy old man from the Sitcom "One Foot in the Grave". There are also times that I'm tempted to follow through with things that irritate, rather than just letting them wash over , but I have learned to let most of them wash over with me. Or rather to think about what irritates me, hold up a mirror, and say "I hope I don't irritate people in that way". But I'm enough of a realist to realise that there will be times that I overlook in myself things that I would critisize in others, and that I probably have my own special ways of doing things which are fine to me, but irritate the living daylights out of others.

Is this a negative post? It's looking a bit that way ... but I can turn it around and add two positives. The first is that something I saw pointed out the other day - that the Victor Meldrew types actually still have all there marbles about them - they're likley to be healthier of mind that the people who just 'roll over and accept it' as they're getting older. And the second is that in and along with those negatives come the positives - the staff member at Aldgate East on Sunday who let me out of the disabled barrier because my luggage was heavy - with a lovely smile that brightened up the end of a rough journey. The check out person at my hotel this morning. And the great bunch of delegates I have on this course.

The text above was written on Wednesday, but during a course where I wasn't able to have my machine on line. It's posted this morning as a quick "stock article" ;-)

I have another very busy day today - Friday - working now in South London; it's a rare pleasure to be able to have two jobs roll into each other without the need to go back to base / restock / wait a weekend somewhere. But even here I have to remember that each customer is King, and that I must not let the fact that I have adsurdly too much hardware doe my Thursday / Friday work interfere with it, not could I restict my provision on Monday to Wednesday due to the fact that having lots of extra stuff would be inconvenient (to put it mildly!) later in the week.

Skyline and looking up in London

For those readers who feel I have a one track (Ruby) mind this week - here are a couple of pictures. This first taken out of the bedroom of the hotel I was in Sunday through Tuesday evenings (I've move on elsewhere in London now to my next job and the outlook is suburban by comparison.

London has changed greatly since I was brought up there; I can still find my way around easily enough, but some of the buildings are very new and looking up gives for fascinating vistas.

Object Oriented Ruby - new examples

We had an "object oriented Ruby" morning on today's course. "But everything's an object in Ruby" you'll tell me and you're sort of right (we could argue about what symbols and code blocks are), but there are certain elements of object orientation which require specific coverage.

Here are links to three examples:

[link] - a straightforward example of a class definintion, no inheritance, nothing fancy (thout it does have an attribute accessor)

[link] - a demonstration that shows inheritance, embedding of classes within a module, static variable and methods, operator overloading, a singleton method and much more. There are plenty of comments in the source code of this example.

[link] - showing the difference between object and class (dymanic and static) variables.

The following new examples have also been added in other areas:

[link] - command line parameters to a Ruby program

[link] - BEGIN and END blocks, and autosplit

[link] - a comparison of symbols and strings in Ruby

[link] Commenting a regular expression in Ruby

Handling unusual and error conditions - exceptions

"I can't answer that question in the way you expect" ... that's something that may be said to you occasionally - you ask someone what suit a playing card is that they're holding and they cannot tell you because it is a joker, or you ask what number is written on a piece of paper when the paper is blank.

The way we handle situations like this in many modern programming languages is through excpetions - where a keyword such as try is used to indicate a block of code that may potentially have a problem, and then we use catch or rescue to tell it what to do if something goes wrong.

Exceptions are blocks, and although you may not think so at first the scope (size) of block is every bit as important as the scope of an if or while block.

Sometimes, exceptions will be processed in there entirety by the function / method in which the problem occurs, but at other times you'll want to throw the excpetion back up to the calling level; in that case, you'll need to put a try block in the calling code too. There's a new example of this (in Ruby) on our web site from today's Ruby Programming Course ... and the example is [here]. We have an example showing how it's coded in Python [here], and one from C++ [here}. We've got examples of the format of exceptions in Java on our site too - see [here].

Ruby collections and strings - some new examples

From yesterday's Ruby course - a whole set of new examples on Ruby collections - arrays (which are ordered lists) and hashes (which are unordered). In other words, you look somethng up in an array by its position number, whereas you look something up in a hash based on its unique key, and the way it's implemented, you cannot sort those keys - if you try, you get an array or arrays of two elements returned!

[example] - Sorting an array and iterating through it in various ways

[example] - Using an anonymous array to swap over two variables easily

[exampe] - Combining arrays with "and" and "or"

[example] - sorting a hash - (you can't, but this shows how to get the same effective result!)

[example] - multidimensional structures in Ruby - a hash or arrays in this case

Here are some other examples from yesterday too - string handling and exceptions:

[example] - lots of ways of defining and handling strings

[example] - cleaning up strings of text that are user input using string methods and regular expressions

[example] Using a constant to turn debug mode on and off - also a very good example of reading from and processing a file in Ruby

Direct access to object variable (attributes) in Ruby

Rather than writing getters and setters, in many Object Oriented languages you can access the variables within an object directly. That can be dangerous for code flexibility for the future, as it means that you're removing the possibility of interspersing code. However, it can be very convenient.

In Java, you'll mark members you want publicly visible as public; in Ruby you specify attr_accessor (for read and write variables) and attr_reader for read only variables. There's an example [here].

If you try to access an object variable for which you have not set up an appropriate accessor in Ruby, an exception will be thrown (I've put that into the example too!)

Passing code to procedures and yield in Ruby

If you want to pass a block of code into a Ruby procedure, you can do so by passing in the code to a variable who's name you start with an & in the procedure definition; you can then run that code using the .call method on the received object. In some ways this is an indirect reference - in Ruby you can's simply assign code to a variable.

You can also pass a curly brace defined block / closure into a procedure by specifying it as the piece of code that the procedure should jump to when it encounters a yield.

There are examples of both of these syntaxes at [this location].

What are Ruby Symbols?

Ruby Symbols are names that follow a : character. They're a quick way of getting at data, where (for example) you have a fixed key or name in a hash, and you'll find them commonly used in many definitions such as in Ruby on Rails. I've put a short example [here]

Comparing floating point numbers - a word of caution and a solution

"Think of a number between 5 and 15.
Double it.
Add fourteen.
Halve it.
Take away the number you first thought of.
Is the result 7?"

We all played games like that as children, thinking of the number 0 ourselves so that we could do an easy calculation and impress our friends. And the result was 7 ... whether you started from 0, 5, 7, 9, 13, 15 ... or any other value.

But if you try such things with floating point numbers in more or less any programming language, you'll not necessarily find that you get back to an exact number, or the number you first thought of. And that's because computers store floating point numbers in a format where they are ever so slighly rounded.

The issue is a common one to most languages, but it happened to come up today on a Ruby course, so I've written an example in Ruby to show the problem. I took a number, divided it, added, multiplied, subtracted, and tested to see if I was back where I started. It wasn't - 1.0 had become 0.999999999999993 and When I tested the result to see if it was equal to one, my program declared that it was NOT!

Be careful when testing floating point numbers in any language - you'll do far better to work with integers much / most of the time. And if you must test a floating point variable on which you have done significant calculations for equality, you'll be well advised to do so by checking to see how much the absolute difference between the two values is, as a proportion of one of the values that you're comparing. Does this sound complex? It isn't, and it can easily be implements as a function:

def approx?(this,that) diff = (this-that).abs return diff < (this * 0.0000000001) end

The full source code of my example, together with some test output, is [here].

Defining a static method - Java, Python and Ruby

Most methods in classes that your write will be run on / applied to individual objects within that class - you'll be asking for the colour of a marker pen, or setting the price of a hotel room. You will NOT - typically - have a model in which all marker pens share the same colour.

But - just occasionally - you need a method that applies to the class as a whole, and this is known as a class (or static) method, and there are various ways of specifying such methods in your code.

In Java, for example, you can simply declare a method to be static (and you are then limited within the code of the method to only handling static variables in the class, as the method hasn't been run on any particular instance of the object).

In Python (from release 2.4), you can use an @staticmethod decorator before you declare your method that's to be static - see this example. You can also - in all versions - use something like
counter = classmethod(getcount), and again we have a full example on the web site here

In Ruby, you declare the method name with a leading self. in front of the method name - so you declare
def getnt()
rather than
def self.getnt()
You can also use the class name in front of the method name - I show the alternative syntax in a full example [here]
but I'm going to discourage that - for if you rename the class, you really don't want to have to start messing about and renaming methods within the class as well!

String to number conversion with error trapping in Ruby

You've read in a string of text in Ruby - perhaps the user's input, or perhaps from a file. You know that string shoould contain a number - so you use the to_i method to do the conversion. But there's a problem - if the string doesn't start with a number, it won't convert correctly - it will just return a zero. And you'll probably want to know there's a problem so that you can deal with it.

The solution is to use the built in Integer function and pass it the string - i.e. replace:
value = stuff.to_i
by
value = Integer(stuff)

When you run that code on a string that does correctly contain a string that converts to an integer, there's no difference to see in how you code works. But if there's an error, Integer throws an exception.

Include your Integer call within a begin and end block .. and within that block, also provide a rescue section. Ruby will start running the main part of the block when it arrives at that piece of the code, and if the main part of the block runs correctly it will skip the rescue section. But if something goes in the main section of the block, it will jump straight away to the rescue code.

Here's an example ... if there is a valid integer in the said string, it is added to total_so_far, and the goodvalues variable is incremented. But if there is not a number in said, then a controlled error message is generated:
begin total_so_far += Integer(said) goodvalues += 1 rescue puts "Sorry - not accepted" end

The complete example, with some sample results, is [here]

Neatly formatting results into a table

Would you like to produce a neat table of results, with the columns just wide enough to take the data? That sounds straightforward, but you can't output anything until you've looked through all the lines to see how long each of them will be.

I was talking about this algorithm today, and this evening I've written it up as an example of formatted printing in Ruby, and posted the complete example on our web site [here]

The trick is to do a "test" format of each of the values in each column first to find the largest, then substitute the column width into the format string for 'real' before you output the data. It's at times like this that you're really glad that a format conversion like %7.2f means "a total width of 7 columns of which 2 are after the decimal place" and not "seven figures before the decimal and 2 after"!

This technique is not appropriate if you're using a variable width font, or if you're generating output for inclusion in a web page (there are some exceptions to this latter)