September 28, 2014
Even in the dark of night, the train comes bearing passengers
I've noticed how we've been getting a full, or nearly-full hotel ... with an empty, or near-empty car park recently. And it's happened again tonight. Four rooms out of five occupied (and full tomorrow night) and yet only one guest has brought her own car. I was down at the station meeting delegates coming on a course in Melksham off the train which got in at 20:30 - the final service of the day. Good to see the new shelter with lights, work being done on new cycle racks, and the foundations for the Ticket Vending Machine going into place.
For the third year of operation of the service (we're still in the first, remember!), we have a target of 20 users per train to, from, or through Melksham. And the 20:30 on an autumn Sunday evening is one of the services that I might expect to be below average. Well - I got that wrong, in the right direction - 6 people got off, 4 people got on and there were some 28 on the train as it pulled out - that means that we're well over the third year average target on a train that I would have expected to be one of the ones that's going to be significantly under ... 34 when I would have been happy with 15 for a very off peak train in order to reach year 3 target. Wild guess - if this keeps up, we'll be around 230,000 journeys for the year April 2014 to March 2015 inclusive. Wow!
September 27, 2014
What can you and I learn from online quizzes?
I see an awful lot of online quizzes on Facebook; what sort of person are you ? / how much do you know? type stuff. And sometimes, I'll have a go - not because I have a huge faith in the results, but rather because they lead me to thought and reflection on myself, on my knowledge, and how such quizzes are constructed, crafted and at times coerce the taker into answers that will give a positive result.
Why do the authors want a positive result most of the time? Because they want you to share the test results on Facebook to your friends and do a little bit of Pyramid selling of their advertisers which the tests carry.
I'm a spoilsport; I don't usually share.
* It's unlikely that my Facebook friends really care that I love the colour green
* I don't want to appear smug [other words could apply] when I post a rather positive result
* I'm not really a "private" person, yet I feel that I would be giving too much of myself away - exposing myself, if you like - were I to post
I have however been logging a lot of my test results, and they've been things that I've tried quite a few of over the summer while the enviornment both at work and home hasn't been conducive to concentrattion (easy to do little things; hard to concentrate on bigger things with - well - customer service and Bobby).
The more serious business of these quizzes, and sharing of them via social media, it that it helps the social media operators build a profile to target advertising and content delivery ...
Just in case ... here's my log from some odd moments over the last month or so ... really not sure I should be putting this anywhere ...
24 out of 25
22 out of 26
8 out of 10
You Are Bashful
Your best feature is your smile!
Your subconscious is obsessed with FOOD!
9 out of 16
Little Miss Wise
You are a Laid-back, Progressive Parent.
Your aura is: Blue
Too Busy You were too busy or you are too busy to remember too many 80's movies.
11 out of 11 - Dr. Know It All
12 out of 15 - A True American!
Which song was written about you?
Your song is "Wild Thing" by The Troggs
At heart you are 19
Your memory score is: 95/100
15 out of 15 - Sponge-like Brain!
Can we guess how long you've been together?
Happily together for: 43 years
You will have 3 kids!
What Word Describes You? You are Narcissistic!
Can You Correctly Answer 10 Trick Question
9 out of 11 - Very Classy!!
8 out of 11 - Fashion Fabulous!
10 out of 11 - A Wise Philosopher
13 out of 13. A Student !!
YOU GOT $7-$12/hour
10 out of 10 Sponge-Like
When you are old, you will be... Wise
Identifying and clearing denial of service attacks on your Apache server
If ... ..... .... I ..... ..... were ...... ... ... ... to . ...... . write .... .... .... . a ... ...... ... ..... sentence, .. ... but ...... .. drip ...... . ..... ...... ... the ..... ... .. ...... ...... words . . .. out ..... .. ...... ..... . slowly ..... ... ...... with .... ... ..... ... long ... . ... ..... pauses ...... . . .... ... between ..... them, .... ...... I ...... ..... ..... ...... could .... .. ... .... burn ... .... up .. . . ...... a ... ..... .. lot ..... ...... ...... of ..... ... ...... . ..... your ..... .. time, . dear ..... .. .. reader, . .... ..... as ...... .... ...... ... you .... . .. parsed .. .. ... ..... ..... through .... ...... . ...... what ..... .... I'm .. .. ..... saying . ... ... and .... .. come ...... ..... ...... ... ... to ...... the .... the . ..... . end ..... ..... .. ..... .... sentence. .... ...... ....
And if I was to ask you a question. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then ask it again and again. Then I would end up burning up a lot of your time again.
Although we're not rude and inconsiderate of each other in this way face to face, such inconsiderateness is often shown - be it accidental, unthought, or intentional by browsers and browsing programs when they visit web sites, and web site administrators must make consideration of such activity, which can be ongoing 24 hours a day, 7 days a week, when they make their web sites visible.
The text above generated by this Python program - you didn't really expect me to handcode all that did you! ... See our Python courses where you can learn this (and other more conventional) uses of the language!
Our web servers get accessed from time to time by thoughless people such as I've described above (and by thoughtful people who intentionally do this sort of thng to lots of people, looking for security holes), and we need to keep an eye on our loadings and watch how our servers are doing. An old blog (about 4 years ago) tells you how we do the monitoring and graphing - it's here and the techniques are still current and valid, and a very recent discussion / item on our First Great Western Coffee Shop Forum - [here] - shows you how we located and overcame some issues a couple of weeks ago, looking at server log files, using Perl scripts (described here) to analyse the daily logs and find the needle that's causing the pain in the haystack of valid traffic.
From the last 48 hours, and again during the night just gone, I noticed some noise in the standard pattern that I expect to see from the server graphs ... here is the current [16:00 update] graph as I write:
And I pick out from that:
a) A big peak one evening. Not a problem, as that's the time that server backups were processing; I would prefer the peak to be not quite so high during this procedure, and indeed I introduced a recovery delay at a couple of points during the backup procedure recently, whilst making sure that each distinct web site is backed up without long gaps so that any content changes during the procedure will not cause a problem ("syncronisation").
b) A rising level of traffic yesterday, with the orange line being noticably above the lines for previous days all the way through the evening. Using the Perl scripts linked to above, I was rapidly able to take a look at the log files through a filter and see that one single IP address was requesting our hotel guest book that goes into each room ... and requesting it again and again - in total there were over 56,000 requests at one second intervals. You'll notice today's black curve in the graph above distinctly drops from around 03:00 when I told our server it could stop answering these requests for 2 Mbytes per second!
c) Sudden upward loading spikes - including one after I had fixed (b) at about 07:40 this morning. Taking a look at the server status pages (which are available to me as server admin), I notice a rather curious pattern during the spike of busyness:
The diagram is showing all the current threads accessing the web server. (Please ask if you would like me to teach you about these things, and / or take a look at your server!). And each of those lines marked "reading" is a remote browser that is dripping a question in, ever so slowly and with lots of pauses in between just as the first text I started this article with. Result - everyone else who's making normal traffic requests is having to wait until there's a thread available, and / or lots more threads are opening on the server and the machine's getting rather full.
The solution - something we've done before on another server - is to make our server a bit less patient, and to give up more quickly on requests that are dripping in slowly. The resulting server status list looks more like
which I can assure you is much more like what I expect so be seeing. As a reader of this article, you might not appreciate just what is and isn't right for these diagrams - they're things to look and and learn on your server, and learn about Python and / or Perl too so that you can do the extra analyses to look for patterns when things aren't quite as you would expect ... and do so soon, rather than waiting to when you have problems to resolve and can't take a dynamic look at the "when it was working" case.
September 26, 2014
Four time target - good news. Four time prediction - poor forecasting.
When Wiltshire Council asked for support through a local sustainable transport fund grant for a three year trial of an appropriate train service on the northern section of the TransWilts line (the Swindon to Westbury section of the Swindon to Salisbury line), they rightly set targets of passenger journey numbers that they wanted to surpass, with those numbers set such they would offer a viable way forward into the future at the end of the three year trial should they be achieved. Those targets are 45,000 passenger journeys on the section of the line which isn't shared with other services in the first year, rising to 108,000 journeys on the same section in the third year.
Separately from the targets, there are predictions of the passenger journey numbers we might expect. Independently, Wiltshire Council, First, the community and the community's consultants have come up with ideas / predictions of the sort of numbers that we might achieve, and those are a range of predicstions - lower figures based on some things not working quite as well as we might expect, and higher figures reflecing the results we would expect if everything goes well together. The predictions are all subjective, and many of them are based on commercially confident data, so it wouldn't be right for me to share them all here.
What I can tell you is where we're headed in the first year, now that we're two thirds of the way through. It's no longer a case of predications based largely on commercial data and theoretical work; rather it's now based on the extrapollation of actual data for the remaining part of the year, and I don't expect we'll be very far from 165,000 journeys - that's 3.6 times our target for the first year, and already 1.53 times our target for the third year.
In the past few days, I have been asked to justify how we got our forecasts so wrong, with some resulting doubt placed on our predictions. But we didn't get any forecasts wrong - the targets were / are the figures we need to reach for their to be no realistic question but that we've got enough traffic, and they are figures we need to reach to help ensure a viable basis for the continuation of the service. They were based on that model ... yes, we checked to ensure that the targets were going to be easy to achieve, and indeed we would have been foolish to set ourselves targets about which there was any real doubt. So we can celebrate being may times over target, with heads held high.
I will admit that our results are coming in very much at the upper end of predictions. And that's because of everything and everyone coming together to make it all work. Wiltshire Council with their support. First Great Western, who have made a number of decisions that have been really helpful, and their fabulous operations team, and some of our community members who have been there on the ground and getting into the hearts and making sure that people know about the services and the options, know where to ask questions, and feel an ownership.
Learning to program in Java - yes, we can help.
Yes, we CAN teach you to program in Java. And we'll teach you with a view to the latest versions of Java too ... at the end of this article's a list of some of the things added to Java 8 (not yet the standard run time environemnt) and Java 7 and we will tell you about those as appropriate. We'll do a darned good job of teaching your newcomers to programming in the up to date language.
So why aren't we offering public Java courses? Because everyone who uses Java want and needs to go on and use some other associated technoogies, and there are simply too any of those associated technologies for us to be experts in them all to the level of being able to provide professional training.
I have a couple of active "Learning to Program in Java" requests active as I write - if you have one too, please get in touch. We'll set your team - or the less experienced members of your team if you've already got some experienced programmers, well versed in Java basics and ready to go on and learn the associiated technologies
New in Java 8
Lambda Expression and Virtual Extension Methods
New date and Time API to handle date and time in a more natural way
New in Java 7
Using strings in switch statements
Automatic resource management
Numeric literals with underscores
Improved exception handling
A new java.nio.file package
File change notifications
Fork and Join
September 25, 2014
Please do not ask me to be the chair!
I'm so glad I don't chair any groups. For having seen, last night, the chairman of a rail user group in action, I think to myself "there, but for the grace of God, go I".
Some 'credits'. Newcomers were being welcomed / chatted to. The women (I make the gender point intentioanlly!) were making sure everyone had a tea or coffee. The meeting was (correctly) started a few minutes after the scheduled time - a bit of a tradition with this group as there are always one or two stragglers late in. Our chair told a personal story that provided a link into our guest speaker, told us a bit about that speaker and told the speaker a bit about the group and what we wanted to hear about from him, and confirmed roughtly how long he was to talk for.
Those of us who wanted to make notes during the presentation could have done with some light - we were placed into pitch darkness prior to the presentation, illuminated only by the glow from the projected slides. And we could have done without the sound of music from elsewhere in the building making it quite hard to hear the speaker.
An hour passes - a pleasant mixture of things I already knew / standard slides but ones which others may well not have seen before, and new / updated interesting material, numbers, comments ... and we get to that "fateful" final slide - "Any Questions".
Yes, of course the chairman has himself a series of questions, and we launch into some of them ... with a pause after a handful to let some favoured members of the audience who he knows ask questions too. There's a tendency for people to ask multiple questions, and for the chair to step back and let off-topic elements ride for quite a while, but in time the chair comes back and adds another question of his own.
The chair *does* go somewhhat around the room, "noticing" his pals, and (good on him) first time attendees to the meeting. Then when there's just one of this group left - I can predict it - "We've time for just one more question". The question is asked and answered. It then turns out that there is time for another question from the chair (must be his 5th or 6th) which is duly asked and answered, then a vote of thanks ... and a quick set of announcements from the chair about upcoming meetings, changes and other things that he wants us to know about, and the meeting concludes. There's one particular member who does a "may I but in and let the group know ..." to get a wider input of his information; he's very knowledgable, he's the late arrival, and he's well known - he "gets away" with it. Many readers will know him, smile and acknowledge his passion, and admire how he gets around and attention even with mobility issues.
For the rest of us? Those who are not favoured by the chair? The user group's membership sales pitch suggests that members can listen to and ask questions of expert speakers. No - that's not the case unless your the privelidged few. These days there are looks back an forth between some of us in the audience who know we won'd be called - watching the chair "at it" again as he does at every meeting. To some extent, I would weep if I wasn't laughing because these days our rail contacts do provide us with other routes where something important needs to be raised, and its scarsely worth joining the scrum of previous questioners as they gather around the speaker after the meeting to re-make, or perhaps carry on the convesation about, the issues they have already raised.
Like I said ... this is not a pitch for a chair role anywhere. I would make the "hogging" mistake at least, I know. Seeing the example does remind me what a good job Chris and Frank, Peter and Colin do at the meetings that I attend from time to time, and I commend them for that!
September 22, 2014
Libre Office - unable to get past REOPEN WINDOWS? question
Little tip for users of Libre Office .... very occasionally, it won't open correctly but offers you a restore:
The last time you opened LibreOffice, it unexpectedly quit while reopening windows. Do you want to try to reopen its windows again?
An no matter whether you choose to ignore or re-open, the question is asked again. It's "modal" so you can't go on. Very frustrating.
Solution? Delete the files / folders in ~/Library/Application Support/LibreOffice which will contain the details of the window that it's trying to restore. Then try again!