The background to write-ins on consultations

It's an excellent idea to allow "write in" answers on surveys - but a nightmare for the statistician to classify and count them. Typically, the survey analysis team will classify the write-ins into groups and perhaps subgroups, with a section highlighting individual answers of significant interest; where a question asks "Do you have any further comments or suggestions that will help us?" you don't just want to count and reject the one-offs ... you want to read them in case someone has fed you a brilliant idea!

Wiltshire Council's consultation responses on the future provision of subsidised bus services are in their cabinet papers and mirrored for ease of finding into the future at [here] . Write-ins have been categorised, and counted by category - mirrored at [here]; that page summarises each write-in category with a very few words. I've not seen a list of individual comments, nor criteria as to how categorisation was decided in borderline cases, nor is it clear whether comments may be counted in multiple categories if they overlap.

I am particularly sensitive to possible misinterpretation of write in data, recalling a consultation some 10 years ago that was geographically split, with one figure given for rail services mentioning "Melksham" and another for responses that talked about "Chippenham to Trowbridge" or similar. Net result was two low figures which were regarded as relatively insignificant, both concerned at the withdrawal of the same service. Added together, they would have come right up the list of "most written in about" topics, and become significant.

When reading write in responses, it's also important to note that the number of times a particular category of answer is written in cannot be compared on a like by like basis with the number of times a box is selected on a category that's explicitly offered; one is a re-active vote and the other is a pro-active action. It should be noted when looking at numbers that "money talks" and options which have paid staff prompoting them are likely to be more popular than spare time volunteer inputs - you've only got to look at the current furore over election expenses in Chippenham to realise just what a difference having a financed campaign is seen to make. Finally, it should be noted that the consultation and offered options were well prepared ahead of time, with correlation to the question numbers, so that the consultation hit the ground running when it opened, with a big press campaign to get it completed (in almost indecent haste!) ... a number of people wrote to me during the consultation and expressed regret that they had not heard of "option 24/7" when filling the form in early on. This effect was probably made more noticeable by the consultation opening 4 days earlier than we had been told about, meaning the resource we had planned to make recommendations from the start was delayed somewhat.

It will sound like I'm grouching. Actually, Wiltshire Council did a very much better job than I've heard reported about from almost any other council in getting interest and a lot of responses, and I'm delighted with that aspect; I think they were somewhat taken aback by the numbers of responses though. I do look forward, hopefully, to further data releases so we can see how the categorisation was decided, and whether particular write-in responses correlate to answers to any of the other questions - there could be vital evidence in there!

Analysis

The following comment / categories of comments are or may be in line with Option 24/7
158 - "option 24/7" written in
146 - Alternatives / other
300 - Alternatives / political
90 - Service Alterations / Better Promotion
199 - Service Alterations / Change Routes
84 - Service Alterations / Co-ordinate Competing Services
166 - Service Alterations / Link to Rail
75 - Subsidies / Other
Which is a total of 1119 responses out of 15,286 that may be supportive

It's even harder to define which other written inputs could potentially be opposing option 24/7; "cut wages / allowances" with 141 votes is most certainly something we would not wish to do if that's bus driver's wages!

Some suggestions are not allowed within the current law, nor are they in the draft Bus Services Bill - "Concession Contribution" at 462, plus "Pay for Passes" at 95 ; we have not considered these possibilities within Option 24/7 as we were fully aware they would be illegal to implement even under the Bus Services Bill, but had the government taken a different position they may have actually helped strengthen the case for Option 24/7.

Many inputs related (as the 'write-in' invite suggested also) why people were using the bus (or wanted to use the bus ... it's not clear) ... and these comments neither support nor reject Option 24/7 ideas as such; Option 24/7 does support getting people to places they need to go for work, business, personal business, social, education and medical reasons - so they could be read as adding some support. "Car Usage" and "Congestion" would need to see a huge swing to bus use to have much of an effect, but again that would be in the supporting direction for a positive bus policy such as Option 24/7

In all cases, there were many more votes for "retain" than "reduce" a category. I note that the categorisation of the comments was not Sunday buses (as in the rest of the consultation) but Weekend buses. I also note that although the weekend numbers were low, when the relatively low cost of providing Sunday buses is taken into account there is a strong case for these being top of the pile to retain - just at there was in my other analysis.

First column - ratio of "retain/improve" to "reduce/stop" written inputs
Second column - ratio of support cost to number of written in retain/improve inputs
6 - 1625 - Evening
19 - 914 - Rural
10 - 823 - Strategic
3 - 962 - Town
5 - 509 - Sunday (or is that weekend?)
So for every 3 people who said the town buses should be retained / improved, 1 suggested that they be reduced / stopped; a popularity measure, with the higher the number meaning the more popular retentions would be. And for each person who wrote in "retain / improve" town buses, Wiltshire Council spent £962.00 last year. Care needs to be taken when reading the Evening and weekend / Sunday results, as the survey consultation was disproportionally brought to the attention of the current passengers in the other three categories, as was explained in the previous article. This means that the evening figure, and the Sunday figure, for costs are probably significantly higher than it should be; an astonishing comment for Sunday services which are already very much the lowest.

Conclusions

The principles of better promotion, looking at arranging the network as a whole and co-ordinating services, and linking to rail all got significant written-in support, and are all elements of Option 24/7 - which when added to the option 24/7 messages come to a considerable number. I can't see any other strategy with as much written in support!

As I would have expected, there's a balance to be struck between the various service groups in providing support. Once again, support for Sunday services appears to be exceptionally good value.

Due to lack of base data to analyse, and lack of data as to how classification of write-ins was done, only general conclusions are possible. I would very much like individual answers to be published just in case there are nuggets of gold in there!

Full set of links to the main report and 5 appendices [here].
(written 2016-06-11, updated 2016-06-15)

 Z407 - Option 24/7 - the way forward for Wiltshire buses
  [4619] Buses - not about the buses, but about the people who use the buses - (2016-01-10)
  [4621] The power of scripting - (2016-01-12)
  [4624] Campaigning for a sensible bus solution and not against anyone! - (2016-01-17)
  [4625] Where does Wiltshire bus subsidy money come from? - (2016-01-19)
  [4630] The story behind the bus cuts, and what they could mean - (2016-01-24)
  [4631] What would the Wiltshire Council bus cuts mean for Melksham? - (2016-01-25)
  [4636] Rail user groups worried about what is happening on the buses - (2016-01-31)
  [4648] Why is bus funding a much hotter topic than a new set of traffic lights? - (2016-02-16)
  [4654] Getting to the Royal United Hospital - the Hopper and the alternatives - (2016-02-24)
  [4657] Rumours of bus changes by First in Wiltshire - what we know and suspect - (2016-03-01)
  [4658] The end of competition on a bus route - the effects from then end of the 234 - (2016-03-02)
  [4660] What is happening on the 231 bus? What are you going to do about it? - (2016-03-04)
  [4667] Wiltshire Supported Bus consultation - final few days - (2016-03-30)
  [4669] Buses - how did we get into the situation we are in? - (2016-04-02)
  [4690] Buses in Wiltshire - potential changes and where we stand at the moment - (2016-05-30)
  [4691] Real life PHP application using our course training MVC example - (2016-06-05)
  [4692] How are bus support payments currently calculated? - (2016-06-07)
  [4693] Wiltshire supported bus service consultation, early 2016 - interim results published - (2016-06-09)
  [4703] What do people use the bus for in Wiltshire - survey interim results. - (2016-07-11)

Back to Wiltshire supported bus service consultation, early 2016 - interim results published

Previous and next or Horse's mouth home

Forward to Melksham Rail Development Group becoming Melksham Rail Users Group

Lisa writes - on the referendum vote today
Month, Day, Year number to day of week and month names in Python - English and Swedish
Programming with random numbers - yet re-using the same values for testing
Melksham Rail Development Group becoming Melksham Rail Users Group
Supported Bus Services in Wiltshire - analysis of written-in comments
Training Course schedule for summer and autumn 2016