Benchmarking in European Service of public Transport (BEST)

1. Benchmarking in European Service of public Transport (BEST) Changing the BEST Survey questionnaire?

2. Background • The BEST Steering committee discussed changes in the questionnaire in a meeting the 25th of August 2008 – based on an evaluation made by the Helsinki University of Technology • The steering committee decided that a small group should propose an “improved” questionnaire – where the number of statements are reduced • In addition the steering committee discussed: • To add more background questions to make it easier to assess sample quality, like: • Household size • No of persons at least 16 years old • Non-response rates – how it is calculated and the effect of decreasing response rates • How to improve documentation of the BEST Survey regarding • Sampling methods • % of interviews conducted by mobile and fixed line • Definition of non-response • This document presents some alternative ways the BEST Questionnaire could be changed. The other issues will be discussed later. Changing the BEST questionnaire?

3. Alternatives Advantages and disadvantages per alternative

4. Alternative ways the BEST Questionnaire could be changed • Five alternatives are proposed: • Alternative 0: No change in the questionnaire • Alternative 1: Only minor changes in wording • Alternative 2: Reduction in number of statements – but keeping most of the indexes • Alternative 3: Reduction in number of statements – and removal of the “index-level” • Alternative 4: Change the wording of the statements – and reducing the number of statements Changing the BEST questionnaire?

5. Advantages: We continue to track a quite detailed information about the public transport service in each city No need to change the use of BEST data in participating companies Some participants has earlier stated that it is important that the survey stays “as it is” – because they need the information. And they want the benchmark data on all items from all the BEST cities. Disadvantages: Does not reduce costs Does not allow adding CIG specific questions without increasing costs The index level makes it difficult to spot changes in a quality dimension – because one item in an index might increase and another item might decrease. Makes it more difficult to see what is happening in each city The index level make it more difficult to discuss changes – because a change in an index can consist of several minor and major changes When respondents are asked to which extent they agree to a specific statement (as in this survey) – we have to define a level of quality that ideally should be comparable across quality items. That is not always easy Alternative 0: No change in the questionnaire Changing the BEST questionnaire?

6. Advantages: We continue to track a quite detailed information about the public transport service in each city No need to change the use of BEST data in participating companies Some participants has earlier stated that it is important that the survey stays “as it is” – because they need the information. And they want the benchmark data on all items from all the BEST cities. Disadvantages: Does not reduce costs If the score on the changed items increases / decreases – we do not know if it is because of the rephrasing or if the actual level of satisfaction has changed Does not allow adding CIG specific questions without increasing costs The index level makes it difficult to spot changes in a quality dimension – because one item in an index might increase and another item might decrease. Makes it more difficult to see what is happening in each city The index level make it more difficult to discuss changes – because a change in an index can consist of several minor and major changes When respondents are asked to which extent they agree to a specific statement (as in this survey) – we have to define a level of quality that ideally should be comparable across quality items. That is not always easy Alternative 1: Only minor changes in wording Changing the BEST questionnaire?

7. Advantages: No changes in wording means we have the history of the remaining quality items Might allow some extra CIG-specific questions Disadvantages: All previous indexes from 2001 to 2008 have to be recalculated (not a very big problem) Does not reduce costs significantly Could be difficult to decide which items that should be removed, and which that should be kept If we want to make a change – it should be more than just a small adjustment. Alternative 2: Reduction in number of statements – but keeping most of the indexes Changing the BEST questionnaire?

8. Advantages: Fewer statements removes the need for an index level. This will make it more interesting and easier to compare results (the index might hide significant changes in the underlying statements) No changes in wording means we have the history of the remaining quality items Might reduce costs significantly (has to be verified) – or make it possible add more CIG specific or questions related to a field of special interest (fare evasion, graffiti/tagging, pricing issues, real-time information etc.) Disadvantages: Could be difficult to decide which items that should be removed, and which that should be kept The level of information is reduced Alternative 3: Reduction in number of statements – and removal of the “index-level” Changing the BEST questionnaire?

9. Advantages: We do not need to define a “quality level” for each item in the survey (i.e. “Public transport mostly runs on schedule”). Instead the respondents are asked to state how satisfied they are with for example “Punctuality / Ability to be on time”) Fewer statements removes the need for an index level. This will make it more interesting and easier to compare results (the index might hide significant changes in the underlying statements) Might reduce costs significantly (to be verified?) Disadvantages: A change in wording might make it difficult to compare with previous results – but we might try to convert old results into a new index score Could be difficult to decide which items that should be included Alternative 4: Ask the respondents to state how satisfied they are – and reducing the number of statements. Changing the BEST questionnaire?

10. Description of the alternatives Items that might be included in the different alternatives

11. Traffic supply Public transport is good for work/school trips Public transport is good for trips e.g. shopping, leisure etc Public transport is good for trips in the (CITY) city centre Public transport is good for trips outside the (CITY) city centre Nearest stop is close to where I live Travel time on public transport is reasonable Waiting time is short at transfers I am satisfied with the number of departures Reliability Public transport mostly runs on schedule Information It is easy to get the information needed when planning a trip The information is good when traffic problems occur The information is good in stops and terminals Staff The staff answers my questions correctly The staff behaves nicely and correctly Personal security and safety I feel secure at stations and bus stops I feel secure on board busses and trains I am not afraid of traffic accidents when using public transport Comfort Traveling with public transport is comfortable Transfers are easy The busses and trains are modern The busses and trains are clean I normally get a seat when I travel with public transport Social Image: More people will travel with public transport in the future Public transport is good for the environment Public transport is beneficial to society Value for money Public transport gives value for money Public transport fares are reasonable Loyalty I gladly recommend travelling by public transport to others Citizen satisfaction How satisfied are you with public transport in general? Alternative 0: No change in the questionnaire Changing the BEST questionnaire?

12. Traffic supply Public transport is good for work/school trips Public transport is good for trips e.g. shopping, leisure etc Public transport is good for trips in the (CITY) city centre Public transport is good for trips outside the (CITY) city centre Nearest stop is close to where I live Travel time on public transport is reasonable Waiting time is short at transfers I am satisfied with the number of departures Reliability Public transport mostly runs on schedule Information It is easy to get the information needed when planning a trip The information is good when traffic problems occur The information is good in stops and terminals Staff The staff answers my questions correctly The staff behaves nicely and correctly Personal security and safety I feel secure at stations and bus stops I feel secure on board busses and trains I am not afraid of traffic accidents when using public transport Comfort Traveling with public transport is comfortable Transfers are easy The busses and trains are modern The busses and trains are clean I normally get a seat when I travel with public transport Social Image: More people will travel with public transport in the future Public transport is good for the environment Public transport is beneficial to society Value for money Public transport gives value for money Public transport fares are reasonable Loyalty I gladly recommend travelling by public transport to others Citizen satisfaction How satisfied are you with public transport in general? Alternative 1: Slight changes in wording – the actual changes to be decided Changing the BEST questionnaire?

13. Traffic supply Travel time on public transport is reasonable Waiting time is short at transfers I am satisfied with the number of departures Reliability Public transport mostly runs on schedule Information It is easy to get the information needed when planning a trip The information is good when traffic problems occur The information is good in stops and terminals Staff The staff behaves nicely and correctly* Personal security and safety I feel secure at stations, bus stops and on board the vehicle I am not afraid of traffic accidents when using public transport Comfort Traveling with public transport is comfortable Transfers are easy The busses and trains are clean I normally get a seat when I travel with public transport Value for money Public transport gives value for money Citizen satisfaction How satisfied are you with public transport in general? Alternative 2: Reduction in number of statements – but keeping (most of) the indexes Changing the BEST questionnaire?

14. I am satisfied with the number of departures Public transport mostly runs on schedule It is easy to get the information needed when planning a trip The information is good when traffic problems occur The staff behaves nicely and correctly I feel secure at stations, bus stops and on board the vehicle Traveling with public transport is comfortable Waiting time is short at transfers The busses and trains are clean I normally get a seat when I travel with public transport Public transport gives value for money / The fare level / ticket price is reasonable ------------------------------------------------------ How satisfied are you with public transport in general? Alternative 3: Reduction in number of statements – but keeping the same wording – and removal of the “index-level” Changing the BEST questionnaire?

15. How satisfied are you with the public transport service in your area regarding…..? The number of departures Punctuality / ability to run on time Information when delays occur The behavior of public transport staff The level of security provided at stations, bus stops and on board the vehicle The comfort when travelling by public transport Waiting time at transfers The cleanliness of busses and trains The possibility to get a seat when travelling by public transport The fare level ---------------------------------- How satisfied are you with public transport in general? Alternative 4: Ask the respondents to state how satisfied they are – and reducing the number of statements. Changing the BEST questionnaire?

16. Questions for group wise discussions: • Discuss the advantages and disadvantages of each of the 5 alternatives: • Do you agree on the listed advantages / disadvantages? • Should some advantages / disadvantages be removed or added to the list? • Rank the alternatives from the most to the least desirable alternative • Discuss the list of included statements in the preferred alternative – is there a need for: • Adding or removing statements? • Minor changes of wording? • Are one or more of the alternatives totally unacceptable to one or more members in the group – and why? • If time allows – are there other changes in the BEST Survey that should be considered? Changing the BEST questionnaire?