Development of a Framework to Measure and Model the Performance of BRT Systems

1. Development of a Framework to Measure and Model the Performance of BRT Systems Project LS1 David A. Hensher Corinne Mulley John Stanley Institute of Transport and Logistics Studies The University of Sydney Business School The University of Sydney December 2010

2. Current status of LS1 objectives (2010-2012) Obj 1: Review and evaluate methodological approaches within which to quantify the salient factors that define the dimensions of performance of BRT systems Completed Obj 2: Identify and review existing data sources to identify gaps in data needs, and a data collection strategy to obtain all necessary data Completed Obj 3: to set out the selected method in sufficient detail so as to ensure that it is capable of estimation so as to deliver an agreed set of performance metrics Completed Obj 4: to undertake preliminary model estimation of data that is readily available in the first 12 months, essentially from secondary sources, Not Commenced Obj 5: to use the evidence to define criteria for selecting BRT systems that should be included by invitation as members of a ‘BRT cities club’. Not Commenced

3. Key Elements completed in the 2010 Work Program • Identifed the influencing dimensions within a framework broadly distinguising internal and external factors. • The external factors emphasised the institutional, cultural, geographic influences including evidence on modal biases present in the population as a whole and key stakeholders. • Identified key externalities (noise, congestion, safety, emissions) and how they should be sourced and measured and built into the performance assessment framework. • Reviewed the broad literature on performance measurement as context in which to develop an analytical framework that will serve to define data needs and the hierarachy of performace indicators. • A hierachical structure is crucial so that we can adopt a bottom up approach in data collection that provides consistency and accuracy, but will also deliver higher level performance indicators as appropriate that can be tracked back to very specific disaggregated indicators, which in combination facilitatesunderstanding of sources of influence on the higher order indicators. • Developed a template for data needs that are candidates for the the SMART data base, what data currently exists (with some assessment of its reliability), what data is missing.

4. Major Activity I • Title: The Dimensionality of Performance Frameworks and Performance Measurement for Bus Rapid Transit Systems • Development of a framework within which performance metrics can be defined and introduced in the context of meeting strategic, tactical and operational objectives in the public transport sector. • We discuss the important matters of definition of performance, data requirements, standards, the hierarchy of integrated partial and global measures of performance and frameworks to compare enterprises, and to explain why there are differences, and what actions might contribute to closing the gap between relatively poor and better overall performance. • The relationship between inputs, outputs and outcomes is central to the performance rubric, as well as an understanding of the processes that underlie the mappings between these three dimensions. • An important aim of the research is to ensure that the data collection activity planned for a global study of the performance of Bus Rapid Transit (BRT) systems is guided by an integrated and comprehensive framework for performance management, measurement, feedback, and improvement. • Ref: Hensher, D.A., Mulley, C. and Stanley, J.K. The dimensionality of performance frameworks, paper prepared for Thredbo 12, Durban, South Africa September 2011

5. Major Activity II • Title: Development of a comprehensive survey instrument to collect primary and secondary data from a sample of 100 BRT corridors • Reviewed literature to establish initial suite of candidate data items. • With Embarq and PUC, reviewed these candidate data items and refine based on key categories of data and extent to which they can be obtained. • Classified all data items as shallow and deep with shallow as a minimum being included in the LS1 performance metrics but recognising the essential need for a number of deep indicators such as those related to costs, revenue, patronage and service kilometres. • Prepared the data base as a survey instrument (through OS activities on Embarq) ready for pilotting in early 2011. • Refs: VREF 1.1_Shallow pilot questionnaire_29 Nov.xlsx Survey instrument 20 Nov 2010.xls Hensher, D.A. and Mulley, C. Minimum ITLS BRT Data Requirements October 22 2010

6. Major Activity III • Title: The relationship between bus contract costs, user perceived service quality and performance assessment • In contract tendering or negotiation, there is a growing interest in identifying and integrating user perceptions of service quality into the determination of contract costs; be they total cost or cost per service vehicle kilometre. • Despite progress in measuring service quality, there is an absence of any structured mechanism for determining the relationship between costs and user perceptions of service quality after accounting for other influences on costs, and in determining what elements of service quality are under the control of the operator, and those that are influenced by factors outside the operator’s control. Furthermore, when the relationship between costs and service quality elements under the control of the operator is identified, a mechanism is needed to quantify the extent of cost inefficiency associated with service quality performance that is below an agreed set of performance standards. • This activity developed a framework within which a cost-service quality relationship is quantified, and then implemented to identify benchmark targets for cost efficiency improvements required to achieve a pre-defined service quality performance target. • We use data from metropolitan and non-metropolitan bus operators in New South Wales to demonstrate the way the method can be used in contract negotiation and ex-post monitoring of performance leading up to contract renegotiation or tendering. • New data collected and compiled in the CoE, (in the observatory and LS1) and the new SQI surveys (also to be developed under LS1) will allow this methodology to be applied to BRT corridors • Ref: Hensher, D.A. The relationship between bus contract costs, user perceived service quality and performance assessment, prepared for Thredbo 12, Durban, South Africa, September 2011.

7. 2011 Work Program with funding already agreed • We see the LS1 activity begun in 2010 continuing throughout 2011 plus other elements of the objective set using the funding of SEK 489,913 ($US71,513 as of 16 Dec 2010) already been agreed. • Key activities in 2011 (as set out in LS1 Proposal 2010-2012): • Appointment of a post-doc position using the funds as part contribution • Continue working with Embarq and PUC in the piloting and refinement of a questionnaire to collect the baseline data for studying the performance of 100 plus BRT systems (at a corridor level) • Undertake pilot data collection on a BRT corridor in Sydney, NSW • Undertake pilot on Brisbane Busway System in February 2011 • Provide feedback on the pilot and contribute to refining the survey form • Advise Embarq and PUC as data collection progresses, on issues that arise in relation to data availability and quality • Commence preliminary modelling using data as it comes in, on models to establish a set of key performance indicators as a means of benchmarking all systems, and developing procedures to meaningfully compare operations given differences in operating environment and what is and is not under the control of the operator, the regulator and the market.

8. 2011 Work Program requiring additional funding • Design a Service Quality Index (SQI) in conjunction with PUC as a way of capturing the dimensions of service quality, and identify BRT systems that are prospective candidates for the SQI survey of passengers. • The intellectual contribution of this task is an objective of LS1 but the survey of cities required to paramatise the model is not funded as part of LS1, although the Lisbon meeting of the CoE identified this as an important contribution. • We suggest 10 systems would initially be appropriate in order to test the portability of preference weights across systems. LS1 is actively seeking cities interested in participating in contributing to this important element of research.

9. Summary 2010 • Prepared the project proposal LS-1 • Prepared a paper on performance frameworks to inform data collection and methodolgy • Designed data needs for the econometric models of comparative performance (we are working on it as I email you) - we want this out there soon so Embarq knows what data we need and this can be compared to what they have already suggested. • Prepared a paper on SQI methodology and its interaction with costs of service delivery and performance standards. Papers prepared as CoE contributions under LS1:Hensher, D.A. and Mulley, C. The Dimensionality of Performance Frameworks, paper prepared for Thredbo 12 and Working Paper LS1-1 for VREF CoE. Hensher, D.A. The relationship between bus contract costs, user perceived service quality and performance assessment, prepared for Thredbo 12, Durban, South Africa, September 2011. 2011 • Appoint postdoc position in BRT • Continue LS1 program • Identify target cities to join SQI development