Bob Kaiser - Michael Baker Jr., Inc. Walt Cherwony - Gannett Fleming, Inc. Anna Lynn Smith, Parsons Brinckerhoff

1. Act 44 Performance Standards Regulation Development April 24, 2009 Bob Kaiser - Michael Baker Jr., Inc. Walt Cherwony - Gannett Fleming, Inc. Anna Lynn Smith, Parsons Brinckerhoff In association with: Draft - Discussion Purposes Only

2. Discussion Outline • Overview of Development Process • Progress to Date • Research • Data and Definitions • Peer Selection Process • Performance Measures Selection • Overall Process • Continuous Improvement • Estimated Timeline of Events (preliminary) • Next Steps • Questions / Discussion

3. Overview • Research & Development Tasks • Background • Legal Req., Research Materials, Gap I.D. • Peer Selection Process • Performance Measure Development Process • Data and Definitions • Performance Review Process • Potential Impacts • Consultation

4. Overview (continued) • Schedule July May Jan March 1 Background 2 Research 3 Peer Selection 4 Performance Measures PA Bulletin 5 Data, Definitions 6 Devel. Overall Process 7 Impacts 8 Consultation

5. Overview (continued) Draft Overview Pennsylvania Public Transportation Agency Performance Review Process Review Cycle Initiated System Notification and Kick-Off Field Work/Data Acquisition & Analysis DOT Planning and Preparation. DOT-Award Recipient Consultation. Peer selection, Data Acquisition & Analysis, Draft Initial Performance Standards Draft Performance Report Review Report with Transit Agency/Finalize Follow-up, Corrective Action, and Monitoring Document Exemplary Performance. Track Corrective Action Plan Implementation. Report Results. DOT-Award Recipient Consultation. Review Results, Discuss Action Plan, Identify Action Plan Elements. DOT-Grant Recipient Meeting. Draft Report of Findings v. Prior-Established Performance Standards. Identify Poor & Exemplary Performance. 3-18-09

6. PPTA Committee Recommendations

7. Overview (continued) • Systems Encompassed • Urban and Rural Public Transit Systems • Modes Encompassed • Fixed Route Public Transportation Modes • Evaluation by Mode • Not being considered at this time: • ADA paratransit • Shared ride • Other, non-public transit

8. Overview (continued) • Trial Applications • Peer Selection Process • Performance Review Metrics Application • Peer and Trend Analyses • Potential Impacts • Process, Schedule, Funding Systems/Modes • SEPTA (MB, HR, CR, LR) • PAAC (MB, LR) • AMTRAN (MB) • Endless Mountains (MB)

9. Progress To-Date • Materials Identification and Review • Legislation: Act 44, Act 3 • PPTA Performance Review Committee Notes • Prior Audits & Processes (Act 3, TFRC) • STAC White Paper • Other States’ Activities • Databases (NTD, dotGrant, Census) • TCRP Synthesis # 56

10. Progress To-Date (continued) • Data and Definitions • NTD and dotGrant Data, Definitions • Applied By Mode • Urban: NTD Rural: dotGrant • Definitions: Act 44 as data permits • Act 44 “Special Considerations” • “Circumstances Beyond Award Recipient’s Control” Events beyond grantee’s control which negatively affect performance, i.e., strikes, infrastructure failures, natural disasters. • Added Consideration of : • New, restructured, revised services • Discontinued services • Man-made disasters • Other (i.e., Special Events)

11. Progress To-Date (continued) • Perf. Review Major Analytical Components • Peer Review • Trend Analysis (intra-agency) • Functional Analysis (as warranted) • Annual Risk Assessment by DOT • Regular Cycle • 3-6 years; average 5 years • More frequent – per performance data • Data Timeliness • 1-2 year NTD data lag may be filled with local or dotGrant data, where feasible.

12. Progress To-Date (continued) • Graduated Approach • Basic Review • In compliance per base metrics and no “best practices” to document. Close out. • Step 2 Review • In compliance; minor issues or Best Practice from Basic Review. Possible Group 2, 3 metrics. Resolve via agency-developed “improvement plan” or document Best Practice. • Step 3 Review • Potential or demonstrated non-compliance. Group 2, 3 metrics; Functional review. May result in Action Plan to rectify non-compliance, potential funding implications per Act; or “Improvement Plan.”

13. Progress To-Date (continued) • Peer Identification • Modal • Number of Peers • 10-12 Initial Selection • 8-10 Final Selection (desired) • 5 Minimum • System Types • Urban compared to Urban (NTD) • Rural compared to Rural (dotGrant) Small urban if nec. to secure minimum number of peers.

14. Progress To-Date (continued) • Peer Review (continued) • Identification Process • Process to identify and select peers based on attributes of system under evaluation. • Includes consultation with subject system. • Approaches Researched & Trialed • Modal Characteristics Primary, Sort by Descending Criteria • Service Area Characteristics Primary, Sort by Descending Criteria • Modal Characteristics Match

15. Progress To-Date (continued) • Peer Review (continued) • Key Criteria from NTD / dotGrant • Revenue Vehicle Hours * • Revenue Vehicle Miles * • Peak Vehicles • Service Area Population • Other Considerations • Modal: • Fixed Guideway: # Stations, Route Miles • Bus: System Design Type, Service Type • Professional Expertise • Major Generators • Special Circumstances / Considerations • Climate * Car miles/hours for FG

16. Progress To-Date (continued) • Peer Initial Identification Method Results #1 Modal Characteristics Primary, Sort by Descending Criteria #2 Service Area Characteristics Primary, Sort by Descending Criteria Both Inadequate: • Insufficient # peers for many modes / systems • Peers disparate from Act 3, TFRC studies • “Spread” inadequate for smaller systems • Over-reliance on “professional judgment” • Add or delete prospective peers

17. Progress To-Date (continued) • Peer Identification Results • Modal Characteristics Match Selected • Developed when #1 and #2 proved inadequate. • Adequate # peers for all modes • Good match-up to Act 3, TFRC peers. • Good “spread” for smaller systems • Many prospective peers • Minimizes need for “professional judgment” to arrive at initial set of candidates • Final peers to be selected via DOT/Agency consultation

18. Progress To-Date (continued)

19. Peers initially identified by selection alternative. Final peers to be determined after consultation with transit agency. Summary:Initial Peer Identification by Method

28. Performance Measures Metrics for the Performance Review

29. Information Sources • Act 44 Section 1513 • Act 3 • STAC Study • PPTA / PennDOT Cmte. • Other States’ Practices • TFRC Transit System Audits • TRB / TCRP

30. Cycle • Regular • 3 – 6 yr intervals (5 year avg.) • As Needed • If warranted based on data • Annual Risk Assessment by PennDOT • Annual dotGrant data • Possibly supplement with NTD submittal • Performed by PennDOT from agency submittals

31. Measures (continued) • 3 Groups of Measures • Group 1 – Act 44 Metrics • Performance Standard associated w/ these • Group 2 – Best Practices Metrics • No performance standard associated w/ these • Supplement and Explain Group 1 Results • Others as appropriate to situation • Calculated by PennDOT during performance review • Group 3 – Customer Service Metrics • PennDOT encourages regular use by agency

32. Measures Group 1 – Act 44 Measures Performance Standards to be Implemented

33. Measures (continued) Group 2 – Supporting Meas. – No Perf. Std.

34. Group 3 – Customer Service & Satisfaction Meas.

35. Group 1 Group 2 Group 3 • Act 44 measures • Operating costs per revenue vehicle hour • Operating costs per passenger • Operating revenue per revenue vehicle hour • Passengers per revenue vehicle hour • Additional metrics for efficiency and effectiveness • Operating expense per revenue vehicle mile • Revenue vehicle hours per employee hour • Revenue vehicle miles per total vehicle mile (percent “live” miles) • Total vehicle hours per peak vehicle • Farebox recovery ratio (operating revenue/operating expenses) • General and administrative costs per total costs • Total vehicle hours to total fleet vehicles • Subsidy per passenger (operating cost minus operating revenue divided by passengers) • Customer satisfaction metrics • On-time arrival and departure • Frequency of service • Span of service • Vehicle cleanliness • Fares • Driver courtesy • Safety • Overall rider satisfaction • Easy Transfers • Bus Stop Locations • Adequacy of routes (i.e., coverage) • Printed and Telephone information • Web page • System map • Number of complaints received per month • Potential other metrics • As Appropriate Types of Performance Review Cycles “Regular” Cycle “As needed” Cycle • Not more than every three years • Not less than every six years • As DOT may determine appropriate Years 1, 6 Years 2, 3, 4, 5

37. Performance Review Standards Process Progress & Thoughts to Date

38. Establishing Performance Standard • Peer Analysis to Establish Standard • Metric • Peer Average, Median, Mode, Other? • Accounting for current performance (good, otherwise) • Allow for Future Events, Progress • Project at current review, or at next review? • Next review in 3-6 years • Trend Analysis • Direction: Improving, Declining, Flat • 5 years in analysis; Most recent 2 years

39. Sample Results: Peer Review Analysis Mode: MOTOR BUS Operating Cost / Revenue Vehicle Hour Passengers / Revenue Vehicle Hour Operating Revenue / Revenue Vehicle Hour Operating Cost / Passenger

40. Sample Results: Trend Analysis Mode: MOTOR BUS Operating Cost / Revenue Vehicle Hour Passengers / Revenue Vehicle Hour Operating Revenue / Revenue Vehicle Hour Operating Cost / Passenger

41. Establishing Performance Standard (cont.) • Measuring Achievement of the Standard • Approaches Under Study • Standard Deviation Approach • X% of Peer Mean • X% of Peer Median • Quartile or Quintile • Other Suggestions? • Incorporate Trend Results ? • Step(s) • One • > One

42. Sample Peer Mean, Median, Standard Deviation, Quintile MOTOR BUS PEER ANAYLSIS Passengers / Revenue Vehicle Hour Operating Cost / Revenue Vehicle Hour Illustrative Only Operating Revenue / Revenue Vehicle Hour Operating Cost / Passenger Peer Average Peer Median 1 Std Dev. 5th Quintile Direction to Pass

43. Passengers per Revenue Vehicle Hour Illustrative Only Peer Average 5th Quintile Direction to Pass Peer Median 1 Std Dev.

44. DRAFT Performance Review Outcomes 2 = Standard Met Trend Declining 1 = Standard Met Trend Improving 4 = Standard Not Met Trend Declining 3 = Standard Not Met Trend Improving Performance Improving Continuous improvement and potential transferability to other agencies. May warrant further research and possible Improvement Plan. Standard Achieved or Exceeded Standard Achieved or Exceeded Standard Not Met Standard Not Met Action Plan required by Act 44. Action Plan required by Act 44. Performance Deteriorating 04-20-09

45. Sample Results: Trend Analysis Mode: MOTOR BUS Operating Cost / Revenue Vehicle Hour Passengers / Revenue Vehicle Hour +4.54 +.57 Illustrative Only Operating Revenue / Revenue Vehicle Hour Operating Cost / Passenger +.12 +.07 +.57 5 year trend / Calculated slope

46. Progress To-Date (continued) • “Continuous Improvement” Approach to Establish Performance Standards • Peer Rev. + Trend Analysis + Functional Rev. • Establish Minimum Perf. Std. by Mode • Possible Performance Targets by Mode • PennDOT – Agency Discussions • Technical Assistance & Measure Progress • Achieve Standard • Achieve Goal Goal Standard Current Performance

47. Overview (continued) Conceptual Process Pennsylvania Transit Performance Review Report to Governor & General Assembly Off Cycle Review Results PennDOT Annual Review Risk Assessment Fail On Cycle Review Pass PennDOT Technical Assistance PennDOT Technical Assistance Waiver Improvement Plan Approved Pass Not Approved Compliance Measurement Action Plan Financial Penalty Fail Draft 3/20/09

49. Estimated Timeline of Major Events - A

50. Estimated Timeline of Major Events – B