1 / 14

Transit Forecasting Accuracy Database: Examining the Impact of Inaccurate Inputs on Demand Forecasts

Explore the relationship between inaccurate inputs and biased transit demand forecasts. Quantify input inaccuracies, assess forecast accuracy, and investigate the role of project variables in forecasting errors.

scullen
Download Presentation

Transit Forecasting Accuracy Database: Examining the Impact of Inaccurate Inputs on Demand Forecasts

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Garbage In…Garbage Out?Do Inaccurate Inputs Cause the Inaccuracy & Bias inTransit Demand Forecasts? David Schmitt, AICP With very special thanks to Hongbo Chi May 16, 2017

  2. Overview Optimistically biased forecasts fall below this line

  3. Motivations • Heard at transit forecaster gatherings... • "Our demand forecasts were wrong because the land use assumptions didn’t come true" • "Our forecasts are always too high because of those inputs they give us to use" • Implications: • Accurate inputs  accurate demand forecasts • Accurate inputs  unbiased demand forecasts • Model specification, validation or other issues are not problematic  Assess whether inaccurate & biased inputs resolve the inaccuracy & bias of demand forecasts Transit Forecasting Accuracy Database: Garbage In…Garbage Out?

  4. Methodology Begin with Transit Forecasting Accuracy Database: 66 projects, with forecasting inputs categorized by accuracy Process: • Quantify the level of inaccuracy for each input/assumption • Compute change in forecasted demand by apply elasticity to corrected assumption • Compute adjusted demand forecast • Compute the adjusted forecast accuracy ratio: • Ratio = actual / forecasted ridership • Ratio < 1.00, optimistically biased • Ratio > 1.00, conservatively biased Mean = 0.65 (n=66) Transit Forecasting Accuracy Database: Garbage In…Garbage Out?

  5. Quantifying Input Inaccuracy • Project service levels • Employment estimates • Project travel time • Population estimates • Project fare • Supporting transit network • Economic conditions • Competing transit network • Auto fuel price • Roadway congestion • Each of the 10 project inputs and exogenous forecasts are placed into 1 of 5 categories: • Also, any differences between the forecast and actual ridership year are adjusted using national historical ridership growth Transit Forecasting Accuracy Database: Garbage In…Garbage Out?

  6. Elasticities Shaded cells reflect selected values Transit Forecasting Accuracy Database: Garbage In…Garbage Out?

  7. Results • Average accuracy ratio: 0.65  0.74 (+14%) • Inaccurate inputs contribute to (0.35-0.26) / 0.35 = 26% of demand forecast inaccuracy Transit Forecasting Accuracy Database: Garbage In…Garbage Out?

  8. Test 1B • Weakness of methodology of Test 1: Exact values of elasticities & values of input inaccuracy are unknown • Perform additional test: randomly vary elasticities and input variability for 10,000 iterations Allowed to vary by ±15ppts Allowed to vary by ±0.3 Transit Forecasting Accuracy Database: Garbage In…Garbage Out?

  9. Mean: 33% Median: 31% Range: 9%, 78% AveDev: 8% Transit Forecasting Accuracy Database: Garbage In…Garbage Out?

  10. Test #2 • Hypothesis: inputs may explain ‘expansion’ project inaccuracy more than for ‘starter’ projects • Uncertain reactions to new modes lowers model’s ability to provide accurate demand forecasts  Input inaccuracies should more fully describe demand inaccuracy for ‘expansion’ projects • Projects divided into starter project (n=31) and expansion project (n=35) groups • Re-ran 10,000 simulations for each group allowing elasticities and input inaccuracy values to vary Transit Forecasting Accuracy Database: Garbage In…Garbage Out?

  11. Mean: 42% Median: 41% Range: 16%, 88% AveDev: 9% Transit Forecasting Accuracy Database: Garbage In…Garbage Out?

  12. Observations • No test confirmed the anecdotal explanations for demand forecast inaccuracy or bias • Input inaccuracies do not appear to explain demand forecast bias • Input inaccuracies ‘explain’ less than 50% of forecast demand inaccuracy • Evidence suggests that other causes of demand inaccuracy and bias exist • Knowledge of travel patterns on modes already in operation within the region seems to heighten the impact of inputs on demand forecast accuracy Transit Forecasting Accuracy Database: Garbage In…Garbage Out?

  13. Thank you! David Schmitt, AICP daves1997@gmail.com dschmitt@ctgconsult.com www.transportforecastaccuracy.com Transit Forecasting Accuracy Database: Garbage In…Garbage Out?

  14. References • Schmitt, David. “Beginning to Enjoy the 'Outside View': A First Glance at Transit Forecasting Uncertainty and Accuracy Using the Transit Forecasting Accuracy Database”. 2015. http://trbappcon.org/2015conf/presentations/143_2015-05-19%20Transit%20Forecasting%20Accuracy%20Database%20Summary%20v5%20-%20with%20script.pptx • Taleb, Nassim Nicholas. Fooled By Randomness: The Hidden Role of Chance in Life and in the Markets: Second Edition. Random House, 2005-08-23. iBooks. • Taleb, Nassim Nicholas. Antifragile: Things That Gain from Disorder. Random House, 2012-11-27. iBooks. • Taleb, Nassim Nicholas. The Black Swan: Second Edition. Random House Trade Paperbacks, 2010-05-11. iBooks. • Transportation Research Board. TCRP Report 95: Traveler Response to Transportation System Changes. 2004. • U.S. Department of Transportation: Federal Transit Administration. Before-and-After Studies of New Starts Projects [annual reports to Congress]. 2007-2016. • U.S. Department of Transportation: Federal Transit Administration. Predicted and Actual Impacts of New Starts Projects: Capital Cost, Operating Cost and Ridership Data. September 2003. • U.S. Department of Transportation: Federal Transit Administration. The Predicted and Actual Impacts of New Starts Projects - 2007: Capital Cost and Ridership. April 2008. • U.S. Department of Transportation: Transportation Systems Center. Urban Rail Transit Projects: Forecast Versus Actual Ridership and Costs. October 1989. • U.S. Department of Transportation: Federal Transit Administration. Travel Forecasting for New Starts: A Workshop Sponsored by the Federal Transit Administration. Phoenix and Tampa, 2009. • U.S. Department of Transportation: Travel Model Improvement Program Webinar: Shining a Light Inside the Black Box (Webinar I). February 14, 2008. • Victoria Transport Policy Institute. “Transit Price Elasticities and Cross-Elasticities”. 2004-2016. http://www.vtpi.org/tranelas.pdf. Transit Forecasting Accuracy Database: Garbage In…Garbage Out?

More Related