1 / 8

He Says vs. She Says Model Validation and Calibration

He Says vs. She Says Model Validation and Calibration. Kevin Chang HNTB Corporation kchang@hntb.com. Model Validation and Calibration Keys to a Successful Simulation Model Model Validation Concept Stories on Model Validation Lessons Learned. Contents.

fabiano
Download Presentation

He Says vs. She Says Model Validation and Calibration

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. He Says vs. She SaysModel Validation and Calibration Kevin Chang HNTB Corporation kchang@hntb.com

  2. Model Validation and Calibration • Keys to a Successful Simulation Model • Model Validation Concept • Stories on Model Validation • Lessons Learned Contents

  3. CALIBRATION – an iterative procedure to fine tune model parameters and settings so that the model can achieve what the modeler wants it to perform. • VALIDATION – an analytical process to verify if the model’s behavior and output statistics can truly represent actual traffic system operations. • PURPOSE – to have a valid simulation model that is able to generate representative numerical results that replicate traffic operations in the modeled network for analyses. Model Validation and Calibration

  4. Model Calibration • Results may be limited by the tool used • Modeler’s knowledge of the simulation tool • Usually the most time consuming process • Model and Model Validation • Law and Kelton (1991) : “a simulation model of a complex system can only be an approximation to the actual system.” • Pegden et al. (1995) : “no model can ever be absolutely correct”. “A model is created for a specific purpose, and its adequacy or validity can only be evaluated in terms of that purpose.” • Fu, M : “Model validation is more an art work than science.” Facts

  5. Use the Right Tool • Modeler’s knowledge on the • System: traffic environment, operations, controls, management, etc. • Tools used • Issues to be addressed • Data availability • Usually the most critical element : availability and accuracy • Model Validation • Right people to review validation results • Selection of validation objects and focus on objectives • Art of work • Sometimes good luck KEYS TO A SUCCESSFUL MODEL

  6. Actual System Highway/Freeway Network Traffic Flow Traffic Operations Modeled System ? = • Model Logic • - Vehicle movements and interactions • Traffic assignment, routing decision • Weaving, merging, lane change • Queuing and delay • Traffic controls and managements • etc. Input Data Output Statistics Model Validation Concept

  7. Inaccurate traffic demand for model inputs • Demand vs. flows (throughputs) • Incompatible performance measures • Delays: definition and collection • LOS criteria • Average, maximum, 90th percentile, etc. • Inconsistent data • Data collected at different times, locations, methods, etc. • Validating MOE against MEMORY • Usually best or worst scenario will be memorized • Always talk to the right person with field experience • MOE selection • Quantifiable and collectable with a clear definition: queue length, delay vs. speed, link density STORIES ON MODELVALIDATION

  8. Know your tools • Clear about project issues, system environment, scope of work • Always budget for data collection and analysis • Select “right” MOEs for model validation and presentation • Talk to the right person • Upgrade hardware and update software –be sensitive to the time required to run your models and to get output stats • Art of work and good luck LESSONS LEARNED

More Related