1 / 27

SNA + MC 2010 Joint International Conference on

Epistemic and systematic uncertainties in Monte Carlo simulation: an investigation in proton Bragg peak simulation. Maria Grazia Pia INFN Genova , Italy. Maria Grazia Pia 1 , Marcia Begalli 2 , Anton Lechner 3 , Lina Quintieri 4 , Paolo Saracco 1 1 INFN Sezione di Genova, Italy

reba
Download Presentation

SNA + MC 2010 Joint International Conference on

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Epistemic and systematic uncertainties in Monte Carlo simulation: an investigation in proton Bragg peak simulation Maria GraziaPia INFN Genova, Italy Maria Grazia Pia1, Marcia Begalli2, Anton Lechner3, Lina Quintieri4, Paolo Saracco1 1 INFN Sezione di Genova, Italy 2 State University Rio de Janeiro, Brazil 3 Vienna University of Technology, Austria 4 INFN LaboratoriNazionali di Frascati,, Italy SNA + MC 2010 Joint International Conference on Supercomputing in Nuclear Applications + Monte Carlo 2010

  2. Quantifying the unknownin Monte Carlo simulation Maria GraziaPia INFN Genova, Italy Maria Grazia Pia1, Marcia Begalli2, Anton Lechner3, Lina Quintieri4, Paolo Saracco1 1 INFN Sezione di Genova, Italy 2 State University Rio de Janeiro, Brazil 3 Vienna University of Technology, Austria 4 INFN LaboratoriNazionali di Frascati,, Italy SNA + MC 2010 Joint International Conference on Supercomputing in Nuclear Applications + Monte Carlo 2010

  3. Epistemic uncertainties Epistemic uncertainties originate from lack of knowledge Relatively scarce attention so far in Monte Carlo simulation Studies in deterministic simulation (especially for critical applications) Possible sources in Monte Carlo simulation • incomplete understanding of fundamental physics processes, or practical inability to treat them thoroughly • non-existent or conflicting experimental data for a physical parameter or model • applying a physics model beyond the experimental conditions in which its validity has been demonstrated Epistemic uncertainties affect the reliability of simulation results Can we quantify them?

  4. Uncertainty quantification • Epistemic uncertainties are difficult to quantify • due to their intrinsic nature • No generally accepted method of measuring epistemic uncertainties • and their contributions to reliability estimation • Various formalisms developed in the field of deterministic simulation • Interval analysis • Dempster-Shafer theory of evidence • Not always directly applicable in Monte Carlo simulation • Adapt, reinterpret, reformulate existing formalisms • Develop new ones specific to Monte Carlo simulation

  5. Benefits of quantifying uncertainties • Epistemic uncertainties are reducible • Can be reduced or suppressed by extending knowledge • New experimental measurements • Uncertainty quantification gives us guidance about • What to measure • What experimental precision is needed/adequate • Priorities: which uncertainties generate the worst systematic effects • Measurements are not always practically possible • Uncertainty quantification to control systematics

  6. Warm-up exercise Epistemic uncertainties quantification in proton depth dose simulation simplicity complexity

  7. Ingredients • pstopping powers • Water ionisation potential • d-ray production • Multiple scattering • Nuclear elastic • Nuclear inelastic • Cross sections • Preequilibrium • Deexcitation • Intranuclear cascade • EGS5, EGSnrc • Penelope • MCNP(X) • PHITS • SHIELD-HIT • FLUKA • GEANT 3 • SPAR, CALOR, CEM, LAHET, INUCL, GHEISHA, Liège INCL, Bertini • d-rayorno d-ray • Preequilibrium or no preequilibrium • Weisskopf-Ewing or Weisskopf-Ewing • Griffin-exciton or hybrid • etc.

  8. Geant4 physics options Water ionization potential set through the public interface of G4Material

  9. “Validation” in the literature • Beam energy (and energy spread) is not usually known with adequate precision in therapeutical beam lines • What matters in clinical applications is the range • Typical procedure: optimize the beam parameters to be used in the simulation by fitting them to experimental data • Determine beam energy, energy spread etc. • Use optimized beam parameter values in the simulation • This is a calibration • This is NOT validation T. G. Trucano, L. P. Swiler, T. Igusa, W. L. Oberkampf, and M. Pilch, “Calibration, validation, and sensitivity analysis: What’s what”, Reliab. Eng. Syst. Safety, vol. 91, no. 10-11, pp. 1331-1357, 2006.

  10. Simulation environment • Realistic proton beam line • Geometry from Geant4 hadrontherapy advanced example • G. A. P. Cirrone, G. Cuttone, S. Guatelli, S. Lo Nigro, B. Mascialino, M. G. Pia, L. Raffaele, G. Russo, M. G. Sabini,“Implementation of a New Monte Carlo GEANT4 Simulation Tool for the Development of a Proton Therapy Beam Line and Verification of the Related Dose Distributions”, IEEE Trans. Nucl. Sci., vol. 52, no. 1, pp. 262-265, 2005 • Water sensitive volume, longitudinal 200 mm slices (through G4ReadoutGeometry) • Proton beam: E = 63.95 MeV, sE = 300 keV • Physics modeling options configured through an application design based on G4VModularPhysicsList • 1 million primary protons • Geant4 8.1p02, 9.1(ref-04), 9.2p03, 9.3

  11. Reference physics configuration Wellisch & Axen Wellisch & Axen

  12. General features electromagnetic electromagnetic + hadronic elastic electromagnetic + hadronic elastic + hadronic inelastic electrons 59.823 MeV peak s=376 keV

  13. Water mean ionisation potential Ep = 63.95 MeV I = 75 eV, 67.2 eV, 80.8 eV Ep = 63.65 MeV(1s from 63.95 MeV) I = 80.8 eV GoF tests Bragg-Bragg p-value = 1 (Kolmogorov-Smirnov, Anderson-Darling, Cramer-von Mises)

  14. Proton stopping powers ICRU49 Ziegler77 Ziegler85 Ziegler2000 Differences would be masked by typical calibration of simulation input parameters

  15. Hadronic elastic scattering U-elastic Bertini-elastic LEP(GHEISHA-like) CHIPS-elastic Wald-Wolfowitz test: p-value< 0.001 p-value (reference: U-elastic) Bertini LEP CHIPS Difference of deposited energy in longitudinal slices

  16. Hadronic inelastic cross sections Difference of deposited energy in longitudinal slices GHEISHA-like Wellisch & Axen Bragg peak profiles p-value > 0.9 (Kolmogorov-Smirnov, Anderson-Darling, Cramer-von Mises) 99% confidence interval for inelastic scattering occurrences in water (Wellisch & Axencross sections): 1688-1849 Occurrences with GHEISHA-like cross sections: 1654

  17. Hadronic inelastic scattering models p-value (reference: Precompound) No visible difference in Bragg peak profiles Wald-Wolfowitz test p-value< 0.001 for all model options except p-value=0.360 for Liège cascade preequilibrium = no preequilibrium

  18. Hadronic inelastic differences secondary p Precompound Bertini LEP Liège CHIPS Bertini LEP Liège CHIPS Difference of deposited energy in longitudinal slices reference: Precompound secondary n Precompound Bertini LEP Liège CHIPS Wald-Wolfowitz test: p-value < 0.001

  19. reference: default Evaporation Nuclear deexcitation Difference of deposited energy in longitudinal slices GEM evaporation Geant4 < 9.3 (bug fix) default evaporation GEM evaporation Fermi break up Binary Cascade Difference of deposited energy in longitudinal slices Fermi break-up

  20. Cascade-preequilibrium Precompound model activated through Binary Cascade w.r.t. standalone Precompound model Difference of deposited energy in longitudinal slices Transition between intranuclear cascade and preequilibrium determined by empirical considerations In Geant4 Binary Cascade model cascading continues as long as there are particles above a 70 MeV kinetic energy threshold (along with other conditions required by the algorithm) systematic effect

  21. Some get lost on the way… 95% confidence intervals July 2006 December 2009 Calibration: 50 and 200 GeV

  22. G4MultipleScattering G4hMultipleScattering Multiple scattering Reference: Geant4 9.3 G4hMultipleScattering Difference: G4MultipleScattering in Geant4 9.3 9.19.2p038.1p02 G4hMultipleScattering, Geant4 9.3 G4MultipleScattering, Geant4 9.3 G4MultipleScattering, Geant4 9.2p03 G4MultipleScattering, Geant4 9.1 G4MultipleScattering, Geant4 8.1p02 Difference of deposited energy in longitudinal slices

  23. Goodness-of-fit

  24. 99.9% CI Total deposited energy Acceptance 9.3 hMS 9.3 9.2p03 9.1 8.1p02 9.3 hMS 8.1p02 99.9% CI 9.3 hMS 9.3 9.2p03 9.1 8.1p02

  25. fresh from the oven… M.G.Pia, M. Begalli, A. Lechner, L. Quintieri, P. Saracco Physics-related epistemic uncertainties in proton depth dose simulation IEEE Trans. Nucl. Sci., vol. 57, no. 5, pp. 2805-2830, October 2010

  26. Conclusions The impact of epistemic uncertainties depends on the experimental application environment • Evaluation of systematic effects associated with epistemic uncertainties • Sensitivity analysis (~interval analysis) • More refined methods: Dempster-Shafer • Methods specific to Monte Carlo simulation? • Complementary statistical methods contribute to identify and quantify effects • Qualitative appraisal is not adequate • Epistemic uncertainties are reducible • Can be reduced or suppressed by extending knowledge • New experimental measurements • Uncertainty quantification gives us guidance about • What to measure • What experimental precision is needed/adequate • Priorities: which uncertainties generate the worst systematic effects

  27. Backup Geant4/examples/extended/electromagnetic/testEm5/mumsc/deviation.ascii

More Related