1 / 21

Expert-Opinion Elicitation

Expert-Opinion Elicitation. Robert C. Patev North Atlantic Division – Regional Technical Specialist (978) 318-8394. Expert-Opinion Elicitation. Subjective Estimation Elicitation Process Background Expert-Opinion Elicitation (EOE) Process Probability Axioms of Probability

jerry-johnson
Download Presentation

Expert-Opinion Elicitation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Expert-Opinion Elicitation Robert C. Patev North Atlantic Division – Regional Technical Specialist (978) 318-8394

  2. Expert-Opinion Elicitation • Subjective Estimation • Elicitation Process • Background • Expert-Opinion Elicitation (EOE) Process • Probability • Axioms of Probability • Medians and Percentiles • Training Example

  3. Subjective Estimation • Uses of one or more experts to estimate a probability (qualitative or quantitative) for use in engineering risk analysis • Good for first estimate of probabilities • Quick, cost effective and efficient method • Problems: • Not a formal elicitation • Usually not well documented • Probabilities may not be repeatable or defendable • Probabilities may be highly subjective and biased • Probabilities have larger uncertainties compared to structured elicitation values

  4. Subjective Estimation • How good are we at quantifying subjective estimates? • Let us see…..

  5. Subjective Estimation • How good are we at quantifying subjective estimates? • Class Example: • How may ships passed through the Panama Canal last year? • Give best estimate

  6. Expert-Opinion Elicitation • Background • Process developed by RAND Corporation in late 1950’s - early 1960’s • Delphi Method • Scenario Analysis • Effects of thermonuclear war • Civil Defense strategic planning • Examine if U.S. population could survive a nuclear attack

  7. Expert-Opinion Elicitation • Background • Definition • A formal (protocol), heuristic (through discussion) process of obtaining information or answers to specific questions called issues • e.g., failure rates or probabilities, and failure consequences

  8. Expert-Opinion Elicitation • Background • EOE is used for preliminary risk evaluation (screening) is not really intended to replace more complex reliability models • EOE has been used by industry and government agencies to develop failure probabilities when there is a lack of failure information

  9. Expert-Opinion Elicitation • Drawbacks • Subjective process • Not consensus building • Inherently contains bias and dominance • Difficult to process result to determine reliability or hazard rates • Assumptions need to be made • Current Usage in USACE • Supplement to other models • Calculate reliability (not for critical components) • Event tree probabilities • Used in consultation with HQUSACE

  10. Expert-Opinion Elicitation • EOE Process • Participants • Experts • Observers • Listeners • Technical Integrator and Facilitator • Peer Reviewers • ITR process and results

  11. Expert-Opinion Elicitation • EOE Process • Identification and Selection of Experts • Strong relevant expertise • Familiarity and knowledge with issues • Willingness to act as impartial evaluators • Willingness to participate, prepare, and provide needed input • Strong communication skills, interpersonal skills, and ability to generalize

  12. Expert-Opinion Elicitation • EOE Process • Inform experts of issues • “Read ahead” materials • Site visits • Train experts • Elicitation • First opinion • Discussion among experts • Second opinion

  13. Expert-Opinion Elicitation • Probability • General expressions • Percent (1% probability of failure) • Fraction (1/100) • Relative frequency (1 out of 1000) • Axioms of Probability • 0 < Pf < 1 • Sum of probabilities over all possible outcomes must equal 1. • This assume events are independent.

  14. Expert-Opinion Elicitation • Statistics • Median • e.g., Median income, median age • Rank value • For odd n, value with rank of (n+1)/2 • For even n, average of value with rank n/2 or (n/2) + 1 • Used to limit extreme values • Average • Sum of Xi divided by sample size

  15. Sample 1 100 100 200 300 400 Median = 200 Average = 220 Sample 2 100 100 200 300 2000 Median = 200 Average = 540 Median Vs. Average

  16. Expert-Opinion Elicitation • Percentiles • A p-percentile value (Xp) based on a sample is the value of the parameter such that p% of the data is less than or equal to Xp • e.g., The median is the 50th percentile

  17. Chickamauga Lock and Dam

  18. Chickamauga Lock and Dam

  19. Chickamauga Lock and Dam

  20. Chickamauga Lock and Dam

  21. Expert-Opinion Elicitation • Class Example • Six experts required • Unknown issue given to experts • Define assumptions of issue • Elicit first values • First results • Expert Discussion • Elicit second values • Show final elicitation results

More Related