A Probabilistic Treatment of Conflicting Expert Opinion

1. A Probabilistic Treatment of Conflicting Expert Opinion Luc Huyse and Ben H. Thacker Reliability and Materials Integrity Luc.Huyse@swri.org, Ben.Thacker@swri.org 45th Structures, Structural Dynamics and Materials (SDM) Conference 19-22 April 2004 Palm Springs, CA

2. Motivation • Avoid arbitrary choice of PDF • Account for vague data • Efficient computational tools • Account for modeluncertainty

3. Probabilistic Assessment • Choice of PDF • Companion paper • Dealing with (conflicting) expert opinion data • Use Bayesian estimation • Efficient Computation • Method must be amenable to MPP-based methods • Epistemic Uncertainty in the decision making process • “Minimum-penalty” reliability level

4. Estimation with Interval Data • Use Bayesian updating • Bayesian updating equation for intervals is

5. Non-informative Priors and the Uniform distribution • Temptation is to assume uniform distribution when nothing is known about a parameter • Non-Informative does NOT necessarily mean Uniform • Illustration: • Choose uniform for X because nothing is known • Choose uniform for X2 because nothing is known • Rules of probability can be used to show that PDF for X2 is NOT uniform • Selecting a uniform because “nothing is known” is not justified

6. Transformation to Uniform • Transformation t exists such that random variable X can be transformed t: X Y where Y has a uniform PDF. • Question is no longer whether a uniform PDF is an appropriate selection for a non-informative prior but under which transformationt: X Y the uniform is a reasonable choice for the non-informative distribution for Y.

7. Data-translated Likelihood • Uniform PDF is non-informative if the shape of the likelihood does not depend on the data • Jeffrey’s principle: uniform PDF is appropriate in space where likelihood is data-translated.

8. Updating with Interval Info • Variable y has a Poisson PDF; estimate mean value of Y • Non-informative prior used • Consider six different updates for mean • Posterior variance decreases as interval narrows • “Weight” of expert depends on length of their interval estimate.

9. Combining Interval & Point Data • Variable y has a Poisson PDF; estimate mean value of Y • Non-informative prior used • Consider five updates for mean • Posterior variance reduces with successive addition of precise observations • Narrow interval contains almost as much information as point estimate • Wide interval estimate still adds some information

10. Conflicting Expert Opinion • Source of conflicting expert opinion • Elicitation questions not properly asked or understood • Correct through iterative expert elicitation process • Each person susceptible to differences in judgment • “Weighting” of expert opinion data has been proposed • Difficult to determine who is “more” right. • Adding weights to experts is therefore a matter of the analyst’s judgment, and should be avoided. • Proposed approach: • Each expert opinion treated as a random sample from a parent PDF describing all possible “expert opinions”. • Weight is related to width of interval • Conflict accounted for automatically in the updating process

11. Bounds reflect epistemic uncertainty 1 0.9 0.8 0.7 As epistemic uncertainty is reduced, bounds collapse to computed CDF 0.6 Reliability 0.5 0.4 0.3 Computed CDF reflects inherent uncertainty 0.2 0.1 0 X Treatment of Model Uncertainty • Separate inherent (X) and epistemic (Q) variables

12. Efficient Computation • Because of model uncertainty Q, b (safety index) is a random variable • Interval estimates with confidence level • Compute CDF of b • Exact confidence bounds determined from CDF • Usually requires numerical tool  NESSUS • First-Order Second-Moment Approximation • Requires only a single reliability computation using the mean value of epistemic variables Q

13. Analytical Example • Limit State Function g = X –Q/100 pf = Pr[g<0] • Assume X is exponential PDF with uncertain mean value l • Q represents model uncertainty: assume Normal(1,s), with s = 0.3 • Estimate the l using 5 interval data (shown) • Reliability b (related to pf) is a function of epistemic parameters l and q

14. Uncertain Reliability Index Confidence bounds shrink when more information is available

15. Decision Making with Epistemic Uncertainty • In a decision making context, a penalty p(b) is associated with using the “wrong” reliability index; the expected value of the total penalty is: • Minimum penalty reliability index minimizes the expected value of the total loss (Der Kiureghian, 1989):

16. btarget Cost function and bmp • Linear penalty function: • k is a measure for the asymmetry of (usually > 1) • Minimum penalty reliability index (Der Kiureghian, 1989) • Normal Approximation

17. Minimum-Penalty Reliability Index • bmp is a “safe” reliability level • This level strongly depends on the severity of the consequence (value k) • bmp increases with the number of experts k = 1 k = 5 k = 20

18. Summary • Proposed method handles both precise and interval (expert opinion) data within probabilistic framework • Conflicting information automatically accounted for • Minimum-penalty reliability index can be estimated from a single reliability computation  Highly efficient • Allows effect of epistemic uncertainties to be determined • Companion paper (tomorrow) will discuss use of a distribution system, whereby the data can determine the shape of the distribution as well as any parameter

19. Model uncertainty Future Work • Amenable to MPP-based solution (future work) • Link to pre-posterior analysis, compute sensitivity of design decision to epistemic uncertainty.

20. Thank You! Luc Huyse & Ben Thacker Southwest Research Institute San Antonio, TX