1 / 126

THE WILD, WILD, WET!

THE WILD, WILD, WET!. SETAC Expert Advisory Panel Performance Evaluation and Data Interpretation. THE PERFECT WORLD. FOCUS ON DATA ANALYSIS. STEP 1: GRAPH THE DATA STEP 2: Analyze the data by EPA flowcharts STEP 3: DO THE RESULTS MAKE SENSE?. SOFTWARE PROGRAMS.

galia
Download Presentation

THE WILD, WILD, WET!

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. THE WILD, WILD, WET! SETAC Expert Advisory Panel Performance Evaluation and Data Interpretation

  2. THE PERFECT WORLD

  3. FOCUS ON DATA ANALYSIS • STEP 1: GRAPH THE DATA • STEP 2: Analyze the data by EPA flowcharts • STEP 3: DO THE RESULTS MAKE SENSE?

  4. SOFTWARE PROGRAMS • Many software packages/programs are available • DO NOT assume they follow the EPA recommended analysis • DO verify the software by running example datasets from the methods manuals

  5. STATISTICAL AND BIOLOGICAL SIGNIFICANCE SETAC Expert Advisory Panel Performance Evaluation and Data Interpretation

  6. TOXIC VS. NON-TOXIC • How are data from a WET test used to make a decision of toxicity? • Two paths: • Decision based on the observed result • Decision based on standard effect

  7. WHO DECIDES WHICH PATH? • The Permit Writers • BOTH approaches are supported by the TSD and the methods manual

  8. OBSERVED RESULT • Data from the test are used to determine if toxicity is present by hypothesis testing • HO: Effluent is not toxic • Ha: Effluent is toxic

  9. STANDARD EFFECT • A preselected level of effect is considered toxic • Acute test: 50 % effect • Chronic test: 25 % effect

  10. THERE ARE INHERENT STRENGTHS AND WEAKNESSES TO BOTH APPROACHES

  11. COMPONENTS WHICH IMPACT THE NOEC

  12. WHAT BIOLOGICAL CONCLUSIONS CAN BE MADE FROM THE STATISTICAL ANALYSIS OF A SINGLE TOXICITY TEST? The biological impact was significant in the beaker

  13. THE LESS THAN PERFECT WORLD

  14. INTRA- AND INTER-TEST VARIABILITY SETAC Expert Advisory Panel Performance Evaluation and Data Interpretation

  15. TYPES OF VARIABILITY • Intra-test : among and between concentrations • Inter-test: within one lab, same method • Inter-lab: between labs, same method • Method specific: within limits of method

  16. INTRA-TEST VARIABILITY

  17. INTRA-TEST VARIABILITY AND ENDPT. UNCERTAINTY

  18. SOURCES OF INTRA-TEST VARIABILITY • Genetic variability • Organism handling and feeding • Toxicity among and between treatments • Non-homogeneous sample source

  19. SOURCES OF INTRA-TEST VARIABILITY • Abiotic conditions • Dilution scheme • Number of organisms/treatment • Dilution water pathogens

  20. SOURCES OF INTER-TEST VARIABILITY • Intra-test sources • Analyst experience and practice • Organism age and health • Acclimation • Dilution water

  21. SOURCES OF INTER-TEST VARIABILITY • Sample quality • Test chamber characteristics

  22. SOURCES OF INTER-TEST VARIABILITY • Replicate volume • Procedures

  23. VARIABILITY AND POINT ESTIMATE UNCERTAINTY

  24. HIGH VARIABILITY - LOW STATISTICAL POWER

  25. LOW VARIABILITY - HIGH STATISTICAL POWER

  26. ACTIONS TO REDUCE VARIABILITY • Increase number of reps/treatment • QA program • Establish and follow strict procedures • Maximize analyst skill • Contract lab selection • Additional QA/QC criteria

  27. EXAMPLES OF ADDITIONAL QC TEST CRITERIA • Region IX: upper MSD limits • Washington: upper MSD limits, change in alpha • N. Carolina: limit control CVs, C. dubia “PSC” • Region VI: limit control CV, increase # replicates, biological significance

  28. SUSPICIOUS DATA AND OUTLIER DETECTION SETAC Expert Advisory Panel Performance Evaluation and Data Interpretation

  29. CONCERNS • Outliers make interpretation of WET data difficult by • Increasing the variability in test responses • Biasing mean responses

  30. IDENTIFYING OUTLIERS • Graph raw data, means and residuals

  31. IDENTIFYING OUTLIERS • Formal statistical test - Chauvenet’s Criterion • Using the previous mysid data, the critical values are: • Mean = .80, Std. Dev. = 0.302, n = 8 • Chauvenet’s Criterion Value = n/2 = 4 • Z-score = 2.054 (two-tailed probability of 4 %) • The calculations are: • Equation 1) (Z-score)(Std. Dev.) = (2.054)(0.302) = 0.620 • Mean  Equation 1 = 0.80  0.620 = 1.42 - 0.18 • Outlier Range is >1.42 or <0.18 • A value of 0.2 is not an outlier.

  32. CAN A CAUSE BE ASSIGNED TO THE OUTLIER(S) ? • Review analyst’s daily observations • Check water chemistry data • Check data entry • Check calculations • If cause can be assigned to outlier, then reanalyze data without outlier

  33. DETERMINE EFFECT ON TEST INTERPRETATION • Keep all data unless cause is found • Analyze data with and without suspect data • Determine effect of suspect data on test interpretation • Results reported will depend on effect of outlier(s) on test interpretation

  34. Insignificant Effect With Outlier IC25 = 131 (96.9-158) ppb NOEC = 100 ppb % MSD = 28.1 % Without Outlier IC25 = 124 (93.6-152) ppb NOEC = 100 ppb % MSD = 20.9 % Report results with suspect data included Significant Effect With Outlier IC25 = 131 (96.9-158) ppb NOEC = 100 ppb % MSD = 28.1 % Without Outlier IC25 = 106 (83.8-126) ppb NOEC = 50 ppb % MSD = 12.2 % Report results from both analyses REPORTING OF RESULTS

  35. HORMESIS ANDNON-MONOTONIC CONCENTRATION RESPONSES SETAC Expert Advisory Panel Performance Evaluation and Data Interpretation

  36. WHAT IS HORMESIS ? • Calabrese and Baldwin, 1998 • General concept • Occurrence • Typical Characteristics

More Related