EPA’s EDSP Validation Strategy

1. EPA’s EDSP Validation Strategy

2. Objectives of Briefing • To communicate the strategy and assumptions EPA plans to use in the development of prevalidation and validation study plans for individual assays • To obtain the EDMVS’s guidance on the strategy and assumptions • To develop a standard vocabulary of key terms to facilitate communication

3. KEY CONCEPTSIN EPA VALIDATION APPROACH • Select core chemicals to permit comparison of assays augmented by chemicals specific to each assay • More chemicals in prevalidation than in validation phase to establish relevance of assay in prevalidation • Select small number of chemicals in validation phase • Empirically validate assay rather than battery • Battery validation will be an analysis of the test results of individual assays

4. Key Terms • Relevance is extent to which a test is related to the effect of interest and the test’s utility for a specified purpose. (ICCVAM) • Reliability is a measure of the degree to which a test can be performed reproducibly within and among laboratories over time. (ICCVAM)

5. Validation Processfor EDSP • Method development and preparation of Detailed Review Paper (DRP) • Pre-validation • Demonstration of relevance • Development of standard optimized protocol • Determination of readiness for validation in consultation with EDVMS and ICCVAM • Validation in multiple laboratories • Demonstrate reliability across labs and over time • Independent peer review of validation effort

6. Detailed Review Paper • Explains basic purpose of the assay and the context in which it will be used • Explains scientific principles upon which the assay rests • Reviews candidate protocols and compares them with respect to meeting purpose, cost and other practical considerations • Identifies their developmental status and information needs • Recommends a protocol for initiation of prevalidation

7. Prevalidation • Types of prevalidation studies depend on state of development of protocol • Protocol demonstration • Confirms results in literature • Gives lead laboratory experience in conducting assay • Uses small number of chemicals

8. Prevalidation • Special studies (as needed) • Address specific questions that arise from the DRP or during the course of the protocol demonstration • Protocol optimization study • Conducted to refine the protocol and eliminate non-sensitive or duplicative endpoints • Multi-chemical study • Primary test of relevance • Should state expected outcomes before conducting • Typically 8-12 chemicals whose mode of action is known • Conducted blind to eliminate bias

9. Prevalidation • For new assays, at least one prevalidation study will be conducted in more than one laboratory to get an estimate of interlaboratory variability • Prevalidation study report • Summarizes and analyzes the results of prevalidation studies • Addresses question of readiness for validation • EDMVS review is essential before beginning interlaboratory validation

10. Validation • Purpose: • Tests transferability of protocol to other laboratories • Determines reliability of protocol • Approach: • Conduct identical studies in 3 laboratories • Use small number of chemicals in studies (one per major mode of action) • Validation studies conducted according to GLPs • Flexibility needed for addition of new endpoints to existing guideline.

11. Validation • Validation report • Will summarize data from each participating laboratory • Analyzes key parameters that allow a determination to be made regarding reliability • Integrated summary report • Summarizes background data, prevalidation report and validation report • Addresses adequacy of validation • Follows ICCVAM BRD format

12. Peer Review • An independent peer review panel will be convened to review groups of related assays • Peer review panel could be convened by EPA (SAP/SAB joint panel), NICEATM, or a contractor • All reports would go to the peer review panel. Raw data would be available upon request.

13. Issues /Answers • Should the primary demonstration of relevance, i.e., the multi-chemical study, be performed during prevalidation rather than during the validation phase? • EDMVS endorsed this concept but noted that since some new chemicals should be used in the interlaboratory validation studies, data on relevance would be acquired at this phase too.

14. Issues/Answers • Is three a reasonable minimum number of laboratories to use during validation? • The answer to this question is really assay specific. Some may require more and it may be reasonable to have fewer than three in other cases. • A power analysis should be conducted at the end of prevalidation to assist in determining the optimum number of laboratories.

15. Issues/Answers • How critical is it to include more than one laboratory at the prevalidation stage? • In general, it is important to get a sense of the transferability of protocols and variability between laboratories before beginning validation. One needs this information to determine how many laboratories are necessary in validation (see power analysis comment). • Validation should be thought of as a confirmation of what one has learned in prevalidation.

16. Issues/Answers • Does EDMVS want to be involved in the design of and review the results of special studies? • The suggestion was made that study plans should be sent by e-mail to permit comment by members. Significant issues could be addressed by conference call or at a subsequent meeting. • We also agreed to revisit this issue after discussion of the avian dosing and fish screening comparison studies.

17. Issues/Answers • Is it reasonable for the validation of the Tier 1 screening battery to be a paper exercise in which the performance of assays on a core group of chemicals is compared? • This is a reasonable expectation and should be the Agency’s goal; however, EPA should be prepared to conduct additional studies in which chemicals are run through the complete Tier 1 battery if validation data on individual assays data do not support a clear determination on the composition of the battery.

18. Issues for Future Discussion • Should laboratories involved in validation be trained in the use of the protocol by the lead laboratory, or should the labs be naïve with respect to the protocol? • Ultimately, the product of the EDSP validation program will be a set of test guidelines. What work needs to be done during prevalidation and validation to allow the construction of test guidelines?

19. Closing Thought It is the mark of an instructed mind to rest satisfied with the degree of precision which the nature of the subject admits and not to seek exactness when only an approximation of the truth is possible. Aristotle