1 / 48

Using Continual Monitoring to Identify, Investigate and Prevent / Remediate ELISA Method Failures

Using Continual Monitoring to Identify, Investigate and Prevent / Remediate ELISA Method Failures. Julie Smith, Ph.D. Product Safety, Protein Analytics and CharaterizationTeam Syngenta Crop Protection May 5, 2016. Syngenta Product Safety.

nmillard
Download Presentation

Using Continual Monitoring to Identify, Investigate and Prevent / Remediate ELISA Method Failures

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Using Continual Monitoring to Identify, Investigate and Prevent / Remediate ELISA Method Failures Julie Smith, Ph.D. Product Safety, Protein Analytics and CharaterizationTeam Syngenta Crop Protection May 5, 2016

  2. Syngenta Product Safety • A global team of approximately 300 experienced scientists dedicated to ensuring the safety of Syngenta products Classification: Public

  3. Syngenta Product Safety • Syngenta Product Safety generates data used to assess the safety of our products to both consumers and the environment • Our products are genetically modified (GM) crops that enable plants to better tolerate herbicides or resist damage from insect pressure Classification: Public

  4. Syngenta Product Safety Characterization and Safety Assessment of Biotech Trait Crops • Internal and external quality systems to ensure high quality • Quality Management System • GLP Standards • Global Guidelines (e.g. OECD) • Over 80 scientific studies are conducted on every one of our products to ensure that they are safe • Event characterization • Safe to eat? • Safe for the environment? • ELISA methods are used to quantitate GM proteins in multiple plant tissues throughout plant development, and multiple plant products throughout processing Classification: Public

  5. Overview • Introduction to protein quantitation • Generating and qualifying an assay control • Monitor method performance • Identifying and investigating aberrant trends • Case studies Classification: Public

  6. Quantitating Protein Expression • Protein levels in the crop are quantitated in different tissues throughout the developmental life cycle of the plant • These data are used to characterize the product and to predict and calculate potential environmental exposure levels Classification: Public

  7. ELISA Basics Add conjugate; wash Add sample; wash Add antibody; wash Capture anitbody Protein of interest binds Detectionanitbody Anitbody conjugated with chromogenic enzyme; wash Add substrate; measure color intensity Classification: Public

  8. Validation Criteria • Validation criteria: • Minimum dilution factor • Protein extraction efficiency • Accuracy (upper and lower limits of quantitation) • Lower limit of detection • Precision (intra-assay and inter-assay) • Curve fit and linearity • Suitability for use in stability testing • Stability (standard, control and samples) • Cross-reactivity • Control qualification Not done yet….. Classification: Public

  9. DMAIC in the Laboratory No aberrant trends; Assay verified Classification: Public

  10. Control Purpose and Characteristics • Used to determine assay validity by challenging: • Standard curve • Operator technique • Equipment • Reagents • Different material than the standard • As much variability as reasonable • Number and experience of operators • Independent executions • Time frame to generate qualification data • Homogeneous if multiple vials Classification: Public

  11. Control Qualification: Homogeneity • Homogeneity testing • Assay control vials from beginning, middle and ending of the control dispensing period • Data should agree within assay variability • Data should be randomly distributed, not trending up or down correlating with dispensing 8% CV among vials Classification: Public

  12. Control Qualification: Specification Range • Specification range • At least 30 independent data points • Relative to qualified standard curve material • Challenge multiple regions of the standard curve • Specification range and acceptable assay failure rate Classification: Public

  13. DMAIC in the Laboratory Aberrant trend No aberrant trends; Assay verified Classification: Public

  14. How to Monitor Method Performance • Positive Assay Control Samples are included on each ELISA plate • Control data are analyzed monthly to monitor assay performance • Aberrant trends include: • 9 consecutive points on one side of the mean • 6 consecutive increasing/decreasing points • 14 points alternating up and down about the midline Classification: Public

  15. How to Monitor Method Performance • Aberrant trend in data signals the need for an investigation, but does not directly identify cause of failure Classification: Public

  16. Introduction to Case Study #1 • A validated quantitative ELISA method is successfully transferred for use at a third party • ELISA method performance is being monitored in-house • The third party laboratory reports that assays are failing • Raw data from both the standard curve and the controls seems low • Third party suspects the ELISA kit plates are the problem and requests assistance in investigation Define Control Classification: Public

  17. Case Study #1: Investigation • Causes of assay failure: • Operator error • Equipment error • Reagents • Standard / control failure • Other Classification: Public

  18. Case Study #1: Investigation • Causes of assay failure: • Operator error • Equipment error • Reagents • Standard / control failure • Other X Classification: Public

  19. Case Study #1: Review Recent Control Data • Data generated at 2 locations • Review at least 3 months of data to identify subtle trends • NOTE: These data are generated in batches of 3 – 10 points per test date • No clear, aberrant trend apparent • No unexpectedly high failure rates • Differences between sites still include batch of ELISA plate kits and potentially storage conditions of standard and control Classification: Public

  20. Case Study #1: Experimental Design • Operator: Testing performed by a single, experienced operator • Equipment: Only use Syngenta equipment • Reagents: Non-assay specific reagents are all from Syngenta • Standard/Control: Direct comparison of both standards and both controls on each batch of plates Syngenta Syngenta External External Standard Curve Standard Curve Control Control Standard Curve Standard Curve Control Control External ELISA Plate Syngenta ELISA Plate Classification: Public

  21. Case Study #1: Results • Everything relative to the External standard curve was invalid, regardless of plate source • External Control is still qualified for use • Why we weren’t surprised • Remediation: Provide new material, reinforce storage specifications Classification: Public

  22. Introduction to Case Study #2 • A validated ELISA method is being used at two laboratories • Both laboratories report unexpectedly high invalid rates • The analysts think the control is the problem • Control specification range was inappropriately set as it did not include data generated from one of the laboratories • The control is unstable and exhibiting vial to vial variability Define Control Classification: Public

  23. Case Study #2: Investigation • Causes of assay failure: • Operator error: • All operators are trained, no calculation errors in preparation or analyses • Equipment error: • Both laboratories have the same models of equipment • All equipment is properly calibrated • Other ELISA methods that use the same equipment are valid • Reagents • Standard / control failure • Other Classification: Public

  24. Case Study #2: Review Recent Control Data • Data generated at 2 locations • Review at least 3 months of data to identify subtle trends • NOTE: These data are generated in batches of 3 – 10 points per test date • No clear, aberrant trend apparent • No unexpectedly high failure rates • Differences between sites still include batch of ELISA plate kits and potentially storage conditions of standard and control Classification: Public

  25. Case Study #2: A Different Perspective Classification: Public

  26. Case Study #2: A Different Perspective • Look at each analyst’s failure rate • The three newest operators account for ~70% of total invalid data, but only ~20% of total data Classification: Public

  27. Case Study #2: Investigate Reagent Handling for Control • Control is stored at a 1:20,000 X concentration • Operators dilute control at time of use in 2 steps • 5µl + 50ml buffer for 1:10,000 dilution • 3ml of the 1:10,000 dilution + 3 ml buffer for 1:20,000 Corrective Actions • Review best-practice for pipetting with operators • Revise SOP to include step by step instructions for dilution series Classification: Public

  28. Introduction to Case Study #3 • A validated quantitative ELISA method is successfully transferred for use in GLP studies • The ELISA method performance is being monitored in-house • Inventory of the ELISA control is low and new material is required • Qualification data for new control material triggers an investigation into ELISA method performance Define Control Classification: Public

  29. Case Study #3: Investigation Data generated by four operators with varying experience Concentration of control: 382 – 2652 µg/ml Variability among control dilutions: 0.3 – 21.1 % CV Classification: Public

  30. Case Study #3: Investigation • Causes of assay failure: • Operator error • Equipment error • Reagents • Standard / control failure • Other Classification: Public

  31. Case Study #3: Investigation • Causes of assay failure: • Operator error • Operators were all trained • Increased variability is not correlated with operator experience • Equipment error • Reagents • Standard / control failure • Other Classification: Public

  32. Case Study #3: Investigation • Causes of assay failure: • Operator error • Equipment error • Increased variability is not correlated with use of specific plate washers, plate readers or pipets • Reagents • Standard / control failure • Other Classification: Public

  33. Case Study #3: Investigation • Causes of assay failure: • Operator error • Equipment error • Reagents • All operators used the same preparations of buffers, diluents and wash solutions • Data suggest whole plate variability, not just control variability • Standard / control failure • Other Classification: Public

  34. Case Study #3: Investigation • Causes of assay failure: • Operator error • Equipment error • Reagents • Standard / control failure • Raw absorbance data for the standard curve points indicate protein standard stability • Aberrant data are very high values which is not indicative of control failure • Other Classification: Public

  35. Case study #3: Investigation • Detection antibody (reagent) is stored at a 1:10,000 X concentration • Very viscous reagent • Operators dilute reagent at time of use: 5µl + 50ml buffer Corrective Actions • Review best-practice for pipetting with operators • Store reagent at 100 X concentration • Operator makes a 100 X dilution into buffer at time of use Classification: Public

  36. Case Study #3: Control • If our remediation solved the problem, then we should be able to resume control qualification • Just in case the first problem we identified wasn’t the only problem, the control qualification experimental design was modified Define Control Classification: Public

  37. Case Study #3: Control • Re-initiate control qualification and determine if the variability is reduced • All operators use the same plate design • Generate 6 control values for each of two plates • Same control and standard on each of two plates • Different preparation of detection antibody for each of two plates Classification: Public

  38. Case Study #3: Control • Data generated by five operators with varying experience • Concentration of control: 476 – 786 µg/ml • Variability among control dilutions: 0.2 – 13.2 % CV • Reagent handling reduced variability to an acceptable range • Now a different aberrant trend is identified: • Assay drift Define Control Classification: PUBLIC

  39. Case Study #3: Investigation #2 • For the 60 control results generated by five operators, the data generated on the left side of the plate were higher than data generated on the right side of the plate in 57 cases • For three results, data from the middle of the plate were used • Most of the results are still within 5% of each other Classification: Public

  40. Case Study #3: Investigation #2 • Is the assay drift due to addition of the enzymatic substrate, the detection antibody, the sample, or a combination of these reagents? • Hypothesis: Plates are drying too much between one or more wash steps, and the wells are not rehydrating equally • Design experiments that will exacerbate the timing variability associated with addition of each reagent Classification: Public

  41. Case Study #3: Investigation #2 (Substrate Addition) Substrate added to top of plate 2 minutes before addition to bottom of plate Results are within assay variability 1 2 3 4 5 6 7 8 The timing of substrate addition is not our primary problem. Classification: Public

  42. Case Study #3: Investigation #2 (Detection Antibody and Sample Additions) Hypothesis: Plates are drying too much between wash steps, and the wells are not rehydrating equally Design experiment to exacerbate timing variability associated with sample addition and detection antibody Sample added to top of plate before addition to bottom of plate Antibody added to left of plate before addition to right of plate Classification: Public

  43. Case Study #3: Investigation #2 (Detection Antibody Addition) Hypothesis: If the sample is not having hydration problems prior to addition of the secondary antibody, results in quadrants 1 and 3 should be similar Conclusion: Results from the left of the plate are not consistently higher than results from the right of the plate. The timing of secondary antibody addition is not our primary problem. Classification: Public

  44. Case Study #3: Investigation #2 (Sample Addition) Hypothesis: If the capture Ab is not having hydration problems prior to addition of the sample, quadrants 1 and 2 should be similar; quadrants 3 and 4 should be similar 1 2 3 4 5 6 7 8 Conclusion: Results from the top of the plate are consistently higher than results from the bottom of the plate. The timing of sample addition is important. Classification: Public

  45. Case Study #3: Remediation • Store detection antibody at a lower concentration and revise SOP to include step by step instructions for dilution series • Minimize plate drying prior to sample loading • Store coated, blocked plates under buffer until just prior to loading • Gently remove buffer from plate just prior to loading • Do not bang out every last drop of buffer • Do not remove buffer from more than one plate at a time Classification: Public

  46. Case Study #3: Control Classification: Public

  47. Summary • Define and monitor relevant measures to verify validity of each assay execution • Define aberrant trends and unacceptable invalid rates • Investigate errors due to Operator, Equipment, Reagent, and Standard/Control, even if the analyst “knows” what the problem is • Thorough investigations can reveal multiple errors that all need mitigation Classification: Public

  48. Acknowledgments • Analysts • Alysha Read • Guoling Luo • Tim Sullivan • Beth Matthews • Karrie Christenson • Steven Testerman • Rich Banach • Contractors at Syngenta and external laboratory • Reviewers • Linda Meyer • Gerson Graser • Scott Young • Protein Analytics and CharacterizationTeam members Classification: Public

More Related