1 / 33

Generalities & Qualitative Testing Plans

Generalities & Qualitative Testing Plans. May 8-10, 2006 Iowa State University, Ames – USA Jean-Louis Laffont Kirk Remund. Objectives. Introduce Acceptance Sampling review assumptions definitions understand strengths & limitations Use with a qualitative assay zero tolerance plans

mallory-knight
Download Presentation

Generalities & Qualitative Testing Plans

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Generalities & Qualitative Testing Plans May 8-10, 2006 Iowa State University, Ames – USA Jean-Louis Laffont Kirk Remund

  2. Objectives • Introduce Acceptance Sampling • review assumptions • definitions • understand strengths & limitations • Use with a qualitative assay • zero tolerance plans • plans that allow deviants • purity testing ISTA Statistics Committee

  3. Challenges: random sampling variability Seed Lot Sample 0.09% 0.07% 0.12% 0.11% 0.05% ISTA Statistics Committee

  4. Challeges: Sampling & Assay Variability Seed Lot Sample Sampling Error 0.12% 0.15% Assay System Error 0.09% Assay (PCR) < 0.10% Sample Prep ISTA Statistics Committee

  5. Benefits of acceptance sampling approach • Manage sampling variability & assay errors • Maintain flexibility: seed pooling schemes, single or double stage testing • Maintain confidence in decisions • “We are 95% confident that the GMO presence in this lot is < 0.1%” ISTA Statistics Committee

  6. Assumption: “Representative” Sample • Definition 1 • “Obtain sample so that each seed has an equal and independent chance of being selected [called a simple random sample (SRS)]” • Index every seed, pick random numbers, obtain indexed seeds • Good idea? • Definition 2: mimic SRS sample • bag sampling (ISTA rules) • probe sampling (uniform grid) • systematic sampling ... 1 2 3 4 5 1,000,000,000 ISTA Statistics Committee

  7. Probe sampling ISTA Statistics Committee

  8. Systematic sampling • Sample a flow of seed on regular time interval • flow from hopper bottom truck • flow from a silo • More samples as heterogeneity increases • Sample collect from cut through entire stream of flowing seed • Caution: Make sure that there is not cyclic behavior in flow that correlates with sampling interval ISTA Statistics Committee

  9. Obtaining Pools to Evaluate Bulk Characteristics Obtain sample seed lot … primary samples composite sample Mix well! submitted sample seed pools (bulks) for testing ISTA Statistics Committee

  10. Assumption: Seed lot is large • Sample size should be no larger than 10% of population • This condition must hold to use Seedcalc or Qalstat • If this assumption is not met we must use methods based on the hypergeometric distribution ISTA Statistics Committee

  11. SEED SEED SEED SEED SEED Acceptance sampling for qualitative assays SAMPLE OF SEEDS SEED LOT X DEVIANT SEEDS FOUND X>C XC Number of deviant seeds is distributed binomial REJECT LOT ACCEPT LOT ISTA Statistics Committee

  12. Definitions • LQL = lower quality limit • highest level of impurity that is acceptable to consumer • “95% confident that seed impurity is below 1%” (LQL=1%) • AQL = acceptable quality level • level of impurity that is acceptable to producer and consumer • Some definitions • Conservative: producer can produce seed at this impurity level or below • Practical: process average • Set in relation to threshold • generally, AQL less than or equal to 1/2 LQL ISTA Statistics Committee

  13. AQL LQL process average Most production between 0% & this value 0.5% 0.15% 0.2% Definitions, cont. % production 0% % impurity ISTA Statistics Committee

  14. Definitions, cont. • Consumer Risk = chance of accepting “bad” lot (lot impurity = LQL) • also called beta () • Producer Risk = chance of rejecting “good” lot (lot impurity = AQL) • also called alpha () ISTA Statistics Committee

  15. Operating characteristic (OC) curve don’t want these whatever want these ISTA Statistics Committee

  16. OC curves, cont., ISTA Statistics Committee

  17. Retest Retest 100% 100% Acceptance Acceptance 80% 80% 60% 60% Probability of Accepting Lot (%) Probability of Accepting Lot (%) 40% 40% 20% 20% 0% 0% 0 0.5 1 1.5 2 2.5 0 0.5 1 1.5 2 2.5 Actual % Impurity in Lot Actual % Impurity in Lot LQL & AQL in relation to threshold LQL = 2 x threshold AQL = ½ x threshold LQL = threshold AQL = what producer can deliver (similar to tolerance approach) threshold threshold ISTA Statistics Committee

  18. Reducing Costs: Testing Seed Pools Rather than Individuals 5 seed pools 300 seeds per pool • Works well in testing for adventitious presence • Assay must be able to detect one GM seed in pool of all conventional seed with high confidence ISTA Statistics Committee

  19. Challenge: setting the threshold Option 1: require true zero threshold result: test all seed in entire lot….. Option 2: “zero tolerance” in sample result 1: hidden non-zero threshold Example: USDA recommendation for Starlink (Cry9c), test 2400 seeds and allow zero positives yields a 0.19% threshold rather than zero. result 2: high cost to producer Throw away a lot of good seed due to false positives and sampling variability ISTA Statistics Committee

  20. Challenge: setting the threshold, cont. Option 3: set reasonable non-zero threshold, allow for some positives result 1: manage consumer and producer risks to acceptable levels result 2: better manage impact of assay errors on results result 3: most seed approved for sale will be much lower than threshold (e.g., 3 or 10 times lower) ISTA Statistics Committee

  21. Zero Tolerance Plans ISTA Statistics Committee

  22. The Perfect Plan Reject 0% of “Good” Lots Accept 0% of “Bad” Lots 1% threshold True Lot Impurity ISTA Statistics Committee

  23. Zero Tolerance Plan - Test one pool of 300 Reject ~20% of “Good” Lots Accept <1% of “Bad” Lots 1% threshold ISTA Statistics Committee

  24. Almost Perfect Plan: Test 6 pools of 300, accept 4 deviants pools or less Reject 5% of “Good” Lots Accept <1% of “Bad” Lots 1% threshold ISTA Statistics Committee

  25. OC curves for two testing plans threshold ISTA Statistics Committee

  26. Hypothetical situation: “Ten seed pools of 300 seeds each are tested from a conventional seed lot and 5 pools test positive for adventitious presence. The lot is labeled as having less than 1% adventitious presence and it is shipped.” Should they have shipped the lot? ISTA Statistics Committee

  27. Yes. INTERPRET WITH CARE!! ISTA Statistics Committee

  28. OC Curves for two testing plans threshold ISTA Statistics Committee

  29. More definitions • False negative rate (FNR) • probability that a positive sample tests negative • PCR failures, DNA problems, … • False positive rate (FPR) • probability that a negative sample tests positive • DNA contamination, … ISTA Statistics Committee

  30. Assay Error Impact (pool size =1) No Errors 10% false negative rate 20%false negative rate 1% false positive rate 2% false positive rate ISTA Statistics Committee

  31. Double Stage Testing Plan N1 X1 ACCEPT LOT REJECT LOT N2 X2 ISTA Statistics Committee

  32. No Pooling Allowed!! Trait Purity Testing • Example: Testing RR Soybeans are above 98% trait purity • Must test individual seeds • DNA or protein assay detects intended trait rather than unintended trait in AP testing • FNR has larger effect on testing plan than FPR • Roles of FNR & FPR reverse in Seedcalc6 and Qalstat programs ISTA Statistics Committee

  33. Introduction to Seedcalc ISTA Statistics Committee

More Related