1 / 44

Should we report a Reference Change Index (RCI) with our test results?

Should we report a Reference Change Index (RCI) with our test results? . AACC Roundtable July 21, 2008 dseccombe@ceqal.com 604-222-3907. “Biological Variation: from Principles to Practice” Callum G. Fraser AACC Press (2001)

alanna
Download Presentation

Should we report a Reference Change Index (RCI) with our test results?

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Should we report a Reference Change Index (RCI) with our test results? AACC Roundtable July 21, 2008 dseccombe@ceqal.com 604-222-3907

  2. “Biological Variation: from Principles to Practice” Callum G. Fraser AACC Press (2001) The most recent and extensive data on biological variation may be found at: www.westgard.com/guest17.htm

  3. The Problem Male – age 55 - routine check-up – total cholesterol was found to be 6.6 mmol/L (255 mg/dL) Physician recommended a “lifestyle modification” program Six months later cholesterol was 5.8 mmol/L (224 mg/dL) Has there been a significant change in this patient’s cholesterol?

  4. What is provided to help them in answering this question? Cholesterol > 6.22 mmol/L (240 mg/dL) (high risk) 5.18 – 6.19 mmol/L (200 – 239 mg/dL) (moderate risk) Reference Intervals 4.0 – 7.1 mmol/L (5th to the 95th percentile) 155 – 275 mg/dL (5th to the 95th percentile)

  5. Potassium RI Results(n = 503) Range of RI Low 3.00 – 3.90 Nordic RI 3.60 – 3.63 (90% CI) Range of RI High 4.50 – 6.20 Nordic RI 4.61 – 4.66 (90% CI) Range of Upper critical limits – 5.2 to 7.0 mmol/L Range of Lower critical limits – 2.0 to 3.6 mmol/L Clin. Biochem. (2000)33:449-456 Nordic Reference Interval Project (http://www.furst.no/norip/)

  6. “Lets look at the problem” What factors are known to contribute to the variation in test results? Pre-analytical factors Analytical factors Human biology

  7. Pre-analytical sources of variation • Patient preparation fed vs fasting starvation exercise altitude stimulants posture

  8. Percentage increase from lying to standing

  9. Pre-analytical sources of variation • Sample collection and processing collection technique type of sample anticoagulant/preservative transport time centrifugation identification/accessioning storage

  10. Percentage increase - prolonged venous stasis(6 minutes)

  11. Pre-analytical sources of variation Conclusion “The impact of pre-analytical sources of variation can be minimized by standardizing the procedures used in sample collection, handling and accessioning”

  12. Analytical sources of variation Analytical variability is of two types – random and systematic (precision and bias) These sources of variation cannot be eliminated but can be minimized by quality lab practices and method selection

  13. When is analytical bias a problem? Method bias can be a big problem particularly when the test is being used for diagnosis, case finding, screening or as a “trigger” for making clinical decisions as recommended in national treatment guidelines

  14. The Impact of Analytical Bias on Medical Decisions Bias (calibration errors) in calcium cost the US Health Care system $60 - $199 million annually (NIST Planning report 04-1: The Impact of Calibration Error in Medical Decision Making)

  15. Impact of Standardization Program on Creatinine Results (RV +/- 10%) 50.4% Pass | 49.6% Fail ; 90.3% Pass | 9.7% Fail Impact on eGFR (RV +/- 10%) 58.7% Pass | 41.3% Fail; 86.6% Pass | 13.4% Fail

  16. British Columbia • Standardization program reduced mean provincial bias from 16.5% to 2.7% • At a maximum the program theoretically reduced false positives by 84% and kept 449,400 people from being “incorrectly classified” as being “at risk”. Estimated savings - $37 million (J Am Soc Nephrol 19:164-169 (2008) )

  17. Analytical Bias Bias is the difference between the results we obtain and some estimate of the true value Known biases should be eliminated before reporting a test result (IFCC, IUPAC)

  18. Analytical Bias There is an underlying assumption that lot-to-lot variation in calibration bias is negligible and bias is therefore assumed to be constant over time “Bias can vary significantly with changes in lot number of calibrator, reagents and consumables”

  19. Variation between IDMS traceable reagent Lots (20080301)

  20. Analytical Precision Random variation is a function of the analytical system and methodology used Precision is measured by replicate analysis of the same sample within and between days (IQC samples) Factors impacting precision - fluctuations in temperature, sample/reagent volume delivered, changes in environment, inconsistent handling

  21. Analytical Precision The difference between two methods used to analyze the same analyte should be less than <0.33CVi to warrant using the same reference intervals for these two methods

  22. Biological Variation Daily biological rhythms Monthly cycles Seasonal rhythms Random biological variation

  23. Biological Variation “measure creatinine repeatedly in ten healthy male subjects over a period of 14 days”

  24. Creatinine Reference Interval(64-120 umol/L)(.72–1.35 mg/dL)

  25. Observations The range of values for anyone individual covers a small part of the reference interval The mean values of all individuals lie within the reference interval and differ from each other

  26. Observations Within-subject variation is the quantitative estimate of homeostasis in man Ample evidence to indicate that within-subject variation is constant Little evidence that within-subject variation changes with age

  27. Observations Homeostatic set points can change with age and pathology but the variation around these set points does not change In general, estimates of CVi are constant irrespective of the number of subjects, the time scale of the study, the methodology and the country in which the study was done

  28. Assessing Performance • CLIA 88 quality requirements appear to be based on what is achievable as opposed to what is clinically and biologically desirable • Performance goals for precision and bias can be defined on the basis of within and between subject biological variation • Three levels of performance can be defined: minimum, desirable and optimal

  29. Assessing Performance Desirable performance for precision CVa < 0.5 CVi (CVi = within-subject biological variation) Desirable performance for bias BA = <0.250 (CVa2 + CVg2)1/2 (CVg = between subject variation) (CVa = analytical CV)

  30. Assessing Performance The precision and bias goals can be combined to produce a desirable total error performance goal (TEa). TEa <0.250 (CVa2 + CVg2)1/2 + 1.65(0.50 CVi) In the case of sodium, the desirable TEa performance goal is 0.9%

  31. Creatinine – CLIA quality specifications Peer group mean +/- 15% or 0.26 mg/dL (23.4 umoles/L) whichever is greater Under CLIA – given a peer mean of 1.13 mg/dL (100 umoles/L) labs will get a “pass” if they are within +/- 23.4% NKDEP recommended minimum acceptable TE performance goal for reporting of eGFR as calculated on the basis of biological variation: Minimum 11.4 %; desirable 7.6%; optimal 3.8%

  32. Measurement of creatinine by clinical laboratories in British Columbia Mean TE = 23.9% Mean Bias = 16.5% (+) Mean CV = 3.02 ; SD = 2.24

  33. The Problem Routine check-up – total cholesterol was found to be 6.6 mmol/L (255 mg/dL) Six months later cholesterol was 5.8 mmol/L (224 mg/dL) Has there been a significant change in this patient’s cholesterol?

  34. Reference Change Value (RCV) Using biological variation data we can determine whether or not the difference between these two cholesterol test results is statistically significant RCV = 21/2 + Z + (CVa2 + CVi2)1/2

  35. Reference Change Value First value = 6.60 mmol/L Second value = 5.82 mmol/L Change as a percent of first result = 12% Lab’s CVa from IQC data = 1.6%; CVi = 6.0 RCV = 21/2 + Z + (CVa2 + CVi2)1/2 = 17.2% (p < 0.05)

  36. RC Index RC index = 12/17.2 = 0.69 If the RC index is greater than 1 – the change is significant (p<0.05). If the RC index is less than 1 - the change is not significant Conclusion: the change in cholesterol in this patient is not statistically significant

  37. Reference Change Value By substituting the actual change (12 %) for RCV in the equation we can calculate a “z score” and see what the probability is that the change is real (82% probability)

  38. Considerations • RCi is calculated in real-time • The RCV for a given test is influenced by the quality of the testing that is being provided by the laboratory. Can it be used as a quality indicator? • Trending of critical markers: would the discriminating value of a test be enhanced if at baseline we determined the patient’s CVi for the marker in question and then used this value in calculating their RCi at subsequent visits?

  39. Other Considerations Uncertainty of measurement vs RCi – would we be better off reporting the RCi ? Your thoughts and feedback Thank You

More Related