1 / 42

POCT, Assessing Method Validity for POCT by NON-Lab Staff

POCT - Assessing Method Validity Brian Smith (Bedlam Dischargee). POCT - Care" - Clinical - Collection" - GeneralClinical Hospitals, GPs, Pharmacies, Self Test.Workplace, Forces, Occ Health, Pre Employment, Random, Fit for Duty", For Cause", Re

angelina
Download Presentation

POCT, Assessing Method Validity for POCT by NON-Lab Staff

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

    1. POCT, Assessing Method Validity for POCT by NON-Lab Staff Brian Smith, Consultant Biochemist Head of Pathology 1990-2003 (Locum 2003-05) Bethlem Royal & Maudsley Hospitals. UKNEQAS Steering Group for Drug Assays Probation Service POCT Pilot/Roll Out 1998 -2000 Draft MDA DB 2002 (02 & 03) Commentator MDA POCT for DAT Evaluation 2002-03 NHS PASA Wk Gp, Nat Contract (DAT POCT)03/05 US NACB Wk Gp, Evidence Based Practice for POCT,Lab Med Practice Guideline (Drugs & EtOH) 2003-2006

    2. POCT - Assessing Method Validity Brian Smith (Bedlam Dischargee) POCT - “Care” - Clinical - “Collection” - General Clinical – Hospitals, GPs, Pharmacies, Self Test. Workplace, Forces, Occ Health, Pre Employment, Random, “Fit for Duty”, “For Cause”, Return to Work. Schools, Prisons (Mandatory,Voluntary), Probation, Police (Roadside, Arrestees, Custody Cells). Non Human Veterinary. Environmental.

    3. POCT - Some Examples Wet Chem, Repeat use Devices (“mini-labs”). (e.g. Blood Gas Analysers). Dry Chem “Detector”, Repeat use Devices. (e.g. Glucose Meters, Rapiscan) ( “Vitros” ?? ). Dry Chem, Single use Devices (“Stix”/Stick Tests) (e.g. Drugs of Abuse, CK/Myoglobin,”Uristix”). If Dry Chem - Within Batch and Between Batch Manufacturing Consistency much more important (each “strip” is Unique).

    4. POCT - some examples (cont) Machine Read, Permanent Detector Blood - Gas,Na,K,Li,ICa,Lact,U,Gluc, INR/PT etc. Breath - EtOH, CO, CO2,O2. Machine Read, Single Use Dry Chem Detector Blood - Gluc, Oral Fluid - Drugs of Abuse(DAT). Visually Read, Single Use Dry Chem Detector Serum - Myoglobin, Saliva/Oral F - EtOH,HIV,DAT, Sweat - DAT, Urine - Hb/RBC,Prot,Gluc,Bili,NO2(Bugs) etc, DAT.

    5. POCT POCT typically involves Rapid On-Site Testing and Result Interpretation, so that Rapid Actions (Clinical Management) are taken. Those Authorising, Managing and Performing the POCT, have a Duty of Care to be able to Demonstrate, that their system Produced & Interpreted the results Reliably. They are both Corporately and Personally Liable for this. POCT should never be initiated Casually.

    6. POCT – Corporate (& Clinical) Governance Requires a Formal : - Cost/Benefit Analysis & Business Plan. Central & Field Evaluation (your Sites,your Operators). POCT Management System (– with accountabilities). System Operating Procedures (SOPs). Operator/Interpretor Competence Testing and Authorisation (Licence to Kill) - and Continuing Competence monitoring including EQA. ( Mirrors routine systems in Accredited Labs )

    7. POCT Evaluation – Guidance (Clinical) Official Guidance is given in MDA DB2002 (03) “Management and Use of IVD Point of Care Test Devices”. (MDA – Medical Devices Agency) – now, the “Medicines and Healthcare products Regulatory Authority” (MHRA). This is the UK “Competent Authority” for IVDs. MDA DB2002 (03) has a status similar to the “Highway Code” – it may not be the Law, but it does define Good Practice – so if you don’t follow it – you are wrong.

    8. MDA DB2002 (03) User/Purchaser (NOT the Manufacturer) must formally : - Assess if a device is Clinically useful. Evaluate the devices (for their particular needs). (e.g. Ketoacidosis interfering with Glucose Meter result) Put in place a formal POCT Procurement, Management & Monitoring system for all POCTs Allow only locally evaluated & Authorised POCTs to be used (continued)

    9. MDA DB2002 (03) - continued POCTs to be operated only by specifically Trained & Competence Tested staff, Corporately Authorised (“Certified”) to perform that POCT,(& non authorised staff prohibited), Using a locally written SOP, including IQC & EQA and incorporating the “Manufacturer’s Instructions” Has a duty to participate in an EQA Scheme

    10. POCT “Manufacturer’s Instructions” - usually include statements such as : - “The user must verify the use of the device for their particular purposes”. “Suspect results should be repeated”. “Suspect results should be verified by another laboratory method”. (How do you know its “suspect”?) “It is good (laboratory) practice to test Positive & Negative Control samples at appropriate intervals” – [ less commonly ] “and to participate in (External) Proficiency Testing / QA schemes”.

    11. Selling POCTs Suppliers usually NOT selling to Lab Staff, & often don’t understand (& don’t promulgate) : - Limitations of their devices Likely Field Error Rates Official Guidance on POCT Management , Staffing & Time needed for Testing and the Ancillary activities. Time, Accountability & ££, for KSF “Certification” Contin Whole System Performance Assessment

    12. POCT Evaluation Overview ( Assessing “Analytical Method Validity” only a Minor part ? ) Literature / Info Sources Search Supplier’s Training & Technical Support Fabric, Safety and Disposal (inc Confidentiality) Connectivity / Result Communication Evaluation (and Continuing) Record Keeping Analytical Performance [“Central” & On-Site(s) ] Ease of Use (robustness) Fault Detection ( including IQC) Ease & Accuracy of Interpretation Cost – including Management,Fabric,Training etc

    13. POCT – Literature / Info Source Search (changing Rapidly – must be recent) Info from Local Path Lab (or Reference Lab). MHRA Evaluation Reports (www.pasa.nhs.uk.evaluation/details.asp?id=1093) Other (Peer Reviewed) Evaluations. NACB Evidence Based Guidelines for POCTs (2006).( www.nacb.org ) Info from EQAs, Info from PASA** (www.pasa.nhs.uk) - POCT Managers Database & Procurement Toolkits. Info from Literature on Known Interferents. Info from Manufacturers – inc list of Users.

    14. Suppliers Training & Technical Support In my experience – Suppliers (“Manufacturers”) are NOT prepared to offer Training and Competence Testing to MDA standards, nor are they prepared to Indemnify the User, for the Trainee as a “Certified” Operator of their POCT – not just for their Device – but for Compliance with the SOP for the Whole process. i.e – the User Organisation has to pick up the tab for Training, Continuing Competence Testing & Authorisation – & the Management & Cost of this.

    15. POCT – Fabric Requirements (you are making “mini”labs – so Each Site) Storage - Records, Consumables, Samples, Samples for Reference method testing. Testing & Recording & Communicating Space, Security(Staff & Records), Confidentiality, Washable Surfaces, Handbasins, Services / Lighting / “Connectivity”, (All H&S and Infection Control approved). Disposal (Clinical Waste – Licenses, Confidentiality – Patient ID on POCTs/Records)

    16. “Connectivity” Validation Recording, Checking, Releasing & Communicating Results Appropriate Space & Services must be present at each site for this. MACHINE READ Devices should meet the NCCLS POCT1-A Connectivity Standard (www.nccls.org). This allows the Device to connect to a Central (e.g. Hospital) Network – allowing - Automated Result Recording (patients,QC,EQA), (approved) Operator ID, Operator performance Audit, etc. VISUALLY READ Devices are more difficult to reliably Read, Record & Performance Manage.

    17. (POCT Method Validation) Evaluation Record Keeping for POCT, QC samples and Reference Method Batch Nos, Expiry Dates, Shelf Life/Storage, Reference Method – EQA data (i.e. – “Reference” lab must be asked, before Eval) Precision, Bias (particularly for “Odd” samples) All Results (POCT & Ref) – Patients, QC, QA, Duplicates. All Analytical Procedures ( & Results) including All Variant “ tried ( & Results) All Training & Competence Testing ( & Results) both for “Central” Eval and, for each Site Eval.

    18. POCT Evaluation – “Central” Trial with Test Analyte in Matrix Imprecision, Linearity, Sensitivity – all in Duplicate at least. typically at - Zero, Below, Mid, Above and “Way Above” (? “Hook effects”) Ref Range (or Path range), & particularly, +/- any “Decision Points”. For Quantitative POCTs say - Zero ( for LOD), 0.2, 0.5 & 1x bottom ref range (for LOQ), mid, 1, 3 & 10x top of ref range.

    19. POCT Evaluation Test Analyte in Matrix (cont) Qualitative & Semi Quantitative POCTs Usually a Neg / Pos Cut-Off or Threshold (T) or, a Neg , + , ++ , +++ result. typically test Imprecision at Zero, - 50%T, - 25%T, +25%T, +50%T, +1000%T and at, - 30% & +30% of ++ and at, - 30% & +30% of +++ Check Batch to Batch variation at Zero, Below , Above and “Way Above” ref range. Zero, -25%T, +25%T, 20xT. - all in Duplicate.

    20. Within and Between Batch Variation Real Examples(with Test Analyte) Vitros Salicylate. Method said, “if Over 500 mg/l, dilute sample with zero calibrator”. Found from EQA, that some batches gave a result of < 400 mg/l , when true result was 600mg/l – so user unaware that sample even needed dilution. MHRA Eval of 16 POCTs for Drugs of Abuse. For two Devices (Acon & DrugSCREEN), some batches gave Negative Opiate results at +25% of Cut Off.

    21. Formal Device / Method Evaluation (“Parallel Testing”) This must involve Parallel Testing of POCTs vs “reference” (local lab) Method, of a WIDE range of Patient samples – plus IQC and EQA samples - first in a local(“Central”) Trial , and then by the “Operators” at Each Site. Sample Collection (& preparation, if any) is part of the POCT Method. Sample often unprocessed & different to the “reference” method sample. Duplicate (precision) testing is also needed – so larger Patient (& IQC,EQA) samples required during Evaluation.

    22. POCT - IQC and EQA (Parallel Testing) Standard “Laboratory” EQA and IQC samples may not be suitable for POCTs. ( Anti Coags, Preservatives, pH, Viscosity etc) A permanent Local Lab based Parallel Testing EQA scheme may be a viable alternative. (COST, £££) MDA DB2002 (03) recommends that POCT services should be Accredited (e.g. by CPA). (COST,£££) The Traceability to National analyte values would be via the Laboratory’s EQA performance.

    23. POCT Evaluation Parallel Testing of Patient Samples Sample Collection and Storage – for Parallel Testing. Sample Vol for Duplicate testing by POCT(s) & Lab. Sample AntiCoag/Preservative for POCT(s) & Lab. Sample Stability (need a “library”) POCT(s) & Lab. e.g. INR, POCT - No AntiCoag, Lab - Citrate. Gluc, POCT - No AntiCoag, Lab - F/Ox Plasma Urine Cannabinoids, Analyte Adsorption (Vol/SA)

    24. Parallel Testing – Patient Samples (cont) Concentration – as wide a range as possible – particularly above & below any “decision points”. Specificity Analyte Variants (from Patients & EQA samples) e.g. Chole, - HDL/LDL,(& different TriG concs). HbA1c, - Hb variants. Analyte Precursors / Metabolites e.g. Heroin, 6MAM, Morph, M3G, M6G etc. any Ethnic, Gender or Age Variants

    25. Parallel Testing – Patient Samples & Other (spiked) Interferents Matrix Variants (– Look at Method Principle & Matrix) “General” Blood - Hct, Viscosity, Haemolysis, TriG, Bili, Urine - Urea, pH, “Colour”, Turbidity, Precipitates. “Possible” (U Gluc) Fruc, Suc, Lactulose, Vit C etc. “Known” (from other Methods) Na conc (Li), DLIS (Digoxin), Labetalol (Amfets)

    26. POCT Method “Robustness” Evaluation (“Central” Trial AND “Site” Trials Effect on Results of Variations in : - Device Storage, Preparation, Humidity. Sample Volume . Device/Testing Temperature . Test Timings . Lighting (Time of Day will cause changes). Ease of Fault Detection (IQC would help) . Ease of Reading result. Ease of Interpretation, Recording, (etc)

    27. POCT – IQC Samples & Test Frequency MDA/MHRA advice is to use 3rd party QC samples - after problems with Manufacturer’s own QCs working, when Patient samples did not. Use Pos & Neg QC before each batch of Patient samples (reason to believe POCT OK when used). (this is what happens routinely in Lab Testing) NB . You are testing the Operator and their Technique, under their Field conditions, at the time performed - NOT just the POCT kits when the Batch is opened – as suggested by some kit Suppliers/Manufacturers.

    28. POCT “Method” & Error Rates real Example 1 Probation Service (PS) DTTO POCT Pilot, 98-2000. PS worried about False Results (go to Prison)(£££). Modified Site, got Clinical Waste Licences, set up “Confidential Chain of Custody” system. In Lab, Trained & Competence Tested 2 P Officers & 2 Senior Drug Abuse Specialist Nurses. Had Written SOPs, + & - IQCs with each batch. Knew all initial samples would be Lab retested by both same POCT (& Chromatog).

    29. Error Rates, Example 1 (cont 1) Despite all the above precautions, Pos & Neg POCT mistakes in 19 of the first 35 sample (54%) Busy setting(frequent distraction) with 6 Drug results to Visually read & Record per sample. Operators needed 10-20 mins per sample for - Client & Sample Registration, POC Testing, Interpretation, & Result Recording – Max of 30 samples per day per Operator – doing effectively Nothing else.

    30. Error Rates, Example 1(cont 2) This left no time for : - Admin Actions, Client Interviews, Counselling Therapy or Court Reporting. By performing POCT, effectively lost one Staff say £3 / sample in Direct Labour (+ O/H & POCT) Work pattern changed, - No distractions(phones) until test recorded - & each test fully completed before next started. (US HHS 2004 same). Error Rate now 6% ( >1 in 20 samples ) !!! – but similar to other UK & US rates.

    31. Error Rates, Example1 (cont 3) After 2.5yr Pilot, PS decided to “roll out” this (POCT) model Nationally in 2000. (i.e. – it was deemed Highly Successful) Within 18 months, vast majority of PS switched to Lab based testing. (Occasional POCT,few sites).

    32. EXAMPLE 2 Similar Experience – Set Up POCT for Drug Abuse In-Patient Ward . (“for Nights, Weekends & Bank Hols”) Little Used – Why ? Because POCT can’t answer several common Clinical queries and, because of Staffing needed (2 on at night) – but caused a 30% increase in their Lab tests.

    33. Example 2, Clinical Queries Qualitative POCT can’t answer Urine Opiates can stay “Positive” for up to 10 days. Query ? Used on Ward (since admission on 8/1) Date POCT Opiates Opi/Creat Ratio 8/1 + 24,000 4,363 10/1 + 2,550 230 14/1 + 3,550 1,044* 15/1 + 2,100 288 16/1 + 1,500 183 17/1 + 350 85 * Gap of 3 days – but Conc (and Ratio) Increase.

    34. Example 2, Clinical Query Qual POCT can’t answer Not “ is he Pos ? ” – But “ is he More Pos ? ” Subject DS, Officially smoking Days 1 – 6 Urine Cannabinoids Positive Days 1 - 8 Days 8.5, 9 Negative Days 9.5, 10, 10.5, 11 Positive Day 11.5 Negative Day 12 Positive Days 13, 14, 15, etc. Negative ? Did he Re-use on Days 9 and 11.5 ??? Again resolved by Quant Cann*/Creat* Ratio ( Two * Quantitative (Lab) Measurements )

    35. Example 3 – Glucose POCT, 2000 Safety Notice MDA SN 9616 issued 1996, says - POCT Operators Must be Competence Tested & use QC Phone call from Adolescent Psych Ward “We need new stix Urgently”. (Lab didn’t supply) For which Meter ?. “For Our Meter”. A Glucose Meter ?. “Yes”. Which Type/Make ?. “Don’t Know, Can’t you send someone over ?” (8 miles away), Get Ward Manager to Phone me. “Glucosense” . Will contact Maker for Stix & Stds. Why are you doing Assays on (Psych) Ward ? “Adolescent (13),DM, misusing Insulin for Attention and to get out of things” .

    36. Example 3 – Glucose POCT (cont 1) ? Training,Supplies,Stds,QC,QA,Record Keeping . “We are all State Registered Nurses” (i.e. Not Psych Nurses). “We’ve done these for years” (i.e. We don’t need Training). “We went through all this with the Sidcup DM Nurse, who came with the Patient & Showed us” . The Stix (& Makers Stds & Instructions) came. I took them to the Ward & asked to see Meter (to check if Stix correct) & Records of Use.

    37. Example 3, Glucose POCT (cont 2) Found beautifully written Records of 10pm levels only. Recorded as 0.7 mmol/l on all 6 occasions – by a Psych HCA (not SRN). [Ward Mgr in meeting] Phoned Ward Manager. Do you have an Action Plan ? What Action Taken ? Why No Daytime Records ?. Do you know about DoH Good Practice Guidance ? How can you get 0.7 on a Meter that only measures down to 1.8 ??? Why are all the values 0.7 ???

    38. Example 3, Glucose POCT (cont 3) No Action Plan, “The Patient (not the Levels) would be discussed at the next Clinical Meeting” “If we thought the Patient was Hypo or Hyper, we would call the Duty Dr.” (Hadn’t Happened). So, Why are you Measuring Glucose on the Ward ?? Said I would contact the SHO & Consultant.

    39. Example 3, Glucose POCT (cont 4) Asked SRN to show me how they used Meter “Humph !!!” . - Do my Glucose. Stick Insufficiently covered with Blood. “See 0.7” Meter shows “LO”, but held upside down, so read as “07”. “OH” .

    40. Example 3, Glucose POCT (cont 5) Immediate DEMAND for Training,Supplies etc. I Said, (a)No Budget or Resources (b)3 Shifts to train, how will they call in Night Staff ? “You’ll have to come when we want” . I said NO !! You will have to send Glucoses to Lab until Staff Trained & Certified. You are NOT actually using Ward POCT Levels Now !!! .

    41. Example 3, Gucose POCT (cont 6) Will discuss Training with your Unit Manager (££) and with Director of Nursing (State Registration ?) . Training will have to be in Sessions – with several Staff together (backfill problem) – who will have to be Certified in the use of the particular Meter the Trust has selected. PLUS, a Ward Manager, (you) Responsible for SOPs, Training, Re-Training, Supplies, Prohibiting use by Non-trained Staff etc.

    42. Example 3, Glucose POCT (cont 7) “We can’t possibly do all that” . Then you can’t do POCT – even by “BM Stix”. “Then we’ll get the Duty Dr.” . No, They would need Certifying. OUTCOME Assays sent to the Lab, inc On Call, (Better Care). No more “Stix” were used. An Action Plan was apparently developed. Patient discharged (alive) some 6 weeks later.

    43. Example 3, Glucose POCT (cont 8) Director of Nursing had discussions with all Senior Nurse Advisors (responsibilities, advance planning) Medical Director (Psychiatrist)(Head of Clin Gov) did not respond. [ Both left within a year ] Next Time a similar case arose, Ward asked for Training,QC etc, In Advance !. 4 separate Recorded Certification/Training sessions (3-4 Staff at each). [For 1 patient in for 3 months(Psych Ward problem)]

More Related