Data Quality Assurance Beverly Musick

1. Data Quality AssuranceBeverly Musick

2. Introduction • Electronic data play a critical role in health care delivery • By providing decision support to health-care providers • By forming the basis for periodic reporting to administrators and funders • By aiding investigators in addressing relevant research questions • Electronic data typically originate with paper collection forms which is why appropriate form design is so critical

3. Quality Assurance • Quality Assurance is the set of processes, procedures, and activities that are initiated prior to data collection to ensure that the expected level of quality will be reached. • Form Design • Data Element Dictionary • Electronic error prevention and control • Anticipation of missing data • Training and informing of clinical staff

4. Form Design Data collection forms must • Employ clear, precise, unambiguous questions • Organize questions into logical groupings • Consider numbering questions/items • Use appropriate responses • Mutually exclusive categories • Consider quantity vs. frequency • Explicit vs. implicit

5. Tick all that apply vs. Tick one Coding Responses 10d. How do you think you were exposed to HIV? (Check all that apply) □ Patient knows spouse or partner is HIV+ □ Suspected exposure in prior relationship □ Blood Transfusion Year of Transfusion □ History of Intravenous Drug Use □ Contaminated Needle Stick □ Unknown □ Other

6. Tick all that apply vs. Tick one Coding Responses 10. What is your current relationship status? (select one) □ Never married and not living with a partner □ Legally married: Number of wives ________ □ Living with a partner □ Separated □ Divorced □ Widowed For “Tick one” responses must be mutually exclusive

7. Coding ResponsesCommon Mistake • Example: How often have you had pain in the past week? □ never □ 1-2 days □ 2-4 days □ 5 or more days • Response categories are not mutually exclusive

8. Appropriate Responses Quantity vs. Frequency How much pain have you had in the past week? □ none □ mild □ moderate □ severe How often have you had pain in the past week? □ never □ 1-2 days □ 3-4 days □ 5 or more days

9. Appropriate Responses Explicit vs. Implicit 10a. □ Yes □ No – Sexually active last 6 months 10b. □ Sexually active last 6 months

10. Form Design Data collection forms must • Employ clear, precise, unambiguous questions • Use appropriate responses • Include units of measurement

11. Include units of measurement • Lab results can have different units of measure depending on assay used • Weights recorded in both kg and lbs

12. Form Design Data collection forms must • Employ clear, precise, unambiguous questions • Use appropriate responses • Include units of measurement • Explicitly identify data

13. Explicitly Identify Data Name: ___________________________ vs. First Name: _______________________ Middle Name: _____________________ Sur Name: _______________________

14. Form Design Data collection forms must • Employ clear, precise, unambiguous questions • Use appropriate responses • Include units of measurement • Explicitly identify data • Avoid open-ended questions

15. Original AMPATH Form Below was the method for collecting ART data Plan/Comments: ______________________________________________________________________________________________________

16. BEGIN TRIOMUNE 30 CHANGE MEDS TO TRIOMUNE CT EMTR REFILL N, Z40, E. CT EMTRI CT N,E CT N, E, Z CT N, Z, E CT N, Z40 CT N, Z30 CT N,Z CT TRIOMUNE EMTRI EMTRI 30 EMTRI-30 EMTRI-40 EMTRI – 40 EMTRI 40 E-40 E - 40 PUT ON TRIOMUNE PUT ON TRIPLE THERAPY RATHER THAN TRIOMUNE REFILL EPIVIR,ZERIT, NEVIRAPINE REFILL N, Z40, E RE-START BACK(2ndTIME) ON TRIOMUNE RESTART TRIOMUNE 30 SWITCH TO TRIOMUNE SWITCH TO T-30 SWITCH TO T-40 TO COMPLETE TRIOMUNE 40 TRY AGAIN T 30 TRIOMUNE TRIOMUNE 30 TRIOMUNE-30 TRIOMUNE 40 T 30 T 40 T-30 T-40 T030 T30 T40 Ways to enter Triomune

17. Alternatives to Open-Ended Item Current Medication Plan: Triomune 30 Triomune 40 or Current Medication Plan: Triomune 30: yes no Triomune 40: yes no

18. Form Design Data collection forms must • Employ clear, precise, unambiguous questions • Use appropriate responses • Include units of measurement • Explicitly identify data • Avoid open-ended questions • Reflect flow of data collection

19. Flow of Data Collection • Who will be collecting data • In what order • How often will data elements be collected (cross-sectional vs. longitudinal variables) • What is the interval of data collection (appropriate windows)

20. Form Design Summary • Well-designed forms should employ clear questions and appropriately coded responses. • Explicitly identify data and include measurement units. • Use of open-ended questions should be avoided. • Consider the overall organization of data both cross-sectional (one-time collection) and longitudinal (multiple observations collected). • Cross-sectional data such as gender and DOB should not be re-collected on follow-up forms. • Information that is collected repeatedly over time should be formulated in the same manner at each time point.

21. PRACTICUM DQA1 Review data collection form

22. Form Implementation • Identify key fields • Select appropriate data attributes (type, length, and format) • Choose meaningful field names • Prepare for missing data

23. Data Element Dictionary • Used to map individual items and responses to the electronic data • A guidebook for understanding the organization and storage of data • Prevents ambiguity and misinterpretation • Example: RefDocs\20050112_ampath_adult_return_Key.pdf

24. Example: Data Element Dictionary

25. Electronic Error Prevention and Checking • Electronic interface should mirror the paper form as much as possible. • Use drop down menus for ease of selecting the desired response. • Force entry of required fields such as patient ID and date before allowing further entry. • Prevent entry of duplicate patient ID’s into the patient registry or duplicate observations on the same patient same visit. • Utilize range restrictions on numeric fields to prevent entry of erroneous data. Include logical checks that conditionally restrict entry.

26. Out of Range Values • Out of range values are those values which are outside the expected scope of response • Can be numeric values which exceed the anticipated minimum or maximum • Can be response categories that were not previously identified • Can be due to changes in the environment such as the availability of new medications • Can be due to data entry errors • Utilize range restrictions on numeric fields to prevent entry of erroneous data. Include logical checks that conditionally restrict entry.

27. Anticipating Out of Range Values Data Collection: • Include response category for ‘Other’ with place to specify what the other is. • Include version numbers on data collection forms so that you can track when a response category was added. Data Entry: • Utilize range restrictions on numeric fields to prevent entry of erroneous data. • Tag out of range values. • Allow data entry clerks to make notes about unexpected values.

28. Dealing with out of range values • Reject: prevent entry of out of range value • Accept unconditionally: allow entry even though out of range • Accept conditionally: allow entry but mark value to indicate out of range • Correct: convert out of range value to upper or lower bound of in-range value

29. PRACTICUM DQA2 Electronic Error Prevention

30. Plan for Missing Data • Before data collection begins, determine how missing data will be recorded and entered • Possibilities for Coding Missing Data • Not Applicable N/A (i.e. Adherence to medications for patient not taking any meds) • Not Available (i.e. variable added to questionnaire at a later date, weight temporarily missing due to broken scale) • Unknown (i.e. HIV status of partner) • Refusal to answer (i.e. questions associated with stigma) • True missing (i.e. Question skipped) • Understand how missing data will be managed in the analysis to help determine how much information is to be gathered about missing data

31. Types of Missingness • Missing Completely at Random – probability of missing data on variable Y is unrelated to the true value of Y or other variables in the dataset • Ex. Water damage to paper forms prior to entry • Missing at Random – probability of missing data on Y is unrelated to Y only after adjusting for one or more other variables • Ex. For really sick patients, clinicians may not draw blood for routine labs • Not Missing at Random – probability of missing data on Y is dependent on value of Y • Ex. Higher income patients may be less likely to report income

32. Documenting Missingness • Embed missing codes and/or variables in dataset and/or on data collection forms Pros: Permanently associated with variable Immediately available for analysis Reduces need to re-look for data Cons: Takes up a lot of digital and physical space Increases time needed to complete forms • Provide explanation in separate meta document Pros: Global explanation of missing data Minimal digital and physical space Cons: Eliminates ability to code subject-level data Tends to get lost/separated from data

33. Benefits of Documenting Missingness • Informs Quality Control reporting • Query data collectors on missingness related to “result not available” but not on “test not ordered” • Allows for full disclosure in publication or presentation of data • Some statistical analysis methods are dependent on Missing Completely at Random or Missing at Random • Useful for methodological research related to missing data

34. Procedures for Minimizing Missing Data • In the clinic: review the data collection forms in the clinic, preferably while the patient is still there. • should be part of the clinical staff training and oversight • Point of data entry: prevent entry of a form that is missing key variables such as patient ID and visit date. Alert data entry clerk about missing fields.

35. Training and Informing Clinical Staff • Regular training in accurate completion of encounter forms is vital to ensuring data quality. • Review of key fields to ensure proper completion by independent individual • Include the provider/user ID of the person completing the forms to encourage and ensure accountability.