Encounter Data Validation: Review and Project Update

1. Encounter Data Validation:Review and Project Update Presenters: Thomas Miller, MA Executive Director, Research and Analysis Team Amy Kearney, BA Associate Director, Research and Analysis Team 1

2. Welcome • About the presenters • Rules for engagement • Presentation overview • The importance of encounter data • CMS protocols • Florida EDV study, including best practices for medical record procurement 2

3. Objectives 3

4. Importance of Encounter Data • Accurate and complete data are critical to success of managed care programs • Essential for overall management and oversight of Florida’s Medicaid program • Ability to monitor and improve quality of care • Establish performance measures • Generate accurate and reliable reports • Obtain utilization and cost information 4

5. Importance of Encounter Data • Used by MCOs and the State for many purposes • Performance measure development and calculation • Performance improvement measurement • Focused studies/quality activities • Rate-setting • Compliance monitoring • Provider practice patterns 5

6. Key Trends • Importance of Federal and State monitoring • Development of core measurement sets • Medicare versus Medicaid • Health care reform • Holding health care accountable • Data, not anecdotes 6

7. Objectives 7 7

8. Objectives 8

9. EQR Protocol • Developed and refined with the evolution of the External Quality Review program 9

10. EQR Protocol • Guidelines for External Quality Review Organizations (EQRO) to use when assessing completeness and accuracy of encounter data. • Data submitted by Managed Care Organizations (MCO) to the State 10

11. EQR Protocol • State establishes standards for encounter data • State must establish the following standards: • Definition of “encounter” • Types of encounters • Data accuracy and completeness • Objective standards for data comparison 11

12. EQR Protocol • Five key activities • Review state requirements • Review MCO’s capability • Analyze electronic encounter data • Review of medical records • Submission of findings and recommendations 12

13. EQR Protocol • Attachment A: Encounter Data TablesTable 2: Data Element Validity Requirements 13

14. EQR Protocol • Five key activities • Review state requirements • Develop understanding of State-specific policies and procedures for collecting and submitting encounter data • Identify data exchange protocols and layouts • Evaluate encounter data system interchange flows, including system edits and submission timelines • Review existing encounter data quality activities, requirements, and performance standards 14

15. EQR Protocol • Five key activities, continued • Review MCO’s capability • Develop, conduct, and review MCO’s Information System Capabilities Assessment • Identification of IS vulnerabilities • Key informant interviews 15

16. EQR Protocol • Five key activities, continued • Analyze electronic encounter data • STEP 1 - Develop data quality test plan to determine: • Magnitude and type of missing encounter data • Overall data quality issues • MCO data submission issues 16

17. EQR Protocol • Five key activities, continued • Analyze electronic encounter data • STEP 2 - Verify integrity of encounter data • Macro-level analysis • Encounter file completeness and reasonableness • Volume and utilization by encounter type and service setting • Internal field consistency • General field completeness and validity 17

18. EQR Protocol • Five key activities, continued • Analyze electronic encounter data • STEP 3 – Generate and Review Analytic Reports • Micro-level analysis • Encounter record completeness and reasonableness 18

19. EQR Protocol • Five key activities, continued • Analyze electronic encounter data • STEP 4 – Compare findings to state-identified standards • Identification of appropriate benchmark population 19

20. EQR Protocol • Five key activities, continued • Review of medical records • Verification of the accuracy of coding • Protocol assumptions • STEP 1 – Determine sampling for medical record review • Identify valid sample size • Encounter- vs. recipient-based samples 20

21. EQR Protocol • Five key activities, continued • Review of medical records • STEP 2 – Obtain and review medical records and document findings • Procurement efficiencies • Abstraction staff and training • Categorization of errors by level, type, and source • Procurement tracking and abstraction tools 21

22. EQR Protocol • Five key activities, continued • Submission of findings • Narrative report summarizing findings from Activities 1-4 • Actionable recommendations for overall encounter data quality improvement 22

23. Questions? 23

24. Objectives 24

25. Objectives 25

26. SFY 2013-2014 Encounter Data Validation (EDV) Study Agency for Health Care Administration Validation of Encounter Data 26

27. Year One Encounter Data Validation (EDV) Study • Four key steps for conducting successful evaluations • Project implementation • Study design • Data collection &analysis • Reporting & recommendations 27

28. Year One Encounter Data Validation (EDV) Study • Study design • Prepared and finalized methodology which included: • Study objectives and research questions • Data source and collection procedures • Measurement methodology • Analytic methods • Timeline 28

29. Year One Encounter Data Validation (EDV) Study • Data collection and analysis • Information systems review • Questionnaire for AHCA • Assessment of AHCA’s policies and procedures for data exchange, its capacity and ability to acquire and process data, and its staff responsible for executing data processing • Questionnaire for MCOs • Assessment of MCOs’ claims processing systems and processes, and its capability to submit encounter data 29

30. Year One Encounter Data Validation (EDV) Study • Data collection and analysis,continued • Information systems review • MCO questionnaire divided into five sections: • Submitting Encounter Data to AHCA • Handling Submission Information from AHCA • Encounter Data Submission from Capitated Providers • Processing and Submission of Medicare Crossover and other Third Party Claims • Policies and Procedures in Processing Payment Information 30

31. Year One Encounter Data Validation (EDV) Study • Data collection and analysis, continued • Information systems review • AHCA questionnaire divided into three sections: • Data Exchange Policies and Procedures • Data Submission Processing Procedures and Personnel • Encounter Data Processing within the Florida MMIS 31

32. Year One Encounter Data Validation (EDV) Study • Data collection and analysis,continued • Information systems review • Documentation will be used to assess encounter data quality • Questions target how data moves through AHCA’s data systems and how the MCOs prepare data files for submission 32

33. Year One Encounter Data Validation (EDV) Study • Data collection and analysis,continued • Information systems review What has been completed? • Questionnaires were approved by AHCA and distributed to AHCA and the MCOs • Received completed questionnaires from AHCA and MCOs What needs to be completed? • Currently reviewing responses from AHCA and MCOs • May conduct additional follow-up for clarification 33

34. Questions?

35. Year One Encounter Data Validation (EDV) Study • Data collection and analysis, continued • Comparative data analysis of AHCA and MCO encounter data • Evaluates the extent to which encounters submitted by MCOs to AHCA are accurate, complete, and reasonable • Included all claim/service types—i.e., inpatient/outpatient, physician visits, dental, and pharmaceutical 35

36. Year One Encounter Data Validation (EDV) Study • Data collection and analysis, continued • Comparative data analysis of State and MCO encounter data • Indicators to measure degree of completeness and accuracy for each encounter type • Overall record matching—percentage of state encounters present in MCO files • Field-level matching—percentage of state encounters with exact value match in MCO file for each select data element 36

37. Year One Encounter Data Validation (EDV) Study • Data collection and analysis,continued • Comparative data analysis What has been completed? • Distributed data submission requirements documents to AHCA and MCOs • Conducted technical assistance sessions with MCOs on 9/16 & 9/17/2013 • Received, processed, and loaded encounter data 37

38. Year One Encounter Data Validation (EDV) Study • Data collection and analysis, continued • Comparative data analysis of State and MCO encounter data What needs to be completed? • Conducting preliminary file review • Ensuring files are sufficient for processing • Completing basic checks • Generate comparative analysis tables and figures for final report 38

39. Phew… Questions? 39

40. Year One Encounter Data Validation (EDV) Study • Data collection and analysis, continued • Medical record review • Represents the “gold standard” • Evaluation of service level accuracyand completeness • Methodology developed: • Only includes MCOs operational as of January 2013 • Year One – SFY 2016: review one-third of plans each year as selected by AHCA • Minimum 50 cases reviewed per plan • Target professional, dental, and inpatient/outpatient encounters 40

41. Year One Encounter Data Validation (EDV) Study • Data collection and analysis, continued • Medical record review • Sample selection methodology • To generate list of randomly selected encounters for medical review, HSAG will use AHCA’s data files from comparative analyses • Two-stage stratified sampling design used to ensure: • Member’s record is selected only once • Number of encounters included in final sample covers all encounter types and proportional to total distribution of encounters 41

42. Year One Encounter Data Validation (EDV) Study • Data collection and analysis,continued • Medical record review • Sample selection methodology • HSAG will evaluate the key data elements below: 42

43. Year One Encounter Data Validation (EDV) Study • Data collection and analysis, continued • Medical record review • Procurement and abstraction process • Based on established policies and procedures • Continually monitored to ensure validity and accuracy • Inter-rater reliability testing & Rater-to-standard testing • All reviewers must achieve 95% accuracy rate • Variety of reports will be generated, i.e., medical record compliance rates 43

44. Year One Encounter Data Validation (EDV) Study • Data collection and analysis, continued • Medical record review – analysis of cases • Verify the service(s) provided on selected data of service and one additional date of service • Each enrollee listed on sample has corresponding selected date of service • Validate services conducted by provider on date of service as compared with encounter data • Reviewers to validate services for additional date of service. 44

45. Year One Encounter Data Validation (EDV) Study • Data collection and analysis,continued • Medical record review – analysis of cases • Analyze record completeness and the accuracy of coding • Four primary indicators for data completeness and accuracy • Medical Record Agreement • Medical Record Omission (surplus) • Encounter Record Omission (missing) • Erroneous 45

46. Year One Encounter Data Validation (EDV) Study • Data collection and analysis, continued • Medical record review What has been completed? • Introductory letter sent to MCOs on 10/1/13 • Conducted technical assistance calls with all participating MCOs on 10/16 & 10/18/13 46

47. Year One Encounter Data Validation (EDV) Study • Data collection and analysis, continued • Medical record review What needs to be completed? • Pull samples and send lists of study cases • Provide letter to send to its providers with sample • MCOs will procure records from provider and accommodate various submission methods • MCOs to submit identified medical records to HSAG for review 47

48. Year One Encounter Data Validation (EDV) Study • Reporting and recommendations • Prepare aggregate EDV report of findings from: • Information system review • Comparative Analysis • Medical Record Review • Preparation of supplemental findings for future evaluation by MCOs • Present statewide and MCO-specific results • Actionable recommendations for improvement 48

49. Objectives 49

50. Questions