The Management and Flow of Data in Drug Treatment Agencies

1. The Management and Flow of Data in Drug Treatment Agencies Jennifer P. Wisdom, PhD MPH James H. Ford II, PhD Using Performance and Outcomes Measures to Improve Treatment March 21, 2008

2. Improving the quality of care requires… • Identifying problem areas • Creative interventions • See if the intervention worked • In other words, you have to capture and track essential measures of performance and quality … which requires staff (or someone*) to collect, analyze, and learn from data

3. Drug treatment agencies have extensive reporting requirements … but few agencies have access to well-developed information systems. • Agencies spend enormous resources collecting data … but very little data is used for clinical decision-making or program planning • For most agencies, it’s just “paperwork.”

4. Research Plan • Step 1: How to drug treatment agencies learn to use data? • Step 1, revised: What do drug treatment agencies need to improve their capacity to collect data? • Step 1, take 3: What on earth are drug treatment agencies doing with data?

5. How do drug treatment agencies learn to use data? • In context of NIATx, we defined success in “using data” as: • Successful use of data in process improvement decisions • Data collection appropriate to identified goal and interpret it correctly • Data reported are internally consistent and reliable • Agency staff have expertise to resolve data inconsistencies and to explain results to others

6. How do drug treatment agencies learn to use data? • Factors associated with success in improving timeliness and retention • Agency leadership values data • Resources committed to infrastructure • Training on data issues • Sharing of change results • Success making data-driven decisions

7. How do drug treatment agencies learn to use data? • Barriers to implementing process-focused data systems • Time and money • System complexity • Expertise • Resistance*

8. What do drug treatment agencies need to improve their capacity to collect data? • Met with stakeholders to inquire about what is needed • Agency staff = 10 • Researchers/evaluators = 6 • Funding agency staff = 1 • An additional funding agency staff provided perspectives.

9. What do drug treatment agencies need to improve their capacity to collect data? • Attendees were asked to consider three main questions: • How can policy makers and funding agencies promote data education and support? • How can agencies that value data help other agency leaders promote and value data and tie data to financial outcomes? • How can educational bodies, licensing boards, and agency continuing education programs foster the value of data?

10. What do drug treatment agencies need to improve their capacity to collect data? • The discussion revealed areas of conflict: • Even though treatment agencies, researchers, and funders agree on the importance of data-reporting and accountability, it can be difficult for addiction treatment agencies to obtain and analyze data to meet those goals. • Infrastructure (including technology and training) is needed, but support of agency infrastructure may not fall within the funder’s / state’s strategic objectives. • Educational bodies teach to licensing requirements, which are difficulty to modify. • And what data do we really need to collect anyway?

11. What are drug treatment agencies doing with data? • Robert Wood Johnson Foundation funded examination of the flow and management of data in drug treatment agencies • 8 drug treatment agencies in 4 states • Staff interviews (n=55) • Flow charts of how information moves • Review of forms

12. What are drug treatment agencies doing with data? • All agencies had at least two levels of care • Sampled broadly for varied characteristics (urban/rural, large/small) • Examined 3 processes • Intake • Transfer within agency • Discharge (or transfer outside of agency)

14. What we’ve learned… What influences agency sophistication • The state data tracking system • Including sophistication of data system • Emphasis of state on tracking/using data • Whether state gives data access to agency • Executive leader recognition of the importance of data • Influences resource allocation • Grant participation

15. Factors that influence what data are collected and how • Always done it this way • State reporting requirements • Financial contracts/referral sources • Level of Care • Informal Agency Protocols

16. Few agencies had written protocols that • Describe the flow of paperwork, • Govern the use of information or • Outline procedures for error checking to ensure data accuracy

17. Flow of information in the agency • Disparity in the amount and type of information collected at intake vs. any other time (discharge or transfer) • How information is shared across the agency depends on the level of sophistication (Electronic distribution vs. written documentation vs. verbal communication)

18. The Issue of Paper… • Despite technology, paper is king! Even with electronic systems, staff like paper. • It appears many forms are completed, filed, and never looked at again. • One agency was attempting to move to a paperless system; staff were challenged to let go of the paper…

19. Implications • There are opportunities to increase efficiency in how drug treatment agencies manage information. • Processes for information collection • Reduce amount of data • Simplify processes • Processes for using information • Increase use of data • Increase focus on outcomes

20. But there is a See Saw Battle

21. And then there is… NOMS

22. Next Steps We need research that • Identifies how the role of data sophistication and the ability to use data impacts an agencies ability to improve and impact outcomes • Verifies organizational “factors” influencing data use and offers advice for improvement • Determines what type of organizational feedback is most useful to help drug treatment agencies use process and outcome measures to enhance quality of care, and • Allows drug treatment agencies to assess their own organizational data readiness

23. Finagle Law of Health Information The information you have is not the information you want, The information you want is not the information you need, The information you need is usually not available, And the Information you can get costs more than you want to pay.

24. Appreciation to: Robert Wood Johnson Foundation Elaine Cassidy PhD Victor Capoccia, PhD Deirdre Mackey, BA Meg Wise, PhD Dennis McCarty, PhD Contact:Jennifer Wisdom, PhD MPHjwisdom@pi.cpmc.columbia.edu

25. Extra slides if there’s time…

26. How can policy makers & funding agencies promote data education and support? • Universal agreement that drug treatment agencies generally are not well-skilled at collecting or interpreting data, and that it would be beneficial for them to increase their ability to do so. • Barriers to system development hamper progress: • unsophisticated state and agency data systems • lack of computer skills of drug treatment staff • general lack of understanding by all parties about common goals and metrics • Stakeholders suggested policy makers use both • incentives (such as providing higher reimbursement rates for demonstrated performance) and • penalties (such as making funding contingent on data collection).

27. How can agencies that value data help other agency leaders promote and value data and tie data to financial outcomes? • Agencies can assist each other by: • promoting the benefits of data collection (e.g., increasing the likelihood of funding) • sharing evidence of how using data to make decisions has improved their financial bottom line • sharing strategies for increasing infrastructure capability and staff expertise.

28. How can educational bodies, licensing boards, and agency continuing education programs foster the value of data? • Stakeholders suggested increasing communication between drug treatment agencies and educational bodies and licensing boards to partner in increasing quality and quantity of training. • Training should specifically focus on information will be directly applicable to graduates’ duties in drug treatment agencies.