The Wonderful World of MIS

1. The Wonderful World of MIS Okay. Maybe not. But it sounds nicer and more approachable than “I am going to lecture you on a boring topic for the next hour while trying not to put you to sleep or make you feel bad about the fact that we all need a major check-in on this topic.” February 13, 2017 Tower Hill Coordinator Meeting 11:15am – 12:15pm

2. Big Ideas • MIS operates on the state fiscal year. It returns to 0 every July. • The total number of people served/reached in any given quarter cannot exceed the total number of people in the community. • The same individuals shouldn’t be counted multiple times within the same strategy. • We want you to make educated estimates when you can’t directly obtain demographics from people. • We need to collectively do a better job of not lumping people into the “unknown” categories.

3. What is MIS? • An antiquated term from an old SAMHSA Management Information System into which the state had to report on numbers of people served through block grant funding. • What does it mean now? It is the part of the MOAPC, SAPC, and PFS quarterly report that asks you to report on the number of people served/reached by each strategy.

4. What Are These Data Used For? • The state reports them to SAMHSA as an indicator of activity conducted with block grant funds. SAMHSA reports them to Congress to justify their annual appropriation. These numbers are taken into account during the appropriation process and in determining eligibility and amounts for discretionary grant projects. • Huh?.... These numbers actually inform funding allocations, funding decisions, and funding eligibility. • No, seriously.

5. So Big Numbers Are Better!!! Right? • Wrong. • Consistent and well-informed numbers are better. • The state looks foolish if we report that 8 million people were served in a state with 6.8 million people. • The state looks foolish if we report that 2 million people were served in 2016 and 6 million were served in 2017. • The state looks foolish if we can’t report on characteristics of those served and if the numbers/characteristics don’t roughly match the services we proposed to deliver or the populations we proposed to serve.

6. Let’s Review the Reporting System • Information on numbers served used to live apart from the narrative report. It is now all integrated together. • Most of the big ideas I want to convey can be described by walking through the system.

7. Quarterly Report Strategy Section Strategies vs. Activities

8. State Fiscal Year [Appropriation] 1. Think in Terms of the State Fiscal Calendar • July 1 – September 30 – Quarter 1 - - - [BEGINNING of the year] • October 1 – December 31 – Quarter 2 • January 1 – March 30 – Quarter 3 • April 1 – June 30 – Quarter 4 - - - [END of the year] Starting July 1 each year. FORGET what you did up to that point in the grant. The counter returns to 0. Everyone is new again.

9. 3. Can only serve people once per strategy per quarter Quarterly Report – Numbers Served 2. Cannot Exceed Community Population Everyone is new in Q1 Keep an Eye on This When Reporting on NEW

10. Quarterly Report – Numbers Served [Incorrect]

11. 4. The Art and Science of the Educated Estimate Quarterly Report – Demographics

12. This is actually an important field Quarterly Report – Description of Estimates

13. 5. We can’t keep doing this Who Are We Reaching? No, really? SAPC – MIS Data FY17Q1-Q2: 86,368 total reached; 75,689 unique individuals

14. Big Ideas Revisited • MIS operates on the state fiscal year. It returns to 0 every July. • The total number of people served/reached in any given quarter cannot exceed the total number of people in the community. • The same individuals shouldn’t be counted multiple times within the same strategy. • We want you to make educated estimates when you can’t directly obtain demographics from people. • We need to collectively do a better job of not lumping people into the “unknown” categories.

15. Pro-Tips • The U.S. Census, DESE profiles, and other community or sub-population estimates are your best friend. • Figure out numerically in as much detail as possible who you are trying to reach. • Few sites are implementing “community-wide” strategies. How many parents of MS/HS kids are there? How many 18-25 year olds? How many MS/HS students? DEFINE your groups. • If there are 48,000 people in your community. You did not reach 48,000 people. • The best approach is always to try to collect these data. Particularly for non-media, direct service strategies. • Reach extends beyond those directly served in some cases (e.g., policy change, train the trainer, changes in practice among prescribers). This still doesn’t mean you reached the entire population of the community. Critically think about how far effects reach – RELATED TO YOUR OUTCOMES OF INTEREST. Don’t count 0-8 years olds when estimating the reach of an alcohol retail strategy.

16. Final Thoughts • We understand this is not easy. • The cluster model makes it even more difficult. • BSAS – and you by extension – is under increased state and federal pressure to demonstrate outcomes and accountability. As you might imagine, this isn’t limited to Massachusetts. • We want this to be a discussion. Call your Contract Manager or call me if you are unclear about what/how to report. • A lot of questions have case-by-case answers. • Doing this well benefits all of us. No, seriously. We’ll talk more about this later today.

17. Let’s Chat • Open floor • Comments? Clarifying Questions? • We will explore this afternoon how this support evaluation. • What do you need from us (including MassTAPP)? • What do we (state/evaluation team/MassTAPP) not understand?