1 / 19

Are Our Programs Successful? Results of MATEC Outcome Evaluation

Are Our Programs Successful? Results of MATEC Outcome Evaluation. Phil Bashook EdD, Marcia Edison PhD, Patricia Aguado LCSW, Barbara Schechtman MPH , Nathan Linsk PhD Midwest AIDS Education and Training Center. Objectives for today’s discussion.

delilah-contreras
Download Presentation

Are Our Programs Successful? Results of MATEC Outcome Evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Are Our Programs Successful? Results of MATEC Outcome Evaluation Phil Bashook EdD, Marcia Edison PhD, Patricia Aguado LCSW, Barbara Schechtman MPH , Nathan Linsk PhD Midwest AIDS Education and Training Center

  2. Objectives for today’s discussion Participants will leave this presentation able to: • Use evaluation data to recognize when a program is successful • Select and implement appropriate evaluation strategies • Describe the impact of some MATEC programs on participants.

  3. Questions for Today’s Discussion • What is success? • How do you know you were successful? • What type of changes do we expect as a result of our training?

  4. We are not discussing • How do participants learn? • How to measure participant satisfaction? • How to measure patient care outcomes?

  5. Was the program successful? • What do we mean by success? • How do you know you were successful? • What evidence would you need? • How would you get the evidence?

  6. What do we mean by “success”? Program participants use what they learned Three levels of use (change) Changed attitudes and knowledge Modified clinical care behavior Influenced patient care in their clinical setting

  7. What evidence do we need? • Examples of practices that participants are doing that they were not doing before (individual behavioral changes) • Examples of system changes that participants influenced the care practices in their clinical environment (system changes)

  8. A Note…AETC impact on practice We need evidence that participation in an AETC program was the likely cause of practice changes.

  9. How can we get the evidence? Ask!! Just ask the program participants.

  10. How can we get the evidence? • How to ask • On-line survey • Telephone Interview • Face-to-face Interview

  11. How can we get the evidence? • What to ask • What are you doing now that you were not doing prior to the AETC program (practice changes)? • Can you give some examples that demonstrate how you used what you learned (triangulation)? • Why did you change what you had been doing? (attribute to AETC program, other events?)

  12. Example from a MATEC Evaluation • Who we contacted • Participants from Level II programs • 539 contacted • 160 responded [28.9%] • When we contacted them • 4-6 weeks after program completion

  13. Summary Results • 160 total responses • 41 individuals (26 %) reported no changes • 119 (74%) reported some changes!!! This means MATEC programs had impact on 3/4 of participants who responded!

  14. Summary results 119 program participants made changes (there were actual 170 changes reported)

  15. Summary results Of the 170 changes: 29 %--attitudes/knowledge 48 %--practice behavior 24 %--care setting (systems) Many individuals made more than one type of change

  16. Changed attitudes and knowledge • 49 Responses (29%) • Example quotes • Better understanding of domestic violence, not as quick to judge friends • More aware to look for risk factors • Inspired to participate in ICTP • Heightened awareness what motivates people to comply/adhere • More self-reflective in terms of personal bias. • Training challenged my thinking • Cultural competence helped in working with my diverse client load

  17. Modified practice behavior • 81 Responses (48%) • Example quotes • Changed content of prevention talks, stages in treatment planning • Planning a community forum for women • Ordering more baseline HIV resistance testing • Ordered more lumbar punctures to rule out neruosyphillis • Following Hepatitis VL and AFP in all patients with co-infection • More direct questions on illicit drug use for harm reduction • Able to discuss co-infection risk with HIV testing counseling

  18. Influence system changes • 40 Responses (24%) • Example quotes • Change in admitting orders reinforced by training • Intake forms modified to include health & dual diagnosis related questions. • Patients interviewed without partner present • Gathered more literature & dispersed throughout office. • Preventive education in clinic q 2 days/week • CD and HIV staff working together • Had in-service at worksite... Targeted 2 staff members to go to training.

  19. Opportunities To Use Evaluation Results • Report outcome data to HRSA • Use for quality improvement & decisions about programming • Conduct further studies • Characterize behaviors & system changes • Estimate number of clients/patients affected by programs

More Related