AgrAbility NTW McGill QOL Webinar November 28, 2012 9-10 AM (MT)

1. AgrAbility NTW McGill QOLWebinarNovember 28, 20129-10 AM (MT) By Robert J. Fetsch, Extension Specialist, Professor Emeritus & Director, Colorado AgrAbility Project, Human Development & Family Studies Colorado State University AANTWMcGillQOL11.2812 (Rev. 11.2712a)

2. Basic Webinar Instructions • Need speakers or headphones to hear the presentation • Meeting > Manage My Settings > My Connection Speed • Dial-up not recommended • Questions about presentation • If you have a web cam/microphone, click the “Raise Hand” icon to indicate that you have a question • We will activate your microphone • If you don’t have a microphone, type into chat window and hit return

3. Basic Webinar Instructions • 4 quick survey questions • Session recorded and archived with PowerPoint files at www.agrability.org • Problems: use chat window or email jonesp@purdue.edu

4. Known Webinar Issues • Increased interactivity = increased technical issues • Be patient while we push our boundaries • Meeting > Audio Setup Wizard • Disconnection with presenters • Hang on – we’ll reconnect as soon as possible • Disconnection with participants • Log in again

5. Current SRAP QOL Impact Measurement& Plans for the Next Four YearsBy Robert J. Fetsch (CSU) With Thanks to Brad Rein for 22+ Years of Leadership (1990-2012)

6. Current SRAP QOL Impact Measurement& Plans for the Next Four YearsBy Robert J. Fetsch (CSU) Welcome to Aida Balsano,National Program Leader—Child and Family DevelopmentDivision of Family and Consumer SciencesStrengthening Families, Farms, Communities and the Economy

7. Part 1: Current State of SRAP QOL Impact MeasurementPart 2:Plans for the Next Four Years

8. Part 1: Current State of SRAP QOL Impact Measurement

9. 2006 2012 2016

10. Our AgrAbility Mission The AgrAbility Mission is to enhance and protect quality of life and preserve livelihoods. It’s about supporting and promoting growth and independence. Ultimately it’s about hope. Source: National AgrAbility Project. (2011). It’s about hope [DVD]. Author: Purdue University.

11. Why Do We Do Program Evaluation? To make sure we’re on track with our mission To find out what works To be accountable and to be good stewards of the tax $ we receive To use valid and reliable instruments to document our AgrAbility program impacts

12. History of National AgrAbility Project Evaluation Committee (NAPEC) Early 2006—Kathryn Pereira, Evaluation Specialist NAP U of WI, invited all SRAP’s to join in an AgrAbility evaluation study. NAPEC met approximately bi-monthly (2007-Present) via teleconference/face-to-face (N = 6-25 participants/meeting).

13. NAPEC Includes: 32 SRAP Team Members 2 NAP Evaluation Team Members—Bob Aherin & Chip Petrea Bob Fetsch as Chair since 2007

14. NAPEC Produced Results Created and tested a practical procedure that 15 SRAPs are using to assess how successful they are at increasing their clients’ quality of life levels (QOL) and their Independent Living and Operating Levels (ILOS).

15. NAPEC Produced Results Began to address SRAPs’ highest need for evaluation related assistance and training by measuring change or impact as a result of AgrAbility services with the McGill QOL and ILOS. This is according to the most current National AgrAbility Project needs assessment & program evaluation summary for 2010 (B. Aherin, personal communication, January 23, 2012).

16. NAPEC Produced Results Increased involvement by the number of SRAPs who used the McGill QOL and ILOS to collect pre-survey and post-survey data from their new clients from 2 in 2008 (VA & WI) to 6 in 2012 (CO, KS, NE, VA, WI, & WV).

17. NAPEC Produced Results Increased the number of SRAPs who are committed to use the MQOL and ILOS from 6 in 2008 to 15 in 2012 (AR, CO, KS, ME, MN, MO, NC, NE, OH, OK, TX, UT, VA, WI, & WV). NAP, UIUC, and CSUE THANK YOU!

18. How many SRAPs are collecting QOL and ILOS data from their new clients?

19. AgrAbility across the U.S. (source: National AgrAbility Project)

20. NAPEC Produced Results Published two refereed journal articles plus a report on the effectiveness of AgrAbility. Christen, C. T., & Fetsch, R. J. (2008). Colorado AgrAbility: Enhancing the effectiveness of outreach efforts targeting farmers and ranchers with disabilities. Journal of Applied Communication, 92(1&2), 1-12. Fetsch, R. J., Helfrich, C. M., Field, W. E., & Olson, E. O. (2010, March 2). AgrAbility: A program that works. Goodwill Industries. Meyer, R. H., & Fetsch, R. J. (2006). National AgrAbility Project impact on farmers and ranchers with disabilities. Journal of Agricultural Safety and Health, 12(4), 275-291. Another manuscript is in preparation.

21. History of NAPEC Who is an AgrAbility Client? An AgrAbility client is an individual with a disability engaged in production agriculture as an owner/operator, family member, or employee who has received professional services from AgrAbility project staff during an on-site visit.

22. History of NAPEC 2 Questions: Do our AgrAbility clients increase their QOL levels? Are our AgrAbility clients more able to live on, operate, and manage their farms/ranches if they choose?

23. History of NAPEC Six SRAP’s conducted a four-year longitudinal study to answer the 2 questions (June 2007-July 2011). CO, KS, NE, VA, WI, & WV

24. Measures Used in CO, KS, NE, VA, WI, & WV Study McGill Quality of Life (QOL) Survey & AgrAbility Independent Living & Operating Survey (ILOS) NAP Demographic Data

25. History of NAPEC By July 13, 2011 CO, KS, VA, & WI entered their 98 matched pre-post-survey data into Excel files and e-mailed them to CSUE for entering and analyzing. KS 49 WI 31 CO 15 VA 3 Total 98

26. Q: Do Our AgrAbility Clients Increase Their QOL levels?A: ?

27. McGill Pre- Post-Survey Changes (Single item Scale, Physical Well-Being, & Physical Symptoms)

28. McGill Pre- Post-Survey Changes (Support, Experiential Well Being & Psychological Well-Being)

29. McGill Pre- Post-Survey Changes (Total Score)

30. Q: Do Our AgrAbility Clients Increase Their QOL levels?A: Yes, they improve on the Total QOL Scale plus on all 6 subscales!

31. Q: Are Our AgrAbility Clients More Able to Live on, Operate, and Manage Their Farms/Ranches if They Choose?A: ?

32. AgrAbility Independent Living & Operating Survey (ILOS) (Manage Farm, Complete Chores, & Operate Machinery)

33. AgrAbility ILOS (Live in Home, Access Workspaces & Modify Machinery)

34. Q: Are Our AgrAbility Clients More Able to Live on, Operate, and Manage Their Farms/Ranches if They Choose?A: Yes!

35. The results of this four-state study show that all of the McGill QOL group mean scores increased from pre- to post-survey plus manage farm, operate machinery, and modify machinery***. We invite you to join us! See me today, send me an e-mail, or give me a call.

36. These results look promising,BUT…

37. How do we know these results are not due to something other than our AgrAbility services?

38. Part 2:Plans for the Next Four Years

39. NAP Priority 5 is Evaluation: Provide an accurate picture of AgrAbility’s outcome and impacts through SRAP SRAP Quality of Life survey How successful are we at increasing customer quality of life?

40. NAP Proposal Reviewer Comments Focus on accountability What’s the impact on clients? What difference does AgrAbility make? Need for greater Evidence-Based information that would support continuation of AgrAbility

41. NAP Proposal Reviewer Comments Collect QOL data and publish results so Brad Rein can show impact. Form a new control group for comparison. Document impacts and publish results.

42. Brad Rein Reported “Good News” and “Bad News.”

43. “Good News” AgrAbility is among the 45 federally funded programs that supported employment for people with disabilities in fiscal year 2010. AgrAbility is among the 10/45 programs with a review or study to evaluate the program’s effectiveness. Source: U.S. Government Accounting Office. (2012). Employment for people with disabilities; Little is known about the effectiveness of fragmented and overlapping programs (GAO Publication No. 12-677). Washington, DC. (p. i).

44. “Good News” “…The Department of Agriculture’s AgrAbility program conducted a review of its activities between 1991 and 2011 and found that 11,000 clients had been served, and that 88 percent of those clients continued to be engaged in farm or ranch activities.” Source: U.S. Government Accounting Office. (2012). Employment for people with disabilities; Little is known about the effectiveness of fragmented and overlapping programs (GAO Publication No. 12-677). Washington, DC. (p. 27).

45. “Bad News” “However, this study did not determine whether other factors may have contributed to participants’ positive outcomes.” “No impact study.” Source: U.S. Government Accounting Office. (2012). Employment for people with disabilities; Little is known about the effectiveness of fragmented and overlapping programs (GAO Publication No. 12-677). Washington, DC. (pp. 27, 80).

46. Brad Rein asked us to help respond. So far 15 SRAP’s are working to collect data from AgrAbility clients with an on-site visit (AR, CO, KS, ME, MN, MO, NC, NE, OH, OK, TX, UT, VA, WI, & WV). We welcome the rest of you to join us.

47. AgrAbility across the U.S. (source: National AgrAbility Project)

49. Experimental Group (N = 200 with matching pre- and post-surveys)Control Group (N = 100 withmatching pre- and post-surveys)

50. Control Group (N = 100) Cannot be receiving any type of AgrAbility program services or onsite visits regardless of whether they are in USDA funded or Affiliate States.