Developing Indicators of Quality for AT Reuse Programs

1. Developing Indicators of Quality for AT Reuse Programs Lindsey Bean Kampwerth and Trish Redmon Moderators: Carolyn Phillips, Joy Kniskern and Liz Persaud Pass It On Center www.passitoncenter.org

2. Agenda Vision and goals of Indicators of Quality Overview of the development process Themes of feedback on first two versions Design of Prototype IQ-ATR Online Assessment Tool Next steps for IQ and Online Tool

3. Vision and Goals for Indicators of Quality for AT ReuseIQ-ATR • IQ-ATR is a living document that grows and develops as AT Reuse Programs develop. • IQ-ATR Online will be a dynamic, interactive tool to assess an AT Reuse Program. • IQ-ATR will link to Pass It On Knowledge Base to provide references to information and models to assist with improving the Reuse Program. 3

4. Vision and Goals for IQ-ATR Successful Practices inform Knowledge Base Knowledge Base supports IQ-ATR Self-Assessment Tool IQ-ATR and Knowledge Base inform Successful Practices IQ-ATR 4

5. This is not the first project:Other Related QI Initiatives • The SMART Exchange • Led by Shelley Kaplan, UCP in Atlanta • 1989-1991, NIDRR grant • Developed indicators for AT use The rationale for many of the SMART Exchange indicators are relevant to AT reuse indicators.

6. Other Related QI Initiatives • QIAT • Quality Indicators for Assistive Technology Services in School Settings • Developed in 1998 QIAT is an outstanding working model of quality indicators for AT use in a specific environment.

7. Validating the IQ-ATR • Have we defined a set of factors or processes known to lead to successful outcomes? • Can we create a review instrument to determine if the factors/processes are in place? • Is the indicator useful for internal quality improvements? • Can the indicator be used as an external measure of quality across programs of the same type and activities?

8. Developing theIQ-ATR • Two Webinars on Quality Indicators • A meeting of program professionals and PIOC team members in Atlanta to develop initial draft • Distribution of draft to developers and interested parties • Editing of draft based on feedback, including alteration of restrictive format • Development of assessment tool prototype

9. Developing the first draft Reviewed past quality indicators Reviewed guidelines for writing quality indicators Small groups edited and added to draft of quality indicators

10. First draft done: Critique time! QI for AT Reuse Development Meeting in Atlanta, March 23 & 24 with occupational therapists, program directors and other key informants. Thank you!

11. Program Operations User Services Sustainability Management Human Resources Marketing Accounting Organizational Structure Emergency Preparedness Partnerships and Collaboration Categories of Indicators

12. Suggestions from reviewers: Reorganize format: categories and sequence Combine some topics; separate some topics Consider strictures of tabular format Add rationale for categories Reorganize into “core” indicators and options based on program type Add specific AT State program indicators

13. Version 2 • Restrictive five-cell response format was abandoned in favor of a flexible checklist of key factors for consideration • Indicators were combined or split as needed • New version was presented to National Task Force – which recommended a thorough language review

14. Version 2.1 • NTF sub-committee (Rik Opstelten, Brian Bard, Kerri Morgan and Lindsey Bean) reviewed the language and recommended changes. • Oops! Name change! QIAT recommended against expansion of their name and possible confusion. (Hence, IQ-ATR.) • Language updated, and some major issues were confronted before building the Online Tool.

15. Major Challenges • Organizing the indicators into CORE indicators and implementing appropriate ‘branching’ based on program type • Weighting the categories and the indicators to create an appropriate “result” measure for the program • Implementing online assessment tool and links to Knowledge Base

16. Organizing the Indicators Variables affecting grouping: • Type of organization • AT reuse activities (reassign, refurbish, exchange, recycle) • Types of equipment (DME, computers, all?) How to deal with the possible combinations?

17. Weighting Categories & Indicators 1. Survey designed and individuals asked to rank: • Categories in order of importance • Indicators in order of importance within categories 2. Ph.D. statistician recommended weighting methodology based on ranking, but it required a standard number of responses.

18. To weight or not to weight? • Based on the ranking of categories and indicators, it is possible to produce a numeric “score” from the online tool. • For the initial assessment tool, it was decided not to produce numeric results, but advisory messages supported by lists of resources for each indicator.

19. Version 3 • Challenge: A return to the five-cell format would lose ease-of-use that made the flexible checklist of key factors so attractive • Challenge: Too many variables to group indicators by program type Solution: • Retain the flexible lists of key factors for consideration. • Create standard set of responses to assess compliance. • Allow program to determine if indicator applies. • No numeric scores – even hidden – at this time

20. IQ-ATR and theOnline Program Assessment Tool • Sharon Meek and the PIOC web developers from REACH, Inc. developed the prototype of the interactive tool www.passitoncenter.org/iq-atreuse

21. Home Page: A brief introduction to how the IQ-ATR tool works

22. Registration Create an account for personal use or register as a guest.

23. Directions explain the response choices.

24. Category in the yellow bar. The name of the indicator and the text of the indicator are followed by the Rationale and the Key Factors for Consideration. The standard set of responses follows. User selects a response and clicks Next.

25. User receives a results page for each category and may stop or continue to the next category. KB or other sources are suggested for indicators that do not meet all Key Factors for the promising practice.

26. Next Steps IQ-ATR: • Share and solicit additional feedback • Revise IQ-ATR based on recommendations • Design a validation process

27. Next Steps Online Program Assessment Tool: • Gather feedback from pilot users • Revise online tool and make “updatable” to keep synchronized with Knowledge Base (KB) • Allow user to choose sequence of categories • Have NTF review KB content for appropriateness and usefulness as recommended at May meeting • Identify or create content to fill reference gaps in Knowledge Base • Decide whether responses should be weighted and tallied for a quantitative (rather than general) result.

28. Not enough knowledge. . . • Referencing Indicators of Quality to content in the Knowledge Base highlighted the gaps in supporting information. • The content that does exist needs to be strengthened with more comprehensive presentation of needed policies and procedures. • The existing content has not been reviewed by NTF panels as recommended in May 2009.

29. How you can help • Review the IQ-ATR and send comments http://passitoncenter.org/content • Try the online tool and send feedback http://passitoncenter.org/IQATReuse • Contribute content or suggest resources for the Knowledge Base Please send comments and suggestions to: Lindsey@passitoncenter.org Trish@passitoncenter.org

30. DISCLAIMER This work is supported under a five-year cooperative agreement # H235V060016 awarded by the U.S. Department of Education, Office of Special Education and Rehabilitative Services, and is administered by the Pass It On Center of the Georgia Department of Labor – Tools for Life. However, the contents of this publication do not necessarily represent the policy or opinions of the Department of Education, or the Georgia Department of Labor, and you should not assume endorsements of this document by the Federal government or the Georgia Department of Labor. Thank YOU!!!