As part of the first activity, please complete the Pre-Test Worksheet and return it to an

1. As part of the first activity, please complete the Pre-Test Worksheet and return it to an NCDDR staff member. NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

2. NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome Overview of Workshop Activities May 31, 2005 10:00 a.m. - 1:00 p.m. - Room 6082 Potomac Center Plaza 550 12 St., SW, Washington, DC 20202

3. Overview of Workshop Activities RERC Workshop - Washington DC March 2, 2005, 1:00 p.m. - 5:00 p.m. • 35 participants representing 22 of 23 current RERCs • 32 pre-tests submitted • 28 post-tests submitted • Mean score increased from 65 points (pre) to 91 points (post) NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

4. Overview of Workshop Activities RRTC Workshop - Washington, DC April 27, 2005, 8:00 a.m. - 12:00 p.m. • 27 participants representing 18 of 36 current RRTCs • 27 pre-tests submitted • 27 post-tests submitted • Mean score increased from 64 points (pre) to 83 points (post) NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

5. Overview of Workshop Activities Dissemination and Outcomes Planning Workshop Washington, DC June 1, 2005, 8:00 a.m. - 5:00 p.m. • 39 participants representing 35 projects • 2 Capacity Building Projects (DRRPs) • 2 DBTACs • 4 Employment Projects • 2 DRRPs • 2 FIPs NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

6. Overview of Workshop Activities Dissemination and Outcomes Planning Workshop Washington, DC (continued) • 17 Health and Function projects • 1 DRRP • 6 FIPs • 9 Model Systems projects • 1 RRTC • 1 Independent Living and Community Integration project • 3 KDUs • 1 DRRP • 2 NIDRR Contracts • 1 State Technology Assistance project • 5 Technology for Access and Function projects (FIPs) NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

7. NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to OutcomeThe Logic Model Approach to Outcomes Planning May 31, 2005 10:00 a.m. - 1:00 p.m. - Room 6082 Potomac Center Plaza 550 12 St., SW, Washington, DC 20202

8. The Logic Model Approach to Outcomes Planning What is a Logic Model? • A logic model is a highly visual method of demonstrating relationships among project resources, activities, outputs, and outcomes. • Logic models are planning tools that indicate the resources a NIDRR project will employ to conduct activities that are intended to produce specific, describable, and measurable changes or results in people, organizations, or the broader physical and social environment. NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

9. The Logic Model Approach to Outcomes Planning Why a Logic Model? • Logic models help researchers target identified goals and desired outcomes by demonstrating how each step in an unfolding program or project is critically linked to the preceding step, whether it is needs sensing, building a conceptual framework, conducting activities and implementing interventions, or performing follow-up. • Logic models to help identify feasible types of data and data collection strategies to suggest progress toward identified project-relevant short-term and/or mid-term outcomes. NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

10. The Logic Model Approach to Outcomes Planning Logic Model Strategies: Backward Chaining • Backward chaining refers to the process of beginning logic model development by considering project outcomes and working in reverse to link the anticipated outcomes to audiences reached, outputs produced, and activities conducted. • Backward chaining enables newer projects to envision the realistic end results of their project work and then plan the activities from their NIDRR-funded scope of work that will achieve such changes. NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

11. The Logic Model Approach to Outcomes Planning Backward Chaining (continued) • Backward chaining is of particular value to so-called “in process” projects further down the road in their NIDRR grant cycle. Such projects might already have outcomes to report or data to suggest progress toward anticipated outcomes. Backward chaining allows these projects to benefit from a logic model in spite of having a shorter time to plan for outcomes. • All projects must remember that backward chaining is a strategy to plan for outcomes, not an opportunity to create new activities outside of their proposed scope of work. Outcomes should still be linked to the activities from the original project proposal. NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

12. The Logic Model Approach to Outcomes Planning Components of a Logic Model • Situation • Inputs • Activities • Outputs • Target Systems • Outcomes • Short-term Outcomes • Mid-term Outcomes • Long-term Outcomes NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

13. The Logic Model Approach to Outcomes Planning Situation • The goal of your NIDRR-sponsored project. • Identifies the problem or priority your project is addressing and the benefit to specific audiences. NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

14. The Logic Model Approach to Outcomes Planning Situation: Key Questions to Answer Clearly describe the purpose of your NIDRR-sponsored project: • What is the overall goal of your project? • Why is this important? • What specific problem or problems are you trying to address with your center or project? • What are you trying to accomplish relative to this problem? NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

15. The Logic Model Approach to Outcomes Planning Example of Situation • Published Personal Attendant Care (PAC) Quality Indicators (QIs) are not designed for low-literate, multicultural populations. (RRTC on Measuring Rehabilitation Outcomes and Effectiveness) • There is a lack of awareness and implementation of Section 508 accessibility standards among e-Learning practitioners. (Accessible e-Learning Authoring System Software and Model Course for Vocational Rehabilitation Services Personnel) NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

16. The Logic Model Approach to Outcomes Planning Inputs • The human and financial resources and systems needed to conduct a high quality, outcomes-oriented program or project. • Types of resources: staff expertise, volunteers, time, money, host institutional support, materials, equipment, technology, research results, previous accomplishments, etc. NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

17. The Logic Model Approach to Outcomes Planning Inputs: Key Questions to Answer What resources are available to your NIDRR-sponsored project to carry out the scope of work? • What human, financial, or other resources are available to apply to the project? • What supports do you have from NIDRR, other contributing agencies, and from your host organization? NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

18. The Logic Model Approach to Outcomes Planning Inputs: Key Questions to Answer (continued) • What resources and planning and management infrastructures do you need in order to conduct high-quality activities, produce outputs, and demonstrate progress toward completing anticipated outcomes? • Do you have sufficient resources and infrastructures in place to meet your goals? NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

19. The Logic Model Approach to Outcomes Planning Examples of Inputs • Previous research findings on the significant inequity in allocation of transportation resources between urban and rural areas. (RTC on Disability in Rural Communities) • Previous expertise in design and testing of accessible e-Learning. • Programming expertise from project collaborator.(Accessible e-Learning Authoring System Software and Model Course for Vocational Rehabilitation Services Personnel) • The data resources of the StatsRRTC. (RRTC on Employment Policy) NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

20. The Logic Model Approach to Outcomes Planning Activities • The action steps, tasks, procedures, and services performed in conjunction with implementing a planned program of research and development, capacity-building, training and/or technical assistance and knowledge translation and dissemination to achieve objectives and demonstrate results. • Project activities include the research, development, training, technical assistance, dissemination, utilization, and other activities specified in your NIDRR grant proposal. NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

21. The Logic Model Approach to Outcomes Planning Activities: Key Questions to Answer • What activities are specified in your NIDRR grant proposal? • What actions, processes, events, services, products, technologies, or other elements will be used to implement your project? • Will your activities be exclusively offered to a specific audience or to members of a specific target system? • Will your activities be organized or “phased” in a particular manner? NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

22. The Logic Model Approach to Outcomes Planning Examples of Activities • Hold scientific conference in the third year. • Conduct 12 to 15 Disability Policy Forums. (RRTC on Employment Policy) • Develop journal articles focusing on technology transfer and research findings. (RERC on Communication Enhancement [AAC-RERC]) • Conduct a national assessment of potential involvement of rural Faith-Based Organizations in providing community transportation for people with disabilities. (RRTC on Disability in Rural Communities) NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

23. The Logic Model Approach to Outcomes Planning Examples of Activities • Build and field-test e-Learning authoring system software. (Accessible e-Learning Authoring System Software and Model Course for Vocational Rehabilitation Services Personnel) • Conduct randomized clinical trial of the “Executive Plus” program. (RRTC on TBI Interventions) NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

24. Connecting Dissemination Activities with Projected Outcomes Activities • Proposed Activities by Area: • Research • Development • Demonstration • Dissemination • Utilization • Technical Assistance NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

25. Connecting Dissemination Activities with Projected Outcomes Activities (continued) • Coordinate all activities to address outcomes. • Dissemination activities should promote outcome accomplishments. • Product orientation (newsletter) - nonstrategic • Activity orientation (web site) - nonstrategic • Outcome orientation - strategic • Outcome planning increases efficiency, effectiveness, and relevance. NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

26. The Logic Model Approach to Outcomes Planning Outputs • The direct results of program activities, such as the discoveries or findings, models, tools, devices, products, publications, events, and/or services produced for external audiences. • Outputs may be expressed in quantitative or qualitative terms. • Outputs are important indicators of productivity and are the essential building blocks of outcomes. • Though outputs contribute to outcomes, not every output will have a corresponding outcome, nor should it. NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

27. The Logic Model Approach to Outcomes Planning Outputs: Key Questions to Answer Describe the services, products or events you anticipate will be the results of your NIDRR-sponsored project activities. • What are the direct services, products or research-based materials that will be developed through your project? • What findings, publications, products, devices, systems, and services) do you plan to produce? • How do these vary for different target systems? NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

28. The Logic Model Approach to Outcomes Planning Examples of Outputs • Conference proceedings. • Papers and corresponding policy briefs based on Disability Policy Forum results. (RRTC on Employment Policy) • Journal articles. • AAC Website. • Writers Brigade Publications. (AAC-RERC) NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

29. The Logic Model Approach to Outcomes Planning Examples of Outputs (continued) • Journal articles to report on the distribution and use of Section 5310 resources and on the extent of adoption of coordinated and voucher model for rural transportation. (RTC on Disability in Rural Communities) NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

30. The Logic Model Approach to Outcomes Planning Measuring Outputs • Outputs are directly observable. • Output measures can be expressed in a quantitative or qualitative manner -- e.g., # of manuscripts published, students trained, patents filed, or the nature of new findings, tools, devices, products, etc. NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

31. Connecting Dissemination Activities with Projected Outcomes Outputs • Generic outputs. • Individualized, tailored outputs. • Clarity of relationship among: • Output • Target system element • Outcome NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

32. The Logic Model Approach to Outcomes Planning Target Systems • The group of individuals, organizations, or other entities expected to be affected by a project. Target systems are specific, realistically measurable consumer audiences in which the group, the commonalities bonding the group, and the context within which the group exists are all defined within a project's scope of work. • The specific members of the overall target audience identified as potential beneficiaries of your project’s work and among whom data collection will occur. NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

33. The Logic Model Approach to Outcomes Planning Target Systems: Key Questions to Answer In what segments of the overall target audience will change be measured through strategic data collection? • Who are the specific members of your target audience from whom data will be collected (such as people with disabilities, service providers, etc.)? • What specific stakeholder groups have been strategically selected in your project dissemination plan to receive project outputs? NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

34. The Logic Model Approach to Outcomes Planning Examples of Target Systems • Members of the RERC-AAC network who currently use AAC and their families and advocates. (RERC-AAC) • Rural Faith-Based Organizations. (RTC on Disability in Rural Communities) • Members of the Congressional Disability Caucus. NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

35. The Logic Model Approach to Outcomes Planning Examples of Target Systems • e-Learning practitioners producing CD-ROM based educational courses • State offices of vocational rehabilitation within Federal Region VI (Accessible e-Learning Authoring System Software and Model Course for Vocational Rehabilitation Services Personnel) NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

36. Connecting Dissemination Activities with Projected Outcomes Target Systems • Relationship to outputs and outcomes • Specificity by: • Type(s) of systems • Geographic location vs. universe • Sub-groups within target systems • Whole or percentage of the whole • Service (receiving services in last 12 months) NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

37. Connecting Dissemination Activities with Projected Outcomes Target Systems (continued) • Age • Gender • Educational level • SES • Primary/secondary conditions • Race/ethnicity NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

38. The Logic Model Approach to Outcomes Planning Outcomes • The effects of R&D and related activities and outputs and constitute changes or improvements in identified target systems. • Outcomes are separated into 3 groups: short-term, mid-term, and long-term. • Outcome goals differ by the type of change or improvement intended (knowledge, action, etc.), the target system in which change is expected to occur, and the level of accountability for which grantees are responsible. • Outcomes are produced in more than one target system and occur sequentially over time, resulting in an incremental “chain of outcomes,” starting with short-term and progressing to mid-term and eventually longer-term outcomes. NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

39. The Logic Model Approach to Outcomes Planning Outcomes: Key Questions to Answer • How will members of target systems’ awareness, attitudes, knowledge, behavior, skills, actions, decisions, policies or level of functioning be measurably changed (benefited) by your activities and outputs? • Short-term outcomes • Mid-term outcomes • Long-term outcomes NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

40. The Logic Model Approach to Outcomes Planning Short-term Outcomes • The anticipated or actual changes or improvements in the knowledge base and/or learning system within an identified target system. • Short-term outcomes are under the direct influence of project activities. • They represent the first level of change that must occur in order to bring about mid-term outcomes. • Types of change: learning, awareness, knowledge, attitudes, skills, opinions, aspirations, and motivations. NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

41. The Logic Model Approach to Outcomes Planning Short-term Outcomes: Key Questions to Answer • What changes or improvements in the identified target system will have occurred under the direct influence of project activities? • What changes or improvements in learning, awareness, knowledge and attitudes will have occurred? NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

42. The Logic Model Approach to Outcomes Planning Examples of Short-term Outcomes • Increased knowledge and awareness among target system about the disproportionate allocation of resources for rural transportation, as observed in focus groups. (RTC on Disability in Rural Communities) • Increase literacy, skills, and ability among sampled individuals who use AAC. (AAC-RERC) NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

43. The Logic Model Approach to Outcomes Planning Examples of Short-term Outcomes (continued) • 50% of VR counselors in Federal Region VI report improved knowledge of utility and accessibility of e-Learning courses. (Accessible e-Learning Authoring System Software and Model Course for Vocational Rehabilitation Services Personnel) • By the end of Year 3, at least 50 percent of the target system will demonstrate an increased understanding of arthritis prevention strategies (25 percent improvement on pre/post measurement). (MARRTC) NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

44. The Logic Model Approach to Outcomes Planning Measuring Short-term Outcomes • Do you have a data collection and tracking system in place to monitor and document progress towards the completion or satisfaction of short-term outcomes? • What data and data sources will you use? • Are they specific enough to provide evidence of your performance and accomplishments? NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

45. Connecting Dissemination Activities with Projected Outcomes Short-term Outcomes • Need for Specificity in Outcome Statements • What will change? • Who/what will manifest the change? • What is the consistency of the change: • All members of the target system? • In all cases? • What outputs do the outcomes derive from? NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

46. Connecting Dissemination Activities with Projected Outcomes Short-term Outcomes (continued) • What is the change in: • Degree of consistency (4 out of 5 times) • Degree of accuracy • Degree of frequency (used 80% of time) • Degree of time • Relationship to the target system and outputs NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

47. The Logic Model Approach to Outcomes Planning Mid-term Outcomes • The anticipated or actual changes or improvements that occur in part as a result of the use or adoption of program outputs. • Mid-term outcomes involve changes in use and, thus, take longer to achieve than short-term outcomes. • Unlike short-term outcomes which occur under the direct influence of program activities, there may be other causal factors contributing to the achievement of mid-term outcomes. • Types of change: actions, behaviors, practices, decisions, policies, system capacity, and social actions. NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

48. The Logic Model Approach to Outcomes Planning Mid-term Outcomes: Key Questions to Answer • What expected or actual changes or improvements in the identified target system will have occurred in part as a result of the use or adoption of project outputs? • What changes in behaviors, actions, decisions, and policies will have taken place in the identified target system in response to short-term outcomes? NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

49. The Logic Model Approach to Outcomes Planning Examples of Mid-term Outcomes • 25 percent of the target system will demonstrate increased implementation of strategies to eliminate disproportionate allocation of resources for rural transportation, as identified through a survey. (RTC on Disability in Rural Communities) • New policies enacted by AAC manufacturers and policymakers encouraging increased use and availability of AAC technologies. (AAC-RERC) NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH

50. The Logic Model Approach to Outcomes Planning Examples of Mid-term Outcomes (continued) • 50% of e-Learning practitioners report increased development of accessible courses. (Accessible e-Learning Authoring System Software and Model Course for Vocational Rehabilitation Services Personnel) • By the end of Year 4, at least 50 percent of the target system will demonstrate an increased use if arthritis self-management techniques as observed in focus groups. (MARRTC) NIDRR Staff Outcomes Planning and Performance Reporting Workshop: From Activity to Outcome May 31, 2005 • NCDDR/RUSH