Chapter 1

1. Chapter 1 The Evaluation Research and Evidence-Based Practice Partnership

2. Practicing Research • Research practitioners are concerned with the practical applications of research findings. • They use or consumeresearch as the basis for making decisions about programs, practices, and policy. • These practitioners/consumers specifically rely on • evaluation researchers for evidence about programs • evidence-based practitioners for methods to assess the validity of the evidence • Research practitioners/consumers also use experience and clinical judgment to match research findings to client needs.

3. The Partners: Part 1—The Evaluators Evaluation Researchers • Evaluation researchers—evaluators--use the scientific method to provide valid evidence on the outcomes, impact, and costs of programs to improve the public’s health, education, and welfare. • Research evidence alone may be insufficient to meet the needs and expectations of clients, patients and other users of the evidence. • Evaluators have identified methods for determining those needs and using their expertise to incorporate user values and expectations into their study’s purposes and methods.

4. The Partners: Part 2—The Practitioners Evidence –Based Practitioners • Evidence-based medicine (EBM) and its sibling, evidence-based practice (EBP), are concerned with using experience and clinical judgment to integrate best evidence from research with patient values and experience. • EBM/EBP practitioners have developed special methods for identifying research studies and grading the quality and strength of the evidence. • By design, EBM/EBP combines the best research evidence and brings it together with clinical and professional expertise and patient/client values to make clinical decisions.

5. Evaluation Research: The Definition Evaluation research aims to provide new knowledge about social behavior by using the scientific method to assess the process, outcomes, impact, or costs of programs or interventions. • Outcomes are the results of program participation. • Impact refers to the magnitude and duration of program effects. • Processevaluations focus on program activities and organization. (They are sometimes called formative or implementation evaluations.) • A program consists of activities and resources that have been specifically selected to achieve beneficial, specified outcomes.

6. Evidence-Based Medicine/Practice: The Definitions • Evidence-based medicine (EBM) is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of patients. • Evidence-based practice (EBP) is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of clients.

7. Evaluation Research and Other Types of Evaluation • Evaluation research is a subdivision of the much larger field of program evaluation. • Evaluation research shares many purposes with program evaluation including improving products and practices, personnel, programs, organizations. • Evaluation researchers always use the scientific method; their ideal is to produce generalizable and new knowledge. • Other evaluators tend to be more concerned with precipitating change or engaging others in the evaluation process; they may adopt business and political methods.

8. Evaluation Research and Other Types of Social Research • Evaluation researchers and social researchers rely on the scientific method . • Evaluation researchers study the effects of programs and interventions on participants. • Evaluations are conducted in “real life’” situations • Evaluations are sometimes referred to as effectiveness studies rather than efficacystudies, which are done under ideal conditions. • Other social researchers focus on studies that describe relationships and predict events.

9. Evaluation Research and Other Social Research: Side-by-Side

10. Hypothesis and Outcomes: Participating (or Not) and Achieving (Planned or Not) A hypothesis is a tentative assumption made in order to draw out and test its logical or empirical consequences.. • Hypothesis: The experimental and control groups will not differ in their outcomes • Hypothesis: Boys in the experimental group will improve signficantly over time when compared to boys in the control group • Outcomes result from program participation. • Beneficial outcomes include improvements in health, education, social well-being and economic prospects.

11. Evidence-Based Medicine: Five Steps • Convert the need for information into answerable questions • Track down the best evidence to answer the questions • Critically appraise the evidence for validity and impact • Integrate the critical appraisal with experience and understanding of values • Evaluate one’s own effectiveness and efficiency in executing steps 1 – 4 and seek ways to improve both next time

12. Evaluation Researcher, Evidence-Based Practitioner and Research Consumer: Compare and Contrast What Does Each Want? Researcher: Produce valid evidence about the process, outcomes, impact, and costs of programs and interventions • Knowledge Practitioner: Identify evidence that matters (valid, meaningful and consistent impact) to provide the best possible clinical care • Care Consumer: Identify high quality evidence-based programs that meet client needs, values, and expectations • Programs that work with patients and clients

13. Evaluation Researcher, Evidence-Based Practitioner and Research Consumer: Compare and Contrast Researcher • Uses scientific method to design studies, collect information, and analyze and interpret data • Provides a report describing and defending the evidence Evidence-Based Practitioner • Uses prespecified and transparent systems for grading the quality and rating the strength of the evidence • Uses the best evidence available to select options for treatment, incorporating patient values and expectations Consumer • Identifies clients’ needs, values and expectations • Identifies high quality evidence-supported programs • Matches clients’ values and high quality programs