None of the

1. None of the interventions are evidence-based but think positive – you get a new and untested intervention

2. IMS APPROACH TO GOOD PRACTICEKnut SundellJenny RehnmanMari Forslundwww.evidens.nu

3. Director-general NBHW Board of IMS (12 stake-holders) IMS director Support & Dissemination (administrative support & dissemination of all that is produced within IMS) Systematic reviews (assessment tools and interventions) Development & Evaluation (assessment tools and interventions) 10 employees 15 projects lasting 1-2 years Additional external researchers 20 employees 30 projects lasting 3-4 years 10 employees 15 projects lasting less than 1 year

4. Sweden • Sweden is the third largest country in Western Europe with nine million inhabitants • Social care service delivery involve agencies of the 290 municipalities, and 20 regional county councils. Median size of municipalities = 15.250 inhabitants • National agencies and research councils, deals with research and development, supervision of service quality and safety, and the delivery of institutional care. • The voluntary sector is considerable • Social welfare states (e.g., low rate of unemployment, poverty, drug use, violence) • General trust among Swedes in collective solutions and in government authorities

5. Research on interventions for womenwith experience of partner violence(Anttila et al, 2007)

6. Research on interventions for womenwith experience of partner violence(Anttila et al, 2007)

7. Research on interventions for womenwith experience of partner violence(Anttila et al, 2007) 0,1 per cent

8. Swedish outcome studies on social work Important dates Official reports of the Swedish government – “Evidence-based social work practice – favouring clients” Agreement between the government and the Swedish Association of Local Authorities and regions on implementing the national guidelines on substance abuse treatment Two articles in Sweden’s largest morning paper by the Director-General of the Swedish National Board of Health and Welfare claiming that social work agency managers did not have any idea whether social care services made any difference to clients’ lives. The first national guidelines on social work (substance abuse treatment) • National action plan for preventing alcohol related harm & National action plan on drugs (lasting to 2007). • 100 million SEK on research • Training of local prevention workers • An inquiry initiated at the NBHW on how to support EBP by • National guidelines, research reviews • National and regional knowledge transfer • Developing criteria for local-follow up and open comparisons Initiative to clarify the concept of EBP among governmental authorities General election and a new government Addiction Severity Index is introduced in Sweden IMS is inagurated

9. Looking for an evidence-based practice

10. Random sample of local authorities (2/3) Managers responsible for budget & staff Autumn 2007 Electronic questionnaire Response rate 88% (n = 834)

11. INTEREST IN EVIDENCE-BASED METHODS (N = 834)

12. PREVALENCE OF EVIDENCE-BASED METHODS (SOMETIMES) 24 per cent

13. PREVALENCE OF EVIDENCE-BASED METHODS (MOSTLY) 7 per cent

14. PREVALENCE OF EVIDENCE-BASED METHODS (SOMETIMES)

15. PREVALENCE OF STANDARDISED ASSESSMENT TOOLS (SOMETIMES) 63 per cent

16. PREVALENCE OF STANDARDISED ASSESSMENT TOOLS (REGUAL) 14 per cent

17. Sweden: What are the political issues that need to be addressed? Keywords: Ethics – Evidence – Transparency not the technical issues concerning internal / external validity Knowledge transfer / dissemination of innovations need extensive support at a regional level Bring forward good examples

18. Stockholm evidence-based Clearinghouse for social work • ”Evidence-based medicine is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients” (Sackett et al., 1996) • The general aim of “the Stockholm evidence-based Clearinghouse for social work” is to build a bridge between professionals and research on “what works” • It was launched on 1st September, 2008 18

19. Stockholm evidence-based Clearinghouse for social work Web-based service (at IMS’ homepage): http://www.socialstyrelsen.se/Amnesord/socialt_arbete/IMS/Metodguiden_index.htm Provides information on interventions, assessment tools and general knowledge (e.g., mechanisms) Target group: Professionals, politicians and policy makers Not recommendations Arranged in a simple, straightforward format reducing the need to conduct literature searches Methodology of the systematic review (e.g., Higgins & Green, 2008)

20. Topics Currently Available on the Website Target groups • Child/family • Social assistance (for welfare recipients) • Disability • Addiction • Elderly Subheadings • Interventions (18 descriptions/5 final evaluations) • Assessment tools (46 descriptions/13 final evaluations) • General knowledge • IMS projects 20

22. The evaluation process • Selection of interventions to evaluate – starts with discussion in IMS research council: Which interventions should we evaluate? Which outcomes are important? What databases should be searched? • All relevant studies with a RCT or QE (including data at baseline) are reviewed • Use of a guide/protocol to assist the review process • Studies are evaluated for to their internal validity (i.e., can we trust the result?) • Two independent reviewers evaluate each study. Any disagreements are settled based on consensus with help from a coordinator 22

23. Judging of internal validity Selection bias e.g., Are there any important differences between the groups? Performance bias e.g., Are researchers, participants and data collectors ”blind”? Attrition bias e.g., Any differences in size and type of attrition between the groups? Detection bias e.g., Are the outcome measures measured in the same way in all groups? Potential bias is evaluated within and across each domain of bias Low risk: plausible bias unlikely seriously alter the results Unclear risk: plausible bias that raises some doubt about the results High risk: plausible bias that seriously weakens confidence in the results (from Higgins & Green, 2008) 23

24. Grading of evidence • The scientific raiting scale is a modified version of a scale developed at California Evidence-Based Clearinghouse for Child Welfare (http://www.cachildwelfareclearinghouse.org/scientific-rating/scale) • Based on statements in Flay et al., 2005 (Society for Prevention Research, SPR) • The scale is devided into five grades - A lower score indicates effectiveness and a higher level of research support • Grading of evidence is based on: • Number of studies with a certain degree of internal validity (i.e., risk of bias) • If the practice is evaluated in usual care • If the effect are sustained over time • If the practice may cause harm • If the practice is replicable 24

25. Effective practice with well-supported research evidence At least two studies with low risk of bias, in different usual care or practice settings, have found the practice to be superior to treatment as usual (TAU). If multiple effectiveness studies have been conducted, the overall weight of the evidence supports the benefit of the practice. In at least one study with low risk of bias, the practice has shown to have a sustained effect at least one year beyond the end of treatment. There is no theoretical or empirical basis indicating that the practice constitutes a substantial risk of harm to those receiving it. The practice has a book, manual, and/or other available writings that specify components of the service and describes how to administer it.

26. 2. Effective practice supported by research evidence At least one study with low risk of bias has found the practice to be superior to treatment as usual (TAU). If multiple outcome studies (at least with medium risk of bias) have been conducted, the overall weight of evidence supports the benefit of the practice. In at least one study with low risk of bias, the practice has shown to have a sustained effect of at least six months beyond the end of treatment. There is no theoretical or empirical basis indicating that the practice constitutes a substantial risk of harm to those receiving it. The practice has a book, manual, and/or other available writings that specifies the components of the practice protocol and describes how to administer it.

27. 3. Practice with promising research evidence At least one study with medium risk of bias has established the practice's benefit over no intervention (or placebo or waiting list) or is found to be comparable to or better than treatment as usual (TAU). If multiple effectiveness studies with at least medium risk of bias have been conducted, the overall weight of evidence supports the benefit of the practice. There is no theoretical or empirical basis indicating that the practice constitutes a substantial risk of harm to those receiving it. The practice has a book, manual, and/or other available writings that specify the components of the practice protocol and describe how to administer it.

28. 4. Practice where the evidence fails to demonstrate effect At least two studies with low risk of bias have found that the practice has not resulted in improved outcomes compared to no intervention (e.g., placebo or waiting list), or that the practice is shown to be less effective when compared to treatment as usual. If multiple effectiveness studies have been conducted, the overall weight of evidence does not support the benefit of the practice. There is no theoretical or empirical basis indicating that the practice constitutes a substantial risk of harm to those receiving it. The practice has a book, manual, and/or other available writings that specify the components of the practice protocol and describe how to administer it.

29. 5. Concerning practice At least one study of low or medium risk of bias shows that the intervention can cause serious harm, and/or there is a reasonable theoretical basis suggesting that the practice constitutes a risk of harm to those receiving it. The practice has a book, manual, and/or other available writings that specify the components of the practice protocol and describe how to administer it.

30. Beyond the five grades the scientific scale includes an additional category (not rated - no number is given). Practice with unknown effect There is lack of studies with a medium or low risk of bias. There is no theoretical or empirical basis indicating that the practice constitutes a substantial risk of harm to those receiving it. The practice has a book, manual, and/or other available writings that specifies the components of the practice protocol and describes how to administer it.

31. Conclusions and summary - Is there a sufficiently robust evidence base to identify good practice? What are the strength and weaknesses? Are there significant gaps that should be addressed? Is there an agreed approach to deciding what counts as evidence? Is there an agreed approach for judging the quality of the evidence base? Is there a scope for working internationally to strengthen the evidence-base?

32. IMS practical delivery mechanisms to promote the adoption of good practice

33. IMS want to reach: • Social workers • Politicians and decision makers • Authorities • Universities and students • Researchers • Clients and others who are interested

34. Target groups • Social work agency managers • Government officials

35. Parts of our communication strategy • Value of direct, face-to-face communication • Web-based services • Tailored products • Short courses for social work agency managers about EBP • Start kit for municipalities for working with EBP

36. IMS STRATEGI Target groups Social workers Politicians and decision makers Governments - - - - - - - - Universitys Goals Increase the interest for EBP Provide support for decision-making Implementation knowledge Support to managers Long-term partnership Activities Lectures Clearinghouse Readiness for change Handbook in leading EBP Pilot-project Train the trainers Ask a researcher Start-up-package IMS network for EBP ETICS INFORMATION TOOLS FOR CHANGE NOT COACHING SPECIFIC METHODS

37. Organizational Readiness for Change Dwayne Simpson Anonymous survey to co-workers, managers and clients

38. Pilot-project Södermalm - Trying different mechanisms to promote EBP • A steering group and 24 IMS-coodinatiors with special drive • Seminars about EBP and implementation • ORC-survey • Focusing on the managers with seminars about leading EBP • the importance of the managers • management research • tools from the handbook • Local seminars about how to find research and assessment tools

39. Training trainers in ASIAddiction Severity Index • Big difference when local support in the new method is given and when the managers get’s support and owns the implementation • 2 persons at IMS, 25 local trainers, 12 county administrative boards and 194 municipalities' • Combination of IMS knowledge in implementation research and the method ASI and the local knowledge about the practice and support to the managers • Success factors: • National support • Local competence • Managers in focus • Networking

40. Start-up- package Focused on Managers Study material as an introduction to EBP Content: • EBP – concepts and implications  • Evidence, ethics and affectivity • Risk- and protective factors • Assessment tools • Research reviews • Dissemination • EBP and cost effectiveness   Including: • DVD-films • The book Evidence based practice in social work • The book To change social work • PowerPoint material • Glossary • Information about IMS website

41. Publications IMS-nytt New layout + more pages 4 numbers annually Special feature issues Research and interviews with professionals and clients Collaboration with a publishing house (Gothia förlag) Adapted to our target group Reading guidances

42. Challenges

43. Challenges • Lack of interest among the Swedish social work academia to support an EBP, and train the future generations of social workers accordingly • Lack of (inter)national agreement on how to grade evidence • Questions on the transportability of evidence-based interventions between countries • Lack of reliable and valid measures of quality of services at a local level (in order to motivate change) • Lack of an infrastructure for diffusion of innovations, and knowledge transfer

44. AIDA – acronym used in marketing A Attention I Interest D Desire A Action