USER LED EVALUATION

1. USER LED EVALUATION By Dagfinn Bjørgen Manager Heidi Westerlund deputy Manager of Competence and Resource Center for Consumer Involvement in Mental Health Service Development.

2. Background • The experience we base this presentation on is 45 projects using the “User Interview User” model (BSB) evalaution method in mental health care services and community based services in Norway. • The largest evaluation was of Community Mental Health services in Oslo (Bjørgen,Westerlund and Johansen 2008). • The method has largely been used to evaluate in and out-patient and services within the community health and social services sector. The projects are common in that they all address quality of services during mental health problems.

3. Founding ideas for BSB. • Respect and acknowledgement of lived experience with services and mental health problems. • Loyalty to the Service-User perspective. Experts by experience. • Empowerment for Service Users and service providers. • Deliberative democratic process in service delivery and development.

4. Documenting Experiences. Reports Documenting Experiences. Reports Documentation Experiences. Reports Documenting Experiences. Reports Users Interviews Users Service user Perspective Through equal terms of communication Ideal speech situation Service user Perspective Through equal terms of communication Ideal speech situation Service user Perspective Through equal terms of communication Ideal speech situation Service user Perspective Through equal terms of communication Ideal speech situation Service user Perspective Through equal terms of communication Ideal speech situation Service user Perspective Through equal terms of communication Ideal speech situation Service user Perspective Through equal terms of communication Ideal speech situation Dialogue and learning Organizational development Organizational development Dialogue and learning Organizational development Dialogue and learning Organizational developement Dialogue and learning Organizational development Dialogue and learning Organizational development Dialogue and learning Organizational developement Dialogue and learning Shared understanding of quality of services and the need of change Shared understanding quality of services Shared understanding quality of services Shared understanding quality of services Shared understanding quality of services Feedback Motivational factors Feedback Motivational factors Feedback Motivational factors Feedback Motivational factors Feedback Motivational factors Feedback Motivational factors Feedback Motivational factors Feedback Motivational factors Feedback Motivational factors Feedback Motivational factors

5. Analyzing process in BSB Report Displays Dialogue conference Process rapport Systematic review in matrices Review of transcribed interviews Interview

6. Objectives • Organizational learning for to decision makers, health professionals, and clients. Focuses on how services is experienced – what is good practice, what is not intended consequences of service delviery, how can we do things differently to improve quality? • To encourage directors, “street level” beaurocrats as well as clients and service users to meet with each other in a way that is constructive and rational. • The conferences are a deliberative arena to: • obtain more credible evaluative conclusions about what how mental health serves function • where and how improvement should be made • how to go about getting them accomplished. do the process after the evaluation. • It is an arena where lower voices can be heard, strengthened, and empowered.

7. Documentation: 1 step: Planning prowith the service unit. 2 step: Interviews with service-users/patients 3 step: Reporting prelimniary findings – process document. 4 Step: Dialouge conference with stakholders 5 step: Final report, based on disussions

8. Public management and BSB • Public services (including mental health care services) are different from other forms of consumer product services. • “The client and citizens act as co-producers and actively influence the service and this implies that the ‘product’ is a negotiated outcome and less controlled by the provider”. Tineke A. Abma and Mirko Noordegraaf ManagementPublic Managers Amidst Ambiguity: Towards a Typology of Evaluative Practices in Public. Evaluation 2003;9 • BSB Dialogue conference deals with “wild problems” , in other words its complex problems. • BSB is emphasizes allowing clients to use their voices in stronger and more constructive way.

9. Create a base for constructive dialog and reflection Make an arena for commitment to follow up the findings Validate the findings. Gather supplementary information for the evaluation. Contribute to new understanding of how services are working. Reduce prejudice between service users and providers. What are the benefits of dialogue conferences?

10. Dialogue conference and empowerment • The process rapport is written in a non-academic form. The participant will see their statements written verbatim to ensure authenticity. • The service users are given the opportunity to participate as rational individuals with valid reflections about their experience of services. • Their voice are first to be focused on • Through the conference process experiences of the users are validated as trustworthy and service personnel and administrative leaders actually discuss the findings.

11. To avoid reinforcing or putting too much focus on conflicts. • Help people to see their problem in a broader perspective and to introduce theory wich can explain phenomenon • Its important to have flexibility in a way that doesn’t restrict the dialog process. Meta discussion about the services - function and accomplishments

12. BSB Evaluation of an early intervention team for families with mental Health problems and substance abuse. • early intervention and familliy planning for pregnant young mothers, in families in risk of not managing the parenthood, from 6 weeks of pregnancy to School age for the child.

13. The Evaluation gave several results; • The organization of the team under the hospital gave less stigma than if they had it in their municipalaty.. • The team team did not reach out to the fathers althoug they had the whole familie as a objective to reach. • Acknowledgment of added nervous reaction during pregancy and birth. Empowering families as good mothers and fathers • The results used as reference in rapport no. 30 (2011-2012) to Stortinget, title; « See Me»

14. BSB evaluation of the National escalation plan for mental health services of Oslo captial.

15. Findings… Relation between service user and providers are the most important factor for experienced quality. Housing, employment and poverty are the main challenges for quality of life. Importance of exercise and daytime activities have been emphasized by the service users.

16. Thank you for your attention . Email dagfinn.bjorgen@kbtmidt.no heidi.westerlund@kbtmidt.no Site www.brukererfaring.no www.kbtmidt.no