Outcome measurement

1. Train the Trainer Program Day 2 - Adult Outcome measurement

2. Overview of clinical measures • Clinician-rated: • Health of the Nation Outcome Scales (HoNOS) • Life Skills profile (LSP-16) • Focus of Care (FOC) • Consumer-rated: • Behaviour and Symptom Identification Scale (BASIS-32)

3. Overview of data collection protocol

4. Background • Health of the Nation (UK) 1992 • The Measurement of Consumer Outcomes in Mental Health (Andrews 1994) • Various implementations including HoNOS • Victorian HoNOS field trial, 1996 • Mental Health Classification and Service Costs Project (MH-CASC), 1996 • Field testing of selected consumer outcome measures, Stedman et al., 1997 • Introduction of routine outcome measurement in adult public mental health services, Victoria, 2000

5. The HoNOS • Key measure of severity • Brief; five minutes to rate • Acceptable and useful to clinicians • Specifically broad spectrum • Satisfactory inter-rater reliability • Change in scores correlate with independent clinical ratings of change • Training required

6. The HoNOSThe 12 scales (adult version) 1. Overactivity, aggression 2. Non-accidental self-injury 3. Problem drinking or drug-taking 4. Cognitive problems 5. Physical illness or disability problems 6. Problems associated with hallucinations or delusions 7. Problems with depressed mood 8. Other mental and behavioural problem 9. Problems with relationships 10. Problems with activities of daily living 11. Problems with living conditions 12. Problems with occupation and activities

7. The 4 HoNOS subscales BEHAVIOUR 1. Overactivity, aggression 2. Non-accidental self-injury 3. Problem drinking or drug-taking IMPAIRMENT 4. Cognitive problems 5. Physical illness or disability problems 6. Hallucinations/delusions 7. Problems with depressed mood 8. Other mental and behavioural problem SYMPTOMS 9. Problems with relationships 10. Problems with activities of daily living 11. Problems with living conditions 12. Problems with occupation and activities SOCIAL

8. The HoNOSExample of item structure Scale 1. Overactive, aggressive, disruptive or agitated behaviour Include such behaviour due to any cause, for example, drugs, alcohol, dementia, psychosis, depression. 0 No problems of this kind during the period rated 1 Occasional irritability, quarrels, restlessness, but generally calm 2 Occasional aggressive gestures, pushing or pestering others; threats or verbal aggression; lesser damage to property (broken cup, window); marked overactivity or agitation 3 Physically aggressive to others or animals (short of rating 4); persistently threatening manner, more serious overactivity or destruction of property 4 At least one serious physical attack on others or on animals; destructive of property (for example, fire-setting); persistent serious intimidation or obscene behaviour

9. HoNOS scoring • Each item is scored: 0 = no problem 1 = sub-clinical problem 2 = mild problem 3 = moderate problem 4 = severe problem 9 = not known • Users are provided with a set of criteria for each rating level

10. Implications of rating a 9 on HoNOS • Only use 9 when you are genuinely unable to make a definitive rating • Wellbeing Reporting Tool will treat all 9 as zero for the purpose of calculating subscale and total scores

11. Requirements for HoNOS rating • Be a mental health professional • Therefore be able to exercise clinical judgment • Have been trained in the use of the HoNOS • Have available information from various sources • So, need to know the consumer’s condition over the past two weeks

12. Sources of information • The measures are not clinical interviews. Information should be gathered from: • The consumer • Direct observation • Information in the medical record • information provided by other staff • information provided by family and friends • information provided by other agencies including general practitioner, housing, police and ambulance staff

13. HoNOS rating rules • Rate each item in order from 1 to 12 • Do not include information rated in an earlier item, minimal item overlap • Rate the most severe problem that has occurred over the previous two weeks • Consider both the impact on behaviour and/or the degree of distress it causes • When in doubt read the glossary

14. From an alternative perspective

15. Important variations in rating guides

16. Practice rating • Use one of the training vignettes for adult mental health services here

17. Offering the consumer self-report measure General rule: always offer the consumer self-report measure. • Complements the clinician rated measures • Completion by the consumer is always voluntary • Consumer self-rating information is subject to the same rules of confidentiality and privacy as all the other information held in their file • Explain why is it important that I complete this questionnaire • Non-completion will not have any detrimental effect on treatment • Encourage them to answer all the questions but accept partial completions • Explain who is going to use the information • Explain what the information is going to be used for

18. When not to offer the self–report measure • Temporary contra-indication Cognitive Distressed Behaviourally disturbed • General exclusion As a result of an organic mental disorder or a developmental disability • Cultural or language issues may make the self-report measure difficult to offer

19. The do’s and don'ts – offering the questionnaire • Request and encourage consumer to fill out the questionnaire • Answer their questions about why they are being asked to complete the questionnaire • Let consumers know that you will be there to assist them if needed. • Encourage consumers to answer all the questions • Do not force or instruct consumers to fill out the questionnaire • Do not tell the consumer that treatment is dependent on their filling out the questionnaire. • Do not minimise the importance of filling out the questionnaire. • Do not accept an incomplete questionnaire without first encouraging the consumer to fill out unanswered questions • If a consumer tells you they cannot do it - accept that they are telling the truth. Do not assume the consumer can do it and just doesn’t want to.

20. The do’s and don'ts – helping with questions • You may read and repeat a question, placing emphasis to help the consumer understand the question • Provide a definition of a single word a person is unfamiliar with • Tell consumers to answer a question based on what they think the question means • Stress that there is no right or wrong answer • Do not paraphrase, rephrase, interpret or explain a question • Do not answer the question for the consumer • Do not tell the consumer how you feel they should answer • Do not allow other people to help the consumer fill out the questionnaire • Do not tell the consumer to go home and get their family to help them

21. Making it worthwhile • Consumer participation means more than just asking consumers to complete self-report questionnaires. • It means: • involving consumers in ongoing mental health education • having a genuine interest in consumers’ views • incorporating consumers’ perceptions into individual care plans • sharing the knowledge gained from the clinician ratings with the consumer • following up discrepancies between consumer and clinician ratings • integrating the results into individual care plans

22. Diagnosis • Principal diagnosis • The principal diagnosis is the diagnosis established after study to be chiefly responsible for occasioning the patient or client’s care during the preceding period of care

23. Mental health legal status • Was the person treated on an involuntary basis (under the relevant mental health legislation) at some point during the preceding period of care

24. Rating the Focus of Care • Assesses the primary goal of care • Based on concept of ‘phase of illness’ in people with psychiatric disorders • Rate main focus of care over whole episode - is therefore a retrospective measure • Single rating item to identify the main ‘focus of care’ • Measures categories not rankings

26. Making sense of outcome measures

27. The Life Skills Profile (LSP-16)

28. The LSP-16 • Key measure of function and disability in people with mental illness • Complements the problem-based HoNOS • Developed by a NSW team in 1989 • Original scale of 39 items; reduced to 16 • Brief; five minutes to rate • Good inter-rater reliability • Sensitive to change

29. The LSP-16 continued • A non-technical instrument - originally designed to require little or no training • Focuses on the person's general functioning - how the person functions in terms of their social relationships, ability to do day-to-day tasks, etc.

30. The LSPExample of item structure 1) Does this person generally have any difficulty with initiating and responding to conversation? 0 No difficulty with conversation 1 Slight difficulty with conversation 2 Moderate difficulty with conversation 3 Extreme difficulty with conversation 2)Does this person generally withdraw from social contact? 0 Does not withdraw at all 1 Withdraws slightly 2 Withdraws moderately 3 Withdraws totally or near totally

31. LSP rating rules • Use all available information, from any source • The LSP-16 is not a clinical interview • Rate the general level of functioning over the last three months

32. The LSP-16 subscales • Withdrawal • Antisocial behaviour • Self-care • Compliance

33. Specific LSP items to note ... • Item 6 - neglect their physical health? Not receiving treatment for a health condition, lead a generally healthy lifestyle • Item 10 - behaviour related to medication adherence • Item 11 - attitude towards medication • Item 12 - cooperate with health services • Item 15 - deliberate intention

34. The present Train-the-trainer course Rater training Implementation of routine outcome measurement The future Induction training Refresher training Delivering training

35. How trainers will work Pleading Worrying and taking the ‘heat’ Thinking Inducing!

36. Tips for trainers • Local ownership of implementation. • From the moment of orientation to the organisation, work towards staff viewing the use of outcomes and casemix measures as part of the culture. • Monitor data quality (in time…) this will flag training misconceptions. • Flexible training, tailored to local needs using multiple modes of delivery, while maintaining consistency in quality and content. • Establishment of a self-sustaining training system.

37. Words of advice • Jump in • Have a clear sense of mission • Build a team • Maintain a flat organisational structure • Pick people with relevant skills and high energy levels • Toss out the rule book • Action feedback model • Flexible priorities • Maintain right to change • Ask for volunteers • Find good team leaders and stay out of their way • Give team members what they ask for in resources • Concentrate dispersed knowledge - issues log