How s Things

1. 1 How�s Things? Quantifying Outcomes in Behavioural Intervention Brian Mc Donald, R.N.I.D.; P.Dip.B.C.Psych. Behaviour Support Specialist 23/5/�07 

2. 2 Rationale National Council for the Professional Development of Nursing & Midwifery � Measurement of Nursing & Midwifery Interventions: Guidance Pack � No Behavioural Measures Included! Interventions should focus on real, tangible, quantifiable, qualitative, applied results for the individual in the 1st instance and the service provider in the 2nd instance Focus should be on what has improved for the person This should be considered in terms of quality of life improvements that are significant to that individual

3. 3 Outcome Measures �OUTCOMES� implies an end to intervention �FUTURE DIRECTIONS� (or similar) might be a better term Outcome Measure is a common management term that is a measure of the result of a system relative to the aim An outcome measure is used to measure the success of a system (e.g. an outcome measure could be percentage reduction in the frequency of aggressive incidents in residential service following a period of intervention)

4. 4 Related Terms Output Measures A measure of activity (e.g. the number of hours individual case-work completed by the behavioural person! A focus on output measures does not inform success. Care should be taken to define good outcome measures to use in determining the success of interventions Process Measure Measures of the performance of a process. Provide real time feedback that can be acted on quickly, when a process first provides data that something is amiss Activity Measure A measure of the level of resources committed to a process Operational Definition Example: Resources allocated to an individual to address severe behavioural difficulties which are operationally defined in a residential unit would represent an activity measure. An in-process measure could be adherence to the behavioural plan. An outcome measure could be the reduction of Episodic Severity by 25%

5. 5 Operational Definitions Description of term (e.g. aggression) as applied to a specific situation When collecting data it is vital to define terms very clearly in order to assure all those collecting and analysing the data have the same understanding Therefore, operational definitions should be very precise and be written to avoid possible variation in interpretations More specifically an operational definition often specifically states how to measure the item being defined Example: How would you count the number of aggressive incidents in a day? The number of physical contacts made, the number of threats, the number of missed attempts at hitting

6. 6 How�s Things??? Generic global statements made by staff teams Anecdotal subjective statements that often have no backing or evidence Should not be discounted as they can be indicative of the mood/morale of the team May indicate an environment conducive or detrimental to behaviour change and therefore a starting point for the work to be done

7. 7 Statistics & Reports Statistics, Means, Standard Deviations, Coefficients, etc. have their rightful place in clinical journals, research, etc. but not at an IPP, PCP meeting, Personal Outcome Reviews, Staff Meetings, etc. Clinical reports must be translated into information that is usable by service-users and their families and carers Scientific & journal quality commentary on behavioural measurement only serves to distance those that are end-users of the service For example: The FI 300-s schedule was then implemented, and hand mouthing increased again to near baseline levels (M = 26%). The reintroduction of time-out (FR-1) again produced a reduction in hand mouthing (M = 4.4%), and responding remained low as the time-out schedule was gradually thinned to the FI 300-s schedule. Across the 57 sessions of the thinning procedure, hand mouthing occurred in an average of 3.4% of the intervals (Lerman, Iwata, Shore & DeLeon, 1997)

8. 8 What do we need to know? Look at this from a number of possible perspectives: Service-user: Ask the individual or, alternatively, consider from their perspective � HOW IS MY LIFE BETTER? Staff team: Is the work environment safer? Service (Day/Res./Other): Is the service of a higher quality? Service provider: Are we getting value for money � 1:1 staffing, extra night staff, assistive equipment, etc. Clinician: Were the interventions employed effective?

9. 9 Formulation Important to constantly evaluate and re-evaluate: Assessment Best approaches to employ Targets for intervention (staff team, environments, behaviours, individual, etc.) It is ethically questionable to address an individuals behaviour without 1st looking at the environment, programmatic and service provision elements

10. 10 Evidence Based Practice Evidence based practice: Most, hopefully all, settings engage in practice that has a sound, contemporary, valid & scientific basis Practice based evidence: The sharing of ideas, innovations and practice initiatives through presentations at seminars, journal submissions, poster presentations, etc. with a view to raising the quality of work of all

11. 11 Why Quantify Outcomes? Quantification of effect = Qualification of effort (service-users & supports)

12. 12 Who Should Quantify Outcomes? Simply put � those that are implementing, or directing the implementation of, interventions Measurement needs to be integrated into working practices Be conscious of pressures already on frontline staff

13. 13 Practical Considerations The goal of records is to obtain accurate & useful information that is gathered in as efficient a manner as possible Design of records should be respectful of the target user Facilitate the user with tick-lists, individualised records, headings, accurate direction, training and supervision/support

14. 14 Interventions Examples of commonly employed models: Applied Behaviour Analysis Positive Behaviour Support Multi-element Behaviour Support Cognitive Behaviour Therapy

15. 15 Applied Behaviour Analysis A discipline devoted to the understanding and improvement of human behaviour (Cooper, Heron & Heward, 1987) Focuses on objectively defined, observable behaviours of social significance Seeks to improve the behaviour under study whilst demonstrating a reliable relationship between the procedures employed and the behavioural improvement Uses the methods of science: Description Quantification Analysis

16. 16 Positive Behaviour Support A bringing together ideas inherent in Applied Behaviour Analysis (a toolkit) and Social Role Valorisation (a values base) P.B.S. has a lifespan perspective that involves long-term planning for each person rather than short-term crisis management Behavioural challenges should be responded to in the least restrictive environment Individuals should be maintained in an environment that allows for maximum contact with the broader community - inclusion & participation

17. 17 Positive Behaviour Support A holistic approach to helping people to engage in adaptive & socially desirable skills whilst overcoming patterns of destructive and stigmatising behaviours Interventions must be based on a full behaviour assessment of the individual and the physical/social environment in which they live and work Interventions must be multi-element & open to review/evaluation on a regular & ongoing basis Staff working with individuals with challenging behaviour must receive the necessary training & support A range of service options should be available based on an assessment of the individual�s support needs Intervention should be of a non-aversive/non-intrusive nature & the least restrictive alternative in the least restrictive environment should be adhered to as a guiding principle Some challenges: http://www.aase.edu.au/2004_Conf_papers/S.Vanderaa.pdf

18. 18 Multi-element Behaviour Support Ecological/Environmental restructuring Direct treatments Reactive/management strategies Positive Programming Mediator interventions Both proactive and reactive elements to the Behaviour Support Plan www.callaninstitute.org

19. 19 Multi-element Behaviour Support Multi-element Behaviour Support plans are non-aversive They involve changes to the environment to achieve a better fit with the needs & characteristics of the person They teach the social & communication skills necessary to overcome challenging behaviours Behaviour Support Plans also include strategies for generating rapid short- term change for behaviours that are difficult or potentially dangerous There are effective reactive strategies, which are used to manage crises without compromising the safety or the dignity of all concerned Finally, Behaviour Support Plans address the need to establish supervisory systems to sustain the achievements gained

20. 20 Cognitive Behaviour Therapy?? Willner, Jones, Tamsy & Green (2002) conducted a study in which 14 individuals with intellectual disabilities were referred for anger management and randomly assigned to a treatment group (group-based anger treatment) or a waiting-list control group (no treatment) They found the approach was effective in decreasing anger Their discussion directs future researchers to investigate whether cognitive behavioural treatment is more effective than purely behavioural interventions

21. 21 Cognitive Behaviour Therapy?? Hagiliassis, Gulbenkoglu, DiMarco, Young & Hudson (2005) describe the evaluation of a group program designed specifically to meet the anger management needs of a group of individuals with various levels of intellectual disability and/or complex communication needs They reported that clients from the intervention group had made significant improvements in self-reported anger levels compared with clients from a comparison wait-list group They also note an absence of measured improvements in quality of life �Very few intervention plans actually teach people with learning difficulties socially acceptable ways of expressing anger or frustration and challenging behaviour may be the one way in which people in such circumstances can exert any control over the way in which they live� (Blunden & Allen, 1987).

22. 22 Measuring Outcomes Baseline - start at the beginning Within treatment measures (process measures) Post-treatment � is there such a thing in intellectual disabilities? (positive behaviour support ethos is that intervention is ongoing)

23. 23 Basic Behavioural Measures Frequency � how often it occurs Duration � how long it occurs for Rate � how many times it occurs over a period of time We may be looking to increase (behavioural deficits) or decrease (behavioural excesses) results based on these measures

24. 24 Episodic Severity Defined: A measure of the intensity or gravity of a behavioural incident Sample Objective Measures: Rate Duration Scaled severity ratings of stress/harm/injury Costs of property repair or replacement/staff replacement, etc.

25. 25 Episodic Severity Quantifiable reductions in behavioural measures do not always imply overall improvement in the situation: 1 tantrum a week lasting 30 mins (injuries/property damage) (low rate/high E.S.) vs. 6 tantrums a week each lasting 5 minutes (screaming) (high rate/low E.S.) Episodic severity gives a global yet case specific picture This allows for increased criteria for success of approaches Less is not necessarily betterLess is not necessarily better

26. 26 Case Study Refer to August 2005 as example of only 5 incidents but highest stress (8.5) as opposed to February 2005 when high incidents but moderate/acceptable level of stress as opposed to August 2006 with high incidents high stress and high biting (most severely aggressive behaviour)Refer to August 2005 as example of only 5 incidents but highest stress (8.5) as opposed to February 2005 when high incidents but moderate/acceptable level of stress as opposed to August 2006 with high incidents high stress and high biting (most severely aggressive behaviour)

27. 27 Instruments & Tools Some examples: Periodic Service Review HoNOS � LD Outcome Rating Scale Self-reporting Basic behavioural measures Episodic severity Graphs

28. 28 Periodic Service Review Manager�s negatives that contribute to poor outcomes: Not spending enough time in the community Not providing enough systematic instruction to the individuals receiving services Not achieving satisfactory outcomes for individuals being served Not collecting data Not being consistent/accurate in data collection Not following agreed-upon procedures Not following the individualised service/education plan Not providing sufficient functional age-appropriate activities Not respecting the individuals being served Not following schedules Not interacting positively with individuals being served Not doing any thing at all

29. 29 Periodic Service Review Reasons offered: High staff turnover Poor staff motivation Poor wages Lack of staff training Lack of staff skills Lack of agreement in philosophy or effectiveness of strategies employed Too little time in the day Dislike of individuals being served Lazy staff Lack of intelligent staff Not enough staff or other resources

30. 30 Periodic Service Review What is it? A management system that monitors & directs improvement across a broad range of staff & agency indicators Used to assess staff quality & consistency Used to evaluate the implementation of behavioural recommendations Can be customised to be employed in residential, educational, day and other services across a number of populations http://www.epsr.com/(bb30li45utlzewzxr2gslt2q)/visitor/home.aspx

31. 31 Periodic Service Review Individual Performance Standards of Behaviour Specialist: Develop behaviour assessment & intervention plans Submit plans on a timely basis Write quarterly progress reports Write termination reports Submit termination reports on a timely basis Provide service as authorised Submit attendance sheets Attend weekly unit meetings Attend 1:1 meetings with supervisor Train staff and key social agents Maintain case record Etc.

32. 32 HoNOS � LD Health of the Nation Outcome Scales HoNOS are a well researched and tested selection of clinical outcome measures developed by the UK Royal College of Psychiatrists� Research Unit (CRU) Measures health and social functioning across 12 items that are individually graded for severity on a 5 point scale related to behaviour, impairment, symptoms and social functioning Primary aim is to measure change in an individual over 2 or more points in time as a measurement of outcome for therapeutic interventions Measures change in the levels of problems that a person has had and is not meant to be a comprehensive description of the individual Also measures no change or deterioration

33. 33 HoNOS � LD (Sample Q�s) Please indicate if the following behaviour problems have been present over the past 4 weeks based on the following rating: 0 � No problem during the period rated 1 � Mild Problem 2 � Moderate Problem 3 � Severe Problem 4 � Very Severe Problem 9 � Unknown Behavioural Problems (External Aggression Towards Others): No behavioural problems directed towards others Irritable, quarrelsome, occasional verbal abuse Frequent verbal abuse, verbal threats, occasional aggressive gestures, pushing or pestering (harassment) Risk, or occurrence of, physical aggression resulting in injury to others requiring simple first aid or close monitoring for prevention Risk, or occurrence of, physical aggression producing injury to others serious enough to need casualty treatment and requiring constant supervision or physical intervention for prevention (e.g. restraint, medication or removal) Behavioural Problems (Aggression Directed Inwards): No self injurious behaviour Occasional self injurious behaviour (e.g. face tapping) Frequent self injurious behaviour not resulting in tissue damage (e.g. redness, soreness, wrist scratching) Risk, or occurrence of, self injurious behaviour resulting in reversible tissue damage and no loss of function (e.g. cuts, bruises, hair loss) Risk, or occurrence of, self injurious behaviour resulting in irreversible tissue damage and permanent loss of function (e.g. limb contractures, impairment of vision, permanent facial scarring)

34. 34 HoNOS � LD Benefits: Clinician friendly instrument that can be used in routine practice irrespective of professional background Global nature of the instrument lands itself to a holistic approach Limitations: Nature of intellectual disabilities is that there is a greater reliance on informant-based data leading to greater opportunities for error Broad nature may be less well suited to detecting the subtle changes in those with chronic conditions

35. 35 HoNOS � LD Supply of the pack is available from:- BILD Publications Book Source32 Sinlas StreetGlasgowG22 5DU Tel No: 0141 5581366Email: Derek@booksource.net

36. 36 Outcome Rating Scale Looking back over the last month, mark the line to indicate how you think the person may be feeling about how well he or she has been doing in the following areas of his or her life, where marks to the left represent low levels and marks to the right indicate high levels Individually: (Personal well-being) I-----------------------------------------------------------------------------I Interpersonally: (Family, close relationships) I-----------------------------------------------------------------------------I Socially: (Work, School, Friendships) I-----------------------------------------------------------------------------I Overall: (General sense of well-being) I-----------------------------------------------------------------------------I What needs to happen to improve these ratings? (Acknowledgement � Brian Mc Clean, Brothers of Charity, Roscommon)

37. 37 Self-reporting Self-report can be a reliable & valid means by which to collect information from people with intellectual disabilities People with intellectual disabilities can deal with abstract concepts providing their knowledge & understanding are assessed & the therapist is prepared to take on a didactic role Self-regulation (therefore generalisation & maintenance of therapeutic gain) can be achieved but only if the environment supports & respects individual rights & opportunities to practice self-determination are present (Kroese, Dagnan & Loumidis, 1997)

38. 38 Modified Anger Log John�s Anger Diary Date & Time? Where? What Happened? Fighting Shouted At Arguing Made Fun Of  Who? Friend Relative Stranger Staff How Angry Were You? Not At All A Bit Angry Very Angry What Did You Do? Shouted Back Ran Off Hit Talked About It Walked Away 

39. 39 Stress/Anger Thermometer

40. 40 Note re Graphs When measures of behaviour are plotted on graphs immediately after an observational period ongoing access to a complete record of the behaviour is provided This allows for constant evaluation and re-formulation Ensures approaches and clinical decisions are responsive to performance Graphs are also a highly effective feedback mechanism to service-users, staff teams, families and management

41. 41 Time-out Room Use Not indicative of duration in time-out which varied considerably - part of the problemNot indicative of duration in time-out which varied considerably - part of the problem

42. 42 Further Research Involves small focus groups of parents and professionals with first-hand knowledge about how autistic behaviours impact on day-to-day functioning for the individual and caregivers The aim is to learn which behaviours pose difficulties or are a concern so that these can be appropriately captured in the measure Also focuses on gathering the list of behaviours generated and narrowing it to the most relevant The measure will then be tested to make sure that it is user friendly, captures behaviours accurately & is statistically sound Future research will be done to assess how well it measures change in behaviour http://www.togetherforautism.ca/client/aso/TFA.nsf/object/Kagan-Kushnir+Developing+New+Behavioural+Measure/$file/Kagan-Kushnir+Developing+New+Behavioural+Measure.pdf

43. 43 References Blunden, R. & Allen, D. (eds) (1987) Facing the Challenge: An Ordinary Life for People with Learning Difficulties and Challenging Behaviour. London: King�s Fund Project Paper, Number 74 Cooper, J.O., Heron, T.E. & Heward, W.L. (1987). Applied Behaviour Analysis. New Jersey: Prentice Hall Hagiliassis, N., Gulbenkoglu, H., DiMarco, M., Young, S. & Hudson, A. (2005). The Anger Management Project: A group intervention for anger in people with physical and multiple disabilities. Journal of Intellectual & Developmental Disability, 30, 86�96 Kroese, B.S., Dagnan, D. & Loumidis, K. (1997). Cognitive Behaviour Therapy for People with Learning Disabilities. London: Routledge LaVigna, G.W., Willis, T.J., Shaull, J.F., Abedi, M. & Sweitzer, M. (1994). The Periodic Service Review � A Total Quality Assurance System for Human Services & Education. Maryland: Brookes Lerman, D.C., Iwata, B.A., Shore, B.A. & DeLeon, I.G. (1997). Effects of Intermittent Punishment on Self-injurious Behaviour: an Evaluation of Schedule Thinning. Journal of Applied Behaviour Analysis, 30, 187�201 Roy, A., Matthews, H., Clifford, P., Fowler, V. & Martin, D.M. (2002). Health of the Nation Outcome Scales for People with Learning Disabilities (HoNOS-LD). British Journal of Psychiatry, 180, 61-66 Taylor, J.L. & Novaco, R.W. (2005). Anger Treatment for People with Developmental Disabilities. New Jersey: Wiley & Sons Ltd. Willner, P., Jones, J., Tamsy, R. & Green, G. (2002). A Randomised Controlled Trial of the Efficacy of a Cognitive-Behavioural Anger Management Group for Clients with Learning Disabilities. Journal of Applied Research in Intellectual Disabilities, 15, 224�235

44. 44 Contact brian.mcdonald@galwayca.ie behavioursupport@gmail.com