Consumer -Responsive Reporting of Psychoeducational Assessments

1. Consumer-Responsive Reporting of Psychoeducational Assessments NASP Convention Mini-Skills Workshop February 17, 2015 Bruce Ecker, PhD Bob Lichtenstein, PhD Bruce Ecker Ph.D. and relevance (“So what?”)

2. Workshop Objectives • Participants will understand and appreciate the purpose, rationale, and principles of a consumer-response approach • Participants will be able to identify how a consumer-responsive approach differs from customary practice • Participants will serious consider adopting this approach, or parts thereof, in ways that increase the value of their clinical work to children, families and educators

3. TYPE 1 TYPE 2 Participants Poll Your observations about typical assessment reports produced by school psychologists… TYPE Written in language that a psychologist will understand Test results are the major focus (primary input, longest section) Tables with test data (e.g, scores, percentiles) included in report Test findings organized by tests administered Interpretations primarily based on test results (e.g., section entitled “Test Results and Interpretations”) Written in language that a parent or teacher will understand Tests results receive equal weight as background info & observations Test data appear in an appendix following the report narrative Test findings organized by theme/area of functioning Interpretations based on convergent data—history, interviews, observations, tests (e.g., section entitled Clinical Impressions”) 3

4. Presentation Topics Underlying purpose, rationale, and principles Conceptualizing from the start Report organization and structure Language and style Focus and relevance Skill-building exercises Implications for due process Oral reporting Practical issues and challenges Questions and comments 4

5. Rationale for a Consumer-Centered Approach Every assessment is an opportunity. The child is at the center of the assessment enterprise. The consumers of child psychological assessment include the child, his/her parents, his/her teachers, school administrators, other helpers. Every school-based assessment must have educational value.

6. Describe: current functioning, including strengths and weaknesses, Explain: functioning, origins, prognosis, and how these strengths and weaknesses influence functioning in the real world. Plan: how to improve functioning via recommendations that are individualized, specific, powerful and feasible. Psychological Reports: The Purpose

7. The Purpose: #2 “…the most useful and ethical psychological reports take into consideration the needs of the consumer (Cates, 1999). This means that the format of the report, the choice of language, the inclusion of test data, and the use of feedback reports should all be decided aftercareful consideration of the consumer’s needs. Reports should also take into consideration the client’s cultural context, ensuring that the report is structured around the client’s culture. They should also provide a balance between positive and pathological aspects in the client’s psychological functioning” (Groth-Marnat, 2006, p. 80).

8. Relevance: Identify and meet consumer needs. Response: Focus on referral question(s). Individualize concrete recommendations. Provide individualized feedback. Relationships: Build lasting relationships with consumers. Results: Continually assess the extent to which consumer needs are met and adjust accordingly. Brenner (2003) and Consumer-Focused Assessment: 4 Rs

9. Applicable Professional Standards NASP Principles for Professional Ethics (2010) Standard II.3.8. School psychologists adequately interpret findings and present results in clear, understandable terms so that the recipient can make informed choices. NASP Model for Comprehensive and Integrated School Psychological Services (2010) Consultation and Collaboration: School psychologists effectively communicate information for diverse audiences, such as parents, teachers and other school personnel, policy makers, community leaders, and others. • NASP Principles for Professional Ethics (2010)

10. Credit Where Credit is Due 1500 children, adolescents and their families My colleague, Bob Lichtenstein Eliot Brenner (2003) Gary Groth-Marnat (2009) Virginia Harvey (2006) Seligman & Darling (2007) Snyder (2006)

11. Our Mantra Client-centered… Consumer-responsive… Referral-driven… Strengths-based

12. Guiding Principle 1: Purpose-Driven Assessment is purpose-driven • Referral questions drive the assessment process • Questions given to us • Questions we develop ourselves • Assessment activities are organized around hypotheses that are derived from the referral questions.

13. Reasons for Referral and Hypotheses

14. Guiding Principle 2: Relevance Procedures and results reporting must be relevant • To the referral question(s) • To the consumers’ lives • To the needs of the referral agent • Select assessment procedures accordingly • Select findings to report accordingly • Apply the “so what” test

15. Assessment is person-focused We describe the person…and not the test results. We relate with those who are important to the assessment process…child/adolescent, teachers, family, administrators Guiding Principle 3: Person-Focused

16. Guiding Principle 4: Strengths-Based • Emphasize strengths without minimizing challenges. • In the child • In the school • In the family • In the community • Consider all available resources • In the child • In the school • In the family (formal and informal) • In the community

17. Guiding Principle 5: The Consumer Positive engagement with the consumer • From initial contact through testing, feedback and follow-up • The report is secondary to the relationship with the consumer. • Implication: shorter reports…with more extensive preparation for meetings and conferences. • This is what makes your work legally defensible. This is your best strategy for minimizing adversarial activity.

18. An understanding of their child’s problems (diagnosis/formulation/ prognosis) Information about intervention, and An offer of treatment Hilton, Turner, Krebs, Volz, & Heyman (2012) Parents’ Desires from Assessment: Content

19. To be put at ease To feel listened to, understood, and reassured, For the clinician to communicate directly with an empathic and authoritative tone To be treated with optimism To have ample time for discussion Hasnat & Graves,2000;Hilton et al., 2012 Parents’ Desires from Assessment: Process

20. Guiding Principle #6: Understandable Reports To be understandable, reports should: • Be cohesively organized around themes of functioning • Focus on the child and not the test result • Bring relevant information together in an integrative Conclusions/Impressions section • Conclude with recommendations that are individualized, specific, powerful, and feasible. • Be of reasonable length • Be written in understandable language

21. Guiding Principle #7: Oral Feedback Oral feedback is as important as the written report “At no time are partnership practices of sensitivity, availability, respect, and honesty more important than when you share evaluation information (Turnbull et al., 2006, p. 245).”

22. Conceptualizing from the Start • Collaboration throughout • Early contact • Client-centered and consumer-responsive • With whom? • Background information • From whom and about what? • Selection of measures • Referral Driven • Hypothesis-Testing • Assessment is an invasive process

23. Report Organization and Structure A consumer-response approach has implications for: • Overall organization and outline • Presentation of test results • Key components: • Clinical impressions • Data Summary

24. Report Organization and Structure Standard Report Outline • Identifying Information • Reason for Referral • Assessment Procedures • Assessment Findings • Background Information • Interviews • Observations • Test Results • Clinical Impressions • Summary and Recommendations • Data Summary (appendix)

25. Report Organization and Structure Test Results section • Comparable status with other sources • Findings only at this point; save interpretations for Clinical Impressions • Theme-based organization • Selectivity (the “So what?” principle) • Scores (numbers) vs. descriptors (words) • Data Summary

26. Report Organization and Structure Clinical Impressions • Interpretive; integrates findings from all sources • Addresses referral questions & related hypotheses • References data • Journalistic style – most important points first

27. Report Organization and Structure Summary & Recommendations • Brief summary of key points (from clinical impressions) • No new information • Sets up the recommendations (because of A, do B) • Recommendations: • Always! • Informs without pre-empting team’s responsibility • Describe what child needs, rather than prescribe a specific service or placement

28. Report Organization and Structure

29. Proposed Test Score Range Descriptors

30. Language and Style • The language used in psychoeducational and psychological test reports is too complex for lay people (and many psychologists) to comprehend. • Jargon-laden. • Multi-syllable words and compound sentences. • Readability per Flesch Index (0-100), low being more difficult; 0-30 is extremely difficult; target is 60-70. • Reports in textbooks: range = 0-48, mean=27, mean grade level =18.5 (Harvey, 2006). • Practicing School Psychologists: mean=33, mean grade level =15.3 (Harvey, 1997) • Clinical Psychologists: mean=25, mean grade level =16.2 (Harvey, 1997) 30

31. Language and Style: Exercise 1

32. Language and Style: Exercise 1

33. Language and Style: Exercise 2

34. Language and Style: Exercise 2

35. Language and Style: Exercise 3

36. Language and Style: Exercise 3

37. Child-Centered Language

38. Child-Centered Language (2)

39. Exercise See Sheet

40. Oral Reporting • Have a conversation rather than a presentation • Invite participation explicitly • Be authoritative and responsive • Bring the child into the room (conceptually…and when you can, concretely) • Limit yourself to main points, titrating the amount of detail to meet the consumer’s needs • Lead with strengths • Present painful material clearly and supportively • Remember primary two functions: formulation and treatment

41. Implications for Due Process • Working assumption: longer, fancier, impenetrable reports have greater influence • No evidence to support this assumption • Anecdotal evidence is mixed, inconclusive • Parents and educators have no basis for comparison • Alternative assumption: Reports that are understandable and useful to consumers will be valued

42. Practical Issues and Challenges • Initial training, and difficulty of transitioning • Time and ease of formula-based format • Expectations or guidelines in given setting • Easier? Faster? NO! Consumer satisfaction is the key to increased use and acceptability!

43. “At no time are partnership practices of sensitivity, availability, respect, and honesty more important than when you share evaluation information (Turnbull et al., 2006, p. 245).” Affect (35%): Clinician was a good communicator, sympathetic, understood parents’ concerns, was approachable, direct Cognition (1.4%): Information easy to remember, information not too technical, understood the information, received enough information Quine & Rutter (1994) Predictors of Consumer Satisfaction

44. Concluding Remarks From initial conceptualization…through choice of procedures…formulation…report-writing…and oral feedback… Client-centered…Consumer-responsive…Referral-driven…Strengths-based

45. Clinical Impressions John is a cooperative and hard-working eighth grade student whose grades have decreased since he suffered a concussion in the middle of seventh grade. Reports from his teachers, classroom observations, his behavior while working with me, and his test results all indicate significant weakness in memory, particularly language-based memory. It is hard for John to remember what he hears and to take meaning from language. It is also hard for him to express himself. These weaknesses are very significant, with related scores ranging from the 1st to 16th %iles, or in the lowest 16% of children his age. Difficulties with memory and understanding language are likely the result of the concussion and very likely the reasons that John is having such difficulty in school. 45

46. References Brenner, E. (2003). Consumer-focused psychological assessment. Professional Psychology: Research and Practice, 34, 240 –247. Groth-Marnat, G. (2009) Five assessment issues you meet when you go to heaven. Journal of Personality Assessment, 91(4), 303-310. Groth-Marnat, G. & Horvath, L. S. (2006). The psychological report: A review of current controversies. Journal of Clinical Psychology, 62, 73–81. Harvey, V. S. (1997). Improving readability of psychological reports. Professional Psychology: Research and Practice, 28, 271–274. Harvey, V. S. (2006). Variables affecting the clarity of psychological reports. Journal of Clinical Psychology, 62, 5–18. Hasnat, M.J. & Graves, P. (2000). Disclosure of developmental disability: A study of parent satisfaction and the determinants of satisfaction. Journal of Pediatric Child Health, 36, 32-35. 46

47. References • Hilton, K., Turner, C., Krebs, G., Volz, C. & Heyman, I. (2012). Parent experiences of attending a specialist clinic for assessment of their children’s obsessive compulsive disorder. Child and Adolescent Mental Health, 17(1), 31-36. • Quine, L. & Rutter, D.R. (1994) First diagnosis of severe mental and physical disability: A study of doctor-patient communication. Journal of Child Psychology and Psychiatry, 35(7), 1273-1287. • Seligman, M. & Darling, R.B. (2007) Ordinary families, special children: A systems approach to childhood disability. New York: Guilford. • Snyder, C.R., Ritschel, L.A., Rand, K.L., and Berg, C.J. (2006) Balancing psychological assessments: Including strengths and hope in client reports. Clinical Psychology, 62 (1), 33-46. • Turnbull, A., Turnbull, R., Erwin, E. and Soodak, L. (2006). Families, professionals, and exceptionality (5th ed.). Mpls, MN: Pearson. 47