2010 SPA Annual Meeting San Jose, California Cross-Informant Assessment: Concepts, Behavioral Science, and Applications

1. 2010 SPA Annual MeetingSan Jose, CaliforniaCross-Informant Assessment:Concepts, Behavioral Science, and Applications Marvin W. Acklin, Ph.D. Honolulu, Hawaii Joshua Dwire, PsyD Grenada, The Caribbean

2. Cross-Informant Assessment:Concepts, Behavioral Science, and Applications • Broad focus of this symposium is the use of others in personality assessment • Using how people see each other as a source of clinical information • Looking at the relationship between how people are seen by themselves and how they are see by others • Focus on applied clinical science

3. Cross-Informant Assessment:Concepts, Behavioral Science, and Applications Data Sources in Psychological Assessment • Current views of psychological assessment endorse the use of multiple information sources. • Psychological assessment is concerned with a variety of test scores, generally obtained from multiple test methods, considered in the context of history, referral information, and observed behavior: unstructured & structured interviews; performance-based cognitive & personality tests, & observer rating scales (Meyer, et al, 2001) • Each information source has both strengths and constraints in the assessment procedure.

4. Cross-Informant Assessment:Concepts, Behavioral Science, and Applications Reliance on Self-Report • Traditional diagnostic practices rely heavily on individual self report as the main source of information in clinical assessment. • There are serious limitations in the validity of self-report, not the least of which is the patient’s motivation to communicate frankly (self-presentational bias).

5. Cross-Informant Assessment:Concepts, Behavioral Science, and Applications Use of Observer-Informant Information • The disciplined, empirically validated use of informants is both recent and innovative. The use of informants is most effective using parallel forms of items for behavior ratings. • The Achenbach System of Empirically Based Assessment (ASEBA) is a life-span approach using self-report and Informant ratings is one highly effective method of obtaining observer ratings. • I see you (observer rating) • I see you, you see me (Criss-cross) • I see you seeing me (metaperception; reflected appraisals)

6. Cross-Informant Assessment:Concepts, Behavioral Science, and Applications Limitations of Self-Report • Self report can be limited by defensive and self-presentational biases (Westen & Weinberger, 2004). • “…people have minimal access to many of their cognitive processes, and they often confabulate explanations for their behavior by applying intuitive attributional theories… • Validity Scales are utilized to correct for self-presentational bias. • Critique of validity scales as a basis for correcting self reporting bias (Piedmont et al, 2000).

7. Cross-Informant Assessment:Concepts, Behavioral Science, and Applications • In order to overcome the drawback of solely using self-reports in psychological assessment, Achenbach, Dumenci, and Rescorla (2005) suggested that cross-informed methods provide a multifaceted approach to assessment in which constructs for evaluation come from the same pool of items.

8. Cross-Informant Assessment:Concepts, Behavioral Science, and Applications Concordance of Information Sources: Cross-Method and Cross-Informant • Research demonstrates surprisingly low levels of concordance between different information sources (Meyer, et al, 2001; Meyer, 2002). • “The substantial independence between sources of information is not a phenomenon that is limited to rarified psychological research on personality…” (Meyer, 2002, p. 75). • “…Campbell and Fiske (1959) noted how relative independence among psychological methods can point to unappreciated complexity in the phenomena under investigation.”

9. Cross-Informant Assessment:Concepts, Behavioral Science, and Applications Concordance of Information Sources: Cross-Informants (Adults) • Research demonstrates surprisingly low levels of concordance between self and other report and concordance in observer reports: .681 substance abuse .428 internalizing disorders .438 externalizing disorders .304 mean .23 self vs. clinician .26 self vs. aggregated peer rating .29 self vs. spouse

10. Cross-Informant Assessment:Concepts, Behavioral Science, and Applications Adults • Self vs. spouse: Personality & mood .29 • Self vs. spouse/partner: Big five .44 • Self vs. peer: Personality & mood .27 • Self versus peer: Big Five Factors .31 • Self vs. significant other: Axes II PD diagnosis .21 • Self vs. clinician: Axis II PD traits .33 • Self vs. clinician: Big Five Factors .32 • Self vs. cognitive test: Memory problems .13

11. Cross-Informant Assessment:Concepts, Behavioral Science, and Applications Children • Research also demonstrates surprisingly low levels of concordance between self and other report and concordance in observer reports ofnchildren: .60 similar informants (e.g., parents) .28 different informants .22 self and other ratings

12. Cross-Informant Assessment:Concepts, Behavioral Science, and Applications Children • Self vs. parent: Behavioral & emotional problems .29 • Self vs. clinician: Behavioral & emotional problems .14 • Self vs. teacher: Behavioral & emotional problems .21 • Parent vs. direct observer of child behavior: .27 • Low concordance suggests situational specificity in informant ratings and the fundamental independence of ratings.

13. Cross-Informant Assessment:Concepts, Behavioral Science, and Applications • One approach to capitalizing on the potential added value of ratings by multiple informants is known as “360° feedback” methodology. This methodology entails comparing assessment data obtained from ratings by multiple informants, as well as from self-ratings. Many businesses have adopted 360o feedback for assessing managers' job performance by having subordinates, peers, and bosses rate the managers' performance for comparison with the managers' self-ratings (Brett & Atwater, 2001).

14. Cross-Informant Assessment:Concepts, Behavioral Science, and Applications • Self-ratings had mean correlations of.19 with peer ratings and.22 with supervisor ratings. And between supervisors and peers, the mean correlation was.34. Despite the relatively small correlations, a meta-analysis of the incremental validity of subordinate and peer ratings led Conway, Lombardo, and Sanders (2001) to conclude that the “results provided justification for the added expense of including subordinates and peers in a rating system”

15. Cross-Informant Assessment:Concepts, Behavioral Science, and Applications Cross-Informant Assessment of Personality Disorders • Personality Disorders: “…diagnoses have only moderate associations when they are derived from self-reports or the reports of parents, significant others and clinicians • Self and/or Other attributional biases in personality disordered reporters or informants may play a role in further reducing concordance.

16. Cross-Informant Assessment:Concepts, Behavioral Science, and Applications Limitations of Observer-Reports • The constraints of observer ratings include parameters of the particular type of relationships (e.g., spouse, coworker, therapist) and the setting in which observations are made. • Appear to be independent sources • Others reports may be systematically biased due to a variety of attributional biases—for example in divorcing spouses.

17. Cross-Informant Assessment:Concepts, Behavioral Science, and Applications Limitations of Observer-report • Theoretical implications of cross-method and cross-informant methods. • Weak support for the concept of consensual reality. The low level of concordance between sources, especially between self and observer report threatens simplistic views of consensual reality. • On the other hand, “…using a composite judge can yield very large validity gains when the redundancy of judges is low…” (Tsujimoto, Hamilton, & Berger, 1990)

18. Cross-Informant Assessment:Concepts, Behavioral Science, and Applications Use of the Achenbach scales in assessment practice The Achenbach System of Empirically Based Assessment comprises a family of forms for assessing adaptive functioning and problems is easy and cost-effective ways. Child Behavior Checklist Teacher-Caregiver Report Form Youth Self-Report Adult Self Report Adult Behavior Checklist Older Adult Self-Report Older Adult Behavior Checklist

19. Cross-Informant Assessment:Concepts, Behavioral Science, and Applications Introduction to Symposium Papers • Cross-informant assess using the Inventory of Interpersonal Problems (Pincus) • Cross-informed assessment of marital compatibility (Dwire & Acklin) • Self and Informant ratings using the NEO-PIR (Kurtz) • Parent personality and infant temperament (Janson & Narde)