1 / 156

New Advances in Measurement

This paper discusses new advances in measurement related to relationship quality and responsiveness to change. Topics covered include IRT optimization, bi-dimensional view, implicit measures, screening for error variance, and attention/effort. The paper also acknowledges the work of various researchers and introduces the Couples Satisfaction Index (CSI). The paper concludes with a discussion on the longitudinal analysis and responsiveness to change over time.

jerryn
Download Presentation

New Advances in Measurement

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. New Advances in Measurement Ronald D. Rogge

  2. TOPICS • RELATIONSHIP QUALITY • T1: IRT Optimization • Study 1 • T2: Responsiveness to Change • Studies 2-5 • T3: Bi-Dimensional View • Studies 6-7 • T4: Implicit Measures • Studies 8-11 • ATTENTION • T5: Screening for Error Variance • Studies 12-16

  3. Acknowledgements • Couples Satisfaction Index • Janette Funk, Mike Maniaci, Maria Saavedra, Soonhee Lee • Positive-Negative Relationship Quality • Frank Fincham, Richard Mattson, Matt Johnson • C.A.R.E. Program • Tom Bradbury, Rebecca Cobb, Matt Johnson, Erika Lawrence, Lisa Story, Lexi Rothman • Implicit Assessment • Soonhee Lee, Harry Reis • Attention / Effort • Mike Maniaci, Janette Funk, Soonhee Lee, Maria Saavedra

  4. Relationship Quality • Relationship satisfaction • Self-report scales (DAS, MAT, QMI) • 30-50 years of research (over 4K studies) • Excellent correlational validity • Level of noise? • Responsive to change over time? • Are these the “best” items?

  5. TOPIC 1: IRT Optimization • Large sample method • N at least 1,000 in smallest group • Large item pool • Unidimensional • Non-redundant • Used by ETS • SAT, GRE, MCAT, LSAT • Quality of each item • Information • Noise • Advantages • Over correlations • Over small sample methods

  6. IRT Approach • Latent scores (q) for each subject • Like GRE scores • Assessing relationship satisfaction • Parameters for each item • Response curves • Higher q’s  higher responses? • Item Responsiveness • How informative? • Where informative? • Creates information profiles • For individual items • For sets of items

  7. Study 1 - Measures • 141 satisfaction items: • DAS, MAT, RAS, KMS, QMI, SMD • 71 additional items • 7 anchor scales: • Neuroticism (EPQ-N) • Conflict / Communication (MCI, CPQ, IAI) • Stress (PSS) • Sexual Chemistry (Eros) • Instability (MSI) • 2 validity scales: • Inconsistency (PAI) • Infrequency (PAI)

  8. Study 1 - Sample • 5,315 online respondents • After removing: • Incomplete or invalid responses • Multivariate outliers • 26yo (10yrs) • 83% Female • 76% Caucasian • 26% High school ed. or lower • $27K average income • 24% married, 16% engaged, 60% dating

  9. Evaluating Previous Scales • IRT results • Simultaneous analysis • 66 items of existing scales • Some very informative items • Many poor items

  10. DAS-31(Degree of happiness, all things considered, of your relationship) Response Curves Information Curve

  11. DAS/MAT 5Agreement on: FRIENDS Response Curves Information Curve

  12. MAT 12In leisure time, do you (and does your mate) prefer to be “on the go” or to stay at home? Response Curves Information Curve

  13. From Items to Scales • A scale’s information = sum of information from each item • How informative Across different levels of happiness

  14. Test Info for Current Measures

  15. Analysis of Existing Measures • Many uninformative items • Particularly for DAS and MAT  noise / error • Modest test information • For all scales • Notably poor for MAT and DAS • Room for improvement

  16. Creating the CSI • 141 item pool • Screen for contaminating items • Screen for redundant items • IRT on remaining 66 items • Select 32 most effective

  17. Test Info for CSI Scales

  18. Basic Psychometrics

  19. Correlations with Anchors

  20. What have we gained? • Identical correlational results • Strong convergent validity • Strong discriminant validity • Strong construct validity  Measuring same thing • Higher information… •  Should have • Lower Noise • Higher Precision • Greater Power

  21. Satisfaction Groups • IRT satisfaction estimates • For each subject • Based on MAT, DAS, & CSI items • (equivalent of GRE scores) • Created satisfaction groups • N = 265 • HIGHLY similar SAT within each group • MAT, DAS & CSI scores also similar?

  22. Precision: CSI-32 vs. DAS

  23. Effect Size • Ability to detect difference • Between groups • Pre – Post • Effect Size = M1 – M2. pooled SD • Difference in SD units • Power for detecting D’s in SAT groups

  24. Power: CSI-32 vs. DAS

  25. STUDY 1 - Conclusions • CROSS-SECTIONALLY • CSI assess same construct • Higher precision • Higher power • NEXT STEP • Longitudinal analysis • Responsiveness to change over time

  26. TOPIC 2: Responsiveness • Detecting change • Assumption • Longitudinal • External Criteria • Treatment effect • Clinician • Interviewer • Global report • SERM (Sdiff) • Noise over time • Estimating • Two main applications • Individual change • Clinically distinct groups

  27. Studies 2 through 4 • Study 2 • 267 online respondents • 1 & 2wk follow ups • 468 change scores • Study 3 • 156 online respondents • 1, 2, 3, 4, 6 & 12mo follow ups • 455 change scores • Study 4 • 545 online respondents • 1, 2, 3, 4, 6, 9 & 12mo follow ups • 1,552 change scores

  28. Studies 2-4: Measures • Relationship satisfaction scales: • DAS-32 • MAT-15 • CSI-32 • CSI-16* • CSI-4* • 3 global relationship change items • Change since last assessment

  29. Individual Change • How many points of change needed? (to show significant change) • SERM in “No Change” • RCI (Jacobson & Truax, 1991) • MDC95 (Stratford et al., 1996) • MDC95 (SD units) = 1.96*SERM. SD • PRESENTING • Meta-Analytic Summary • Standardized Units

  30. Reliable Individual Change C C C B A

  31. Detecting Change • Individual Change • IRT optimization • Longer scales • Distinct Groups • Can scales distinguish? • Mild deterioration • No change • Mild improvement

  32. Perceived Change • How much have these changed? • Overall happiness in the relationship • Feeling close and connected • Stability of the relationship

  33. Perceived Change • Averaged responses • Alpha = .92 • Created change groups

  34. Distinct Change Groups

  35. Quantifying Group-Level Responsiveness • MCID (Guyatt, Walter & Norman, 1987) • Noise over time (SERM) • Effect Sizes: (Avg Change)IMPROVE – (Avg Change)NO CHANGE SERM (Avg Change)DETERIORATE – (Avg Change)NO CHANGE SERM

  36. Analytic Strategy • Improving method • Multi-wave data • Global change continuous • Moderation • HLM • PV: Global change score • Moderators: • Gender • T0 Satisfaction • DV: Change scores on scales (n = 2475)  Change scores ≈ 1pt global change  MCID effect sizes • Meta-Analytic Summary

  37. Responsiveness in Dissatisfied (1SD below M) A B C C C D C B A A

  38. Responsiveness in Satisfied (1SD above M) A A B B B E D C B A

  39. Responsiveness Conclusions • Can be quantified • Scale selection • Power estimates • Responsive scales • Greater power • Individual • Group • Cross-sectional  Longitudinal • Precision & Power translate • NEXT STEP  Treatment Effects

  40. Topic 3: Bi-Dimensional View • Uni-Dimensional view • Positive feelings opposite negative feelings • Bi-Dimensional view • Pos/Neg independent • Moderately “dissatisfied” • Ambivalent • Indifferent • Uni-Dimensional obscuring?

  41. Background • Fincham & Linfield (1997) • PN-QIMS • Two 3-item scales • Qualities of spouse • Feelings toward spouse • Feelings about marriage • Considering only (pos/neg) • Separated in survey • CFA in 123 couples • Unique information

  42. Study 5 • Mattson et al. (under review) • New pos-neg scale • 7 SMD items of CSI • Pos / neg separately • Large online sample • Ambivalent • Indifferent

  43. Study 5 - Sample • 1656 online respondents • Demographics • 28yo (7yrs) • 94% Female • 87% Caucasian • 30k income • 5% ≤ high school • Romantic relationships • 38% married (6.5yrs) • 19% engaged (3.6yrs) • 41% dating – exclusive (2.4yrs)

  44. Ambivalence vs. Indifference • Median Splits

  45. Ambivalence vs. Indifference • Median Splits

  46. Uni-Dimensional Satisfaction

  47. Negative Conflict

  48. Negative Affect

  49. Study 6 • IRT Optimized Positive & Negative Scales • Item Pools • 20 positive items • 20 negative items • Large sample • UG respondents • Analyses • EFA • Redundancy • IRT • Precision / Power / Validity

  50. Study 6 - Sample • 1,814 undergrad respondents • Demographics • 19yo (2yrs) • 77% Female • 72% Caucasian • Together 2.6yrs • 26% dissatisfied • Close relationships • 54% romantic partners • 38% friends • 5% family members • 3% roommates • Romantic relationships • 76% dating – exclusive • 21% dating – non-exclusive

More Related