Cricket Mitchell, PhD CIMH Evaluation Consultant

1. CiMH Palette of Measures Evaluation Training: Parenting Scale Cricket Mitchell, PhD CIMH Evaluation Consultant

2. Palette of Measures Evaluation: What You Will Need (slide 1 of 2) • Palette of Measures Data Entry Shell v2 (Excel file) • Developed by CiMH and customized for each participating agency • Holds all data for clients served thru the Palette of Measures project • Demographics, service delivery information, pre- and post- outcome measure data • Palette of Measures Data Dictionary v2 (Word document) • A guide for using the associated data entry shell • Defines each column in the excel file

3. Palette of Measures Evaluation: What You Will Need (slide 2 of 2) • Outcome measures from the two-pronged approach • General Outcome Measure • Target-Specific Outcome Measure(s) • For example… • Parenting Scale • Public domain, no fee for use

4. Overview of Training • Brief Overview of Palette of Measures evaluation protocol • Parenting Scale: Target-specific measure when the focus of treatment is disruptive behaviors • Administration • Scoring • Clinical Utility • Instructions for Palette of Measures data entry and data submissions • Data entry: Parenting Scale • Data entry: Demographics & Services • Data submissions to CiMH

5. Brief Overview of Palette of Measures Evaluation Protocol

6. Outcome Assessment • Palette of Measures providers will track outcomes using data from pre- and post- administrations of standardized measures of functioning • Pre- and Post- a “dose” of treatment / an intervention interval • General measure of youth mental health functioning (e.g., YOQ/YOQ-SR, CANS, Ohio Scales) • Target-specific measure linked to focus of treatment/intervention (e.g., AQ, ECBI, PHQ-9, PTSD-RI, RCADS) • Providers may choose to administer mid-course assessments as well • e.g., at 3-month intervals in usual care

7. A note about the use of standardized assessment measures… (slide 1 of 2) • Assessment is the beginning of developing a relationship with the child and family • Demonstrates a desire to know what the child and family are experiencing • By incorporating standardized assessment measures of functioning, the efficiency and thoroughness of assessment is enhanced

8. A note about the use of standardized assessment measures… (slide 2 of 2) • Using standardized assessment measures of functioning… • Assists in initial clinical impressions • Provides valuable information to guide treatment/interventions • Assesses sufficiency of treatment delivered • Demonstrates treatment-related improvements in child functioning

9. Parenting Scale

10. CiMH Parenting Scale Training • Information on the administration, scoring, and clinical utility of the Parenting Scale was obtained from • Arnold, O’Leary, Wolff, and Acker (1993). A Measure of Dysfunctional Parenting in Discipline Situations. Psychological Assessment, 5 (2), 137-144. • Rhoades and O’Leary (2007). Factor Structure and Validity of the Parenting Scale. Journal of Clinical Child and Adolescent Psychology, 36 (2), 137-146.

11. Parenting Scale Description • Target-specific measure for disruptive behavior • Measures parenting and discipline styles, particularly those that are found to be related to the development and/or maintenance of child disruptive behavior problems • Parent/caregiver report for children ages 1-12 • Adequate validity and reliability • Available in the public domain, provided to partner agencies by CiMH

12. Parenting Scale Description • 30 items • 7-point Likert scale • Two opposing responses are anchored at either end of the scale for each item* • One representing “ideal” parenting (1) • One representing parenting “mistakes” (7) *Some items are reverse-scored Total Scale Score 12

13. Example: Items from the Parenting Scale • When my child misbehaves… I do something right away…………I do something about it later • I threaten to do things that… I am sure I can carry out………………I know I won’t actually do • After there’s been a problem with my child… I often hold a grudge………….things get back to normal quickly 13

14. Parenting Scale Administration • Administer pre- and post- a dose of treatment, or an intervention interval, focused on disruptive behavior • Parenting Scale completed by mother/mother figure or father/father figure (when available) • Both can be completed for clinical purposes; however, only one should be entered for outcome tracking purposes • Select parent/caregiver who spends most time with client • Some agencies may choose to administer mid-treatment assessments as well

15. Let’s take a look at the Parenting Scale...

16. Parenting Scale Administration • 5-10 minutes to complete • Parents complete independently • Written at a 6th grade reading level • Can be read aloud if necessary/desired • Encourage honest responses • Encourage responses on all items • There are no “right” or “wrong” answers

17. Parenting Scale Scoring • The Total Score represents the mean item response • Total Score reflects the mean, or average, of all item responses and ranges from 1-7 • The scoring protocol is complicated and does not readily lend itself to hand-scoring • Depending on which side of the page the “ideal” parenting response is located, the values for item responses will either range from 1-7 left to right or from 7-1 left to right

18. Parenting Scale Scoring • The Palette of Measures Excel Data Entry Shell will automatically calculate the Total Score as data are entered • Until all items are entered, the Total Score in the file is not valid

19. Clinical Utility of the Parenting Scale • Factor Scores • When the measure was first developed there were three factor scores identified, representing three discipline styles • Laxness (11 items) • Permissive discipline • Over-reactivity (10 items) • Displays of anger, meanness and irritability • Verbosity (7 items) • Lengthy verbal responses or reliance on talking • Four of the 30 items did not load on to any factor • Two of the items loaded on to more than one factor

20. Clinical Utility of the Parenting Scale • Factor Scores • Recent studies have not supported the original factor structure, and three recently revised factors have been proposed • Laxness (5 items) • Permissive, inconsistent discipline • Over-reactivity (5 items) • Harsh, emotional, authoritarian discipline and irritability • Hostility (3 items) • Use of verbal or physical force • 17 of the 30 items do not load on to any factor

21. Clinical Utility of the Parenting Scale • Factor Scores • Given that the proposed factors have not proven to be reliable and the current factor structure excludes the majority of items, CiMH does not currently support the use of Factor Scores in outcome evaluation analyses

22. Clinical Utility of the Parenting Scale • Total Score • 30 items reflecting counter-productive discipline strategies • The higher the score, the more likely the parenting and disciplinary strategies are contributing to child disruptive behavior problems • Mean scores from the initial validation study: • Clinic group: scores range from 2.8 to 3.4 • Non-clinic group: scores range from .7 to 1.0

23. Clinical Utility of the Parenting Scale • Comparisons of pre/post scores reveal treatment-related improvement in parental discipline strategies • e.g., • Does the Total Score decrease substantially? • Is the Total Score most similar to the clinic or non-clinic groups from the initial validation study? • Before treatment? • After treatment?

24. Palette of Measures Data Entry and Data Submissions

25. Palette of Measures Data Entry Shell • There is a separate spreadsheet in the excel workbook (aka database) for each type of data: • Demographics & Services • Pre- General outcome measure • Post- General outcome measure • Pre- Target-specific measure(s) • Post- Target-specific outcome measure(s) • Specific outcome measure spreadsheets included in each agency’s database varies across Palette of Measures project participants

26. Palette of Measures Data Entry Shell • In addition to the spreadsheets that hold data... • There is an Instructions spreadsheet • Basic data entry instructions • Contact information for T.A. (Cricket Mitchell) • There is a Data Lists spreadsheet at the end of the workbook that you will not use • Data Lists populate the pull-down menus in other spreadsheets • There is a Formulas spreadsheet at the end of the workbook that you will not use • Automatically calculates the Parenting Scale Total Score

27. Palette of Measures Data Entry: Parenting Scale … • There is a separate spreadsheet for Pre-ParScale data, Post-ParScale data, and Mid-ParScale data • In each spreadsheet, there is a separate field for all 30 item responses • The Total Score will automatically calculate • ParScaleTot Scores will not be valid until all items are entered

28. Palette of Measures Data Entry: Parenting Scale … • For agencies who will conduct mid-treatment assessments, indicate the Assessment Interval in the Mid-ParScale spreadsheet by selecting from the available pull-down menu • e.g., 1st mid-treatment assessment, 2nd mid-treatment assessment

29. But, before you enter any outcome data, you’ll enter Demographics & Service Delivery Information...

30. Palette of Measures Data Entry: Demographics & Services (1 of 2) • Use a unique identifier for Client ID# • Categorical variables will have pull-down menus from which you’ll select an option (e.g., gender, ethnicity, language) • Dates should be entered as xx/xx/xxxx • Axis I diagnoses s/b the numeric DSM-IV code • Therapist ID is optional

31. Palette of Measures Data Entry: Demographics & Services (2 of 2) • Select Focus from pull-down menu (e.g., anxiety, depression) • The shell will hold data for up to 4 foci, or treatment targets • Enter Date of First Session • The remaining fields are to be completed at the end of treatment targeting this particular focus (e.g., Date of Last, # Sessions) • 2 levels of “Completed?” • Treatment targeting this particular focus • Overall service delivery

32. Palette of Measures Data Submissions (slide 1 of 3) • Data submissions to CiMH will occur twice a year throughout the duration of the project • The end of each May (reflecting all clients served from the initiation of the project through the end of that April) • The end of each December (reflecting all clients served from the initiation of the project through the end of that November) • Note that this is the anticipated schedule; actual data submission dates may vary slightly • An email notice will be sent to Palette of Measures site leads approximately one month in advance of each data submission deadline

33. Data Submissions (slide 2 of 3) • Providers may choose from among the following methods for submitting their Palette of Measures Excel databases to CiMH: • Use YouSendIt, or another secure web-based transfer site, to submit data electronically • YouSendIt (www.YouSendIt.com) is a vendor that supports the secure transfer of electronic data (encrypted and password-protected) • Mail a password-protected CD to CiMH and submit the password separately (via email or phone) • Email an encrypted, password-protected file(s) to CiMH and submit the password separately (via email or phone)

34. Data Submissions (slide 3 of 3) • After data are submitted, sites continue to enter new data into the same database • Always reflects an ongoing, historical record of clients served through the Palette of Measures project • Every effort is made to distribute reports within two months of each data submission • Aggregate and site/agency-specific reports

35. Questions

36. The End • Contact Information • Cricket Mitchell, PhD • Email: cmitchell@cimh.org • Cell phone: 858-220-6355