Internal Consistency of Current Situation Rating Scale

1. Validation of English and Spanish Versions of a Client Progress Measure for Use in Healthcare-based Domestic Violence ProgramsAuthors: Jeanne Hathaway, MD, MPH; Bonnie Zimmer, LICSW; Sue Chandler, MPH, MSW; Lisa Hartwick, LICSW; Jennifer Robertson; Lisa Tieszen, LICSW; Jay Silverman, PhD Background and Study Objectives Internal Consistency of Current Situation Rating Scale Main Sections of Client Feedback Form Acceptability and Clarity • 1) Current Situation Rating Scale • Covers a range of broad categories (personal safety, empowerment, emotional support, etc.) • Uses 1-5 likert rating scale • List other formal supports • 2) Barriers to Change / Healing • Abuse and controlling behavior • Court and legal issues • Other potential barriers (lack of resources, health concerns, etc.) • 3) Effects of Advocacy or Support Groups • Better/Worse ratings of same categories as Current Situation Rating Scale • Detailed questions re: health/health care effects • Two open-ended questions: Ways most helpful? Anything worse? • 4) Client’s Goals • Two open-ended questions: Current goals? Ways advocate can help? Methods: Assessment of the acceptability and clarity of the Client Feedback Form were obtained through a brief 3-item survey with both open and closed-ended questions. Sample: n = 105 (49 – English; 56 - Spanish)Main analyses: Frequencies and content analysisResults: How did it feel to complete the Client Feedback Form?- Responses did not differ significantly by language version. - Discomfort was mainly attributed to being reminded of past abuse. Background: Despite the increasing number of domestic violence (DV) programs based in health care settings, few of these programs have been evaluated. This is partly due to a lack of appropriate outcome measures. To address this gap, the Conference of Boston Teaching Hospitals (COBTH) DV Council has developed a new progress measure, the DV Program Client Feedback Form for use in healthcare-based DV programs. Study objectives: To conduct initial tests of the reliability and validity of English and Spanish versions of the “Client Feedback Form.” Purpose of Client Feedback Form: The Client Feedback Form is designed to be used for both research and advocacy purposes. In terms of research, the DV Council intends to use the form to evaluate how well healthcare-based DV advocacy programs are meeting their programs’ goals. The four common goals of these programs are to: • Methods: We measured the internal consistency of the Current Situation Rating Scale of the Client Feedback Form. This test indicates how similar ratings are between items on each person’s survey. • Sample: n = 155 ( 98 - English; 57 - Spanish) • Main analytic test: Cronbach’s alpha • Results: Cronbach’s alphas for the Current Situation Rating Scale were .86 (English) and .87 (Spanish). • Conclusions: The internal consistency of the Current Situation Section is uniformly high on both English and Spanish versions of the Client Feedback Form. • Revisions to Client Feedback Form: None based on these results. • Improve clients’ safety, • Improve clients’ health and well-being, • Connect clients to needed resources, and • Increase clients’ ability to advocate for themselves. Photos are for illustration only and do not represent actual program clients. How administered: The Client Feedback Form is completed by the client (with or without advocate assistance), then reviewed with the client’s advocate. The form is currently completed on paper, however some programs are considering computer-administration. We decided that clients should complete forms, rather than advocates or healthcare providers, since clients are the best source of information on how well the program is achieving its goals and how program advocates can best assist them. When administered: The Client Feedback Form is completed 3 and 6 months after intake, then every 6 months thereafter. The “Initial” version of the form, which asks about lifetime partner abuse, is used at the first administration. The “Follow-up” version of the form, which asks about partner abuse in the past 6 months, is used subsequently. Length: The Client Feedback Form is 6 pages and takes approximately 15 - 20 minutes to complete (not including review with the advocate.) Languages: Currently available in English and Spanish Study Components Response Bias In addition to program evaluation, the DV Council also intends to use the Client Feedback Form as an advocacy tool. We therefore designed the form to help assess clients’ needs, help clients recognize their strengths, and help advocates focus their work with clients. Efforts were made when developing the Client Feedback Form to minimize any ways in which completing it could be harmful to clients or the client-advocate relationship. This was done through attention to the wording, order, and length of each section. We also developed an instruction guide for administering the form, since how the form is introduced and used will also affect clients’ experiences. In summary, the Client Feedback Form tested in this study was developed over a period of two years with input from: We administered written surveys at one time point to women receiving services from healthcare-based DV programs to assess the following aspects of the Client Feedback Form: • Were any parts of the Client Feedback Form confusing or unclear? • 15% of respondents (7 – English; 9 - Spanish) reported that some aspect of the form was confusing or unclear. Of these 16, six did not specify what was unclear and no more than three respondents cited confusion over any single item or section. Some comments were: Methods: We measured response bias by comparing responses on Client Feedback Forms reviewed WITH a DV program advocate to responses on Client Feedback Forms NOT reviewed with an advocate. Sample: n = 48 “With Advocate” group; n = 50 “No Advocate” group; (English only) The “With Advocate” group was slightly more likely than the “No Advocate” group to be of “Other” race/ethnicity (p = .09), born outside the U.S. (p = .04), and to have some income (p = .09). Groups did not differ significantly in terms of types of abuse, court or legal issues, or other barriers to change/healing reported. Main analytic test: Linear regression Results: Controlling for race/ethnicity, immigration status, and income, we found no significant association between group and clients’ ratings of their “Current Situation” (p = .75) or how participation in advocacy or support groups had affected their lives (p = .52). • Concurrent Validity of Five Key Concepts • Internal Consistency of Current Situation Rating Scale • Response Bias • Acceptability and Clarity • “I needed my advocate to explain it to me because English is hard for me.” – Asian respondent • “Court and legal issues [were confusing] – some still ongoing, but not necessarily active or court-related.” Concurrent Validity of Five Key Concepts • Methods: We measured the concurrent validity of five key constructs in the Client Feedback Form (partner abuse, physical health, mental health, self-efficacy, and quality of life) by comparing Client Feedback Form responses to scores on standardized measures covering similar constructs. • Sample: n = 100 (43 – English; 57 – Spanish) • Main analytic test: Spearman Correlation Coefficients • Results: • Healthcare-based DV program directors and advocates • Focus groups with White, African-American, and Latino clients of healthcare-based DV programs • Findings from previous research with clients from one of the healthcare-based DV programs involved (HAVEN) • Intake and activity forms from healthcare-based DV programs • Other measures, including the DV Survivor Assessment by Dienemann and Campbell Study Sample Conclusions: The Client Feedback Form was acceptable and clear to the majority of respondents, regardless of language version. Approximately one in four respondents experienced some discomfort, however, mainly due to the abuse questions. Minor changes should be made to clarify certain items or sections. Conclusions: We found no evidence of response bias, that is, Client Feedback Forms reviewed WITH a DV program advocate did not have “better” (or “worse”) responses than forms NOT reviewed with an advocate. Revisions to Client Feedback Form: None based on these results. We used a convenience sample of 155 women receiving services from four healthcare-based DV programs in the Boston area. Eligible clients were female, age 18 or over, spoke English or Spanish, had 3 or more phone or in-person contacts with a DV advocate, and were not in acute emotional distress (as determined by the DV advocate). Revisions to Client Feedback Form: We developed an instruction guide that outlines precautions advocates can take to minimize clients’ risk of emotional discomfort and reminds advocates that some clients may need additional support after completing the form. We also revised the directions to the Court/legal Section and replaced one open-ended question on informal supports with two closed-ended questions on emotional and material support in the Current Situation Section. Additional minor revisions were made to the Client Feedback Form following this study based on: • Average age: 35 y.o. (range: 18 – 59 y.o.) • Average length time in program: 1 year and 4 months Race/ethnicity Immigration Status • Responses written in under “Other” in several sections • Missing descriptions of change in the Program Impact Section • Responses to Current Goals questions • Back-translation of the form from Spanish to English Marital Status Photos are for illustration only and do not represent actual program clients. Summary • How did it feel to complete the Client Feedback Form? • “It felt good to let out some of the bottled feelings, especially writing out on paper was a slight relief.” • “Me dado de cuenta de muchas cosas que no veia antes.” • [It gave me insight into things I didn’t see before.] These initial assessments indicate that English and Spanish versions of the DV Program Client Feedback Form are generally acceptable and clear to respondents, have good validity of items covering partner abuse and quality of life, and that review of client’s responses with a DV advocate does not cause significant response bias. Future Research: Additional studies are needed to validate items covering self-efficacy on both language versions and mental and physical health on the Spanish version. Future research should also assess the reliability and validity of the Client Feedback Form among different populations, including other racial/ethnic groups, adolescents, people in same-sex relationships and male victims. It may also be worthwhile to assess utility of the Client Feedback Form in other types of domestic violence services. Photos are for illustration only and do not represent actual program clients. Conclusions: In both language versions, correlation coefficients were high for partner abuse and quality of life. Correlations were also high for mental and physical health on surveys completed in English. Correlations were low for self-efficacy in both language versions, as well as for mental and physical health on surveys completed in Spanish. Revisions to Client Feedback Form: Added new self-efficacy/ empowerment item to Current Situation Rating Scale and modified wording of mental and physical health items on Spanish version. • How did it feel to complete the Client Feedback Form? • “Good - but I’m a little depressed because there are things that happened that I don’t want to remember.” • “Me siento un poco triste. Triste porque hay cosas que pasan que uno no quisiera recordar.” • [I feel a little sad. Sad because there are things that happened that I don’t want to remember.] Sources of Income Type Health Insurance