ADCANP/CTF Program Evaluation: Grantee Training August 30, 2011

1. ADCANP/CTF Program Evaluation:Grantee TrainingAugust 30, 2011 Debra Nelson-Gardell, Ph.D., LCSW M. Felicia Woerner, MA The University of Alabama School of Social Work

2. UA Evaluation Project Mission • To support DCAP/CTF in the successful delivery of child abuse and neglect prevention services through an effective, efficient, and feasible data collection, management, analysis, and reporting system.

3. Significance of Evaluation • Funding and Impact • Without a solid evaluation, impact cannot be determined. • An objective, outside, evaluation supports yours and your clients’ confidence about your services. • Without some determination of impact, the argument for increased funding becomes difficult. • Your part in the evaluation contributes to ADCANP’s ability to obtain increased money to support your programs.

4. DCAP/CTF Overall Goal: A Reminder • Programs will work to decrease risk factors and increase protective factors related to child maltreatment. • Congruent with current, empirically supported, best practices • Same goal for all programs • Implementation differs across program types • Goals by type of program are listed in the RFP

5. Overview of the Evaluation • Process Evaluation • Number and characteristics of clients served • Frequency and duration of services • Public outreach and awareness activities • Client-level Outcome Evaluation relying on pre- and post-tests • Parent-Focused & Home Visiting Programs • Protective Factors Survey • Respite Care Programs • Perceived Stress Scale • Child-Focused Programs • Custom Created Child-Focused Measure of Pro-social Behaviors

6. Protective Factors • Nurturing and attachment • Knowledge of parenting and of child and youth development • Parental resilience • Social connections • Concrete supports for parents

7. Risk Factors • Parent or caregiver factors • Personality characteristics/mental health • History of abuse • Substance abuse • Child rearing approaches • Teen parents • Family factors • Family structure • Domestic violence • Stressful life events • Child factors • Birth to age 3 • Disabilities • Environmental factors • Poverty and unemployment • Social isolation and social support • Violence in communities

8. Important Definitions • Pre-test • Administered within two weeks of first service delivery date • Post-test • Any instrument administered beyond the two week pre-test time-limit • Must be administered to clients at least once per contract year • Either at end of service delivery, or (whichever comes first) • Before the end of the contract year • Courtesy reminder sent by evaluation staff in early May regarding impending end of contract year

9. Pre- and Post-Test Instruments • Instruments were selected to • Maximize data quality • Minimize instrumentation burden on clients and service delivery staff • Parent-Focused & Home Visiting Programs • Protective Factors Survey (PFS) • Respite Care Programs • Perceived Stress Scale (PSS) • Child-Focused Programs • Assessments of Prosocial and Self-Protective Behaviors (APSB) • Kindergarten to 2nd Grade • 3rd Grade – 5th Grade • 6th Grade – 12th Grade

10. Special Instructions • Instruments are available at http://dcapdata.ua.edu. • Do NOT download them from another source! • Instruments have been customized for CTF programs.

11. Important Reminders • Race and ethnicity are not the same. • Ethnicity = Hispanic or Not Hispanic • Race = Black, White, Native American, etc. • Race is not Mexican, Hispanic, etc. • Mexican or Hispanic is not a race, it is an ethnicity.

12. Important Reminders • Read the website User Guide. • It will save you time. • Supporting your ability to serve clients • It will save evaluation staff time. • Supporting timely report generation • Supporting potential website enhancements

13. Important Reminders • Read the User Manuals for the outcome instruments. • They tell you: • When to administer • What materials to prepare • How to administer • About consent • User Manual includes templates that can be customized to your agency/program • Re: Protective Factors Survey • For clients’ children’s ages under 1, use the Age Conversion Table found under the “Additional Resources” tab on the DCAP website.

14. Important Reminders • How do I know my program’s monthly report has been successfully submitted and received by the UA evaluation team? • The online forms tell you. • How do I make corrections? • Send email detailing needed corrections to dcapdata@bama.ua.edu. • Include either your program’s contract number and/or agency ID.

15. Important Reminders • Client and Agency forms must be completed and finalized for every month your program is funded. • Client form must be completed and finalized every month even if your program did not serve clients that month. • Exception: programs that are ONLY Public Awareness Programs • FYI: All report finalizations are time and date stamped. • All successful log-in attempts are also time and date stamped.

16. Updates • Client Form Enhancement • Target availability early January 2012 • Button streamlining • Concentration of available functions into one area • A change regarding active and inactive client display • Faster to mark clients as active or inactive • Faster loading

17. Outcome Data Analysis • Your diligence in obtaining and entering data is crucial. • If enough data are not entered, we cannot do a meaningful analysis. • This affects the credibility of what ADCANP can claim regarding the effectiveness of its programs. • Your actions affect everyone!

18. Thank you! • Debra Nelson-Gardell, Principle Investigator • dnelsong@ua.edu • 205-348-2990 • M. Felicia Woerner, Project Manager • dcapdata@ua.edu • 205-348-8011 • This presentation available for download under the “Additional Resources” tab on the evaluation website.