James Bell Associates (JBA): Who We Are…

1. Technical Assistance on Evaluation to Child Welfare Discretionary Grantees2012 Abandoned Infants Assistance Comprehensive Support Services for Families Affected by Substance Abuse and/or HIV/AIDSOverviewNovember 2012James Bell Associates

2. James Bell Associates (JBA):Who We Are… • Small business located in Arlington, VA. • Conducting research and evaluation in public health and child welfare since 1979. • 30+ employees and nationwide pool of consultants. • Experts in program evaluation, technical assistance and training, applied research. • DHHS clients include ACF, HRSA, NIH, ASPE, and SAMHSA

3. JBA’s Evaluation TA Contract • TA provider to CW discretionary grantees since 1999. • Number of grantees served annually varies (currently ~100). • Other priority areas supported include Adoption Opportunities Programs, Child Abuse/Neglect Prevention, Integrating Trauma-Informed and Trauma-Focused Practice in CPS, Promoting Safe and Stable Families, Diligent Recruitment, Child Welfare and Early Education/Education Partnerships, Improving Service Delivery to Youth in CWS. • TA is also provided to Family Connection grantees under a separate cross-site evaluation contract.

4. Technical Assistance Goals • Support the Children’s Bureau in disseminating knowledge and promoting the use of effective and evidence-based practices in the child welfare field. • Promote the implementation of rigorous, useful, and practical evaluations. • Strengthen the quality of evaluation reports and products available to the child welfare field.

5. JBA’s Evaluation TA Contract: What We Provide • Responsive TA to individual grantees upon request. • Proactive TA to individual grantees upon request of FPO and/or JBA recommendation. • Group TA to grantee clusters (e.g., facilitating conference calls & Webinars regarding research/evaluation topics of common interest).

6. JBA’s Evaluation TA Contract: Important Facts • JBA is not a program monitor or auditor; our role is to work in partnership with the Children’s Bureau to assist you in implementing a quality evaluation. • We do not conduct evaluations under this contract. • JBA’s services are available at no cost to grantees.

7. TA Approach • Introductory mailings • Evaluation TA packet • Kickoff Meeting sessions/calls • Initial conference calls • Develop TA plans • Ongoing communication • Web-based portal/information sharing • Ongoing evaluation document review

8. TA Modalities Site-Specific TA Cluster TA • Grantee meeting presentations and facilitated discussions • Webinars and teleconferences • Summary documents • Web-based portal • Conference calls • Document review • E-mail • Site visits • Proactive and Responsive

9. More About Site-Specific TA • Tailored to the specific interests and needs of grantees. • Responsive TA: Grantee contacts JBA for advice or assistance with a specific evaluation issue. • Proactive TA: FPO asks JBA to contact grantee if a specific concern or TA need is identified.

10. More About Cluster TA • Initiated based on interests and needs of FPO and grantees. • Focus is on peer learning, networking and information sharing. • Examples: • Facilitate collection and analysis of data on common outcome measures. • Facilitate discussion and information sharing regarding research and evaluation issues or challenges of common interest.

11. Now that you have won your grant … • Enhancements to the evaluation described in your proposal are encouraged. Components to revisit include: • Logic Model • Research/Evaluation Design • Outputs/Outcomes • Output/Outcome Measures • Data Collection Tools/Instruments

12. Consider Revising Your Evaluation Design If: • Expected data sources are not available. • Inadequate referrals pose challenges for random assignment or comparison group designs. • Original evaluation plan/logic model focused on the “wrong” outcomes or measures. • Assessment instruments are invalid, unreliable, or otherwise inappropriate. • Some interventions/activities were never implemented and cannot be evaluated. • Original evaluation plan was too ambitious.

13. Back to the Drawing Board: Steps to Take Before Changing Your Evaluation • Discuss your proposed changes with JBA. • Contact your FPO to obtain approval before you make any changes. JBA cannot approve changes! • Document changes in subsequent progress reports to CB and JBA.

14. Include JBA in the Distribution of Semi-Annual Progress Reports • Send a copy directly to your JBA TA liaison every six months (electronic copies preferred). • Allows JBA to stay informed about your project and identify possible TA needs.

15. Questions? Kim Keating, MEd Sr. Research Associate/TA Liaison keating@jbassoc.com 703/528-3230 James Bell Associates 3033 Wilson Boulevard, Suite 650 Arlington, VA 22201