Teenage Pregnancy Prevention: Replication of Evidence-based Programs (Tier 1)

1. Teenage Pregnancy Prevention: Replication of Evidence-based Programs (Tier 1) U.S. Department of Health and Human Services Office of Adolescent Health April 14, 2010 3:00-5:00pm ET 1

2. Purpose of Today’s Call • Review the Teenage Pregnancy Prevention: Replication of Evidence-Based Programs Funding Opportunity Announcement, including the purpose, eligibility requirements, and how to apply for funds • Will not be answering questions specific to individual applications 2

3. Introductions Office of Adolescent Health Evelyn Kappeler, Acting Director Alice Bettencourt, Acting Deputy Director Allison Roper, Public Health Analyst Amy Margolis, Public Health Analyst Jennifer Gannon, Program Specialist Miryam Gerdine, Public Health Analyst Office of Grants Management Karen Campbell, Director April 14, 2010 3

4. Overview of the Office of Adolescent Health & Teenage Pregnancy Prevention Initiative

5. Office of Adolescent Health (OAH) Consolidated Appropriations Act, 2010 directed that a new OAH be established Responsible for implementing and administering new grant program to support evidence-based teen pregnancy prevention approaches Coordinates adolescent health programs and initiatives across the U.S. Department of Health and Human Services April 14, 2010 5

6. Office of Grants Management • Official signatory for obligating federal grant funds • Official signatory for all grant business • Monitor all business/financial transactions on grants for compliance to Federal Regulations (including interpretation of Federal Regulations)

7. Secretary Kathleen Sebelius Office of Public Health and Science Deputy Assistant Secretary for Health (Science and Medicine) Anand Parekh, MD, MPH Deputy Assistant Secretary for Health (Healthcare Quality) Don Wright, MD, MPH ASSISTANT SECRETARY FOR HEALTH Howard Koh, MD, MPH Principal Deputy Assistant Secretary for Health Wanda Jones, DrPH Senior Advisor to the ASH Rosemarie Henson, MPH, MSW Regional Health Administrators Regions I-X Office of Communications Dori Salcido Office of the Surgeon General Surgeon General Regina Benjamin, MD, MBA VADM, USPHS National Vaccine Program Office Deputy Assistant Secretary Bruce Gellin, MD, MPH Office on Women’s Health Director Frances Ashe-Goins, RN, MPH (Acting) Office of Minority Health Deputy Assistant Secretary Garth Graham, MD, MPH LCDR, USPHS Office of HIV/AIDS Policy Director Christopher Bates, MPA Office for Human Research Protections Director Jerry Menikoff, MD, JD Office of Research Integrity Director Don Wright, MD, MPH (Acting) Office of Population Affairs Director Susan Moskosky (Acting) Office of Adolescent Health Director Evelyn Kappeler (Acting) Office of Disease Prevention and Health Promotion Deputy Assistant Secretary Penelope Slade-Sawyer, PT, MSW RADM, USPHS President’s Council on Physical Fitness & Sports Executive Director Shellie Y. Pfohl, MS Office of Commissioned Corps Force Management Director Denise Canton, JD, RN RADM, USPHS

8. Location 8

9. Teenage Pregnancy Prevention Initiative • Consolidated Appropriations Act, 2010 (Public Law 111-117)- $110 million • $75 million - replicate program models proven effective through rigorous evaluation (Tier 1) • $25 million - research and demonstration grants to develop, replicate, refine, and test additional models and innovative strategies (Tier 2) • $10 million - training and technical assistance, evaluation, outreach, and additional program support activities 9

10. Introduction to TPP Tier 1: Replication of Evidence-based Programs Funding Announcement

11. Purpose of Tier 1 Funding Announcement • To support the replication of evidence-based program models that are medically accurate, age appropriate, and have proven through rigorous evaluation to reduce teenage pregnancy, behavioral risks underlying teenage pregnancy, or other associated risk factors. 11

12. Target Populations • Individuals 19 years of age or under at program entry • Applicants should clearly define target populations by age groups or priority populations within a defined geographic area with high teen birth rates. 12

13. “Funds made available … shall be for making competitive contracts and grants to public and private entities”(Consolidated Appropriations Bill, 2010) Nonprofit organizations For-profit organizations Universities and colleges Research institutions Hospitals Community-based organizations Faith-based organizations Federally recognized or state-recognized tribal governments State and local governments State and local school districts Political subdivisions of States Who’s eligible to apply? 13

14. Funding Ranges for Tier 1 • Range A - $400,000 to $600,000 per yr • Range B - $600,000 to $1,000,000 per yr • Range C - $1,000,000 to $1,500,000 per yr • Range D - $1,500,000 to $4,000,000 per yr * Applicants may only apply for one funding range under this announcement. 14

15. Cost-Sharing or Matching • Neither cost-sharing nor matching are required • Applicants are encouraged to include participation by stakeholders in the community as an indicator of community support for the project • An indication of institutional support from the applicant and its collaborators indicates a greater potential of success and sustainability of the project 15

16. Award Information • Type of Award: Cooperative Agreements • Number of awards: Up to 150 • Project Period: Up to 5 years • Funding Range: $400,000 - $4,000,000 • Start Date: no later than September 30, 2010 • Applicants may only submit one application for consideration 16

17. Any Questions? April 14, 2010 17

18. Identification of Evidence-based Programs Eligible for Replication

19. Identification of Evidence-based Programs • Independent, systematic review of evidence base conducted by Mathematica Policy Research, under contract to HHS • Steps of the Review • Find potentially relevant studies • Screen studies to review • Assess the quality of studies • Assess the evidence of effectiveness • More detailed information available at http://www.hhs.gov/ophs/oah 19

20. Finding Relevant Studies • Review of reference lists from earlier research syntheses • Search of relevant research and policy organizations’ websites • Public call for studies to solicit new and unpublished research • Keyword search of electronic databases 20

21. Screening Criteria • Use quantitative data and statistical analysis and hypothesis testing to measure impacts • Measure impact on at least one sexual risk behavior or its health consequences: sexual activity (initiation, frequency, number of partners), contraceptive use, sexually transmitted infections, pregnancies, or births • Focus on youth ages 19 or younger in the United States at start of program • Been conducted or published since 1989 21

22. Assessing Study Quality • High, Moderate, or Low Rating based on: • Study design - Reassignment • Attrition - Confounding factors • Baseline equivalence • High rating = random assignment studies with low attrition and no sample reassignment • Moderate rating = quasi-experimental designs with well-matched comparison groups at baseline; certain random assignments that didn’t meet all criteria for high rating 22

23. Assessing Evidence of Effectiveness • Supported by at least one high- or moderate-rated study showing a positive, statistically significant impact on at least one priority outcome (delay in sexual activity; increase in contraceptive use; decrease in STIs, pregnancies, or births) for either the full study sample or a key subgroup (defined by gender or baseline sexual experience) 23

24. Program Models Eligible for Replication • 28 Evidence-based program models currently identified as eligible for replication with Tier 1 Funding • http://www.hhs.gov/ophs/oah/prevention/research/programs/index.html • Appendix A of Funding Announcement • All studies that were reviewed but didn’t make the Tier 1 list and the rationale for why the study didn’t meet the review criteria are available in a searchable database at www.hhs.gov/ophs/oah 24

25. Aban Aya Youth Project Adult Identity Mentoring (Project AIM) All4You! Assisting in Rehabilitating Kids (ARK) Be Proud! Be Responsible! Be Proud! Be Responsible! Be Protective! Becoming a Responsible Teen (BART) Children’s Aid Society (CAS) – Carrera Program Comprehensive Abstinence and Safer Sex Intervention ¡Cuídate! Draw the Line/Respect the Line FOCUS HIV Risk Reduction Among Detained Adolescents Horizons It’s Your Game: Keep it Real Making a Difference! Making Proud Choices! Project TALC Promoting Health Among Teens! Reducing the Risk Rikers Health Advocacy Program (RHAP) Safer Sex Seattle Social Development Project SiHLE Sisters Saving Sisters Teen Health Project Teen Outreach Program What Could You Do? Program Models Currently Eligible for Replication

26. Where to Learn More About the 28 Program Models • Intervention Implementation Reports on OAH website (www.hhs.gov/ophs/oah) • Intervention Name and Developer • Program Description • Target Population • Curriculum Materials • Training and Technical Assistance • Research Evidence

27. Can an organization apply to replicate program models not in Appendix A? Yes, but only if all of the following criteria are met: • Research or evaluation of program model was not previously reviewed (previously reviewed evidence will not be re-reviewed) • Research or evaluation meet the screening and evidence criteria • Application must include all relevant research and evaluation (not included in 100 page limit) • Application must be received by May 17, 2010 27

28. Any Questions? April 14, 2010 28

29. Program Expectations

30. Overview of Program Expectations • Implement an evidence-based program model • Maintain fidelity to the program model • Address the target population • Ensure medical accuracy and age-appropriateness • Engage in phased-in implementation period • Collect and report performance measurement data • Adhere to evaluation expectations 30

31. Maintaining Fidelity to Program Model • Must maintain fidelity to the “core components” of the original evidence-based model that led to the outcomes associated with the program • “Core Components” are those parts of the curriculum or its implementation determined by the developer to be the key ingredients related to achieving the outcomes associated with program 31

32. Can the Program Models be Adapted? • Minimal adaptations are allowed: • Changing names or details in a role play • Updating out-dated statistics • Adjusting reading and comprehension levels • Making activities more interactive • May propose adaptations to make program more relevant to ethnic, racial, or linguistic characteristics of the population to be served as long as core elements aren’t affected • Significant adaptations (adding activities, changing sequence of activities, replacing supplementary materials) are not allowed under Tier 1 32

33. Demonstrating Ability to Replicate Program Models with Fidelity Demonstrate effectiveness of replication strategy Ensure facilitators delivering program have been or will be formally trained Receive training on acceptable adaptations or propose adaptations for approval Monitor and document program implementation to ensure fidelity Provide MOUs stating that all partners have agreed to implement with fidelity April 14, 2010 33

34. Medical Accuracy & Age-Appropriateness • Funded programs will need to ensure that information provided is age appropriate, and scientifically and medically accurate • Materials will be reviewed by OAH prior to implementation to ensure medical accuracy • Full curricula should NOT be submitted with the application. Program materials will be submitted to OAH for review and approval during the phased-in implementation period of the first grant year. 34

35. Phased-In Implementation Period • Funded recipients will engage in a planning, piloting, and readiness period for the first 6 to 12 months of funding • During this period, grantees will: • Continue to assess needs and resources • Finalize goals, objectives, and logic model • Assess program fit • Build organizational capacity • Finalize implementation plans • Pilot test program • The length of the phased-in implementation may vary by grantee depending on implementation readiness • OAH approval is required before full-scale implementation 35

36. Evaluation Strategies • Monitoring and reporting on program implementation and outcomes through performance measures for all grantees • Grantee-level evaluations for projects funded in Ranges C and D • Federal evaluation of a selected subgroup of all grantees 36

37. Performance Measures • Developed by OAH during first year of program • All grantees will be expected to collect and report on common set of performance measures to assess program implementation and outcomes • Training and technical assistance will be provided by OAH • Anticipated categories for measures: • Output measures • Fidelity/adaptation • Implementation and capacity building • Outcome measures • Community data 37

38. Evaluation for Funding Ranges A & B Must be able to demonstrate ability to collect and report on common set of performance measures to assess: program implementation and outcomes Not expected to conduct rigorous independent grantee-level evaluation April 14, 2010 38

39. Evaluation for Funding Ranges C & D • Rigorous independent grantee-level evaluation design unique to proposed project • Use either random assignment or quasi-experimental design • OAH will review and assess proposed evaluation designs • OAH approval required before implementing evaluation plan • OAH will provide training & TA on evaluation – general & project specific. • See Appendix C in the FOA for detailed guidance • Budget 20-25% to support evaluation activities, but not more than $500,000 per year 39

40. Federal-level Evaluation of TPP • Grantees from all funding ranges are required to participate in Federal evaluation, if selected • Will agree to follow all evaluation protocols established by HHS • Will no longer be expected to have separate grantee-level evaluation and will be required to redirect evaluation budget to support activities related to the Federal-level evaluation • Decisions of which grantees will participate in the Federal evaluation will be made by end of the 1st grant year 40

41. Items Requiring OAH Approval • OAH approval required for: • Medical accuracy of curricula and program materials • Proposed adaptations • Evaluation plans • Full-scale implementation • Full curricula should NOT be submitted with the application. Program materials will be submitted to OAH for review and approval during the phased-in implementation period of the first grant year. April 14, 2010

42. Any Questions? April 14, 2010 42

43. Application Contents

44. Contents of Application Submission • Abstract (one-page) • Project Narrative (no more than 50 pages) • Organizational Capability Statement • Project Management • Need Statement • Model to be Replicated and Project Approach • Target Population • Program Goal(s), Objectives, and Activities • Work plan and Timetable • Collaborations and Description of MOUs • Performance Measurement • Evaluation • Appendices • Budget Narrative/Justification No more than 100 pgs 44

45. Project Abstract (1 page) • Clear, concise description of the project that can be understood without reference to other parts of the application. Should include: • Project title • Applicant contact information • Type of organization applying • Overarching goal(s) • Evidence-based program model to be replicated • Geographic area to be served • Target population 45

46. Project Narrative Formatting • Double-spaced • 8 ½ x 11 inch (letter-size) pages • 1-inch or larger margins on top, bottom, and both sides • At least 12 point font • All pages, charts, figures, and tables should be numbered 46

47. Project Narrative:Organizational Capability • Current capability to organize and operate effectively and efficiently • Decision-making authority and structure • Organization’s experience, expertise and previous accomplishments in the area of teen pregnancy prevention • Previous partnerships and strategies used to address teen pregnancy prevention • How various sites and outside resources will be managed logistically and programmatically 47

48. Project Narrative:Project Management • Plans to govern and manage the execution of the overall program • Governance structure, roles/responsibilities, operating procedures, composition of committees, workgroups, terms and associated leaders, and communications plans • How plans and decisions are developed and documented and issues/risks managed • Specify mechanisms to ensure accountability among community participants and incremental progress in achieving milestones 48

49. Project Narrative:Need Statement • Geographic area to be served • Benefit for the target population • Incidence of teen births in the area • Documentation of sexually transmitted disease rates, socio-economic conditions including income levels, existing services and unmet needs in the service area • Unique challenges and barriers facing proposed population 49

50. Project Narrative:Model to be Replicated • Rationale for choosing program model for replication and how approach is based on previous practice and community needs assessment • Lessons learned from previous projects of this type • Implementation site(s) • Plans to implement model with fidelity • Proposed adaptations to program model • Plans to train staff and obtain implementation materials • Plans to coordinate, integrate, and link to existing services within the service area • Program management plan 50