1 / 17

NIH Loan Repayment Program Evaluation: Achievements and Impact Analysis

This report evaluates the impact of NIH Loan Repayment Programs on career outcomes and compares them to other NIH award programs, analyzing applicant demographics, success rates, debt loads, and grant participation trends from FY03-FY07. Findings include insights on early career researcher recruitment, gender, race/ethnicity distribution, and debt disparities across various degree types. Outcome analysis using regression discontinuity method compares funded and unfunded applicant cohorts, showing the program's influence on grant participation and career progression within the NIH extramural workforce.

lalma
Download Presentation

NIH Loan Repayment Program Evaluation: Achievements and Impact Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. NIH LOAN REPAYMENT PROGRAM EVALUATION NIH Loan Repayment Programs Extramural LRPs FY 2003-2009 Milton J. Hernández, Ph.D. Director Division of Loan Repayment OEP, OER, OD National Institutes of Health Bethesda, MD Laurel Haak, Ph.D. Chief Science Officer Discovery Logic, Inc. Rockville, MD

  2. NIH LRP Evaluation Questions • Question 1: Who is applying to the NIH LRP? • Question 2: Does participation in the LRP affect career outcomes? • Question 3: How does the LRP compare to other NIH career development award programs? NIH Loan Repayment Programs

  3. New and Renewal LRP Applications and Awards Trends Analysis, FY03-FY07 RENEWAL NEW The renewal program has decreased the success rate for new applications but has not affected the number of new applications submitted. The success rate for new applications from FY05-FY07 is 40% and for renewals is 70%. NIH Loan Repayment Programs

  4. Gender Distribution of New LRP Applicants Trends Analysis, FY03-FY07 • The ratio of women to men LRP applicants is reversed from the ratio in recent graduating MD and PhD classes. • Success rates are not different. NIH Loan Repayment Programs

  5. Gender Distribution of New LRP Applicants, by Degree Trends Analysis, FY03-FY07 About the same number of men and women applicants are MDs, but 2.5 times more women applicants are PhDs, and twice as many men applicants hold dual degrees. p<0.001 p<0.001 NIH Loan Repayment Programs

  6. Race/Ethnicity of New LRP Applicants Trends Analysis, FY03-FY07 • More Blacks and fewer Asians applied as compared to recent MD and PhD graduating classes. • Success rates did not differ from each other or from the overall success rate. NIH Loan Repayment Programs

  7. Educational Debt, by Degree Type Trends Analysis, FY03-FY07 Among LRP applicants, academic doctorates have a higher debt load than recent PhD graduates, and physician doctorates have a lower debt load than recent MD graduates. *Source: Survey of Graduates, American Association of Medical Colleges (2003-2007). †Source: Survey of Earned Doctorates, National Science Foundation (2003). Prepared by NIH OER/OEP/DLR

  8. Years Since Degree Trends Analysis, FY03-FY07 Up to 3 Years (35%) 4-6 Years (37%) 10+ Years (10%) 7-9 Years (18%) • LRP attracts early career researchers • 72% of new applicants apply within 6 years of receiving their degree • 90% of new applicants apply within 9 years of receiving their degree • Academic doctorates and physician doctorates apply at different times • 67% of academic doctorates apply within 3 years of their degree • 68% of physician doctorates apply 4-6 years after receiving their degree NIH Loan Repayment Programs

  9. NIH LRP Evaluation: Outcome Analysis • Question 1: Who is applying to the NIH LRP? • Question 2: Does participation in the LRP affect career outcomes? • Question 3: How does the LRP compare to other NIH career development award programs? NIH Loan Repayment Programs

  10. Outcome Analysis Methodology • Outcome analysis: Regression Discontinuity Analysis. • We compared two nearly-identical applicant groups. LRP applicants with very similar priority scores were identified, about half of whom were funded and half of whom were not funded. Priority Score “Bubble” Funded Not Funded NIH Loan Repayment Programs

  11. Identifying the Comparison Cohort LRP Applicant Cohort, FY03-FY06 Funded Priority Score range, e.g. 190-214 Identify bubble applicants foreach IC and each fiscal year Not Funded F259 Combine individual bubbles NF254 Outcome Analysis Subsequent publications and NIH grant participation F8 F9 F8 F11 NF9 NF8 NF9 NF14 IC-FY Bubble513 NIMH, FY03 NIDDK, FY04 NICHD, FY04 NIAID, FY05 A 50-50 mix of funded and not funded applicants with closely-matched priority scores NIH Loan Repayment Programs

  12. IC-FY Bubble, Demographic Distribution Outcome Analysis, LRP Applicant Cohort, FY03-FY06 The demographics of the funded and not funded IC-FY Bubble cohorts are indistinguishable, hence they can be used as comparison groups. There are some notable differences compared to the total new applicant group.

  13. NIH Grant Participation: IC-FY Bubble Overall p=0.0102 13

  14. NIH Grant Participation: IC-FY Bubble, by Gender Outcome Analysis, LRP Applicant Cohort, FY03-FY06 Male Female p=0.0474 p=0.0147 Women LRP awardees are less likely than men to have applied for an NIH award or to have a subsequent observable role in the NIH extramural program. NIH Loan Repayment Programs 14

  15. NIH Grant Participation: IC-FY Bubble, by Degree Outcome Analysis, LRP Applicant Cohort, FY03-FY06 Physician Doctorate Academic Doctorate p=0.0324 p=0.0106 p=0.0470 Participation in the LRP program increased the likelihood that a physician doctorate applied for a subsequent NIH award and was retained in the NIH extramural workforce. It also increased the likelihood that an academic doctorate was awarded an NIH grant. NIH Loan Repayment Programs 15

  16. Question 3: Comparison with Other NIH Programs Identification of Comparison Cohorts K Cohort K01, K08, K23 LRP Cohort LRP+K Cohort Funded Type I awardees, FY03-FY06 Top 10% priority score, each FY/IC K awards only Funded Type I awardees, FY03-FY06 Top 10% priority score, each FY/IC L and K awards* Funded Type I awardees, FY03-FY06 Top 10% priority score, each FY/IC L awards only LRP+K Top-10% n = 96 LRP Top-10% n = 215 K Top-10% n = 218 Demographicparameters Researchaccomplishments Publications Outcome analysis Subsequent NIHgrant participation *L award between FY03-FY06 anda K01/K08/K23 award at any time. Awardees were in the top 10% of priority scores for both award mechanisms. NIH Loan Repayment Programs

  17. NIH Grant Participation: K and L Programs Outcome Analysis, 0-6 Years Since Degree, Top-10% Comparison p=0.0003 p=0.0176 p=0.0003 Even after balancing for years since degree, the LRP recipients show less likelihood of applying for subsequent NIH funding than K or LRP+K awardees. LRP awardees are also less likely than K awardees to have a subsequent role on NIH awards. NIH Loan Repayment Programs 17

More Related