Jamelle E. Banks, MPH Sarah Glavin, PhD

1. It’s a Small World After All: Describing and Assessing NIH Funded Research in the Context of a Scientific Field Jamelle E. Banks, MPH Sarah Glavin, PhD Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD), NIH

2. NIH v. the World: What that Comparison Will NOT Allow You to Do • Are we doing a good job selecting investigators, projects, etc.? • Does the work we support meet quality standards? • Is a P01 better than 3 R01s? • If we issued a grant and nobody published, would it make a sound? • Should we support more research in area X v. area Y?

3. NIH v. the World: What that Comparison WILL Allow You to Do • What is the NIH “market share”? • Who are the other funders in the field? • What do the other funders support, compared with NIH? • Overlap/duplication • Areas to collaborate and potential collaborators • Where are the gaps in the research? • Where are most of NIH publications along the research continuum (basic, clinical, translational)? • Who is using NIH findings?

4. NIH v. the World: How do you do it? • Start with “the world”, and then pull NIH as a subset OR • Start with NIH, then search for “the world”

5. NIH v. the World: How do you do it? • Start with “the world”, and then pull NIH as a subset • Advantages: • Conceptually cleaner/less biased • Less time and resources • Disadvantages: • Difficult to characterize an NIH program as such, because you often do not capture the whole program • Most likely to miss more basic research, so may understate NIH impact

6. NIH v. the World: How do you do it? • Start with NIH, then expand to “the world” • Advantages: • Easier to check whether searching “the world” properly captures NIH publications • Can build criteria for “the world” search to best match NIH program • Disadvantages: • More time and resource intensive • May be biased in favor of including NIH research

7. An Example: The SCCPRIRs Program • Specialized Cooperative Centers Program in Reproduction and Infertility Research (SCCPRIRs) • 16 center grants, $19.6 M in FY 2008 from NICHD • Largely basic science research in infertility and reproduction and some clinical research • Annual publications range from 108 to 170 over past 6 years

8. An Example: The SCCPRIRs Program • How do SCCPRIRs fit into the world of infertility and reproduction research? • Develop list of SCCPRIRs publications from progress reports and grant acknowledgements – n=121 • Use SCCPRIRs keywords to search ISI database for other literature in the field • The world’s n ranged from 6,316 to 30,880 articles (depending on search parameters)

9. An Example: The SCCPRIRs Program • SCCPRIRs “market share” 0.4% to 2% • Areas of research and journals were fairly similar between SCCPRIRS and the broad literature • Broad literature included more publications specifically focused on animal reproduction • Broad literature emphasized reproductive cancers to a greater degree than SCCPRIRs program

10. An Example: The SCCPRIRs Program

11. An Example: The SCCPRIRs Program • Other funding sources for reproduction and infertility research • SCCPRIRs grants are not co-funded, but SCCPRIRs researchers received other NIH funding from 7 ICs • Non-NIH research projects on reproduction and infertility were funded by: • Private foundations (small numbers each); • Japan’s Ministry of Science, Education, and Culture • Japan’s Society for the Promotion of Science (quasi-governmental) • Small number of private industry sources

13. Projected v. Actual Citations for SCCPRIRs-Associated Articles, 2007

14. NIH v. the World: How Could You Use These Type of Results? • Identify collaborators • Look for possible overlap and duplication • Identify research gaps • Help identify impact of research results • Develop more realistic expectations and understanding of NIH’s role in the larger research community

15. Thank you! • Questions? • Contact Information: • Sarah Glavin – glavins@mail.nih.gov • Jamelle Banks – banksj@mail.nih.gov