1 / 17

Barry Bozeman, Arizona State University Jan Youtie, Georgia Institute of Technology USA

Credibility and Use of Scientific and Technical Information in Policy Making: An Analysis of the Information Bases of the National Research Council’s Committee Reports. Barry Bozeman, Arizona State University Jan Youtie, Georgia Institute of Technology USA. Motivation.

zaynah
Download Presentation

Barry Bozeman, Arizona State University Jan Youtie, Georgia Institute of Technology USA

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Credibility and Use of Scientific and Technical Information in Policy Making: An Analysis ofthe Information Bases of the National Research Council’s Committee Reports Barry Bozeman, Arizona State University Jan Youtie, Georgia Institute of Technology USA

  2. Motivation • Contribution: little empirical understanding of use of STI • Much literature on use of formal information in decision-making • No literature on use of STI in science, technology and innovation (S&T) policy • Research questions for study • Does the perception of the limited use of formal scientific and technical information (STI) accord with empirical reality? • What types of information “compete” with STI for inclusion in science policy-making (where one might expect greatest receptivity)? • How does the choice of various types of information relate to the use and impacts of science policy reports and recommendations?

  3. Definition of STI • Open scientific and technical literature appearing in peer-reviewed academic journals or proceedings. • STI used in a narrow sense v. typical in the literature (McClure, 1988; Walker and Hurt, 1990)

  4. National Research Council (NRC) • Performs research work for the production of reports on science and technology issues within the National Academies • National Academy of Science (NAS) • National Academy of Engineering (NAE) • Institute of Medicine • National Academies serves as advisor about science and technology intensive policy issues to Congress • Little research on the NRC • Ellefson (2000): single case on non-forest federal land management • Policansky (1999): an ecologist and staff member at the NRC, argues that well-constructed committees with high level of trust given a precisely constructed policy question are most successful • Shapiro and Guston (2006): bureaucracies will shirk their duties, relying on the peer review process to for correction • Fein (2011): NRC plays an increasingly important role in regulatory peer review • Parascandola (2007): conflict of interest policy history in the NAS and NRC • Martin and Irvine (1989): lack of priority setting in NRC reports

  5. National Research Council Process National Academies (2006) Our Study Process: Ensuring Independent, Objective Advice.

  6. Group-based Credibility Model

  7. Sample • All National Academies reports published 2005-2012 • Exclusion of workshop, narrow/very particular studies: Transportation Bureau (NAE), Health and Safety (IOM), repeat Congressionally authorized standing studies • Focus on board appointed/empaneled single shot studies (mostly NAS) • Results=589 reports

  8. Method • Report variables: publication year, policy area, National Research Council department, Congressional authorization, # pages, # committee members, # reviewers by sector (business, government, academia, etc.) and geographic location (domestic, foreign), # cited references, # congressional briefings • Committee variables: chair versus member, terminal degree and concentration, industry affiliation by three-digit NAICS, sectoral affiliation (business, government, academia, etc.), title (e.g., president, secretary of agency, gov’t laboratory director) • Committee publications: # publications in the Web of Science and Scopus, # citations, primary Web of Science category, self-citation in reference list • Report outcomes: # Web of Science and Scopus papers, newspapers, industry trade press, newsletters, magazines, news transcripts, hearing transcripts, Congressional record daily issues, bills, House and Senate reports, Congressional Research Service reports. • Reference variables: STI (journal article, published proceeding) v. government report/document, book or book chapter, working paper, industry and nonprofit organization document, interview/testimony, website, expert opinion

  9. Database Linkages • NRC annual reports • Web of Science • Scopus • LexisNexis (news, web) • Proquest, UNT Digital Library (hearings, legislation, bills, Congressional reports)

  10. Preliminary Findings: Reports • 14% Congressionally authorized • Natural Resource and Defense areas most common and most likely Congressionally authorized (~ 30% of studies in these areas)

  11. Average Number of Pages Declining • Average number of pages=188

  12. Policy Areas Vary in Use of References in NRC reports (2005,2006)

  13. Preliminary Findings: Outcomes • NRC reports have greater impact than expected • 37% self-reported Congressional communications • Informal briefings (30%) more common than formal testimony (15%) • 20% reports have independent presence in Congressional documents • Not necessarily overlapping with self-reports (only 28% of reports with any Congressional impact are independent + self reports) • Taken together 45% of NRC reports have either (or both) of the above impacts

  14. Preliminary Findings: Reports and Outcomes • ~ over time (except for independent outcomes being much higher, and self-reported outcomes lower, in 2005) • Congressionally authorized studies are associated with self-reported briefings (29% versus 6% for nonauthorized) and independent outcomes (26% versus 12% for nonauthorized) • Longer NRC studies tend to have self-reported briefings and independent outcomes (briefings=216 mean # pages versus 171 for no briefings; outcomes=213 mean versus 181 for no outcomes)

  15. Preliminary Findings: References and Outcomes • Inclusion of references makes little difference to outcomes

  16. Plans and some issues • Planned analysis Impact f(STI, active researchers on committee, other committee information, report characteristics) • Limitations • Not every NRC report has references; reference formats differ • We are not analyzing report content • We would like to do some interviews

  17. Acknowledgements • This work was supported by the US National Science Foundation, Science of Science and Innovation Policy, Award #1262251. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the sponsors. • For more information http://stip.gatech.edu/credibility-and-use-of-scientific-and-technical-information-in-science-policy-making/

More Related