1 / 28

Program Level Assessment of Outcomes and Impacts of Research of Centers

This project aims to assess the outcomes and impacts of 15 Nanotechnology Science and Engineering Centers (NSECs) in relation to the National Nanotechnology Initiative (NNI) objectives and goals. The study examines publication output, co-authorship patterns, geographical spread, multidisciplinarity, and industry collaborations. Various data analysis methods are used, including bibliometrics, network analysis, GIS, and qualitative analysis.

crooks
Download Presentation

Program Level Assessment of Outcomes and Impacts of Research of Centers

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Program Level Assessment of Outcomes and Impacts of Research of Centers Juan D. Rogers School of Public Policy Georgia Tech Project: Assessment of 15 Nanotechnology Science and Engineering Centers’ (NSECs): Outcomes and Impacts: Their Contribution to NNI Objectives and Goals, NSF 0955089.

  2. Objectives • Publication output and impact of the NSECs • Co-authorship patterns change over time • Geographical spread or concentration of NSEC research collaboration • Evolution of areas of research as reflected in keyword changes • Multidisciplinary footprint of the centers • The case for centers AEA, Annaheim CA

  3. Methodology Outline • Qualitative-quantitative concurrent design • Qualitative component • Multiple embedded case studies of mechanisms for center outcomes • Follow up field level impact with quantitative analysis • Quantitative component • Bibliometrics (productivity, citation, co-authors, etc.) • Personnel and funding data analysis • Intellectual property instances (patents, licenses, etc.) • Links with business and commercial applications • Methodologically: Final results are generalizations to theory from qualitative analysis • We offer analysis of combined quant-qual data • Interpretive schemes for quantitative findings on centers AEA, Annaheim CA

  4. Data and methods • Acquisition of 85 center annual reports from all 15 NSEC centers • Extraction and clean up (duplicate removal) of publication lists • Extraction of NSEC articles from Web of Science (n=3,500) • Look up and extraction of articles citing NSEC articles (n=75,000) • Clean up and classification of collaborating organizations • Clean up, look up, identification, and matching of NSEC authors in author listings • Analyses of different type, multiple tools: • Growth, shares, and overall trends (tabular) • Networks and collaborations (Gephi) • Geographical spread, GIS (ArcGIS) • Keywords and topics (VantagePoint) • Multidisciplinarity and science maps (Pajek) AEA, Annaheim CA

  5. Publications NSEC publication activity grows in three waves 2001-04 2005-06 2007-08 Notes: *Publication data not reported by all NSEC centers; last column reports average annual change for rows with change data. Source: ISI-WoS publication data based on NSEC annual reports by center. AEA, Annaheim CA

  6. Industry collaborations One out of 10 NSEC publications has an industry co-author Total unique firms co-authoring articles with NSEC (2001-2010): 146 Total unique firms maintaining other types of collaborations with NSEC (as of 2010): 275** Notes: * Publication data not reported by all NSEC centers; last column reports average annual change for rows with change data. **The type of collaborations are not specified by centers (only number of industry partners was provided). Source: ISI-WoS publication data based on NSEC annual reports by center and lists of industry partners provided by NSEC centers. AEA, Annaheim CA

  7. Comparative Impact of NSEC Papers (Citations Cohort 2001) AEA, Annaheim CA

  8. Comparative Impact of NSEC Papers (Citations Cohort 2002) NSEC papers appear to have higher impact measured by citations: median and mean citations grow faster than the cohort with window length AEA, Annaheim CA

  9. Rank in Cohort of Top 20 NSEC Papers NSEC papers rank highly in their cohort of Nano papers. Total Cohort 2001: 30462 papers. NSEC Cohort 2001: 66 papers Total Cohort 2002: 34971 papers. NSEC Cohort 2002: 128 papers AEA, Annaheim CA

  10. Comparative Impact of NSEC Papers (Citations Cohort 2003) The distributions parameters show NSEC high position in the field but the top paper is still not by the centers. N = 40813 CN = 222 AEA, Annaheim CA

  11. Comparative Impact of NSEC Papers (Citations Cohort 2004) N = 48952; CN = 259 AEA, Annaheim CA

  12. Comparative Impact of NSEC Papers (Citations Cohort 2005) N = 55998; CN = 499 AEA, Annaheim CA

  13. Comparative Impact of NSEC Papers (Citations Cohort 2006) N = 62351; CN = 512 AEA, Annaheim CA

  14. Increasing JIF of Target Journals AEA, Annaheim CA

  15. Increasing JIF of Target Journals AEA, Annaheim CA

  16. Leveraging Support Reflected in Publication Support Acknowledgment AEA, Annaheim CA

  17. Co-authorships NSEC co-authorship networks grow and become more widespread Co-authorships 2001-2006 Co-authorships 2007-2010 Notes: Nodes represent authors. Node size represents number of publications for the period. Node colors represent 15 NSEC centers. Line colors are those of the centers that maintain each co-authorship as found in publication databases. Source: ISI-WoS publication data based on NSEC annual reports by center. AEA, Annaheim CA

  18. Co-authorships Multiple productive and collaborative authors NSEC collaboration networkspresent diverse patterns of co-authorship (2001-2010) Central and well connected authors Highly collaborative projects AEA, Annaheim CA

  19. Collaboration and places Wide geographic spread of NSEC research with concentration in specific locations/regions Note: Number of NSEC publications from 2001-2010 = 3509; number of citing publications = 75335. Citing publications, 2001-2010 exclude all NSEC publications. AEA, Annaheim CA

  20. Collaboration and places Co-authoring extends beyond the NSECs to nearly all US states AEA, Annaheim CA

  21. Research topics • Clusters of top keywords • (21 cluster solution reported) More specialized terms More linked terms

  22. Research topics Top 20 Terms Across 15 NSECs(relative position of 30 most common, 2001-2006 v. 2007-2010) AEA, Annaheim CA

  23. Agriculture Environ Sci Geosciences Ecology Chemistry Clinical Med Matls Sci Engineering Neurosciences General Med Computer Sci. Physics Source: Rafols, I. and Meyer, M. (2009) Diversity and Network Coherence as indicators of interdisciplinarity: case studies in bionanoscience. /Scientometrics/, 81(2), in print; Leydesdorff, L. and Rafols, I. (2009) A Global Map of Science Based on the ISI Subject Categories. Journal of the American Society for Information Science and Technology, 60(2), 348-362. Map of Science: Subject Categories within all Nano(2009) AEA, Annaheim CA

  24. Multidisciplinarity NSEC research is multidisciplinary with focus areas in materials science, chemistry and biomedical sciences Ecology Agriculture Geosciences Global Map of Science, 2006 175 SCI Subject Categories Infectious Diseases Biomed Sci Environ Sci Clinical Med Chemistry Matls Sci Engineering Neurosciences Physics Computer Sci. General Med Map source: Rafols, I., Meyer, M. (2009) Diversity and Network Coherence as indicators of interdisciplinarity: case studies in bionanoscience. Scientometrics, 81(2), in print; Leydesdorff, L., Rafols, I. (2009) A Global Map of Science Based on the ISI Subject Categories. Journal of the American Society for Information Science and Technology, 60(2), 348-362. AEA, Annaheim CA

  25. Multidisciplinarity A range of disciplines beyond focal areas cite NSEC works Ecology Agriculture Geosciences Global Map of Science, 2006 175 SCI Subject Categories Infectious Diseases Biomed Sci Environ Sci Clinical Med Chemistry Matls Sci Engineering Neurosciences Physics Computer Sci. General Med AEA, Annaheim CA

  26. Conclusions Implications for Center Policy • NSEC publication growth rate indicates rapid take-off by new centers. • NSEC mechanism allows for greater involvement of authors over time in diverse locations. • NSEC research involves some foundational  as well as some emerging (and some maturing) topical areas. • NSECs work in multiple disciplines and their citation influence involves many more disciplines (especially in Biotechnology related fields). AEA, Annaheim CA

  27. The Case for Centers • Reporting by PIs on crucial role of centers: • Unique incentive to go deeply across disciplinary boundaries • Significantly reduced cross-disciplinary transaction costs • Unique research experience for graduate students • Rich and diverse research infrastructure enables more risk taking • Exposure to greater number of high quality scientific contacts • Exposure to unique industry contacts • Accelerator of promising young researcher development • Center as recruiting tool of top talent • Diverse mentoring opportunities for rapid career development • Unique infrastructure possibilities • Leverage of resources for shared new facilities • Design of new unique instruments and experimental arrangements AEA, Annaheim CA

  28. Some Program Challenges • Some scientific contributions are difficult to explain to the lay public • Important for long term support of the enterprise • The special benefits for development of faculty and graduate students should be scaled up to reach the rest of the university community • The pre- and extra- university education efforts are dependent on the center program and are difficult to institutionalize without it. • Their sustainability should be a program concern • Similar sustainability issue is raised by specialized infrastructure that may go to waste if centers are discontinued AEA, Annaheim CA

More Related