1 / 28

Information Retrieval to Informed Action with Research Metrics

Information Retrieval to Informed Action with Research Metrics. Thomson Scientific Research Services Group 2007. Challenge of Managing and Evaluating Research. Science is very dynamic and growing exponentially Research is happening in a global and multidisciplinary context

nathaniel-carney
Download Presentation

Information Retrieval to Informed Action with Research Metrics

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Information Retrieval to Informed Actionwith Research Metrics Thomson Scientific Research Services Group 2007

  2. Challenge of Managing and Evaluating Research • Science is very dynamic and growing exponentially • Research is happening in a global and multidisciplinary context • Emerging fields make it difficult to determine relevancy • Selectivity becomes more challenging, e.g. how to find the right journals Responsibilities of leaders in research • Keep up to date on new and developing science • Nurture new research through direction and funding • Measure the impact of scientific efforts • Access timely, objective and informed information to make policy and budget decisions

  3. Metrics – How to measure science? • Scientists communicate through the literature in academic journals • Citations are considered the “coin of the realm” in science • Bibliometrics uses statistical analysis to draw a landscape of scientific communication and productivity. • Citation analysis: • examines the elements of bibliographic records (citations) • studies relationships between citations • measures the amount of scientific activity in a subject category, journal, country, topic or other areas of interest.

  4. Why Metrics? • Uses readily available data • Employs transparent and reproducible methodology • Provides quantitative information that policy makers can apply to their decision process “If you can measure that of which you speak, and can express it by a number, you know something of your subject; but if you cannot measure it, your knowledge is meager and unsatisfactory” - Lord Kelvin (William Thomson)

  5. Elements of Evaluation using Bibliometric Methods • Theory of citation: • Citations are a form of peer recognition • Basic measures: • Papers, citations, citations per paper (impact) • Relative measures: • impact relative to baselines; • expected citation rates for articles; • percentiles • Units of analysis: • papers, authors, institutions, countries, journals • Measures of interaction: • research fronts, interdisciplinarity, collaboration

  6. A National view of Turkish Research

  7. Economics informs but citation analysis provides more detail Turkey Turkish research motivation is seen in the OECD records on percentage of GDP dedicated to Research & Development. Turkey is surging and soon to exceed Portugal. Source: OECD Factbook 2008 Science and technology - Research and Development (R&D) - Expenditure on R&D

  8. Research Growth and Impact in Turkey, 1981-2007 Turkish research has grown steadily over the last 27 years. On average, Turkey produced over 14,000 scientific papers per year from 2003-2007, with large movement in aggregate citations between 2003 and 2007. Source: National Science Indicators, 1981-2007 – WOS version

  9. Total Papers Annual Research Growth in Turkey, 1981-2007 In 2003, Turkey broke through to go above 10,000 research papers produced in a single year. Source: National Science Indicators, 1981-2007 – WOS version

  10. Total Papers Research Growth in Turkey, 1981-2007, compared to Greece Among the countries in the region, Greece is the closest in publications but continues to lag behind Turkey’s strong growth trend. Source: National Science Indicators, 1981-2007 – WOS version

  11. % Papers In World Percentage of papers in the world, 1981-2007 Turkey also exceeds Greece in overall contribution to research in the world. Source: National Science Indicators, 1981-2007 – WOS version

  12. Avg. Cite Per Paper Baseline=1 Research in Turkey – Citation Impact Citation impact is the average citation rate: total cites/total papers. Turkey’s average citation rate has risen consistently since the mid-1990’s and is trending to continue its steep movement into the world scientific community. Source: National Science Indicators, 1981-2007 – WOS version

  13. % of Papers Cited Research in Turkey – Percentage Cited While publication growth is strong, Turkey is still working to exceed Greece in percentage of papers cited. Source: National Science Indicators, 1981-2007 – WOS version

  14. Turkey: Field Specialization The following 10 fields make up over 85% of Turkish scientific research Source: National Citation Report on the nation of Turkey (ESI), 1981-2007

  15. Turkey: Field Specialization Generally Turkey contributes 1.48% of all world papers in 2003-2007. The following fields exceed that level and indicate a national strength for Turkey in world contribution Field% papers in field in the world Agricultural Sciences 2.76 Clinical Medicine 2.62 Engineering 1.94 Plant & Animal Science 1.89 Field% papers in field in the world Ecology/Environment 1.59 Pharmacology 1.54 Materials Science 1.50 Source: National Science Indicators, 1981-2007, ESI version

  16. Turkey: International Collaboration Turkish research is written more often within the country. Here are the top 20 countries represented in Turkish collaborations since 2000. Source: National Citation Report on the nation of Turkey, 1981-2007

  17. Key Universities in Turkish Research

  18. University Publications These 17 universities published 1000 or more papers over the period 2002-2006. This illustrates academic contribution to the national performance Source: National Science Indicators customized for Turkish Institutions, 1981-2006

  19. Instanbul University Hacettepe University Ankara University Ege University Gazi University Growth of Publications in Specific Universities In the 2002-2006 period, these 5 leading Turkish universities fit into the mid-range of publication counts for the US, UK, Australia, and Canada university indicators. Source: National Science Indicators customized for Turkish Institutions, 1981-2006

  20. Relation of University Strengths to National Strengths Source: National Science Indicators customized for Turkish Institutions, 1981-2006

  21. Field Specialization, 2002-2006 Other fields of Turkish strength like Engineering are represented by a unique set of universities. Source: National Science Indicators customized for Turkish Institutions, 1981-2006

  22. Growth of Publications in Leading Technical Universities, 1981-2007 Highlight comparisons between Middle East Technical University and Instanbul Tecknick Source: National Science Indicators customized for Turkish Institutions, 1981-2006

  23. Engineering subfields at METU Source: National Science Indicators customized for Turkish Institutions, 1981-2006

  24. Engineering at METU: ChemE journals Source: National Science Indicators customized for Turkish Institutions, 1981-2006

  25. Engineering at METU: Authors in ChemE Source: National Science Indicators customized for Turkish Institutions, 1981-2006

  26. Engineering at METU: A well cited ChemE paper from Dr. T. Dogu Source: National Science Indicators customized for Turkish Institutions, 1981-2006

  27. Comments • Because science is such a dynamic enterprise, no one metric is adequate for evaluation. • Evaluation is the culmination of quantitative and qualitative considerations, within a particular environment. • Citation analysis can help highlight researchers and the work that impacts the overall impact of an organization, group, or nation. • Examines the elements of publıcatıon records • Studies relationships between citations • Allows for the measurement of the amount of scientific activity in varıous areas, e.g. Organızatıons, nıdıvıduals, journals, subject areas

  28. Thank you!

More Related