1 / 80

Gary Liang Thomson Scientific Regional Manager Gary.liang@thomson

「發現經典，追求卓越」系列講座 ( 二 ) Bibliometrics & R esearch E valuation: multiple measures to maximize our potential. Gary Liang Thomson Scientific Regional Manager Gary.liang@thomson.com. 學術評鑑制度的建立. 上海交通大學高等教育研究所「世界大學學術排行榜」 10% - 校友得諾貝爾獎、費爾茲獎 (Fields medals) 得獎人數

maggy-schultz
Download Presentation

Gary Liang Thomson Scientific Regional Manager Gary.liang@thomson

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 「發現經典，追求卓越」系列講座 (二)Bibliometrics & Research Evaluation:multiple measures to maximize our potential Gary Liang Thomson Scientific Regional Manager Gary.liang@thomson.com

  2. 學術評鑑制度的建立

  3. 上海交通大學高等教育研究所「世界大學學術排行榜」上海交通大學高等教育研究所「世界大學學術排行榜」 10% - 校友得諾貝爾獎、費爾茲獎 (Fields medals) 得獎人數 20% - 任職本校時得諾貝爾獎、費爾茲獎 (Fields medals) 得獎人數 20% - 各學科中 Highly Cited「高度被引用」的教師人數 (資料來源 Thomson Scientific – Highly Cited.com) 20% - 發表於SCIE、SSCI、AHCI 的論文數 20% - 發表於「Nature」或「Science」雜誌的論文數 10% - 上述指標得分的教師數 國內外學術評鑑 – 回顧 (1)

  4. 英國泰晤士報高等教育增刊 (The Times Higher Education Supplements, THES) 「全球前兩百名大學排行」評比指標 40% - 同儕審查(Peer Review) 20% - 每位教師論文平均被引次數(Research Impact) 20% - 師生數比(Faculty-to-student Ratio) 10% - 招募者的意見(Recruiter Review) 5% - 國際教師數(International Faculty) 5% - 國際學生數(Overseas Students) 國內外學術評鑑 – 回顧 (2)

  5. 國內外學術評鑑 – 回顧 (續) 英國學術研究評估（Research Assessment Exercise)美國新聞與世界報導（US News and World Report） 美國最佳大學排名 (America’s Best Colleges 2007)美國新聞週刊（Newsweek）「世界前一百大排名 」 …… ……

  6. First and Foremost…

  7. Research Evaluation - process • Data Source – (e.g. SCIE, SSCI, etc.) • Reliable and Unbiased Data • Comprehensive Coverage • Quality and Steady Collections • AnalyticResults – • Nation(e.g. Essential Science Indicators, National Science Indicators) • Institution (e.g. Institutional Citation Reports, University Science Indicators) • Researcher (e.g. Personal Citation Reports) • Peer Review Process – Important!!

  8. Peer Review System • Experts reviewing the work of their colleagues for decisions • Publication • Funding • Promotion, etc. • Known problems • “Old-boy” network • Bias by past achievements, award history, etc. • Peer review is still foundation of evaluation, and should NOT be replaced by quantitative analysis per se.

  9. Why bibliometric indicators? • Science has grown faster than our ability to support it • Quantitative Analysis: an attempt to make human decisions more rational, scientific • Peer review system CAN be enhanced with good supporting evidences.

  10. Who needs bibliometric indicators?

  11. Evaluation by evaluators • Bibliometric studies conducted by governments, research funding agencies, or accreditation bodies are: • Selective • Because they have limited resources to allocate. • Simple (or sometimes simplistic) • Because they need to reach general consensus. • Cost-bound • Because their time & cost for evaluation work is often limited.

  12. Evaluation for researchers • Bibliometric approach CAN be used for empowering researchers: • Strengths • Of authors, institutions, or countries • Self-appeal • To evaluators, open public • Strategies • In research, organizational policies

  13. General reminders General, but IMPORTANT!!!

  14. 10 rules for publication and citation analysis • Consider whether available data can address question. • Choose publication types, field definitions, and years of data. • Decide on whole or fractional counting. • Judge whether data require editing to remove “artifacts” • Compare like with like. • Use relative measures, not just absolute counts. • Obtain multiple measures. • Recognize skewed nature of citation data. • Confirm data collected are relevant to question. • Ask whether the results are reasonable. Compiled and presented by David Pendlebury, Thomson ISI Symposium, Oct 2002, Tokyo

  15. Variety of indicators • Publication count • Citation count • Time-series data • Field-sensitive data • Relative indicators • Co-citation analysis • Collaboration analysis

  16. Examples

  17. Example (1): Time-series data Courtesy of National Institute of Materials Science, Japan Essential Science Indicators, January 1995 – June 2005

  18. Example (2): Time-series data Which research(ers) should be funded? Custom Citation Report [sample]

  19. Example (3): Field-sensitive data University Science Indicators 1981-2005 Deluxe version

  20. Example (4): Field-sensitive data Courtesy of Prof. Teng Ching Kung, Hong Kong Polytech Univ Web of Science, analyzed on May 15, 2006

  21. 從學校整體的表現不易察覺其中表現優異的學科領域從學校整體的表現不易察覺其中表現優異的學科領域 • 強調優勢的學科領域以獲取最大之研究資源及成果

  22. Taiwan appears in all 22 fields of Essential Science Indicators country ranking. Taiwan ranks 18th by number of publication, published during January 1996 – August 2006. Among Asian countries,Taiwan ranks 5th by number of papers; but ranks 2nd by citations per paper. Example (5): Relative Indicators Essential Science Indicators, January 1996 – August 2005

  23. Number of papers by field • Clinical Medicine, Engineering, Physics are the top 3 fields in terms of “number of papers” published in the last 5 years. Chemistry, Materials Science, Biology & Biochemistry follow. National Science Indicators 1981-2005 Standard Edition

  24. Impact relative to the world • Agricultural Science is the strongest in terms of “citation impact” relative to the world. Geosciences, Materials Science and Mathematics follow. National Science Indicators 1981-2005 Standard Edition

  25. Institutional analysis by top 1% papers • Highly Cited Papers • Top 1% papers determined by category and by publication year • Among 82,989 Highly Cited Papers (2006 3rd update), there were 494 papers with at least one “Taiwan” in author address. • Analyzed by institution, the following is the top 5 in Taiwan. Essential Science Indicators, 2006 3rd update

  26. Example (6): Relative Indicators Essential Science Indicators, January 1995 – December 2005

  27. A P B A B C Example (7): Co-citation AnalysisResearch Front (研究前鋒) • When paper A and B are“co-cited” by paper P, A and B are likely to havetopical relevance. • When this “co-citation” isfrequent, it forms a groupof papers that are “topicallyassociated” to one another. Small, H. Co-citation in scientific literature: new measure of relationship between two documents.” Journal of the American Society for Information Science, 24 (4): 265-269, 1973.

  28. There are approx. 120,000 papers published from Taiwan in 1996-2006. 494 papers are ranked within the top 1% by year and by category. With which countries did Taiwan produce most number of such papers? See right. Example (8): Collaboration analysis Source: Institutional Citation Report, January 1996 – August 2005

  29. Example (9):H-Index for author

  30. 透過引文分析探討學術表現 - 以國立成功大學為例

  31. 成功大學名人堂 • 從基礎科學指標 (Essential Science Indicators, ESI) • Thomson Scientific Citation Laureate Award 2006 • ESI Top Papers Which SPECIALTIES is Natl Cheng Kung Univ contributing most? Who are the leading researchers in those SPECIALTIES?

  32. 張定宗 教授國立成功大學 醫學院內科學科 • 研究專長: 內科、腸胃肝膽科、肝炎、病毒學 • 榮譽、獎項及其他有關事項: • 中華民國消化系醫學會優秀論文獎(1993) • 國立成功大學醫學院優秀論文獎(1996) • 宋瑞樓教授學術基金會優秀論文獎(1996) • 亞太地區B型肝炎醫療策進會委員(2004~2006) • 台灣地區B型肝炎醫療策進會首任會長(2004~2006) • 國立成功大學發表國際頂尖期刊獎勵(2006) • 台灣消化系醫學會第36屆年會會長(2006/3/17~19) • 國家衛生研究院第五屆病毒性肝炎及肝癌會議召集人(2006/3/24~25) • 現職 • 國立成功大學醫學院內科學科教授 • 國立成功大學醫學院附設醫院內科部 主治醫師 • 行政院衛生署肝炎防治委員會委員 • 行政院衛生署葯事審議委員會

  33. Research Front #513 • Topic: CHRONIC HEPATITIS B • Most cited paper in this front:Title: Effects of extended lamivudine therapy in Asian patients with chronic hepatitis B Author(s): Liaw YF, Leung NWY, Chang TT, Guan R, Tai DI, Ng KY, Chien RN, Dent J, Roman L, Edmundson S, Lai CL Group Author(s): Asia Hepatitis Lamivudine StudySource: GASTROENTEROLOGY 119 (1): 172-180 JUL 2000 • Liaw YF, Chang Gung Mem Hosp, Liver Res Unit, 199 Tung Hwa N Rd, Taipei, 105 Taiwan • Chang TT, Natl Cheng Kung Univ Hosp, Dept Internal Med, Tainan 70428, Taiwan • 283 times cited • Chang TT was also recognized as the most cited author in this front (752 times)

  34. Documents Specialties Disciplines Research Front Research Font (研究前鋒)– finding scientific specialties • Research Front is a group of papers based on the frequent co-citation patterns • finer subdivision than analyzing by discipline • more contextual than analyzing individual papers or journals

  35. A P B A B C Research Fronts methodology • When paper A and B are“co-cited” by paper P, A and B are likely to havetopical relevance. • When this “co-citation” isfrequent, it forms a groupof papers that are “topicallyassociated” to one another. Small, H. Co-citation in scientific literature: new measure of relationship between two documents.” Journal of the American Society for Information Science, 24 (4): 265-269, 1973.

  36. Thomson Scientific Citation Laureate Award 2006 • 使用 Research Front 分析法進行遴選 • Research Front：高度被引用文獻及共被引關係交叉分析後之結果 • 資料庫來源： Essential Science Indicators (ESI)，近10年至2006年第3次雙月更新之 Research Front • 選出 82,989 篇文獻、5,514 個 Research Front • 119 篇包含台灣作者的 Research Front

  37. Thomson Scientific Citation Laureate Award 2006評選基本標準 • 每個入選的研究前鋒中至少有4篇核心文章 • 其中至少 2 篇為台灣機構所發表 • 前 3 大高度被引用文獻至少出現一個台灣地址 • 符合上述條件，該篇文章作者即可獲獎 • 若文章為兩位以上作者共同著作，進一步分析每位作者在同一研究前鋒之核心文獻被引用次數，被引用次數最高之者亦選為得獎者 • 選出之頂尖文獻自我引用超過 50%，該研究前鋒不列入遴選 • 若研究前鋒為多國共同研究之成果，台灣在研究前鋒中需入前三名研發國家 • 選取的主題若出現多個研究前鋒(如：LED、B型肝炎、Arabidopsis)，只選取其中一個台灣在文獻和引文數量最強的研究前鋒

  38. Essential Science Indicators (ESI) - Top Papers 成功大學進入 ESI 的學科領域 (共 7 項) • Clinical Medicine • Engineering • Chemistry • Physics • Material Science • Environment /Ecology • Computer Science

  39. 平均被引用次數 總發表文章數 總被引用次數 ESI – Clinical Medicine (NCKU)

  40. ESI – Clinical Medicine (NCKU) Number of Papers Number of Citations Average Citations per Paper

  41. ESI – Clinical Medicine (NCKU) – Top Paper 成功大學醫學院 張定宗 教授

  42. 總文章數 總被引用次數 平均被引用次數 ESI – Engineering (NCKU)

  43. ESI – Engineering (NCKU) Number of Papers Number of Citations Average Citations per Paper

  44. ESI – Engineering (NCKU) – Top Paper 成功大學工程科學系 廖德祿 教授

  45. 總文章數 總被引用次數 平均被引用次數 ESI – Chemistry (NCKU)

  46. ESI – Chemistry (NCKU) Number of Papers Number of Citations Average Citations per Paper

  47. ESI – Chemistry (NCKU) – Top Paper 成功大學化學工程學系 吳季珍 副教授

  48. 總被引用次數 平均被引用次數 總文章數 ESI – Physics (NCKU)

  49. ESI – Physics (NCKU) Number of Papers Number of Citations Average Citations per Paper

More Related