1 / 13

Multi-Dimensional Factors in Academic Research Evaluation

Explore the impact of strong research on universities, measuring research production and impact, identity building, acceptable research production, effects of university rankings, and major sources of rankings in the US.

jstaggs
Download Presentation

Multi-Dimensional Factors in Academic Research Evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Multi-Dimensional Factors in Academic Research Evaluation Xiaodong Zhang Ohio State University

  2. Impact of Strong Research to Universities • A foundation of improving undergraduate and graduate programs • Timely updating contents in classrooms • Creating research environment and opportunities to students • Bringing competitiveness and recognition • New knowledge, discoveries, and innovations. • Continue to attract strong students, excellent faculty, and research grants.

  3. How is Research Measured? • Research Production is Typically Measured by • Publications in refereed conferences and journals • Software and prototypes • Invited lectures • Patents • Research Awards and honors • Input of Research • Research proposal writing, and research project planing • Research grants • Recruiting and forming teams.

  4. How is Research Impact Measured? • Research results are useful • Citations of the publications • Widely used software and prototypes • Technology transfer from original research results • Useful patents • Influence in the research community • Leading new directions in the field • The quality of research is essential • Number of publications should not be a major metric • Our research should be Impact-driven

  5. Where to Publish? • Why publish in top venues? • Each field has its own flagship and leading conferences/journals defined by reputations • Most researchers only read papers there. • Conferences versus Journals • In certain fields, such as systems, architecture, networking, and databases, top conference papers are prestigious, highly visible, with 4-5 or more reviews in depth, and low acceptance rate. • SCI and EI should NOT be used as a guidance • An SCI/EI entry does not necessarily reflect its quality in the field • The quality should be judged by the peers in the field. • The impact factor sometimes is field-size dependent.

  6. Establishing Identity and Reputation • Conducting research on focused topics in depth • Making names behind some important innovations • US NSF Career Program for Junior Faculty • A 5 year research/education plan, aiming at conducting research to solve a small set significant problems. • A strong department should have several identities • Paying attention to building on existing strength. • Rewarding to strong research identity/reputation

  7. What is an ``Acceptable” Research Production? • How to quantify research productions? • Number vs. quality of publications • Research grants/expenditure (mission vs. basic research) • Supervising Ph.D. students: productivity and quality of placements. • Impact of research after many years. • Conference/journal organizations • Not part of the research, but professional service. • Each department has its formula • Resources dependent (teaching load, et. al.)

  8. Effects of University Rankings • Rankings by society have to be a consideration • Rankings significantly affect student recruiting • University rankings influence parents • Faculty recruiting is also affected. • However, these rankings are a reference not an absolute indicator.

  9. Major Sources of University Ranking in US • US News & World Reports (University ranking) • Published every August. • Based on 6 criteria to quantify comprehensive ability. • Quality of undergraduate students is an important consideration. • US News & World Reports (Graduate school ranking) • Published every March. • Different school rankings are based on different disciplines. • Quality of graduate programs is the major factor. • National Science Foundation (funding statistics) • Research expenditure of last year published every August • Ranking the research scale of universities. • National Research Council (Department ranking) • Published every 10 to 15 years • Ranking all the Ph.D. granted departments in all majors

  10. US News & World Reports’ 6 Criteria • Undergraduate student selectivity • SAT scores, high school ranking, and other activities. • Acceptance rate. • Faculty academic reputation • Research activities and accomplishments • Faculty-student ratio • Reflecting the efforts and attention to students, e.g. class size. • Student retention and graduation rate • Seriousness of the students and management of curriculum. • Financial resources • University revenue and endowed funds. • Alumni donation rate • Reflecting graduates’ proud, passion, and care to the university.

  11. Major Themes of Graduate Program Ranking • Scholarships and impacts • Quality of faculty publications and their influence to the field • Citations rates and technology transfers • Ph.D. student production and placement • How many per year, and where they go after graduations. • Ph.D. alumni achievements • The group distinguishing themselves in the fields. • Faculty resources • Award winners, members of the academies, and others. • Research scale • Measured by the amount of expenditure by external grants

  12. Foundations of Academic Excellence • Excellent undergraduate programs • Highly selected students with comprehensive talents. • Providing highly quality education and research environment. • All top U.S. universities have top undergraduate programs. • Excellent research and education infrastructure • Students are able to effectively learn and research. • Excellent faculty • World-class professors who are well established in their fields. • High standards in research and education • Rigorous training to students and doing impact driven research. • Ambitious students who will play leadership anywhere • Enter: grow the wisdom; Depart: serve the better country and the kind. • Should not be too narrowly focused.

  13. Mr. Hu’s 4 Conditions of an ``Established University” • In 1947, Hu Shi urged the government to invest 5 existing universities to make them be ``established and independent” in the world within 10 years. • 1. Providing first-class education environment. • 2. Being able to retain young researchers. • 3. Being able to solve country’s problems in science, industries, health, and national defense. • 4. Being able to collaborate with foreign scientists to solve open and hard problems.

More Related