1 / 10

The NRF rating system and its appeals: some facts and some fallacies

The NRF rating system and its appeals: some facts and some fallacies. Duncan Mitchell. ● I am a theoretical physicist and the panel consists only of experimental physicists, so they can’t assess me. ● My h index is better that that of five B-rated

melissa-becker
Download Presentation

The NRF rating system and its appeals: some facts and some fallacies

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The NRF rating system and its appeals: some facts and some fallacies Duncan Mitchell

  2. ● I am a theoretical physicist and the panel consists only of experimental physicists, so they can’t assess me ● My h index is better that that of five B-rated researchers I know, but the committee gave me a C ● I was elected Vice-Chairman of the International Society of Orchid Botanists so I must have an international reputation ●My research has improved but my rating has gone down

  3. Reviewers ● The only assessors of research output ● Your choices and the specialist committee’s choices ● Excluding potential reviewers Pick a likely reviewer and write your application for her/him

  4. The reviewers’ focus ● Your best outputs of the review period ● Peer-reviewed journal articles ● Other legitimate research outputs ● Postgraduate students Be careful out about conference proceedings Teaching textbooks are not research outputs

  5. Scoring the reviewers’ reports ● Familiarity with the research field ● Evidence-based assessment of the applicant’s research in the review period ● Bias and hostility ● Halo effect ● Congruence with data in the application Good reports count more

  6. Arriving at the rating ● Enough usable reports? ● Search for the language: …… coherent body of work? …… promise, leadership? …… significant international impact? ● Agreement between committee, assessor and chair Count up the A, B, C, P, Y, RU reports (weighting the good reports)

  7. The Appeals Committee ● Mandate ● Membership ● Evidence: ……. appeal letter ..…... RRAP sheets ……. documents that served before the specialist committee/ Executive Evaluation Committee ……. and it has the right to call for more reports

  8. The Appeals Committee’s deliberations ● Is the appeal legitimate? ● What issues in the letter need a response? ● Did the rating process follow the rules? ● Were there enough usable reviewers’ reports? ● Was the set of reviewers (with usable reports) balanced? Does the Appeals Committee agree with the rating decision?

  9. The difficult issues ● Applicants on a cusp ● Established early-career researchers ● Members of large research teams ● Researchers competent in more than one field of research ● Researchers still active but past their peaks

  10. ● I am a theoretical physicist and the panel consists only of experimental physicists, so they can’t assess me ● My h index is better that that of five B-rated researchers I know, but the committee gave me a C ● I was elected Vice-Chairman of the International Society of Orchid Botanists so I must have an international reputation ●My research has improved but my rating has gone down

More Related