Assessment and Examination

1. Assessment and Examination Dianne Ford Graduate School Faculty of Medical Sciences University of Newcastle

2. Assessment • Why? • Monitor student progress • Identify problems with • Project • Supervisor(s)-student relationship • Completion rate • ‘exit’ strategy for struggling students • Student re-registration • Increased focus on the timeline

3. Assessment http://www.ncl.ac.uk/fms/postgrad/documentation/documents/Restud2012-13_final.pdf • When?

4. Assessment http://www.ncl.ac.uk/fms/postgrad/documentation/documents/Restud2012-13_final.pdf • When?

5. Assessment • What? • First assessment

6. Assessment • Intermediate progress reviews

7. Assessment • Final progress review

8. Before the interview • Read all the documents! • Read the student’s research report • Read the report from the student for the progress panel

9. Read the report from the supervisor

10. Your report

12. Exit strategy • If you feel a student is struggling do not give him or her the benefit of the doubt! • Students who fail to complete often show weak 1st reviews • But in some cases a prod in the first year can be very beneficial! • If a student “exits” within 12 months, they don’t appear on our overall completion statistics. • And some (not all) grant bodies regenerate the funds for a replacement student

13. The Viva Voce

14. Purpose of an examination • For the University • To assess and maintain quality • To mark ‘completion’ of the degree programme • For the Student • Potentially leads to award of a degree • Is an important and memorable life event • can be a real emotional roller coaster (for everyone involved)

15. Who needs an oral examination? • All PhD and MD candidates • Students need one internal and one external examiner • All staff candidates require two external examiners and an internal ‘moderator’ • Not all MPhil candidates • Same criteria for examiners as above • Oral examination held at examiners request • Not only for weak students • With a good candidate can be fun for everyone!

16. Appointment of examiners • See nomination forms on Graduate School http://medical.faculty.ncl.ac.uk/postgraduate/internal/staff.html • Completed by supervisor and Head of school/Institute • Examiners must: • be cognisant of standards • have subject knowledge (need CV) • command authority • not have played a role in the research • (if external) not have been a member of Newcastle staff for at least 3 years • be able and willing to examine

17. Internal examiners • Do NOT organise the exam, food, accommodation, travel, etc • This is the supervisors responsibility. • Ensure the examination complies with Newcastle University protocol • Ensure appropriate report forms are completed and submitted in a timely manner • Provide balance, fairness and ensure good examination conduct • Potentially play a role during any appeals process

18. External examiners • Often is the scientific specialist • Is an experienced examiner • Chicken and egg…. • Often thought to have the ‘casting vote’ • But there are procedures for disagreement • Maintains inter-university quality • Report provides important feedback to Graduate School (and QA etc) • May not be completely ‘up to speed’ with local regulations!

19. What if 2 external examiners? • Need to appoint an internal moderator • This person need not read or understand the thesis • Role is to provide advice on Newcastle examination process • May be required to present records of the exam if the result is questioned (appeal process).

21. Criteria – all theses • Should be: • Authentic • Scholarly • Professional • Well-structured, written and presented

23. MPhil candidates • Should • Demonstrate advanced knowledge • Have good knowledge of literature • Theses need not • Demonstrate consistent originality • Be worthy of publication

24. PhD/MD candidates • Should • provide evidence of adequate industry • demonstrate ability for originality • understand relationship with wider field • thesis should contain material worthy of publication

25. Types of thesis • ‘Standard’ • Divided into chapters with results and interpretations • By publication • A review and a series of ~5 related papers • Can be difficult to examine as papers have already satisfied external referees! • Staff candidates only

26. Examiners handbook http://www.ncl.ac.uk/students/progress/staff-resources/pg-research/examiners.htm

27. Reading a Thesis • Are you a proof reader or a scientist? • You will need to provide a list of corrections if you require them • For a good thesis, I (and most colleagues) usually stick “post-it” notes to the margin to localise my questions within the thesis • Be sensitive though; hundreds of these can look very scary! • If the thesis is poor, it might be better to have a more detailed critique with lists of specific questions and problems.

28. Preliminary Report • Regulations vary between institutions -read them! • Many (but not all) institutions require examiners to independently produce reports before the examination • Some need these to be submitted (well) before the examination to flag up potential problems • Some don’t require submission of these reports until after the examination! • But they should be exchanged with the other examiner’s report on the day.

29. Newcastle

30. Imperial college

31. How to conduct the examination - 1 • Arrange the room • Often good to have pencils and paper to draw on • At the start candidates can be very nervous! • Put them at their ease if possible with a soft start (but don’t anticipate the result!) • Remember to arrange refreshment breaks • Consider the candidate’s bladder

32. How to conduct the examination - 2 • Agree a plan with your co-examiner • Remember: • Oral examination of a good candidate can/should be one of the most pleasurable academic experiences for all involved • Examination of a poor thesis/candidate can be truly awful! • You need to devise different strategies for both situations

33. Questioning • The soft start • “what result in your thesis are you most proud of?” • “what led you to choose this study” • “what are you doing now?”

34. How long should the exam last • No fixed duration but >3 hours is exhausting for everyone. • Often examination of good students will last longer! • Use your judgement

35. At the end (if all has gone well) • Ask the candidate to withdraw for a few minutes • REMEMBER you do not award the degree!! • Work out what you wish to say, then invite the candidate back • Tell the candidate what recommendation you will be making to the higher degrees committee (or other appropriate authority). • Smile and shake hands

38. Examination Results 2011

39. Can you do more to recognise excellence? • In many countries, 1st rate PhD students can be defined. • the French system allows PhDs to be awarded as "honourable (not very good)", "very honourable" (average) and "very honourable with felicitations" (top 5%). • there is no similar recognition in the UK. • In Newcastle we ask examiners (on a separate report sheet) to indicate whether they consider the thesis to be in the top 10% of theses they have examined. • A committee considers this recommendation and prizes are awarded. • We hope this will be good for the student’s CV.

40. At the end (if it has NOT gone well) - 1 • Make absolutely sure you know what your options are (read the regulations) • You are not obliged to tell the candidate anything (although you will feel some pressure to do so) • the candidate will receive written confirmation of the outcome in due course • You may wish to speak to the supervisor • You might need clarification of the regulations (Graduate School)

41. At the end (if it has NOT gone well) - 2 • You will need to fill in the report form with very comprehensive details of any changes you require • This outcome results in much more effort in the future for the examiners! • All report forms look different. • Make sure you know the precise significance of a tick in every optional box! • For example, ticking box 3 (pass) at Newcastle can produce a very different outcome from box 3 at Imperial College (fail)!

42. The appeal process www.ncl.ac.uk/spo/AppealsForm.pdf

43. Invitation to examine • This is not necessarily an honour! • Not even a valued career move? • Why you and not someone else? • Are you a ‘soft touch’? • Are you the supervisor’s best friend? • You will/should see the abstract of the thesis at the time of invitation • Think about the thesis • You have the right to REFUSE!

44. Case study • X agreed to act as external examiner for Y’s PhD thesis. • The thesis was of marginal quality but X’s preliminary report suggested that with modification and a satisfactory oral examination it was likely that a positive recommendation could be made. • Close scrutiny of the thesis immediately before the examination revealed references to an earlier MD thesis by the same candidate (at the same university). X contacted the internal to request that a copy of this thesis should also be available for scrutiny (it is reasonable to see all reference sources cited in a thesis). • Before meeting the candidate on the day of the oral, X and the internal discovered that the earlier thesis reported about 70% of the data presented in the PhD thesis (about 45 pages of the introduction to both works was identical). • X and the internal examiner commenced the examination. Discuss the potential outcomes.