Introduction to Research Supervision

1. Introduction to Research Supervision Margaret KileyCEDAM

2. Program • Supervision • Supervisory relationships • Supervisory contexts • Supervisor roles and responsibilities

3. ENVIRONMENT/CONTEXT: Candidates and supervisors interact and learning within a research learning environment discipline CANDIDATE CHARACTERISTICS e.g. gender, age, enrolment, previous academic /research experience, motivation, intellectual capacity, research topics, conceptions of research OUTCOMES • Timely progression and completion • Career prospects • Publications • Quality thesis • Research and generic skills • Research understandings • Changed world view • Changed perception of self as researcher and learner • ‘Becoming’ a researcher • Contribution to field of research • Social Value institutional protocols LEARNING AND RESEARCH EXPERIENCES e.g. modification/development of expectations, conceptions, approaches to research & supervision research culture ASSESSMENT AND EXAMINATION government policies SUPERVISOR CHARCTERISTICS e.g. previous doctoral experiences, gender, age, previous experience supervising/ examining, conceptions of research global developments university policies

4. Supervision rather than Supervisor • Research suggests that there are a range of roles that need to be filled in terms of research supervision, in addition to knowledge of the topic and methodology, these are: • Mentor • Coach • Facilitator of candidature • Sponsor • From Pearson, M & Kayrooz, C. (2004). Enabling critical reflection on research supervisory practice. International Journal for Academic Development, 9(1), 99-116

5. Mentor • The mentoring role requires specific subject expertise and includes mentoring students’ so they can complete the research project itself, but also mentoring the intellectual development of the student, i.e.: • Encourages publishing • Encourages networking • Helps with seminar and conference presentations • Assists with career goals

6. Coach • The “coach” role involves helping candidates develop their research expertise while they are actually doing their research project. The coaching role often is performed by a range of people. This role includes: • Helping students with identifying the research question and theoretical framework • Helping plan and refine the project • Advising on critical aspects of research • Being directive when needed

7. Progressing/facilitating Candidature • The “progressing the candidature” role can be thought as facilitation-related functions, this includes: • Monitoring progress • Periodically reviewing supervision arrangements • Negotiating availability and initiating contact • Devoting sufficient time to the student

8. Sponsor • The sponsor is one who, for example, will: • Ensure candidates have access to basic resources • Ensure, or advise on how, students can access funding for conferences, field work etc • Keep students current with policies & procedures • Identify administrative procedures that students need to meet • Provide access to expertise and full participation in the research ‘practice’ including alternative sources of expertise.

9. Can/should one person do all this? • While there are a few outstanding people who can fulfill all of those roles, generally we find that one person is strong in one area and not so strong in another. • In your group, discuss: • Which of the roles reflect your strengths? • Which roles would you need someone else to fulfill? • What can/should you do about it? • How can you encourage your candidates to seek out people who can fulfill the four roles?

10. Tools • Expectation Scale • Learning needs analysis • To assist students identify strengths that they bring to their candidature and where they need to develop new skills and knowledge • Supervisory alignment • To assist students and supervisors gain a sense of where the ‘other’ is situated • Memorandum of Understanding • Helpful for even just discussing the aspects of supervision without necessarily turning into an MoU

11. Expectations • Expectation scale • Read and decide where you fit on the scale for each of the points • Discuss with the group

12. Communication Techniques • In groups, discuss techniques that you have used, your supervisor used, or you know from others that have helped communication (st/su, st/panel, st/st, st/other) e.g. • Agendas for, and Notes from, individual and panel meetings • Memorandum of Understanding • Email/web-based discussion • Group meetings (with different candidates getting practice at chairing, noting etc) • Regular meeting times/Open-door policy/Meetings as needed • Meetings with others in the Centre/disciplines

13. The ‘hard facts’ • Code of Practice • HRD Enrolments, progression and completion (RTS) • Evaluative data re the HDR experience & supervision

14. Research Training Scheme • The Government funds a certain number of ‘RTS places’ to universities annually. Some places are high cost e.g. some sciences, others low cost e.g. humanities • Funding is determined by students enrolled i.e. reaching target, student completions, and other research income e.g. from staff research grants and publications • Funding is calculated on a four-year candidature (FTE) i.e. if a student takes longer than four years then, the University is not being paid for supervising that student.

15. Graduate Destinations (2005) % within University Type

16. Postgraduate Research Experience Questionnaire (PREQ) • Developed by Graduate Careers Council of Australia in 1998 • Distributed by each University after graduation with results collated nationally by GCCA and then sent to each university • 28 Statements clustered into six scales • Supervision (#1, 7, 13, 21, 24) • Skill development (#6, 10, 14, 20, 26) • Intellectual climate (#5, 9, 16, 22, 23) • Infrastructure (#3, 8, 12, 18, 27) • Thesis Examination (#2, 15, 25) • Goals and Expectations (#4, 11, 19) • Overall Satisfaction (#28)

17. Supervision Scale (Agree 4+5) 2002 70.8% 2003 72.7% 2004 72.7% Skill Development (Agree 4+5) 2002 89% 2003 89.6 2004 91.1 Intellectual Climate (Agree 4+5) 2002 54.7% 2003 56.3% 2004 57.7% Infrastructure Scale (Agree 4+5) 2002 65.8% 2003 67.2% 2004 68.5% Thesis Examination Scale (Agree 4+5) 2002 75.0% 2003 75.1% 2004 76.7% Clarity of Expectations (Agree 4+5) 2002 87.5% 2003 89.3% 2004 90.1% PREQ 2004 (National) Overall Satisfaction: 2002 = 80.8%, 2003 = 82.3%, 2004 = 83.8%

18. What does this instrument say about the postgraduate research experience? • In groups look through the statements that students are asked to consider in the PREQ. • What sort of picture are they implying of the research experience? • How does that relate to your own experience? • What might it mean for you as a supervisor?

19. Intellectual Climate • From the PREQ results we can see that one of the scales which is consistently low is “Intellectual Climate” or research culture • In pairs work through the sheet in your folder titled Developing a vibrant research culture among your postgraduate students • How might you contribute to, and encourage your students to also contribute to, a positive research culture in your discipline/school?

20. Evaluation Strategies • Supervision is very difficult to evaluate given the small number of students involved and lack of anonymity • On the other hand we need some sort of feedback on performance for both formative (improvement) and summative (going for promotion) purposes • Consider the options that are available at Newcastle • Look at the Alternative Evaluation Strategies sheet and the case study and discuss in groups whether any of those ideas would work for you.

21. Stages of Candidature • Recruitment and selection • Framing the candidature e.g. needs analysis, expectations, topic selection/refinement, establishing the panel, literature review and methodology • Guiding and monitoring progress e.g. ensure formal requirements met, writing, development of networks, feedback on progress • Completing e.g. when to ‘stop’, feedback, nomination of examiners, support during and after examination • The post graduation phase e.g. preparing a research and publication plan

22. Case Study • In your folder you have Case Study 1(a) and 1 (b) • In your group work through one of the case studies and discuss the questions at the end

23. Case Study (cont) • Now work through the other case study in the folder • Can you see how the misunderstandings might have developed? • How might they have been avoided?

24. Supervisory Panels • Discuss how you might work with a candidate on the task of constructing a panel/committee taking into account the various roles (Mentor, Coach, Facilitator, Reflective Practitioner and Sponsor) and the different stages of candidature and the related responsibilities. (Supervisory Framework might be useful as a guide)

25. The Proposal Seminar • Many universities in Australia only allow students an interim enrolment until they have successfully completed their proposal seminar • There is evidence to suggest a correlation be a high quality research proposal and successful PhD submission and completion • Who at Newcastle is responsible for organising the the seminar? • What guidance is the candidate given?

26. Feedback on work • One of the most common complaints from candidates is the lack of timely and useful feedback from supervisors • These complaints come through in national surveys e.g. the Postgraduate Research Experience Questionnaire (PREQ) • What makes for useful feedback? • What is timely?

27. In small groups • How do supervisors give feedback to research candidates? E.g. did your supervisor give you verbal, hand written on written work that has been presented, via email etc • How often do you think a supervisor should provide feedback and how can they make the time to do it? • What advice can supervisors give to candidates about submitting work for feedback e.g. suggest that candidates give them drafts to read as they are about to head off on a long flight? • What do supervisors expect candidates to do with the feedback? e.g. does the supervisor expect them to act on it or is it for advice only?

28. Monitoring Progress • Monitoring progress has been shown to be critical in candidature. • Most Australian universities have a system (e.g. Annual Progress Reports) where the supervisory panel/committee discusses with the candidate their progress over the past 12 months and plans for the next 12 months • Reflecting on, and discussing progress, have been shown to have a positive affect on progress. It can also be the time when changes are made to the panel. • What happens at Newcastle?

29. Examination • The Supervisory Panel is responsible for proposing names of potential examiners to the Head of School (or equivalent) • Encouraged to discuss potential names with the candidate, but generally candidates do not to know final names • Generally 50% of all Australian dissertations are sent overseas • The aim is to find examiners who are knowledgeable in the area and who will give a fair and balanced opinion • The mean time for examination of • ‘accept as is’ theses is 0.35 of a year • minor revision 0.39 of a year (although 17% took more than six months) • major revision 0.53 of a year and • revise and resubmit 1.39 years. (Courtesy Sid Bourke)

30. Experienced Examiners report that they… • Expect the student to pass as they open the thesis • Are very reluctant to fail a student with most experiencing considerable distress if they do so • Come to a decision about the quality of a PhD by about the end of Chapter 2 • Have a formative rather than summative view of thesis examination • Believe that there is a risk attached to sending theses to inexperienced examiners • Are reluctant to take much notice of institutional criteria when examining • See ‘Nobel Prize’ paper for more detail

31. Experienced Examiners appear to… • Be fiercely independent in their views • Hold varying views about the purpose of the PhD. (Is it the thesis or the student being examined?) • Consider professional duty as the main reason for examining, followed by the fact that they are going to be needing examiners for their own students! • Devote considerable time to examining each thesis • Have surprisingly broad approaches to methodology/ paradigm • Demonstrate few discipline differences in their responses, other than regarding publications

32. Inexperienced Examiners • Have a high level of confidence in their ability to examine (which is not always reflected in what they say in response to other questions) • Frequently talked about experience from supervising & examining Honours students and theses • Adopt a similar approach to the actual process of examining as do their more experienced colleagues, although they are more likely to focus on the ‘steps’ or components of a PhD rather than the whole • See their role as maintaining standards and performing their summative assessment role correctly

33. Inexperienced Examiners … • At a surprisingly high rate, wanted to fail first thesis or said it was ‘awful’ • Are more prone, than experienced examiners, to follow institutional criteria. • Felt (some of them) that they were being examined too • Suggest that one of their main difficulties is their inability to benchmark • Seem to have very high expectations of the supervisors of the thesis being examined

34. Is there a difference? • From work of Trafford (2003) from 130 vivas it was possible to determine that: • Experienced examiners tended to ask questions that can be defined as ‘Defending doctorateness, contributing to knowledge, critique of research, synthesizing concept’ • Inexperienced examiners tended to ask more ‘technical’ questions • Trafford, V. (2003) Questions in doctoral vivas: Views from the inside, Quality Assurance in Education 11(2) pp 114-122

35. Trafford’s Categorisation Innovation and DevelopmentHIGH C. Questions generally related to issues such as research question, choice of topics, location of study D. Defending doctorateness, contributing to knowledge, critique of research, synthesizing concept Scholarship & Interpretation A. Types of questions include resolving research problems, content, structure B. Implications, awareness of, and familiarity with wider literature LOW

36. Strategies for examining • Different examiners approach the task differently, but most: • Begin by reading the Abstract, Introduction & Conclusion to gauge the scope of the work and whether what candidates say they are going to do is actually done • Look at the references to see what sources have been used and whether they need to follow up on any of them • Then read from cover to cover taking detailed notes, finally go back over the thesis to check whether their questions have been answered or whether their criticisms are justified

37. The reports demonstrate… • A less than ideal thesis has • Too much detail with lack of analysis • Lack of confidence, energy & engagement by the candidate • Lack of argument and rigour • Shoddy presentation (typos etc) • Lack of critique of own analysis/ sweeping generalisations based on opinion rather than analysis • Inadequate or poorly expressed methodology & scope A ‘good’ thesis has • Critical analysis & argument • Confidence & a rigorous, self-critical approach • A contribution to knowledge • Originality, creativity & a degree of risk taking • Comprehensiveness & scholarly approach • Sound presentation & structure • Sound methodology

38. Resources • Papers from all seven biennial Quality in Postgraduate Research conferences http://qpr.edu.au • Australian University consortium For Improving Research Supervision Training (fIRST) http://www.first.edu.au (you will need a Username and Password from University Contact) • Australian Deans and Directors of Graduate Studies http://www.ddogs.edu.au/cgi-bin/index.pl • SORTI web site at the University of Newcastle has information on examining theses particularly in the performing/ visual arts http://www.newcastle.edu.au/centre/sorti/publications.html