1 / 16

Examining research theses: What needs to be considered?

Examining research theses: What needs to be considered?. Margaret Kiley CEDAM, The Australian National University Margaret.kiley@anu.edu.au. Questions for you. Where did you do your own PhD? What was the examination process like?

maalik
Download Presentation

Examining research theses: What needs to be considered?

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Examining research theses: What needs to be considered? Margaret KileyCEDAM, The Australian National University Margaret.kiley@anu.edu.au

  2. Questions for you • Where did you do your own PhD? • What was the examination process like? • How many doctoral dissertations have you examined? What about Honours? Masters? • Why do you think you might be asked to examine? • What do you think your weaknesses as an examiner might be? • What do you think your strengths as an examiner might be?

  3. Selection of Examiners (Kiley, 2009) • Recent research suggests that the selection of examiners varies depending on the perceived quality of the candidate and the dissertation. • Experienced supervisors report that there are two sorts of criteria that they use: • Professional / academic • Personality attributes. • The aim is to find examiners who are knowledgeable in the area and who will give a fair and balanced opinion.

  4. Professional / academic considerations • Topic/methodology fit • Understanding of the system: country, type of doctorate (professional doctorate, exegesis) • Examiners outside the academy • Experience versus inexperience (avoid retirees with too much time on their hands) • Availability

  5. Personality Issues • High standards but fair (avoid the pedants) • Intellectual courtesy and respect (avoid the Smart Alecs) • Reliability (avoid the ‘gunnas’)

  6. Why might you be selected to exam? • Topic knowledge? • Methodology expertise? • You are a ‘mate’ of the supervisor? • Because they think you might be a ‘soft touch’? • You have research projects that might provide post doc positions for the candidate if their work is good? • You are considered reliable, fair, balanced and courteous? • Other?

  7. Analysis of reports demonstrates that… (Kiley, 2004) • A ‘less than ideal’ thesis has: • Too much detail with lack of analysis • Lack of confidence, energy & engagement by the candidate • Lack of argument and rigour • Shoddy presentation (typos etc) • Lack of critique of own analysis/ sweeping generalisations based on opinion rather than analysis • Inadequate or poorly expressed methodology & scope A ‘good’ thesis has: • Critical analysis & argument • Confidence & a rigorous, self-critical approach • A contribution to knowledge • Originality, creativity & a degree of risk taking • Comprehensiveness & scholarly approach • Sound presentation & structure • Sound methodology

  8. Discussion • What does the above analysis mean for you: • when advising your own candidates? • examining a thesis? • writing the report?

  9. Strategies for examining • Most experienced examiners: • Begin by reading the Abstract & Acknowledgements • Introduction & Conclusion to scope the work and to see if what candidate says they are going to do is done. • The references to see what sources have been used and if they need to follow up (and to check if they were cited). • Read cover to cover taking detailed notes. • Finally go back to check if their questions have been answered or their criticisms justified. • Then think about writing the report

  10. Inexperienced Examiners … • Felt (some of them) they were being examined too. • Suggest a main difficulty is inability to benchmark. • See their role as maintaining standards and performing their summative assessment role ‘correctly’.

  11. Finalising the first part of the report • The ‘tick a box’ • If you suggest minor changes – how minor is minor? • What about major changes? What’s the difference between minor and major? • Revise and submit - submit to you or….? • Fail – seek advice from an experienced colleague before thinking of ticking this box!

  12. The written report (Average 2.5 pages) • To whom are you addressing the report? Why? (Analysis suggests the audience often ‘shifts’) • If you have ticked ‘minor or major changes’ what are they and more importantly – have you made it clear the difference between suggestions and requirements • If a review and resubmit, have you made it clear what needs to be done for it to be acceptable

  13. Resources Bourke, S., Hattie, J., & Anderson, L. (2004). Predicting examiner recommendations on PhD theses. International Journal of Educational Research, 27(4), 178-194. Carter, S. (2008). Examining the doctoral thesis: A discussion. Innovations in Education and Teaching International, 45(4), 365-374. Council of Deans and Directors of Graduate Studies in Australia (2005). Framework for best practices in doctoral examination in Australia, 14 November 2005, from http://www.ddogs.edu.au/cgi-bin/index.pl Denicolo, P. (2003). Assessing the PhD: A constructive view of criteria. Quality Assurance in Education, 11(2), 84-91. Grabbe, L. (2003). The trials of being a PhD external examiner. Quality Assurance in Education, 11(2), 128-133. Hansford, B. C., & Maxwell, T. W. (1993). A masters Degree Program: Structural components and Examiners' comments. Higher Education Research and Development, 12(2), 171-187. Hartley, J. (2000). Nineteen ways to have a viva. PsyPag Quarterly Newsletter 35(June), 22-28. Hartley, J., & Fox, C. (2004). Assessing the mock viva: The experiences of British doctoral students. Studies in Higher Education, 29(6), 727-738.

  14. Holbrook, A. (2004). Examiner reflections on the fine arts higher degree examination process, Australian Association of Research in Education. Melbourne. Holbrook, A., & Bourke, S. (2004). An investigation of PhD examination outcomes in Australia using a mixed method approach. Australian Journal of Educational and Development Psychology, 4, 153-169. Holbrook, A., Bourke, S., Fairbairn, H., & Lovat, T. (2007). Examiner comment on the literature review in Ph.D. theses. Studies in Higher Education, 32(3), 337 - 356. Holbrook, A., Bourke, S., Lovat, T., & Dally, K. (2004b). Qualities and characteristics in the written reports of doctoral thesis examiners. Australian Journal of Educational and Development Psychology, 4, 126-145. Johnston, S. (1997). Examining the examiners: An analysis of examiners' reports on doctoral theses. Studies in Higher Education, 22(3), 333-346. Joyner, R. (2003). The selection of external examiners for research degrees. Quality Assurance in Education, 11(2), 123-127. Kiley, M. (2004). What examiners' comments can tell us about the postgraduate learning environment. In C. Rust (Ed.), Improving student learning: Theory, research and scholarship (pp. 213-222). Hinckley, Leicestershire: The Oxford Centre for Staff and Learning Development. Kiley, M. (2009). Rethinking the Australia doctoral examination process. Australian Universities' Review, 51(2), 32-41.

  15. Kiley, M. (2009). You don't want a smart alec: Selecting examiners of doctoral dissertations Studies in Higher Education, 34(8), 889-903. Kiley, M., & Mullins, G. (2004). Examining the examiners: How inexperienced examiners approach the assessment of research theses. International Journal of Educational Research, 41(2), 121-135. Kiley, M., & Mullins, G. (2006). Opening the black box: How examiners assess your thesis. In C. Denholm & T. Evans (Eds.), Doctorates downunder: Keys to successful doctoral study in Australia and New Zealand (pp. 200-207). Melbourne: ACER. Lovitts, B. (2007b). Making the implicit explicit: Creating performance expectations for the dissertation. Sterling, Va: Stylus. Mullins, G., & Kiley, M. (2002). 'It's a PhD, not a Nobel Prize': How experienced examiners assess research theses. Studies in Higher Education, 27(4), 369-386. Pitkethly, A., & Prosser, M. (1995). Examiners' comments on the international context of PhD theses. In C. McNaught & K. Beattie (Eds.), Research into Higher Education: Dilemmas, Directions and Diversions (pp. 129 - 136). Melbourne: HERDSA, Vic. Powell, S., & Green, H. (2003). Research degree examining: Quality issues of principle and practice. Quality Assurance in Education, 11(2), 55-63.

  16. Powell, S., & McCauley, C. (2003). The process of examining research degrees: Some issues of quality. Quality Assurance in Education, 11(2), 73-83. Powell, S. D., Green, H., McCauley, C., & Shaw, M. (2000). Research degree examining: Common principles and divergent practices. UK: UK Council for Graduate Education. Sankaran, S., Swepson, P., & Hill, G. (2005). Do research thesis examiners need training? Practitioner stories. The Qualitative Report, 10(4), 817-835. Tinkler, P., & Jackson, C. (2000). Examining the Doctorate: institutional policy and the PhD examination process in Britain. Studies in Higher Education, 25(2), 167-180. Tinkler, P., & Jackson, C. (2004). The Doctoral Examination Process: A handbook for students, examiners and supervisors. Buckingham: SRHE and Open University Press. Trafford, V. (2003). Questions in doctoral vivas: Views from the inside. Quality Assurance in Education, 11(2), 114-122. Wisker, G. (2005). The good supervisor: Supervising postgraduate and undergraduate research for doctoral theses and dissertations. Basingstoke: Palgrave Macmillan. Wright, T., & Cochrane, R. (2000). Factors influencing successful submission of PhD theses. Studies in Higher Education, 25(2), 181-195.

More Related