The Scholarship of Teaching and Learning: implications for its assessment

1. The Scholarship of Teaching and Learning: implications for its assessment Michael Prosser Centre for the Advancement of University Teaching The University of Hong Kong

2. Introduction • The Scholarship of Teaching, and the UK Research Assessment Exercise • Example of Pedagogic Research • What is pedagogic research? • How does it relate to the scholarship of teaching? • How does pedagogic research relate to other forms of scholarly inquiry in teaching and learning in higher education? • How is pedagogic research assessed in the UK RAE

3. Growing Interest in various forms of Scholarly Inquiry in Teaching and Learning in Higher Education • Carnegie Academy for the Scholarship of Teaching and Learning (CASTL) Program in Higher Education - USA • Carrick Institute for Learning and Teaching in Higher Education – Australia • Higher Education Academy – UK – The Academy • Research Assessment Exercise – RAE – UK Conferences, Research and Development Journals, Graduate Certificates in Higher Education, PhD programs in T&L in Higher education The UK Professional Standards Framework for teaching and supporting learning in higher education: ‘the ability to incorporate research, scholarship and/or professional practice into those activities’

4. The Higher Education Academy in the UK supports various forms of scholarly inquiry into teaching and learning in higher education, and has developed analytical distinctions to support its aims of supporting and encouraging the various forms. A concern is that the RAE in the UK will distort the forms of inquiry, by institutions encouraging all forms of inquiry to be consistent with and to meet the definition of research for the RAE.

5. Boyer’s four scholarships (Scholarship Reconsidered, 1990) Discovery – more traditional UK RAE type research Integration – making connections across the disciplines and placing things in larger context Application (or now often termed engagement) – goes beyond application and develops an interactions that each informs the other (research and application) Teaching – research informed, critical, evidence based, collegial reflection on practice to improve practice Separate, but overlapping

6. Analytical distinction between forms of educational research and inquiry

7. Investigative or evaluative study Theoretical study Scholarship of teaching

9. Definition of research for the UK RAE ‘Research’ for the purpose of the RAE is to be understood as original investigation undertaken in order to gain knowledge and understanding. It includes work of direct relevance to the needs of commerce, industry, and to the public and voluntary sectors; scholarship*; the invention and generation of ideas, images, performances, artefacts including design, where these lead to new or substantially improved insights; and the use of existing knowledge in experimental development to produce new or substantially improved materials, devices, products and processes, including design and construction. It excludes routine testing and routine analysis of materials, components and processes such as for the maintenance of national standards, as distinct from the development of new analytical techniques. It also excludes the development of teaching materials that do not embody original research.

10. Definition of pedagogic research for the RAE (generic statement) • ‘60. Pedagogic research in HE will be assessed where it meets the definition of research for the RAE. It is research which enhances theoretical and/or conceptual understanding of: • • teaching and learning processes in HE • • teacher and learner experiences in HE • • the environment or contexts in which teaching and learning in HE take place • • teaching and learning outcomes in HE • • the relationships between these processes, outcomes and contexts. • Reports of studies providing descriptive and anecdotal accounts of teaching developments and evaluations do not constitute pedagogic research. • Pedagogic research is firmly situated in its relevant literature, and high quality pedagogic research makes a substantial contribution to that literature.’

11. UOA 35, Accounting and Finance 16. Teaching materials and non-text output may be included in submissions where they contain original research. Such materials will be assessed in the same way as other work. Where the research aspects of submitted work may not be clear to the sub-panel, departments should describe this briefly (in up to 50 words) in the ‘Other relevant details’ field in RA2. 17. Where this is relevant to the UOA the sub panel welcomes the submission of research into the teaching and learning of the field or sub-field, at HE or other levels. This will be assessed in the same way as other work.

12. This does not imply that that theoretical and conceptual understanding cannot be developed through investigative studies or the scholarship of teaching – but that the outcomes of the study enhances theoretical or conceptual understanding – it is then the outcomes that bare judged.

13. Training of UK RAE Reviewers of Pedagogic Research in the Disciplines The RAE and the Higher Education Academy organised a training day for members of the discipline panels responsible for pedagogic research in disciplines The Academy identified what it considered to be high quality published papers, and asked the attendees to rate and discuss the papers in terms of the definition of high quality pedagogic research for the RAE (4*). 4 the Academy judged to have met the criterion for high quality pedagogic research 1 was more a description of the results of a survey 1 was more a description of practice 1 was an unconceptualised case study 1 was the report of an investigation

14. The attendees independently judged the 4 judged by the Academy to be of high quality, to be of high quality The other 4 were judged to be research, but not of high quality. The issue here is that the research needs to be well contextualised in its appropriate literature and / or field and be seen to be making a significant contribution to that literature and / or field.

15. Overview of the student learning perspective Figure 1: Model of Student Learning

16. STUDENT APPROACHES TO LEARNING • Surface Approach • Intention to reproduce • - rote memorise information needed for assessment • - failure to distinguish principles from examples • - treat tasks as external impositions • - focus on discrete elements without integration • Deep Approach • Intention to understand • meaningfully memorise information for later use • - relate new ideas to previous knowledge • - relate concepts to everyday experiences • relate evidence to conclusions 

17. Approach to learning represents the relationship between the student and the task or subject Approaches vary between subjects Approaches vary between tasks within subjects Approaches not stable but variable Surface and deep approaches describe a fundamental variation in the way students approach learning – they do not describe the rich detail in the way students approach their learning

18. STUDENTS’ PERCEPTIONS OF THE LEARNING CONTEXT • Research by Entwistle and Ramsden (1983) using interviews and questionnaires identified a number of student perceptions relating to the way they approached their studies • Student perceptions of: • Quality of teaching – including quality of feedback • Clearness of goals of course and standards of assessment • Workload so high that it was not possible to understand everything • Assessment measuring reproduction and not understanding • were found to relate to how they approach their studies and to learning outcomes (exam results and other indicators)

19. RELATIONSHIP BETWEEN PERCEPTIONS AND APPROACHES Study of 8837 students in 51 first year subjects around Australia (3 first year accounting courses) Amongst the data collected were students responses to: contextualised Ramsden's Course Experience Questionnaire and a contextualised Biggs Study Process Questionnaire 1994-1996: Australian Research Council ; Academic Departments and the Quality of Teaching and Learning; Paul Ramsden, Griffith University, Elaine Martin, RMIT, Michael Prosser, La Trobe University, Keith Trigwell, UTS

20. Approaches to Study Surface Approach 32. Although I generally remember facts and details, I find it difficult to fit them together into an overall picture 35. The best way for me to understand what technical terms mean is to remember the textbook definitions Deep Approach 28. I try to relate ideas in this subject to those in other subjects, wherever possible 34. In trying to understand new ideas, I often try to relate them to real life situations to which they might apply. Biggs Study Process Questionnaire

21. Student Perceptions Of Learning Context Good Teaching 15. The staff made a real effort to understand difficulties students might be having with their work. Clear Goals and Standards 1. It was always easy to know the standard of work expected 6. I usually had a clear idea of where I was going and what was expected of me in this subject. Appropriate Workload 25. The sheer volume of work in this subject meant that it couldn't all be thoroughly comprehended (-). Appropriate Assessment 8. To do well in this subject, all you really need is a good memory (-). Student Independence 20. Students had a great deal of choice over how they learned in this subject. Ramsden’s Course Experience Questionnaire

22. Factor Analysis of Perceptions of T & L Context and Approach to Study – accounting students _________________________________________________________________ Scale Factors ____________________________ 12 _________________________________________________________________ Perceptions of Context Good teaching .83 Clear Goals and Standards .58 (.38) Appropriate Workload .73 Appropriate Assessment (.36) Student Independence .80 Approach to Study Surface Approach -.83 Deep Approach .63 -.37 _________________________________________________________________ Principal Components, Varimax Rotation, n=463

23. 1.   A deep approach is associated with perceptions that the teaching is good and the goals and standards are clear • 2.   A surface approach is associated with perceptions that the workload is too high and assessment tests reproduction • That is, variation in individual students’ perceptions of the learning context across all subjects is associated with the approaches to study (and learning outcome) • 1994-1996: Australian Research Council ; Academic Departments and the Quality of Teaching and Learning; Paul Ramsden, Griffith University, Elaine Martin, RMIT, Michael Prosser, La Trobe University, Keith Trigwell, UTS

24. Overview of the student learning perspective Figure 1: Model of Student Learning

25. Having looked at the way students approach their learning, how do university academics approach their teaching? How does the approach to teaching relate to approach to learning and how does? Prosser, Martin, Trigwell, Ramsden, Lueckenhausen (’05) In a series of phenomenographic interview studies aimed at identifying key aspects of the differences in the way university teachers approach their teaching, the following key differences were identified:

26. A: Teacher focused, teacher activity with an intention of transferring information B. Teacher focused, student activity with the intention of transferring information to students C. Teacher focused, student activity with the intention of student acquiring the concepts of the discipline D. Student focused, student activity with the intention of students developing their own conceptions E. Student focused, student activity with the intention of students changing their conceptions

28. ATI scales measuring: • Information Transmission / Teacher Focused (8 items) • It is important to present a lot of facts to students so that they know what they have to learn for the subject • Conceptual Change / Student Focused (8 items) • A lot of time in this subject should be used to question students’ ideas

29. Trigwell, Ramsden, Prosser and Martin (‘98) – individual class unit of analysis (n=55 first year courses) Correlations between Approach to teaching and Approach to learning

30. Information Transfer / teacher focused approach to teaching in first year associated with student adopting more reproductive approaches to learning Conceptual change / student focused approaches to teaching associated with students focusing more on meaning and understanding

31. Eleven of those courses were first year business studies courses. The following scatter plots show the relationship between the two approaches for those courses

34. There is a clear relationship between the way student approach their learning in first year business courses and the way academics approach their teaching. We know that there is a relationship between the way students approach their learning with the quality and quantity of what they learning Is there a relationship between the way academics see their teaching and how they understand their subject matter Substantial amount of ongoing work, including that of Lynne Leveson’s, who has addressed this issue in accounting: More than the classroom: A study of approaches to teaching in accounting and its relationship to lecturers’ disciplinary conceptions

35. In her study Lynne interviewed 30 accounting academics about the way they see accounting as a discipline and the way they approach their teaching in first year curses. The study was again a phenomenographic study in which the aim was to identify key differences in the way these accounting academics saw they discipline and approached their teaching

36. Accounting Academics Conceptions of Accounting

37. The variation in the way they saw their discipline ranged from a focus in the discipline as substantially technical in orientation to seeing the discipline going beyond the technical and seeing it as being socially and culturally constituted and open to critical inquiry and analysis. But the interesting outcome of this study was in the relationship between these ways of seeing and experiencing the discipline and that way they approached their teaching.

39. A relatively strong association between the way accounting academics see their subject matter and the way they approach their teaching. Seeing accounting in terms of technical processes only is associated with more teacher focused information transfer approaches to teaching Seeing the accounting in its broader social and cultural context is associated with more student focused conceptual change approaches to teaching

40. Is it too long a bow to draw to say that if we wish: • to engage our student in more meaningful approaches to study, • to enhance their perceptions of the quality of teaching and learning in accounting and • to improve their learning outcomes, • accounting academics need • to adopt more conceptual change and student focused approaches to teaching, which may entail them seeing the discipline they teach not purely or substantially in technical terms but in its broader social and cultural context.

41. Conclusions The UK RAE judges pedagogic research in terms of its contribution to enhancing theoretical and conceptual understandings through pure, applied, policy and practice-based research. Accounts of such research need to go beyond being descriptive and anecdotal, and to show how it is making an important contribution to the literature of the field. A high rating would mean that it is judged to be comparable to the best work in the field worldwide. Research in teaching and learning in higher education is highlighting the importance of the way academics see and understand their subject matter on the way they teach and, in turn, on the way their students learn.