Inclusive Assessment and SOAS

1. Inclusive Assessment and SOAS Hong Bo Professor in Financial Economics School of Finance & Management Associated Director for Teaching Quality (UG) SOAS University of London

2. SOAS inclusive assessment project Stage I: SOAS vision and principles in inclusive assessment, July 2017-June 2018 The implementation of stage I was led by HEA consultants, which contains two main activities: Firstly, the build-up of SOAS vision and principles in inclusive assessment. The SOAS Inclusion Working Group (SIWG) was formed, three workshops were carried out by SIWG. The participants were mainly the school level learning and teaching leadership team members, including the Pro-Director (L&T), Associated Directors (L&T), and Learning and Teaching Coordinators (LTCs), and the Student Union representative. Secondly, two HEA consultants-led assessment labs. The exercise is to demonstrate HOW to optimise assessment and feedback designs to be consistent with inclusive principles. Three SOAS programmes had fully completed the exercise: BA Art History, BA Chinese (Modern and Classical), and BA Economics.

4. SOAS inclusive assessment project Stage II: SOAS internal capability of implementing inclusive assessment: train the trainers, November-December 2018 The main activities include two full-day workshops on inclusive assessment led by HEA consultants They were designed to build up SOAS internal capabilities which can implement inclusive assessment at SOAS. Hence, “train the trainers” The train the trainers assessment workshops provided more technical knowledge/skills required in designing inclusive assessment and feedback, including its rationale, the programme-focused approach, models of inclusive assessment design, etc. The participants were the Pro-Director (L&T), Associated Directors (L&T), and all Leaning and Teaching Coordinators (LTCs) from departments, which covered the entire leadership group in Learning and Teaching at SOAS

5. SOAS inclusive assessment project Stage III: SOAS inclusive assessment labs, Term 2-Term 3 in the 2018-2019 academic year The aim is to implement the programme level restructuring of assessment internally at SOAS based on the vision and principles of SOAS inclusive assessment A school-wide inclusive teaching workshop is in preparation, incorporating decolonising curriculums, unconscious bias, and inclusive assessment A series of assessment labs will be organised. These labs will focus on current practice of assessment at SOAS based on programmes of five selected departments as a test pilot.

6. SOAS inclusive assessment project

7. SOAS inclusive assessment project

8. Underlying theories I: Programme Level Assessment Rationale for programme level assessment Integrative learning is about making connections within a programme or major, between fields, across curriculum and co-curriculum, and between academic knowledge and practice. Fostering students’ abilities to integrate learning is arguably, according to Huber et al. (2007) ‘one of the most important goals and challenges of higher education’. (Healey, et al., 2013) (1) What are the differences between programme-level assessment and module-level assessment in terms of students learning experience and outcomes? (2) What are the transferrable skills students can benefit from programme level assessment? Semester-long modules with an average of two assessment events are unlikely to provide the learning architecture for cycles of continuous reflection” (Jessop, El Hakim, and Gibbs, 2014) “The whole is greater than the sum of its parts” (Jessop, El Hakim, and Gibbs, 2014)

9. Underlying theories I: Programme Level Assessment A programme view of assessment: (a) What are the academic programme aims and learning outcomes: overall and each module (b) What are students learning outcomes: graduate attributes and/or employability, generic and transferable skills assessed across and along the programme, learning and learner development for progressively more independent, self-regulating learners (c) What are the patterns of assessment and feedback on the programme level? (d) It has implications on both academics and quality assurance

10. Underlying theories I: Programme Level Assessment Features of assessment on the programme level (Gibbs and Dunbar-Goddet, 2009) the variety of assessment methods the volume of summative assessment the percentage of marks derived from examinations the volume of formative-only assessment the volume of oral feedback the volume of written feedback the timeliness of feedback the explicitness of goals, criteria and standards the extent of alignment of assessment with learning outcomes

11. Underlying theories I: Programme Level Assessment An example: Ishiyama (2005) documents that a better structured programme results in better student outcomes. How to measure the structure of the programme (a) Whether or not courses /modules are sequenced into a coherent whole (b) Whether or not the curriculum includes a senior seminar or capstone course (c) Whether or not a research methodology course is required, and when it is required in the sequence of course Students outcomes are measured by both test scores and employability Evidence supports the notion that a programme should be structured ! There should be an integrated and sequential course of study rather than a disconnected set of individual courses. The purpose of such an integrated curriculum is to provide for sequential learning, with the knowledge acquired in one course being extended and developed in a subsequent course (Ishiyama, 2005) .

12. Underlying theories I: Programme Level Assessment Programme level assessment requires Connectivity along two dimensions: horizontal (module) level and vertical (progression) level “A unified assessment plan has the potential to benefit student learning when there is a logical and coherent flow of assessment tasks across and through the whole programme” (Jessop, El Hakim, and Gibbs, 2014) The iterative cycles of assessment, feedback, reflection and action implied by assessment for life-long learning are only plausible at the programme level (Boud and Falchichov 2006)

13. Underlying theories I: Programme Level Assessment Connectivity in assessment design (both horizontal and vertical) (1) Variety of assessment stakes (low, medium and high) both within the module and across modules (2) Reasonable distribution of student work load across modules (3) Good distribution of assessment timing to reduce bunching of student effort both within the module and across modules (4) Build sequential coherence into the assessment tasks (5) Map out the variety and timing of different assessment modes on the programme level (6) How the work to develop skill A in a module will help students’ performance in other modules (7) Students demonstrate achievement of cross-module skills (8) Feedback from one module can be used to support students’ learning in another module: discipline related or generic skills (9) Include feedforward in the assessment feedback; students refer back to this feedback when they come to tackling the next assessment task in the sequence

15. Underlying theories II: Learning-Orientated Assessment Rationale for Learning-Orientated Assessment ---The literature describes three main purposes of assessment: (a) to measure achievement (summative); (b) to engender learning (formative); and (c) to develop graduate attributes which enable students to put real-life skills to work in employment and life settings (long-term learning) (Boud and Falchichov, 2006). ---The module-based practice: too much the certification function of assessment, too little the learning function ---The trade-off between inclusive assessment and academic standards ---The connection between learning-orientated assessment and inquiry-based learning The key is to engage students: students as partners in assessment One of the most effective ways of linking the doing and the thinking is engaging students in inquiry-based learning (Healey et al. 2010; Lee 2012). ---Many ways of students’ participation in assessment, Inquiry-based learning is an effective one ---Inquiry-based learning enhance students transferrable skills and employability Problem or inquiry-based learning in community and workplace settings has proven to be ‘an effective approach to teaching as real-life, problem-based learning provides opportunities for students to find academic activities meaningful and worthwhile’ (Lee et al. 2010, 563).

16. Underlying theories II: Learning-Orientated Assessment Synoptic assessment provides opportunities for inclusive assessment Example 1: Patchworks Patchwork assessment processes provide a cumulative set of formative assessment opportunities which can then be stitched together to produce a fully-justified summative account; with formative assessment thoroughly integrated within the learning and teaching process “The essence of a patchwork is that it consists of a variety of small sections, each of which is complete in itself, and that the overall unity of these component sections, although planned in advance, is finalized retrospectively, when they are ‘stitched together’.” Winter (2003: 112) It is a continuous formative assessment, reflecting the process of learning It stresses the personal and individual construction of Knowledge, which is in line with inclusive principles Articulation of experience is at the centre of the learning process in which the episodes of learning are placed within a wider context by the student ‘stitching’ together a justified meaning of their theory and practice The patchwork assessment process is designed so that the student can experience continuous assessment which encourages deep and transformative learning and helps to develop an understanding of complex inter-relationships.

17. Underlying theories II: Learning-Orientated Assessment Example 2: Capstone projects (1) What are capstone projects (2) Objectives of capstone projects (3) Types of capstone projects: research-based vs. non-research-based capstone projects (4) The delivery of capstone project modules (5) Assessment of capstone projects (6) Limitation of implementing capstone projects

18. Underlying theories II: Learning-Orientated Assessment Healey, et al. (2013) The term ‘capstone project’ is commonly used in North America and Australasia for a project in the last year or semester of the degree programme which provides opportunities for students to synthesise and apply their knowledge and experiences from their whole programme. It helps them to negotiate successfully the transition to the next stage of their career, whether to the workplace or further study. Rowles et al.(2004, 14) Capstone experiences should be a culminating set of personal, academic, and professional experiences, and as such, the primary focus of capstone experiences should be on the synthesis, integration, or application of previously acquired knowledge rather than on the acquisition of new knowledge or skills. “Such a course serves as an impetus to review, integrate, extend and apply the materials presented in the curriculum; it allows us to foster a pragmatic orientation toward sociology in our students" (Wallace 1988:34 in Hauhart and Grahe, 2010)

19. Underlying theories II: Learning–Orientated Assessment Objectives of Capstone projects: (1) Pedagogical reasons (e.g. Hauhartand Grahe,2010; Mckinney & Busher, 2011) (a) Students engagement in learning (Self-directed learning) (b) Deep-learning (iterative cycles of learning; integrative learning ) (2) Students learning experience (a) Transferrable skills : analytical and critical thinking skills, presentation skills, communication skills, research skills (literature review, writing, research method), teamwork skills, etc. (b) Employability Top 4 Objectives supported by evidence (e.g. HauhartandGrahe, 2010; Mckinney & Busher, 2011) -- Review and integrate learned material -- It helps students extend and apply learned material -- Integrate theoretical work across the field ---It fosters a pragmatic orientation toward the discipline

20. Underlying theories II: Learning–Orientated Assessment Types of Capstone projects: (1) Research-orientated capstone projects e.g. Independent Study projects (ISPs) (2) Non-research-orientated capstone projects --- Community-based capstone projects: schools, hospitals, etc. --- Industry (work)-based capstone projects: internship, R&D projects, etc. ---University internal non-research-orientated capstone projects: student journals, associations, clubs, etc. Why are capstone projects inclusive?

21. Example 3: Cornerstone projects Underlying theories II: Learning-Orientated Assessment