Learning outcomes

1. Learning outcomes London College of Fashion Sally Brown April 2013 http://sally-brown.net Emerita Professor, Leeds Metropolitan University, Adjunct Professor, University of the Sunshine Coast, Central Queensland and James Cook University Queensland Visiting Professor: University of Plymouth and Liverpool John Moores University.

2. Designing effective learning outcomes. They should be: • Specific, Measurable, Achievable, Realistic and Time-constrained; • Appropriate in number and level of detail; • At the right academic level: so progression can be demonstrated; • Expressed in language that students can relate to and that external examiners deem acceptable; • Reasonably robust and able to withstand external interrogation and the test of time.

3. Reviewing learning outcomes • Are assignments constructively aligned with learning outcomes? • Are your learning outcomes over-specified? Is there room for flexibility in interpretation? • Do you make space for brilliant yet unexpected outcomes? • Is the language excessively woolly? (e.g. over use of words like ‘understand’).

4. Constructive alignment (after Biggs, 2003): • Clarify what knowledge and skills your students need to be able to demonstrate by the end of the programme; • Identify content and processes for effective curriculum delivery; • Select a range of appropriate delivery means; • Identify authentic ways by which they can demonstrate this achievement; • Decide how best to evaluate how successful you have been in achieving the above.

5. Integrating assessment into learning • Do your assignments align with learning outcomes? • Are they sensibly spaced throughout the programme? • Are they varied to allow diverse students to shine without being overwhelming in variety? • Are feedback opportunities, plentiful, timely (so students can use them) and productive? • Are you over assessing (or more rarely under-assessing?)

6. Manageability of assessment • Across a programme, have you avoided log jams for students and staff? • Have you mapped out marking time necessary (and hence costs of marking)? • Can you access the necessary rooms, equipment and software particularly for exams? • Can you comfortably meet university requirements in terms of exam boards, moderation etc?

7. Differentiating knowledge and skills. It’s all in the verbs! Task: Chose from the following verbs that are likely to relate to knowledge, skills or both: Perform, demonstrate, outline, describe, show, present, make, practice, differentiate, define, argue create, design, develop effect, utilise, enact, act, carry out, differentiate, define, argue, analyse, synthesise, evaluate, articulate, specify, propose, compare, choose, challenge, identify.

8. Useful verbs: Assessing knowledge after Kennedy et al 2011 • Arrange, collect, define, describe, duplicate, enumerate, examine, • find, identify, label, list, memorise, name, order, outline, present, • quote, recall, recognise, recollect, record, recount, relate, repeat, reproduce, • show, state, tabulate, tell.

9. Assessing Comprehension • Associate, change, clarify, classify, construct, contrast, convert, decode, • defend, describe, differentiate, discriminate, discuss, distinguish, • estimate, explain, express, extend, generalise, identify, illustrate, indicate, • infer, interpret, locate, paraphrase, predict, recognise, report, • restate, rewrite, review, select, solve, translate.

10. Application • Apply, assess, calculate, change, choose, complete, compute, construct, • demonstrate, develop, discover, dramatise, employ, examine, • experiment, find, illustrate, interpret, manipulate, modify, operate, • organise, practice, predict, prepare, produce, relate, schedule, select, • show, sketch, solve, transfer, use.

11. Analysis • Analyse, appraise, arrange, break down, calculate, categorise, classify, • compare, connect, contrast, criticise, debate, deduce, determine, • differentiate, discriminate, distinguish, divide, examine, experiment, • identify, illustrate, infer, inspect, investigate, order, outline, point out, • question, relate, separate, sub-divide, test.

12. Assessing Synthesis • Argue, arrange, assemble, categorise, collect, combine, compile, compose, • construct, create, design, develop, devise, establish, explain, • formulate, generalise, generate, integrate, invent, make, manage, • modify, organise, originate, plan, prepare, propose, rearrange, reconstruct, • relate, reorganise, revise, rewrite, set up, summarise.

13. Assessing Evaluation • Appraise, ascertain, argue, assess, attach, choose, compare, conclude, • contrast, convince, criticise, decide, defend, discriminate, explain, • evaluate, grade, interpret, judge, justify, measure, predict, rate, recommend, • relate, resolve.

14. References & further reading Biggs, J. (2003) Teaching for Quality Learning at University, Buckingham: SRHE & Open University Press. Bristol University Characteristics of excellent teaching http://www.bris.ac.uk/esu/academicdevelopment/prizes/excellentteacher.html Brown, S. Rust, C. & Gibbs, G. (1994) Strategies for Diversifying Assessment, Oxford Centre for Staff Development. Browne, J. (2010) Securing a sustainable future for higher educationwww.independent.gov.uk/browne-report Brown, S. and Denton, S. (2010) Leading the University Beyond Bureaucracy in A practical guide to University and College management (Eds. Denton, S and Brown, S) New York and London: Routledge. Brown, S. (2012) Managing change in universities: a Sisyphean task Quality in Higher Education, Volume 18 number 1. Brown, S. (2011) Bringing about positive change in the higher education student experience: a case study, Quality Assurance in Education, Volume 19 Number 3 p 195-207. Carroll, J. and Ryan, J. (2005) Teaching International students: improving learning for allRoutledge SEDA series

15. References: 2 Kennedy, D., Hyland, A. and Ryan, N. Implementing Bologna in your institution, Using learning outcomes and competences: Writing and Using Learning Outcomes: a Practical Guide http://sss.dcu.ie/afi/docs/bologna/writing_and_using_learning_outcomes.pdf Northedge, A. (2003) Enabling participation in academic discourse Teaching in Higher Education, Vol. 8, No. 2, 2003, pp. 169–180. Newton, J. (2003) Implementing an Institution-wide learning and Teaching strategy: lessons in managing change. Studies in Higher Education Vol 28 No 4. Nicol, D. J. and Macfarlane-Dick, D. (2006) Formative assessment and self-regulated learning: A model and seven principles of good feedback practice, Studies in Higher Education 31(2) 199-218. Peelo, M. and Wareham, T. (eds) (2002) Failing Students in higher education Buckingham, UK, SRHE/Open University Press. Race, P. and Pickford, R. (2007) Making Teaching work: Teaching smarter in post-compulsory education, London, Sage Race, P. et al (2009) Using peer observation to enhance teaching Leeds metropolitan Press Leeds.

16. References 3 Renfro, W. L. and Morrison, J. L. (1983) Anticipating and managing change in educational organisations, Educational Leadership Association of Supervision and Curriculum Development Roxa, T. and Martensson, K. (2009) Significant conversations and significant networks- exploring the backstage of the teaching arena Studies in Higher EducationVol 34 no 5 p547-559 Rust, C., Price, M. and O’Donovan, B. (2003). Improving students’ learning by developing their understanding of assessment criteria and processes. Assessment and Evaluation in Higher Education. 28 (2), 147-164. Ryan, J. (2000) A Guide to Teaching International Students, Oxford Centre for Staff and Learning Development. Sadler, D. R. (1989) Formative assessment and the design of instructional systems Instructional Science 18, 119-144. Yorke, M. (1999) Leaving Early: Undergraduate Non-completion in Higher Education, London: Routledge.