ESCalate Research Project

1. ESCalate Research Project ‘Using reflective dialogue to assess professional learning’ website http://escalate.ac.uk/6333 Ruth Pilkington RMHPilkington@uclan.ac.uk

2. Non-formal Learning Formal Learning NEW HE STAFF EXPERIENCED HE STAFF Informal, on-the-job development Professional Dialogue Assessment Process A CPD FRAMEWORK FOR UCLan showing formal and informal learning routes to professional development and accreditation

3. The Uclan Approach Formal Courses • Toolkit • PGCert LTHE • Certificates/other PGCerts • PGDiploma • MEd • EdD Informal Process • Experienced • Discipline-specific • Reflective • Developmental • Discursive • Meaning-making = Professional Dialogue

4. Informal Learning Route:The ‘Professional Dialogue’ • Normally takes place between an experienced ‘other’ and the ‘student’. • It involves a 3-stage process: • an initial set-up dialogue, • an exploratory dialogic process, and • the assessed dialogue. • Stage 3 is recorded and involves a second assessor to act as moderator or second marker. • In Stage 3, one assessor comes from the subject or ‘field’ and acts as mentor through the preparation phase of dialogue, the other may have a broader, objective, generic teaching and learning perspective

5. Assessment and Reward: the Professional Dialogue showing Timeline, Input and Process Mentee Participant Mentor Set up Dialogue: high Mentor input; low Mentee ownership Dialogue 1: high Mentor leadership; increased Mentee ownership Dialogue 2: shared ownership; increased Mentee input and determination Assessed Dialogue: celebration of Mentee role and practice Adapted and adopted by Ruth Pilkington from the process outlined by Brockbank and McGill (2007)

6. Benefits • Ethos of process • Peer-led, supportive, celebratory, affirmative • Status to teaching and learning; space • Sharing and exchange • Communication medium: reliability • Rigour of process • Can work with teams – culture change • Meaning-making, reflective, subject and individual focus • Not ideal for new staff or directed development needs

7. The Professional Dialogue Model • Applies a model by Brockbank & McGill (2007). They outline specific conditions supporting a collaborative approach to process-oriented, active, peer-supported, reflective learning: • Ascertain a reflective dialogue has taken place (ideally with others); • Establish evidence of learner’s participation in dialogue; • Identify evidence of developmental process over time, regardless of the start / end point; • Ascertain evidence that process review has taken place, enabling students to take away understanding of the learning process and replicate it elsewhere. (p194) • Boud & Falchikov (2007) use a cognitive model of coaching particularly relevant to peer assessment of professional learning called ‘cognitive apprenticeship ‘. • Cognitive apprenticeship focuses on dialogue over an indefinite period using a structured process of ‘modelling, scaffolding, fading and coaching’ (Collins et al 1991:2, cited Boud & Falchikov, 2007:130). • Coaching forms a thread that runs throughout ensuring that the collegial relationship is built on trust, confidentiality and empowerment. • Meta-cognitive apprenticeship skills are developed in partnership with the experienced ‘other’ who initially models behaviours, scaffolds professional learning using reflective processes, encourages interrogation and engagement with self assessment, and then gradually fades out as ‘expert’.

8. The UK Professional Standards Framework

9. Project Process • Four UK Institutions • Professional Dialogues used • To assess academic leadership (Standard Descriptor 3) • To assess new lecturers wanting to meet Standard Descriptor 1 • To assess experienced lecturers seeking Standard Descriptor 2 (core lecturer role) • Dialogues recorded and transcribed • Analysis of themes emerging (Tag Cloud, thematic identification, reflective cyclical review). attentiveness to issues – a personal response, discussion with colleagues – strengthen and derive codes • Structure of dialogues and management of interaction; facilitation and support; • Evidence of reflection and evidence of meeting Standard Descriptors • Phatics , use of small talk • Rigour of dialogue as assessment tool • Judgement and power; performance and control • Storytelling, narrative of meaning, cosntruction • Reflection by team and review of data, experience • Interview data collected from participants and assessors on process

10. Group Activity • Explore within groups how this might work within your context • Examine examples provided to discuss issues of dialogue within the process • Can you use the model for your assessment? • (Share outlines from institutions and documents) • What would be the issues for you in terms of assessing professional learning?

11. Issues and Outcomes Power, Judgment Training and Support for Assessors Afternoon session

12. Review of Dialogues • Modelling of dialogue • Emergence of issues of power • Analysis of different types of dialogue; ways that power emerges • Evidence of judgment – a complex thing

13. Use of Questions • Open Qs / Direct Closed Qs / Open Qs • Redirect issues or focus • Closed questions move things on; focus attention • Open Questions probe and explore • Affirmatory and Summative statements

14. Dialogue – power and parity • Power (assessor): scene setting, introductions; interruptions, questioning, probing & direction, judgment • Power (assessee): taking ownership by managing exploration, redirecting questions, pursuing own line of thought, own judgments • Reflecting back at assessee; affirmation; noise; comments on issues; sharing own thoughts; empathy – support; meta-cognitory unpacking • Equality: shared exchanges; shared feelings; asides, exploratory detours

15. Enhancement components • Reflection: description, thinking things through, exploring own experience, reasons behind, recognition of own values • Storytelling: evidence, authenticity, entry to reflection, role and identity • Relationship building: sharing, humour, use of phatics, empathy • Meaning making: exploration, explanation, interpretation, reflecting back, meta-analysis, positioning within outcomes

16. Dialogue - Types • Parity, Equality • Assessor control monologue dialogue presentation interview Incr. Control by assessee Incr. Input by assessor interview supported monologue interrogation Incr. Input by assessee Incr. Control from assessor

17. QUESTION NEW QUESTION 1 PROBE Dialogue Cycles SHARE REVIEW SUMMARISE REFLECT REPEAT CYCLES FINAL STAGE REVIEW GAP CHECK FEED BACK 2 INTRODUCTION CONCLUSION TOPIC SET CONTEXT NEW TOPIC EXAMPLE AFFIRM RESPONSE LINK MADE JUDGMENT PROMPT / PROBE DESCRIBE

18. Institution A • Preparatory dialogue and final dialogue • 2nd final dialogue • Peer dialogue • Video of final dialogue by deaf studies • Mentoring process • Co-construction • Story telling • Turn-taking • Power and management • Open v. closed questions • Judgment

19. Assessment, Reflection and Professional Judgement within Dialogue – an exciting outcome Dialogue ICEBERG An Interesting ‘Sub-text’ • Assessment of professional knowledge • Judgment • Assessment of values • Use of reflective models to assess professional learning

20. Judgment • Expertise and experience of assessor • Comfort and familiarity with outcomes (UK PSF) and field (L&T), and appreciation of subject (of assessee) • Familiarity with setting: evidence, locus, complexity, authenticity – areas of activity • Reading body language, examples, stories, language – truth, reliability, values • Contextualising: alignment to outcomes; tacit v. explicit; hidden text • Gap analysis: Is it enough? What more is needed?

21. Use of Dialogue • Purpose of Dialogue • Position of Dialogue • Process • Training of Assessor, Participant

22. Brockbank A & McGill I 2007 Facilitating Reflective Learning in HE SRHE/OUP 2nd Ed. Bowen Clewley L ‘Assessing against competency standards in the workplace’ in 207-227 in Arguelles & Gonczi (2000) Eraut M 1997 Professional Learning and Competence Ghaye T & Lillyman S 2006 Learning Journals and Critical Incidents RP for HE Professionals 2nd Ed Quay Books London O’Donovan B, Price M and Rust C (2004) ‘Know what I mean? Enhancing Students understanding of assessment standards and criteria’ in Teaching in HEVol 9, No 3, July 2004 Knight P & Yorke M (2003) Assessment Learning and Employability SRHE/OUP Boud D and Falchikov N EdsRethinking assessment in HE 2007 Routledge Bryan C & Clegg K 2006 Innovative Assessment in HERoutledge Goodfellow R & Lea MR (2007) Challenging E-learning in the University: a literacies perspective SRHE / OUP Larrivee, Barbara(2008)'Development of a tool to assess teachers' level of reflective practice', Reflective Practice,9:3,341 — 360 Moon J (2004) A Handbook of Reflective and Experiential Learning: Theory & Practice RoutledgeFalmer Pedler M (1996), Action Learning for Managers,The Learning Company Project Shulman LS ‘Knowledge and Teaching’ pp61-77 in Leach J, Moon B (eds) (1999) Learners and Pedagogy Sage/ Paul Chapman pubs