U21 Educational Innovation Conference University College Dublin 31 October – 01 November 2013

2. U21 Educational Innovation Conference University College Dublin 31 October – 01 November 2013 Exploring the Cost & Benefits of Online Innovation Diana Laurillard London Knowledge Lab Institute of Education

3. Exploring the costs and benefits of online innovation in education Outline of the argument • Global demand for education • Why we need to understand costs and benefits • What it takes to teach with technology • Teaching on the largescale • Tools for teachers as designers • Modelling costs and benefits

4. The global demand for education The new UNESCO goals for education: • Every child completes a full 9 years of free basic education … • Post-basic education expanded to meet needs for knowledge and skills … (Draft for UNESCO post 2015 goals) By 2025, the global demand for higher education will double to ~200m per year, mostly from emerging economies(NAFSA 2010)  Implying significant graduate and teacher training growth for this level of schooling, FE and HE 1:25 staff:students??

5. Why we need to understand costs and benefits The global demand for HE is a demand for new pedagogies capable of educating millions – far from the current model Teaching has to adapt continually to rapid changes in opportunities from digital technology innovation, changing student needs, capabilities and expectations • teachers have to discover the digital pedagogic forms that scale up for large-scale high quality teaching The iterative and adaptive nature of design must be at the core of teaching innovation in HE in this rapidly changing world

6. Understanding high quality TEL MOOC Preparation time (fixed costs) MOOCvsstandard online course • Adaptive feedback (sim/models/games) • Expositions (lecture videos) • Automated grading (MCQs, quizzes) • Readings (pdfs) • Collaboration activities (wiki) • Peer group discussion (forums) • Peer grading against criteria (tests) • Tutored discussion (forums) • Tutor feedback (e-portfolio) • Adaptive feedback (sim/models/games) • Expositions (lecture videos) • Automated grading (MCQs, quizzes) • Readings (pdfs) • Collaboration activities (wiki) • Peer group discussion (forums) • Peer grading against criteria (tests) • Tutored discussion (forums) • Tutor feedback (e-portfolio) • Adaptive feedback (sim/models/games) • Expositions (lecture videos) • Automated grading (MCQs, quizzes) • Readings (pdfs) • Collaboration activities (wiki) • Peer group discussion (forums) • Peer grading against criteria (tests) • Tutored discussion (forums) • Tutor feedback (e-portfolio) Support time (variable costs)

7. What it takes to teach with technology The teaching workload is increasing in terms of Planning for how students will learn in the mix of the physical, digital and social learning spaces designed for them Curating and adapting existing content resources Designing the activities, tools and resources that afford all types of active learning Personalised and adaptive teaching that improves on traditional methods Providing flexibility in blended learning options Guiding and nurturing large cohorts of students Using learning technologies to improve scale AND outcomes BUT: Institutions and teachers do not typically plan for the teaching workload implied by these learning benefits nor for the need to collaborate to innovate with technology

8. The MOOC as ‘large-scale’ pedagogy MOOCs are not large scale – Duke University Completed = 2% of enrolment, 25% of ‘engaged’ Duke University Report 2012

9. The MOOC as ‘large-scale’ pedagogy Average student numbers per course - Edinburgh 51500 20500 15000 6000 5500 Completed = 10% of enrolment, 37% of ‘engaged’ MOOCs @ Edinburgh 2013 – Report #1

10. The MOOC as undergraduate education Not for undergraduates 72% have degrees Enrolled students Duke University Report 2012

11. The MOOC as undergraduate education Not for undergraduates 70% have degrees 40% 30% 17% 10% Enrolled students 3% MOOCs @ Edinburgh 2013 – Report #1

12. What it takes to teach a basic MOOC vs the Duke MOOC Total teaching time The variable cost of high quality teaching does not achieve economies of scale if you maintain the same pedagogy Prep time = 420 Preparation time = 420 hrs Duke University Report 2012

13. Modelling the benefits and costs It’s important to understand the link between the pedagogical benefits and teaching time costs of online learning – especially for the large-scale What are the new digital pedagogies that will address the 1:25 student guidance conundrum? – How to shift variable cost support to fixed cost support? Can we develop a viable business model that will make HE affordable for novice learners?

14. Pedagogies for supporting large classes Concealed MCQs The (virtual) Keller Plan The vicarious master class Pyramid discussion groups Conceal answers to question Ask for user-constructed input Reveal multiple answers Ask user to select nearest fit Introduce content Self-paced practice Tutor-marked test Student becomes tutor for credit Until half class is tutoring the rest 240 individual students produce response to open question Pairs compare and produce joint response Groups of 4 compare and produce joint response and post as one of 10 responses... 6 groups of 40 students vote on best response Teacher receives 6 responses to comment on Tutorial for 5 representative students Questions and guidance represent all students’ needs

15. Pedagogies for supporting large classes Concealed MCQs The (virtual) Keller Plan The vicarious master class Pyramid discussion groups Laurillard, 2002 Keller, 1974 Mayes et al, 2001 Gibbs et al, 1992 The traditional pedagogies for large classes could be redesigned as digital formats

16. Tools for academics as learning designers • Library of learning designs indexed by learning outcome & topic • Three different subject instances for each to promote migration of good pedagogy across domains • Academic ‘adopts’ a design and adapts it as needed using edit tools & links to other resources – creating a computationally interpretable design • Library of OERs for academics to link to from learning designs • Feedback on ‘learning experience’ created, total learning time, and teacher preparation and contact time • Prompt to include ‘production’ activity for collecting learning analytics on outcomes • Design sent to Moodle to test with students – collects data • Students can annotate design for detailed evaluation • Academic redesigns as needed – tests again – publishes to Library • Library of learning designs indexed by learning outcome & topic • Three different subject instances for each to promote migration of good pedagogy across domains • Academic ‘adopts’ a design and adapts it as needed using edit tools & links to other resources – creating a computationally interpretable design • Library of OERs for academics to link to from learning designs • Feedback on ‘learning experience’ created, total learning time, and teacher preparation and contact time • Prompt to include ‘production’ activity for collecting learning analytics on outcomes • Design sent to Moodle to test with students – collects data • Students can annotate design for detailed evaluation • Academic redesigns as needed – tests again – publishes to Library

17. Tools for teachers as learning designers Teachers as design scientists need the tools for innovation http://tinyurl.com/ppcollector To find or create new ideas Adopt Adapt Test To collect learning analytics Redesign Analyse Publish Creating knowledge about effective blended and online pedagogies

18. Tools for academics as learning designers http://tinyurl.com/ppcollector

19. Tools for academics as learning designers • Library of learning designs indexed by learning outcome & topic • Three different subject instances for each design to promote migration of good pedagogy across domains • Academic adopts’ a design and adapts it as needed using edit tools & links to other resources – creating a computationally interpretable design • Library of OERs for academics to link to from learning designs • Feedback on ‘learning experience’ created, total learning time, and teacher preparation and contact time • Prompt to include ‘production’ activity for collecting learning analytics on outcome • Design sent to Moodle to test with students – collects data • Students can annotate design for detailed evaluation • Academic redesigns as needed – tests again – publishes to Library

20. The design cycle for teaching Build on others’ tested designs Make links to existing content resources Redesign existing content resources? Building teaching community knowledge

21. Similar to the design cycle for science What is the teaching design equivalent of the journal paper? Building scientific knowledge

22. A learning design for Ed students Make links to existing content resources Check the feedback on the overall distribution of learning activity Export to Word or LMS/VLE Add link to an OER, e.g. a digital tool for practice The learning design as a ‘content shell’ ready to receive content products?

23. Export to Moodle for Ed students • Interprets metadata to assign activity types in Moodle (or other LMS) • Attaches resource links • Inserts study guidance from text in the pattern • Collects data on student performance on TEL-based activities

24. Reversioned for Med students • Same pedagogical pattern • Same study guidance except for subject content terms and resources • Different resources attached • Same type of evidence data (?)

25. Modelling the pedagogic benefits A computational representation can analyse how much of each learning activity has been designed in Categorised learning activities Conventional Blended Analysis shows more active learning

26. The Course Resource Appraisal Model … Run No. of students Run 1 15 Run 2 20 Run 3 20 Run No. of students Run 1 15 Run 2 30 Run 3 60 • Run 1 Run 2 Run 3 • Students 15 20 20 • Profit -£27k £4k £11k • Run 1 Run 2 Run 3 • Students 15 30 60 • Profit -£27k £11k £38k

27. Modelling the teaching time costs An interactive learning design tool can analyse how much design and teaching time is needed Design hrs Teaching hrs Design hrs Teaching hrs Yr1 Yr2 Yr3 Yr1 Yr2 Yr3 40 40 40 40 80 160 Students Students

28. Managing the fixed costs of teaching What academics need Do publishers help? • Create and test professional content • Curate existing professional content • Adapt and customise existing content • Select and organise community content • Create reusable learning designs as ‘content shells’ • Adapt and customise existing learning designs ✓✓✓ ✓✓✗ ✓✗ ✗ ✗ ✗ Could the learning design interface to the VLE, for creating ‘content shells’ suggest a new kind of content product?

29. Further details… tinyurl.com/ppcollector http://buildingcommunityknowledge.wordpress.com Teaching as a Design Science: Building pedagogical patterns for learning and technology (Routledge, 2012) d.laurillard@ioe.ac.uk

30. Teaching as a design science: Summary The global demand for education requires investment in pedagogic innovation for high quality large scale teaching We need to design, test, use, and reuse high quality open educational resources that amortise high fixed costs over large student cohorts We need to invent variable cost pedagogic innovations that supports students at a better than 1:25 staff-student ratio Teachers need the tools to design, test, gather the evidence of what works, and model benefits and costs Teachers are the engine of innovation – discovering the means by which we fulfill our social responsibility of doing more for less to meet that global demand

31. Break-out questions • Should our universities play any role, or take any responsibility for meeting the global demand for HE? [Slides 2, 3] • Can universities and teachers plan for the teaching workload implied by the learning benefits that technology can confer? [Slide 4] • Can academic teachers play a part in discovering the digital pedagogic methods that will scale up to provide large-scale high quality teaching in order to reduce the costs to students/government of meeting the national demand for HE and lifelong learning? If not – who? [Slide 5, 6] • Can we use pedagogy-driven learning analytics to understand better the relationships between teaching and learning? [Slide 6] • Can university teachers collaborate to innovate with technology? [Slide 7] • Do we know the real costs of current teaching as a business model with a related return? How can we understand the new cost models for moving to online courses if we do not have a clear activity-based cost model for current teaching? [Slides 8, 9]