Insight research, design research and impact on practice Engineering research in education

1. Insight research, design research and impact on practice Engineering research in education Hugh Burkhardt Shell Centre School of Education, University of Nottingham

2. Outline • Paradigms for research and development • Delivering effective tools and processes • Mathematical literacy – a case study • Cost effectiveness • Building a design community

3. The Shell Centre Team • Malcolm Swan, Daniel Pead, Rita Crust, Alan Bell, HB, with many associates • Does ‘engineering research’ – design and development of: • teaching materials and processes • assessment tasks • professional development materials and processes • Tools and strategies for system change with some associated ‘insight research’ • Based in the University of Nottingham School of Education • Work with many others, notably Michigan State, Berkeley and school systems in UK and US • Contact: Hugh.Burkhardt@nottingham.ac.ukwww.mathshell.com

4. Research in Education Three research traditions, and products: • Humanities: insight focus > papers • Science: insight focus > papers • Engineering: impact focus >new tools, processes + papers R<–>P Theme: for research-based practice we need a better balance of research effort – more engineering (cf also medicine,…)

5. Influence on policy and practice • Humanities • dominates policy making (all can play) plausibility ~ ‘common sense’ – often fails • Science • dominates research, “informs” policy, but no body of agreed results > little influence • Engineering • still marginal, except for tests* some systematic work in some places eg here

6. What is design research? Design Research has emerged as an accepted part of educational research, with a strong input from Cognitive Science. Key features include: • insight focus > products and papers • realistic classroom situations • exploring teaching and learning • theory building but with • atypical teachers • exceptional support • no claim to wider usability >no direct impact Engineering research sees the products as first drafts

7. What is design research? For more, see e.g. Educational Design Research eds Jan van den Akker, Koeno Gravemeier, Susan McKenney, Nienke Nieveen Routledge 2006 “pragmatic, grounded, interactive, iterative and flexible, integrative, and contextual”

8. R<–>P process – key ingredients • body of reliable research • exemplars of tools and processes • funded development programs • stable design teams • systematic iterative development • clear range of effectiveness • comparative evaluation-in-depthfunded by clients who understand the process see HB+AHS Educ Researcher 32 (2003)

9. …but the academic value system Favours • new ideas over reliable research • new results over replication and extension • trustworthiness over generalizability • small studies over major programs • personal research over team research • first author over team member • disputation over consensus building papers overproducts and processes

10. What is good design? Good design: • makes users more powerful via tools and processes that fit their purposes • gives users, and others, pleasure • does these in a cost-effective way

11. Types of contribution Good design can: • make something more widely availableprinting press, Model-T Ford, www,..slates, paper, ‘lesson study’, ….. • lead to continuous improvementcars, TV, medical treatments,…teaching materials,….. • enlarge possibilities for practicetelephones, search engines, arthroscopescomputers, geometric software, microworlds • provide existence proofs

12. What do good designers do? They know how to • use research results and design skills to • improve ‘best practice’ • tackle new challenges effectively • pass on their knowledge to • other practitioners • novices through their materials.

13. Approaches to Innovation • Craft-based innovation • Builds on authors’ successful practice • Simple and inexpensive process • Research-based innovation • Builds on ‘best practice’, research, design skills • Slow, complex and expensive R&D processthat uses trialling to build robust innovations ie ambitious innovations that work well

14. Craft-based development model The normal process of authorship • Write draft materials • Circulate to an expert group • Discuss and revise • Publish

15. Craft-based development model Questions: • How reliable is the extrapolation of author's professional experience to others’ needs in the new situation? • How effective is the communication of the author's intention to the target community of users? OK for small changes Extrapolation and communication unreliable for real innovation

16. Research-based development model • Input from: • earlier research and development worldwide • craft skills of best practice • Design skill and creativity • led by designers of exceptional achievement • well-defined locus of ‘design control’ • Systematic development iterative rounds of developmental trials with: • rich, detailed feedback guiding • revision of the materials in each round so that intentions and outcomes converge • Continuing customer feedback

17. Educational design principles Heuristic, phenomenological theory: • Some based on ‘insight research’, eg • active learning • constructive • build multiple connections • Others design-based, eg • role shifting • cognitive conflict • student ‘ownership’ Design theory is not often discussedin enough detail to be useful

18. Design beyond just principles Design brilliance is more than these: • ‘Surprises’ that are clearly ‘right’ • Handling complexity simply • Controlled innovation • Balance in all aspects We know it when we see it – iPod,…

19. The design process • Seek excellence in design. What is it? • a coherent plan, influenced by research • lessons that work well with target S • materials that communicate to target T • robustness across circumstances of use • enjoyment in use by S and T • Develop in realistic circumstances • Cover all the main goals • Sell to many users

20. Mathematical Literacy exemplar development need ML enables non-specialist adults, if they are taught how,to benefit from using mathematics in their everyday lives to better understand the world they live in, and to make better decisions. “The sophisticated use of, often elementary, mathematics” also calledfunctional mathematics, quantitative literacy, numeracy … Post-11 mathematics is non-functional for most people

21. “Mathematics is useless”

22. Who needs it? Plan • Potential secretaries asked to critique and complete the spreadsheet for planning a conference budget

23. Sudden Infant Deaths = Murder? • In the population as a whole, about 1 baby in 8,000 dies in an unexplained "cot death". The cause or causes are at present unknown. • Three successive babies in one family have died. • The mother is on trial. An expert witness says: "One cot death is a family tragedy; two is suspicious; three is murder. The odds on just two deaths in one family are 64 million to 1" Discuss the reasoning behind the expert witness' statement, noting any errors, and write an improved version to present to the jury.

24. The modelling process The real world Problem Report Validate Formulate Solve Interpret Mathematics

25. Problem Report Validate Formulate Interpret Dysfunctional math curriculum The real world Solve Mathematics

26. Oft-neglected Design Aspectsa view • Is this outward-looking mathematics? • few students will become mathematicians • math can give them power in their lives • does this curriculum do that? for all? (cf ELA) or is it “just math” (RPF) symptoms: all topic focus, no modelling, tasks • What ‘dimensions of engagement’? • many students lack interest in math itself • is “make the math interesting” all this does? symptoms: variety of activities, of tasks (cf ELA)

27. Oft-neglected Design Aspects a few more • Does this develop student autonomy? • reliable imitation is not enough to do math • what ‘transfer distances’ do the tasks cover? • how long are the chains of reasoning? • …involving, which problem solving phases? symptoms: no linked phases, similar tasks together • Does this give teachers enough support? • student centered teaching is difficult • it is easy to overload the teacher • what design tactics are used to avoid this? symptoms: teacher in hot seat, centre-stage; no support tactics; too much innovation at once; ……

28. Modelling • Joe buys a six-pack of coke for $3 to share among his friends. How much should he charge for each bottle? • If it takes 40 minutes to bake 5 potatoes in the oven, how long will it take to bake one potato? • If King Henry 8th had 6 wives, how many wives had King Henry 4th?

29. Plan a trip: fault finding and fixing Meet at Loughborough station at 7.23 am. Buy tickets and then catch the train to Derby. This arrives at 7.51 am. At Derby, catch the 8.20 am train to Cromford. This arrives at 8.41 am. Here are the instructions for getting to the Cycle Hire centre: “Turn left as you come out of Cromford station, walk along by the river and down Mill road. Cross over the A6. Walk up Cromford hill for about 1/2 mile and you will see.. Alison and two friends has planned a cycling trip around Derbyshire on Saturday. Here is their plan for the day. Read through the plan and the information sheets (next page). If you find a mistake, or realise something has been forgotten, write it down and say how they should change the plan.

30. Authentic information sheets

31. Design a tent - open task

32. Design a tent - scaffolded task It must be big enough for two adults to sleep in. The sloping sides and the two ends will be made from a single, large sheet of material. Write down the height and breadth (shoulder to shoulder) of a typical adult. Write down suitable measurements for CD, DE and AP. Remember to leave space for baggage. Draw a sketch to show how you will cut the material from a single piece. Label your drawing with letters A to G. Use Pythagoras theorem to calculate the length of AD. Calculate angles <ADP and <ACD using trigonometry.

33. Design a tent - final version Your task is to design a tent like the one in the picture. It must be big enough for two adults to sleep in (with baggage). The tent should be big enough so that someone can move around while kneeling down. Two vertical tent poles will hold the whole tent up. Estimate the relevant dimensions of a typical adult. Estimate the dimensions of the base of your tent. Estimate the length of the vertical tent poles you will need. 3. Show how you can make the top and sides of the tent from a single piece of material. Show all the measurements clearly. Calculate any lengths or angles you don't know. Explain how you figured these out.

35. “Bowland Maths” • ~20 “case studies, including: • Reducing road accidents • How risky is life? • “You reckon?” • Alien invaders • Professional development • 5 module package, activity based • Delivered on-screenon web June 17 www.bowlandmaths.org.uk

36. Body of reliable research needed • RF: universities, journals, funders encourage: • team research with long timescales • on available treatments as well as new • coherent in-depth work to give diverse data Mathematical Literacy • Extend current research, which shows students need: • knowledge of concepts and skills • strategies for problem solving and modelling • metacognitive control of their reasoning, and the • disposition to view situations mathematically

37. What development skills are needed? • The team needs: • Systematic methods of observation • Interview skills • Protocols related to the design goals • Methods for analysing observation reports, student work, interviews • Design skill in using this rich feedback systematically to improve the materials. ie as products are developedin other fields

38. Stable design teams need • long term coherent programs in each field • developing a community that works towards • higher standards of design and development • training and career progression for designers • Wider recognition of its importance • recognising the intellectual value of designas in medicine, engineering, fine arts Mathematical Literacy needs all of these.

39. Clear range of effectiveness • Trials will give an indication of conditions for successful use, including: • Prior learning of students • Teacher skills and support • Time and support for absorbing the change • Feedback from widespread use will refine this Most research fails to explore boundaries of validity– essential for design

40. Comparative evaluation-in-depth • Guides choice of materials and approaches • Informs improvement and future development To do this well, evaluation needs to: • look at widely available treatments • across variables: users, use, outcomes • compare alternatives – many dimensions Still rare in all countries Mathematical Literacy will need all these in due time – typical time from agreeing goals to stable curriculum implementation ~ 10-15 years

41. Math Literacy ~ proof of concept For Mathematical Literacyunits so far, it seems: • All students succeed and enjoy the work • FM narrows the range of performance • Many, but not all, teachers can handle this work with just the materials – more with live PD training • 1 or 2 new three-week units per year is digestible More research needed, across more exemplar units to warrant such general statements

42. Long term growth of design skill • Freudenthal Institute • 40 years, 40 fte people, coherent support • US: EDC, TERC, LHS,.….. • 40 years, ~ 100 people, one-off projectsno coherent support • Shell Centre • 40 years, ~ 5 fte people, strands of work, no coherent support Such groups are rare worldwide currently, you need luck and cunning to survive

43. How much does engineering cost? Engineering research is more expensive. • NSF curriculum projects • ~$1,000,000 for ~160 hours >> $6,000/ hour • Shell Centre curriculum units • 10-15 hours ~ $30,000 per classroom hour. Is this good value?

44. How much does engineering cost? Analysis (US context) • whole school curriculum ~15,000 hours 14 years*200 days*5 hours • 3 parallel developments ~ $1.5 billion 15,000 hours*3*$30,000/hr • Over 10 years ~0.05% of running cost$1.5 Bn/($300Bn/yr*10 years) Other changing fields? R&D typically 5-15%80% development, 20% basic research

45. Why is design flair ignored? • Tradition that, for most, math and science are necessary evils – and boring • Focus on design principles but details matter(as in all art) • No comparative evaluation in depth of learning or attitudes (beliefs and motivation) • The ‘classics’ of math and science are not digestible K-12 (cf music, literature), so • Major burden on educational design, but • No profession of educational designer some systematic work in some places eg here

46. Building a Design Community • Improve standards of design by: • sharing methods and experience • developing theory of design • providing training • Increase recognition in: • government via payoff >> funding • academia via argument, funding.. • Better career paths – will follow • ISDDE, a growing group, is making a start International Society for Design and Development in Educationwww.ISDDE.org

47. ISDDE Conference 2008 • FI @ Egmond aan See, NL • June 30 – July 2 • “Linking design research to professional educational design” • Working groups: • curriculum design • designing professional development • assessment design • educational software • Contact Peter Boon, Conference Chair p.boon@fi.uu.nl http://www.fi.uu.nl/isdde/welcome.html

48. Summary • Design and development – society needs to develop the engineering research approach • ‘Functional Mathematics’ – we need to make good our claims of power for all • Build a design community to get higher standards, and improved recognition for designers