An experiment in institutional change

1. A new model for transforming the teaching in post-secondary education focus on science but likely general An experiment in institutional change 1. Motivation 2. Goals and brief description 3. Some early results and observations 4. Underlying ideas/rationale for design

2. Science (physics) education research results (as branch of physics research) • Quantifying failures of traditional teaching and assessment practices • Good validated assessments in widespread use • (conceptual mastery) • Identifying specific student difficulties (“misconceptions”) and how to address • Quantifiably and reliably better instructional approaches -- everyone, not just future scientists • Effective uses of technology consistent with principles from learning research

3. Major advances past 1-2 decades Consistent picture  Achieving learning brain research classroom studies cognitive psychology

4. Average learned/course 16 traditional Lecture courses Fraction of unknown basic concepts learned Measuring conceptual mastery • Force Concept Inventory- basic concepts of force and motion 1st semester physics Ask at start and end of semester-- What % learned? (100’s of courses) improved methods On average learn <30% of concepts did not already know. Lecturer quality, class size, institution,...doesn't matter! Similar data for conceptual learning in other courses. R. Hake, ”…A six-thousand-student survey…” AJP 66, 64-74 (‘98).

5. How to make these more effective research-based teaching practices the norm in every post-secondary classroom? The goal of CUSEI 3 yrs CWSEI 2 yrs Detailed goals Summary of results How being implemented Design principles Lessons learned

6. Attempting to change an ancient culture “There is nothing more difficult to take in hand, more perilous to conduct, or more uncertain in its success, than to take the lead in the introduction of a new order of things. For the reformer has enemies in all those who profit by the old order, and only lukewarm defenders in all those who would profit by the new order, this lukewarmness arising partly from fear of their adversaries … and partly from the incredulity of mankind, who do not truly believe in anything new until they have had actual experience of it.” – Machiavelli SEI finding #1 IS EXACTLY CORRECT-- true for faculty members, administration, students, alums, funders, ...

7. 1. Detailed goals Changing major research universities  set standards for teaching & learning science Focus on the departmental level  decide what and how to teach in science unit of change

8. Department- For each course & program offered “SEI Holy Trinity” 1st: Decide on learning goals. (what should studentsbe able to do?) 2nd: Rigorously assess student learning on an ongoing basis. 3rd: Apply research-based teaching techniques to improve student learning. Measure progress! (technology to improve effectiveness & save time) Ensure materials, data, assessment tools, software,… saved, reused, improved. (like scientific research)

9. semi-hidden change Student learning to become departmental rather than individual instructor responsibility

10. Ultimate goal… SEI model for all universities • Incentives to copy? • 1) convincing data that improves student learning • 2) responds to calls for accountability-- shows what • students are learning per course and program • How we make it cheaper for others? • online data base of new designed courses. • materials, assessment tools, evidence, etc. • freely available.

11. 2. Summary of results --what has been accomplished? U. Colorado--3 years into 5-6 year plan. Int. Physio, Geo. Sci., MCDB-- Over 80% of faculty have some involvement. About 60% have significantly changed how they teach. ~80% of their undergrad credit hours. Physics and Chemistry & Biochem-- Substantial impacts, many faculty members. • Establishing learning goals (what students can do)-- better guidance to students & coherence in curriculum • Rigorous assessments: learning & student attitudes • Improved teaching methods-- engagement, interactivity, feedback, higher level thinking • TA training and faculty development • Research on learning: content, concepts, and attitudes Improved 53 courses, > 10,000 student-courses/year

12. What accomplished at UBC (2 years into 6 year plan, ramping up incrementally) After 2 years, dramatic progress in Earth and Ocean Sciences Dept. at UBC. Most faculty, many courses changing. Several other departments ramping up Not always so successful! Some depts. much slower progress Some science departments not participating or only doing small projects

13. 3. How SEIs being implemented • 5 departments, selected competitively on basis • of proposals. Commitment and readiness to pursue “SEI Holy Trinity” & sustainability plans/structure. • Focused $$$ ($1-2 M) and guidance to Department. • Support over 6 yrs-- then self-sustaining (no ongoing extra $$) one-time investment permanent impact $5 M at Colorado, $12 M Cn at UBC CW--“Hands-semi-on” funding agency

14. “Science education specialist” approach not required, but dominates $$$ and activity. SES expert in discipline, hired by department, trained in sci. ed., research and assessment (by me). Work collaboratively with 1-3 faculty to implement changes in teaching and assessment for a course (SEI Trinity). Provide 1) Expertise 2) Labor (design, implement, assess, sustain) 3) Push research agenda and methods

15. 4. Design principles of the SEIs • Many elements common to any organizational change • Incentives at level of both the individual and the relevant unit for change • Change requires investment (time, ... =$$$)

16. What is the unit for change? The department (in research university). Decide what gets taught, who teaches it, how it is taught. If want change at that scale, must focus resources to match. Major research department budget $20-40 M/yr • Incentives at individual and departmental level. • Otherwise, never overcome ability of old (or young) curmudgeon to block change.

17. How much should it cost? Can’t you change the culture of a major science dept. for $100-200 K, instead of $1-2 M+? Change costs time=money. Industry-- major shift involves 5-10% or more reallocation of annual budget. SEI based on 5% annual budget for 5 yrs-- Looks about right. 1) Gets full department attention, 2) Roughly (?) covers cost of labor required to transform most courses (= involve most faculty).

18. Other underlying ideas-- • 1. Much easier to get people to adapt existing culture/identity than invent new one. • (scientist-- teaching as scientific activity with • data and research based principles and practices) • 2. Physics Ed. research has impacted instruction. • Primary Drivers: • backed by data and publications, • fairly good easy to use standard assessment tools • topic and course specific proven interventions, many easy to incrementally adopt • advocates who know subject, speak same language • 3. Evidence is necessary. • (although far from sufficient)

19. 4. Competing for funding drives change MUCH better than simply making resources available. 5. Require good measurement of learning, but department decides on what should be learned and develops/accepts the assessment instruments. Acknowledges their expertise and ownership.

20. 5. What learned so far from SEI experiments? • I. Some aspects working well-- • SES model works well • Extensive dept-wide conversations on undergrad ed. • Meaningful learning objectives being developed • and assessments being rethought (but slooowly) • (but 1/2 the battle) • Many long overlooked opportunities for improvement and streamlining emerging • Faculty members enjoy talking to each other about • teaching in substantive ways--provide forum • Also enjoy approaching teaching as research • Steadily growing number of converted faculty who can be wonderful advocates

21. II. some major challenges(ideology) 1. Tyranny of content-- data only way to respond (?) 2. Threatening to a few faculty in variety of ways 3. “Students are lazier and less well prepared than last generation” (4000+ years and counting) 4. Individual course ownership 5. “Better instruction ≠ more work, more $$!” Efficiencies are possible. Its ok to use other’s material. 6. Incentive systems not connected to better student learning or using best practices.

22. 7. Discipline related cultural oddities (from N=2  ) ? intro geology chemistry biology x Can’t teach x, until cover a, b, c, d ...t, u, v, and w! d c b C.S. education??? a Physics Computer science

23. III. Some neutral lessons • 1. Lots of little stuff is important. • Teaching assignments • TA requirements and training, ... 2. Very high degree of similarity and challenges across the two universities and the various disciplines. What basic differences there are between universities are smaller than differences between disciplines.

24. a work in progress-- stay posted www.cwsei.ubc.ca (extensive set of resources for faculty and students, course archive (sei.ubc.ca) where course materials assessments, etc. preserved for dissemination) www.colorado.edu/sei/