750 likes | 799 Views
Koli’2007 – Keynote. Constructive Alignment for Teaching Computer Science. Claus Brabrand ((( brabrand@itu.dk ))) ((( http://www.itu.dk/people/brabrand/ ))) Associate Professor, IT University of Copenhagen Denmark. 1. 2. 3. 4. 5. 6. utline. O. Introduction:
E N D
Koli’2007 – Keynote Constructive Alignment for Teaching Computer Science Claus Brabrand ((( brabrand@itu.dk ))) ((( http://www.itu.dk/people/brabrand/ ))) Associate Professor, IT University of Copenhagen Denmark
1 2 3 4 5 6 utline O • Introduction: • Background, Motivation, and Expectations • The Theory of Constructive Alignment: • “Teaching Teaching & Understanding Understanding” • From Theory to Practice: • “From content to competence” --- short (10’) break --- • Implementing Alignment (case study): • Implementing alignment in Teaching Computer Science • Computer Science Analysis: • Preliminary Analysis of DK experiences (~new grade scale) • Open discussion: • Q’n’A / open debate / discussion, …
T First: exercise • Before we start: Post-It exercise: • Write down answer to: • "what is good teaching?" • 2) Swap Post-Its...
Background (~ this talk) • Concurrency 2004+2005: • "Pre-alignment" • Exposure to teaching/learning theories: • “Constructive Alignment” • “The SOLO Taxonomy” • Concurrency 2006+2007: • "Post-alignment"
1 2 3 4 5 6 utline O • Introduction: • Background, Motivation, and Expectations • The Theory of Constructive Alignment: • FILM: “Teaching Teaching & Understanding Understanding” • From Theory to Practice: • “From content to competence” --- short (10’) break --- • Implementing Alignment (case study): • Implementing alignment in Teaching Computer Science • Computer Science Analysis: • Preliminary Analysis of DK experiences (~new grade scale) • Open discussion: • Q’n’A / open debate / discussion, …
Let's watch the short-film... ((( ))) Teaching Teaching & Understanding Understanding Inspired by: "Teaching for Quality Learning at University", John Biggs Available on DVD through Aarhus University Press: ((( http://www.daimi.au.dk/~brabrand/short-film/ ))) Features Epilogue by John Biggs, DVD menu, and subtitles inEnglish, French, Spanish, Portuguese, Italian, German, and Danish Won “The Golden Ratio 2006” Award for “Best Educational Video” (~4000 DVDs sold)
1 2 3 4 5 6 utline O • Introduction: • Background, Motivation, and Expectations • The Theory of Constructive Alignment: • FILM: “Teaching Teaching & Understanding Understanding” • From Theory to Practice: • “From content to competence” --- short (10’) break --- • Implementing Alignment (case study): • Implementing alignment in Teaching Computer Science • Computer Science Analysis: • Preliminary Analysis of DK experiences (~new grade scale) • Open discussion: • Q’n’A / open debate / discussion, …
From Content to Competence • The “pre-alignment” Concurrency course aims: • Given in terms of a 'content description': • Essentially: • The goal is...: • To understand: • deadlock • interference • synchronization • ... This is a bad idea for 2 reasons...!
Problem with 'content' as aim • What is the problem with 'content'as learning objectives ?!? analyze ... theorize ... analyze systems explain causes explain deadlock describe ... • Objective: • To understand: • deadlock • interference • synchronization • ... Stud. C agreement tacit knowledge from research-based tradition (not known by stud.) name solutions recite conditons Teacher analyze systems explain causes Stud. B BUT, even if it were possible to agree, we know that the exam will dictate the learning anyways. Stud. A Censor
Problem with 'understanding' • Why not use 'understanding'as learning objectives ?!? • Objective: • To understand: • deadlock • interference • synchronization • ... concept of deadlock ?! The answer is simple: It cannot be measured (!)
[ Competence := knowledge + capacity to act upon it ] 'Competence' as objectives ! • 'Competence' as learning objectives ! • Evaluation = Have the student do something, and then measure product and/or process • Objective ! • To learn to: • analyze systems for... • explain cause/effects... • prove properties of... • compare methods of... • ... Note:'understanding' is (of course)pre-requisitional (!) Note': inherently operational (~ verbs) 'SOLO' = Structure of the ObservedLearning Outcome
T Neighbour Discussion Discuss with neighbour: "does this make sense ?!?" (content competence)
SOLO 5 to generalize to hypothesize to theorize ... "extended abstract" to relate to compare to analyze ... SOLO 4 "relational" to classify to combine to enumerate ... SOLO 3 "multi-structural" to identify to do procedure to recite ... SOLO 2 "uni-structural" no understanding irrelevant information misses point ... SOLO 1 "pre-structural" Advantages of 'SOLO' • Advantages of 'SOLO': • Constructed for research-based (university) teaching • Converges on research (at SOLO 5) depth (qualitative levels) surface (quantitative levels)
Note: the list is non-exhaustive Graphic Legend immediately relevant aspects – given! related or hypothetical – not given! irrellevant or inappropriate student response x R x R' x R R'' x x R R1 R2 R R3 R SOLO (elaborated) QUALITATIVE QUANTITATIVE SOLO 2 ”uni-structural” SOLO 3 “multi-structural” SOLO 4 “relational” SOLO 5 “extended abstract” • define • identify • count • name • recite • paraphrase • follow (simple)instructions • … • combine • structure • describe • classify • enumerate • list • do algorithm • apply method • … • analyze • compare • contrast • integrate • relate • explain causes • apply theory (to its domain) • … • theorize • generalize • hypothesize • predict • judge • reflect • transfer theory (to new domain) • …
Concrete Example andConcrete Recommendations (4x) 1) Use 'standard formulation': a) puts learning focus on the student b) competence formulation: "to be able to" Intended Learning Outcomes [Genetics 101] After the course, the students are expected to be able to: locate genes on chromosomes do simple calculations : (e.g., recombination frequencies, in-breeding coefficients, Hardy-Weinberg, evolutionary equilibria). describe and perform connexion-analysis describe fundamental genetic concepts: (e.g., mutation variation, in-breeding, natural selection). describe and analyze simple inheritancies analyze inheritance of multiple genes simultaneously 4)Avoid 'understanding-goals': "To understand X", "Be familiar with Y", "Have a notion of Z", ...! V N N V N V V V N V V N V 3) Use 'Verb + Noun' formulation: What the student is expected to dowith a given matter . 2) List sub-goals as 'bullets': Clearer than text N V
Concurrency: analyze for deadlock compare models T Post-It exercise Write down 1-2 key competences (i.e., verbs) (for your course)
1 2 3 4 5 6 utline O • Introduction: • Background, Motivation, and Expectations • The Theory of Constructive Alignment: • FILM: “Teaching Teaching & Understanding Understanding” • From Theory to Practice: • “From content to competence” --- short (10’) break --- • Implementing Alignment (case study): • Implementing alignment in Teaching Computer Science • Computer Science Analysis: • Preliminary Analysis of DK experiences (~new grade scale) • Open discussion: • Q’n’A / open debate / discussion, …
10' Break Please put the Post-Its on the wall "What is good teaching?" Key competences (in your course)
1 2 3 4 5 6 utline O • Introduction: • Background, Motivation, and Expectations • The Theory of Constructive Alignment: • FILM: “Teaching Teaching & Understanding Understanding” • From Theory to Practice: • “From content to competence” --- short (10’) break --- • Implementing Alignment (case study): • Implementing alignment in Teaching Computer Science • Computer Science Analysis: • Preliminary Analysis of DK experiences (~new grade scale) • Open discussion: • Q’n’A / open debate / discussion, …
Disclaimer The point of this part is: • not to exhibit aperfectly aligned course; • but to show how the principles of alignment can be put to use (esp. howILO’s may serve as guidelines for exam and teaching form).
Implementation Process • Process(course specific): 1) Think carefully about: overall goal of course (what are the stud. to learn?) 2)Operationalize these goals: and express them as intended learning outcomes alignment learning incentive learning support 3)Choosecarefully the form(s) of examination (~ intended learning outcomes) 4)Choosecarefully the form(s) of teaching (~ intended learning outcomes)
Starting Point • Content description (Concurrency '04+'05): What is the overall goal of the course...? (what are the students to learn)
Overall Course Philosophy • Model-Based Designfor Concurrency:
Implementation Process • Process(course specific): 1) Think carefully about: overall goal of course (what are the stud. to learn?) 2)Operationalize these goals: and express them as intended learning outcomes alignment 3)Choosecarefully the form(s) of examination (~ intended learning outcomes) 4)Choosecarefully the form(s) of teaching (~ intended learning outcomes)
#2 #1 . #3 . S M . I Model-based design for Concurrency T Intended Learning Outcomes • Intended Learning Outcomes(based on The SOLO Taxonomy): Note:explicitly included as a non-goal
Implementation Process • Process(course specific): 1) Think carefully about: overall goal of course (what are the stud. to learn?) 2)Operationalize these goals: and express them as intended learning outcomes alignment learning incentive 3)Choosecarefully the form(s) of examination (~ intended learning outcomes) 4)Choosecarefully the form(s) of teaching (~ intended learning outcomes)
On Aligning the Exam (~ ILOs) • Pre-alignment (Concurrency 2004+2005): • Group Project (50%) • Individual Multiple-Choice Test (50%) • Post-alignment (Concurrency 2006+2007): • Group Project (50%) • Individual Multiple-Choice Test (50%) 'Inherited' from pre-2004: Because it seemed like a good idea to do a project Added in 2005: Politically motivated: exam must have individual part! However; BIG differences...! Coincidentally: Carefully designed (~ILOs): Project good for evaluating model-based design process Carefully designed (~ILOs): MC-test good for evaluating analytical skills (~problem): to analyze/compare models
Project (pre- vs. post-alignment) • 2004 Project: "The Beer Factory": • 2006 Project:"The Banana Republic": • No explicit learning objectives (only 'list of contents') • No explicit project grading criteria result • Some student projects with no appearantmodel impl. relationship (at least, to me)!
The Banana Republic Project designed(~ ILO's): • (a) Construct unsafe model (w/o controller); • (b) Test model - observe that collisions with 'El Presidente' can occur; • (c) Define safety property NO_CRASH; • (d) Verify that collisions can occur; • (e) Construct a controller (such that collisions can no longer occur); • (f) Verify that collisions can no longer occur; • (g) Define liveness property ('El Presidente' can eventually leave); • (h) Implement model in Java. • Grading (of the report): • constructmodels... • apply common solutions... • relate specmodel... • test model... • define properties... • verify model wrt. properties... • implement model... • relate modelimpl... • All ILO's except: • analyze models • comparemodels Better evaluated on MC-test
MC-test (pre- vs. post-alignment) • 2004 MC-test: • 2006 Project: (a bunch of seemingly reasonable questions): Bad Alignment Carefully designed (~ ILO's): • analyze models (and programs) wrt. behavior • compare models (and program) wrt. behavior
Example: analyzemodels Good Alignment
Example: compare models Good Alignment
Implementation Process • Process(course specific): 1) Think carefully about: overall goal of course (what are the stud. to learn?) 2)Operationalize these goals: and express them as intended learning outcomes alignment learning support 3)Choosecarefully the form(s) of examination (~ intended learning outcomes) 4)Choosecarefully the form(s) of teaching (~ intended learning outcomes)
On Aligning the TLA (~ ILOs) [ TLA :=Teaching/Learning Activities ] • Pre-alignment (Concurrency 2004+2005): • Lectures (2-3 hrs/week) • 'Theoretical Exercise Classes' (2 hrs/week) • 'Programming Lab' (2 hrs/week) • Post-alignment (Concurrency 2006+2007): • Lectures (2-3 hrs/week) with activation exercises • 'Theoretical Exercise Classes' (2h/w) apply common solutions • 'Programming Lab' (2 hrs/week) hands-on training for project • Weekly hand-ins (every week) train for project (w/ feedback!) • MC-test sample questions (given early) train for MC-test essentially teacher-centric "monologues" [ Idea due to colleague Thomas Hildebrandt at ITU ] student-centric
Intended learning outcomes TLA's (~ ILOs) • construct models… • apply common solutions... • relate specmodel... • test model... • define properties... • verify model wrt. properties... • analyze models… • comparemodels… • implement model... • relate modelimpl... Student-centric: • 'Th. Ex. Classes' (2h/w) apply common solutions • 'Programming Lab' (2 hrs/week) hands-on training for project • Weekly hand-ins (every week) train for project (w/ feedback!) • MC-test sample questions (given early) train for MC-test Teacher-centric: • Lectures (2-3 hrs/week) with activation exercises { apply common solutions } { construct, implement, test, verify, define, apply } { construct, implement, relate } { analyze, compare } introduce fundamental concepts/problems/solutions (in terms of models & impl)
Implementation Process • Process(course specific): ? 1) Think carefully about: overall goal of course (what are the stud. to learn?) 2)Operationalize these goals: and express them as intended learning outcomes alignment 3)Choosecarefully the form(s) of examination (~ intended learning outcomes) 4)Choosecarefully the form(s) of teaching (~ intended learning outcomes)
Conclusions (pre vs. post) • Subjectively: • Constructive Alignment (!!!): • To the point that I bothered making a film about it :) • Own behavior changed: • From 'intuition' to conscious choices;awareness of alternatives and of consequences of choices (~ student learning) • My students' behavior changed (from my perspective): • More focusses on learning the objectives (esp. 'to relate') Disclaimer: (many factors involved that vary from-year-to-year) • Student background and prerequisites; • The "Susan/Robert ratio"; • Teacher's experience gain; ... ...and many more
Objectively (I/III):(Questionnaire at end, 7-step scale) self-reported • Student satisfaction: • "slightly more satisfied"..or • "constructive alignment doesn't compromize student satisfaction" • Student proficiency: • More useful figures (~learning)! • However: I only havepost-alignment data :( • Thus: "inconclusive" :( Pre ('04-'05) Pre ('04+'05) Post ('06-'07) Post ('06+'07)
Objectively (II/III):(Competences explicitly tested & trained) • Competences(tested and trained for): • Conclusion: • "Substantial SOLO-level increase" (~ good teaching)! • Much better projects (esp. 'modelimpl' relationship)!
Objectively (III/III):(Qualitative data from 2006 eval) • Anonymous student in 2006 evaluation: Overall: “This course has been awesome! It took me a while to be able to think in models, but I saw the light along the way.” Teaching: “Lectures have been great, the theoretical exercise classes have been rewarding and the feedback has been immense and insightful” Exercises: “I did not have a lot of time to do the exercises, but they seemed relevant from week to week.” Project: “The mini project was a good and solid exercise in analyzing a problem, making a model and implementing it. A very good exercise!”
1 2 3 4 5 6 utline O • Introduction: • Background, Motivation, and Expectations • The Theory of Constructive Alignment: • FILM: “Teaching Teaching & Understanding Understanding” • From Theory to Practice: • “From content to competence” --- short (10’) break --- • Implementing Alignment (case study): • Implementing alignment in Teaching Computer Science • Computer Science Analysis: • Preliminary Analysis of DK experiences (~new grade scale) • Open discussion: • Q’n’A / open debate / discussion, …
The New Danish Grade Scale Conversion (between EU countries): 7 steps: 4 steps 4 steps 8 steps 8 steps ECTS 10 steps 10 steps SCALE ... ... ... ... ... 21 steps 21 steps A, B, C, D, E, Fx, F • Problems (comparability ~ EU nations): • Information loss (10 steps 7 steps): • (13,11) A; (9,8) C; … • The “13” (“exception grade”); doesn’t exist in other scales! • Some places only access if you have top grade (~ 13) • …and a number of other motivations pigeon hole principle
“The Danish 7 Step Scale” For an excellent performance which completelymeets the course objectives, with no or only a few insignificant weaknesses. 12 A Excellent For a very good performance which meets the course objectives, with only minor weaknesses 10 B Very good For a good performance which meets the course objectives but also displays some weaknesses 7 C Good Grade := Degree of realizationof course objectives! For a fair performance which adequately meets the course objectives but also displays several major weaknesses 4 D Fair 02 E Adequate For a sufficient performance which barely meets the course objectives 00 Fx Inadequate For an insufficient performance which does not meet the course objectives -3 F For a performance which is unacceptablein all respects Unacceptable
Intended Learning Outcomes • Consequence: • Every course has to explicitly define…: Intended Learning Outcomes (!) :)
Collect data... (1000 courses!) • Systematically collect data (i.e. competences) • Quantifiable via The SOLO Taxonomy: Note: Work in progress (with Bettina Dahl Søndergaard, STENO/AU)
Analysis: ”Nature of Subjects” • Analyzing for diff.’s in ”nature of subjects”: • i.e., CSvs. Mathvs. Physicsvs. Biologyvs. Chemistryvs. Geology vs. Statistics vs. …) * *) Tool used forentering ILO’s Note: Work in progress (with Bettina Dahl Søndergaard, STENO/AU)
Analysis: ”Progression” • Analyzing for ”progression”: • i.e., ”undergraduate” vs. ”graduate” courses * Note: Work in progress (with Bettina Dahl Søndergaard, STENO/AU)
Top 15 Competences • Top 15 Competences: • Computer Science (at Aarhus University): * Note: Work in progress (with Bettina Dahl Søndergaard, STENO/AU)
Statistics: Computer Science (DK) • Danish Universities (~ Computer Science) • (excl. AAU/Aalborg, DTU/Copenhagen, RUC/Roskilde): • (Note: much more systematic impl. processundertaken at IMADA/SDU and DAIMI/AU.) * Note: Work in progress (with Bettina Dahl Søndergaard, STENO/AU)
…and Identify Potential Problems • E.g. course: ”Databases”(at RUC/Roskilde): • Note: almost entirely non-operational(!) • i.e. measure how?! • obtain knowledge aboutthe structure of database systems; • be familiar with design of databases by useof special notations like E/R and analysisthrough normalization; • get an overview of the most important database models and a detailed knowledge about the most important model - the relational model as well as the language SQL; • get an overview of database indexing and query processing; • obtain knowledge about application programming for DB systems. Familiar with ?! Note: Work in progress (with Bettina Dahl Søndergaard, STENO/AU)