1 / 73

Constructive Alignment for Teaching Computer Science

Koli’2007 – Keynote. Constructive Alignment for Teaching Computer Science. Claus Brabrand ((( brabrand@itu.dk ))) ((( http://www.itu.dk/people/brabrand/ ))) Associate Professor, IT University of Copenhagen Denmark. 1. 2. 3. 4. 5. 6. utline. O. Introduction:

sophie
Download Presentation

Constructive Alignment for Teaching Computer Science

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Koli’2007 – Keynote Constructive Alignment for Teaching Computer Science Claus Brabrand ((( brabrand@itu.dk ))) ((( http://www.itu.dk/people/brabrand/ ))) Associate Professor, IT University of Copenhagen Denmark

  2. 1 2 3 4 5 6 utline O • Introduction: • Background, Motivation, and Expectations • The Theory of Constructive Alignment: • “Teaching Teaching & Understanding Understanding” • From Theory to Practice: • “From content to competence” --- short (10’) break --- • Implementing Alignment (case study): • Implementing alignment in Teaching Computer Science • Computer Science Analysis: • Preliminary Analysis of DK experiences (~new grade scale) • Open discussion: • Q’n’A / open debate / discussion, …

  3. T First: exercise • Before we start: Post-It exercise: • Write down answer to: • "what is good teaching?" • 2) Swap Post-Its...

  4. Background (~ this talk) • Concurrency 2004+2005: • "Pre-alignment" • Exposure to teaching/learning theories: • “Constructive Alignment” • “The SOLO Taxonomy” • Concurrency 2006+2007: • "Post-alignment"

  5. 1 2 3 4 5 6 utline O • Introduction: • Background, Motivation, and Expectations • The Theory of Constructive Alignment: • FILM: “Teaching Teaching & Understanding Understanding” • From Theory to Practice: • “From content to competence” --- short (10’) break --- • Implementing Alignment (case study): • Implementing alignment in Teaching Computer Science • Computer Science Analysis: • Preliminary Analysis of DK experiences (~new grade scale) • Open discussion: • Q’n’A / open debate / discussion, …

  6. Let's watch the short-film... ((( ))) Teaching Teaching & Understanding Understanding Inspired by: "Teaching for Quality Learning at University", John Biggs Available on DVD through Aarhus University Press: ((( http://www.daimi.au.dk/~brabrand/short-film/ ))) Features Epilogue by John Biggs, DVD menu, and subtitles inEnglish, French, Spanish, Portuguese, Italian, German, and Danish Won “The Golden Ratio 2006” Award for “Best Educational Video” (~4000 DVDs sold)

  7. 1 2 3 4 5 6 utline O • Introduction: • Background, Motivation, and Expectations • The Theory of Constructive Alignment: • FILM: “Teaching Teaching & Understanding Understanding” • From Theory to Practice: • “From content to competence” --- short (10’) break --- • Implementing Alignment (case study): • Implementing alignment in Teaching Computer Science • Computer Science Analysis: • Preliminary Analysis of DK experiences (~new grade scale) • Open discussion: • Q’n’A / open debate / discussion, …

  8. From Content to Competence • The “pre-alignment” Concurrency course aims: • Given in terms of a 'content description': • Essentially: • The goal is...: • To understand: • deadlock • interference • synchronization • ... This is a bad idea for 2 reasons...!

  9. Problem with 'content' as aim • What is the problem with 'content'as learning objectives ?!? analyze ... theorize ... analyze systems explain causes explain deadlock describe ... • Objective: • To understand: • deadlock • interference • synchronization • ... Stud. C agreement tacit knowledge from research-based tradition (not known by stud.)  name solutions recite conditons Teacher analyze systems explain causes Stud. B BUT, even if it were possible to agree, we know that the exam will dictate the learning anyways. Stud. A Censor

  10. Problem with 'understanding' • Why not use 'understanding'as learning objectives ?!? • Objective: • To understand: • deadlock • interference • synchronization • ... concept of deadlock ?!  The answer is simple: It cannot be measured (!)

  11. [ Competence := knowledge + capacity to act upon it ] 'Competence' as objectives ! • 'Competence' as learning objectives ! • Evaluation = Have the student do something, and then measure product and/or process • Objective ! • To learn to: • analyze systems for... • explain cause/effects... • prove properties of... • compare methods of... • ... Note:'understanding' is (of course)pre-requisitional (!)  Note': inherently operational (~ verbs) 'SOLO' = Structure of the ObservedLearning Outcome

  12. T Neighbour Discussion Discuss with neighbour: "does this make sense ?!?" (content  competence)

  13. SOLO 5  to generalize  to hypothesize to theorize  ... "extended abstract" to relate  to compare  to analyze  ... SOLO 4 "relational" to classify  to combine  to enumerate  ... SOLO 3 "multi-structural" to identify  to do procedure  to recite  ... SOLO 2 "uni-structural" no understanding  irrelevant information misses point  ... SOLO 1 "pre-structural" Advantages of 'SOLO' • Advantages of 'SOLO': • Constructed for research-based (university) teaching • Converges on research (at SOLO 5) depth (qualitative levels) surface (quantitative levels)

  14. Note: the list is non-exhaustive Graphic Legend immediately relevant aspects – given! related or hypothetical – not given! irrellevant or inappropriate student response x R x R' x R R'' x x R R1 R2 R R3 R SOLO (elaborated) QUALITATIVE QUANTITATIVE SOLO 2 ”uni-structural” SOLO 3 “multi-structural” SOLO 4 “relational” SOLO 5 “extended abstract” • define • identify • count • name • recite • paraphrase • follow (simple)instructions • … • combine • structure • describe • classify • enumerate • list • do algorithm • apply method • … • analyze • compare • contrast • integrate • relate • explain causes • apply theory (to its domain) • … • theorize • generalize • hypothesize • predict • judge • reflect • transfer theory (to new domain) • …

  15. Concrete Example andConcrete Recommendations (4x) 1) Use 'standard formulation': a) puts learning focus on the student b) competence formulation: "to be able to" Intended Learning Outcomes [Genetics 101] After the course, the students are expected to be able to: locate genes on chromosomes do simple calculations : (e.g., recombination frequencies, in-breeding coefficients, Hardy-Weinberg, evolutionary equilibria). describe and perform connexion-analysis describe fundamental genetic concepts: (e.g., mutation variation, in-breeding, natural selection). describe and analyze simple inheritancies analyze inheritance of multiple genes simultaneously 4)Avoid 'understanding-goals': "To understand X", "Be familiar with Y", "Have a notion of Z", ...! V N N V N V V V N V V N V 3) Use 'Verb + Noun' formulation: What the student is expected to dowith a given matter . 2) List sub-goals as 'bullets': Clearer than text N V

  16. Concurrency: analyze for deadlock compare models T Post-It exercise Write down 1-2 key competences (i.e., verbs) (for your course)

  17. 1 2 3 4 5 6 utline O • Introduction: • Background, Motivation, and Expectations • The Theory of Constructive Alignment: • FILM: “Teaching Teaching & Understanding Understanding” • From Theory to Practice: • “From content to competence” --- short (10’) break --- • Implementing Alignment (case study): • Implementing alignment in Teaching Computer Science • Computer Science Analysis: • Preliminary Analysis of DK experiences (~new grade scale) • Open discussion: • Q’n’A / open debate / discussion, …

  18. 10' Break Please put the Post-Its on the wall "What is good teaching?" Key competences (in your course)

  19. 1 2 3 4 5 6 utline O • Introduction: • Background, Motivation, and Expectations • The Theory of Constructive Alignment: • FILM: “Teaching Teaching & Understanding Understanding” • From Theory to Practice: • “From content to competence” --- short (10’) break --- • Implementing Alignment (case study): • Implementing alignment in Teaching Computer Science • Computer Science Analysis: • Preliminary Analysis of DK experiences (~new grade scale) • Open discussion: • Q’n’A / open debate / discussion, …

  20. Disclaimer The point of this part is: • not to exhibit aperfectly aligned course; • but to show how the principles of alignment can be put to use (esp. howILO’s may serve as guidelines for exam and teaching form).

  21. Implementation Process • Process(course specific): 1) Think carefully about: overall goal of course (what are the stud. to learn?) 2)Operationalize these goals: and express them as intended learning outcomes alignment learning incentive learning support 3)Choosecarefully the form(s) of examination (~ intended learning outcomes) 4)Choosecarefully the form(s) of teaching (~ intended learning outcomes)

  22. Starting Point • Content description (Concurrency '04+'05): What is the overall goal of the course...? (what are the students to learn)

  23. Overall Course Philosophy • Model-Based Designfor Concurrency:

  24. Implementation Process • Process(course specific): 1) Think carefully about: overall goal of course (what are the stud. to learn?) 2)Operationalize these goals: and express them as intended learning outcomes alignment 3)Choosecarefully the form(s) of examination (~ intended learning outcomes) 4)Choosecarefully the form(s) of teaching (~ intended learning outcomes)

  25. #2 #1 . #3 . S M . I Model-based design for Concurrency T Intended Learning Outcomes • Intended Learning Outcomes(based on The SOLO Taxonomy): Note:explicitly included as a non-goal 

  26. Implementation Process • Process(course specific): 1) Think carefully about: overall goal of course (what are the stud. to learn?) 2)Operationalize these goals: and express them as intended learning outcomes alignment learning incentive 3)Choosecarefully the form(s) of examination (~ intended learning outcomes) 4)Choosecarefully the form(s) of teaching (~ intended learning outcomes)

  27. On Aligning the Exam (~ ILOs) • Pre-alignment (Concurrency 2004+2005): • Group Project (50%) • Individual Multiple-Choice Test (50%) • Post-alignment (Concurrency 2006+2007): • Group Project (50%) • Individual Multiple-Choice Test (50%) 'Inherited' from pre-2004: Because it seemed like a good idea to do a project Added in 2005: Politically motivated: exam must have individual part!  However; BIG differences...! Coincidentally: Carefully designed (~ILOs): Project good for evaluating model-based design process Carefully designed (~ILOs): MC-test good for evaluating analytical skills (~problem): to analyze/compare models

  28. Project (pre- vs. post-alignment) • 2004 Project: "The Beer Factory": • 2006 Project:"The Banana Republic": • No explicit learning objectives (only 'list of contents') • No explicit project grading criteria  result • Some student projects with no appearantmodel  impl. relationship (at least, to me)! 

  29. The Banana Republic Project designed(~ ILO's): • (a) Construct unsafe model (w/o controller); • (b) Test model - observe that collisions with 'El Presidente' can occur; • (c) Define safety property NO_CRASH; • (d) Verify that collisions can occur; • (e) Construct a controller (such that collisions can no longer occur); • (f) Verify that collisions can no longer occur; • (g) Define liveness property ('El Presidente' can eventually leave); • (h) Implement model in Java. • Grading (of the report): • constructmodels... • apply common solutions... • relate specmodel... • test model... • define properties... • verify model wrt. properties... • implement model... • relate modelimpl... • All ILO's except: • analyze models • comparemodels Better evaluated on MC-test

  30. MC-test (pre- vs. post-alignment) • 2004 MC-test: • 2006 Project: (a bunch of seemingly reasonable questions): Bad Alignment  Carefully designed (~ ILO's): • analyze models (and programs) wrt. behavior • compare models (and program) wrt. behavior

  31. Example: analyzemodels Good Alignment

  32. Example: compare models Good Alignment

  33. Implementation Process • Process(course specific): 1) Think carefully about: overall goal of course (what are the stud. to learn?) 2)Operationalize these goals: and express them as intended learning outcomes alignment learning support 3)Choosecarefully the form(s) of examination (~ intended learning outcomes) 4)Choosecarefully the form(s) of teaching (~ intended learning outcomes)

  34. On Aligning the TLA (~ ILOs) [ TLA :=Teaching/Learning Activities ] • Pre-alignment (Concurrency 2004+2005): • Lectures (2-3 hrs/week) • 'Theoretical Exercise Classes' (2 hrs/week) • 'Programming Lab' (2 hrs/week) • Post-alignment (Concurrency 2006+2007): • Lectures (2-3 hrs/week) with activation exercises • 'Theoretical Exercise Classes' (2h/w) apply common solutions • 'Programming Lab' (2 hrs/week) hands-on training for project • Weekly hand-ins (every week) train for project (w/ feedback!) • MC-test sample questions (given early) train for MC-test essentially teacher-centric "monologues" [ Idea due to colleague Thomas Hildebrandt at ITU ]  student-centric

  35. Intended learning outcomes TLA's (~ ILOs) • construct models… • apply common solutions... • relate specmodel... • test model... • define properties... • verify model wrt. properties... • analyze models… • comparemodels… • implement model... • relate modelimpl... Student-centric: • 'Th. Ex. Classes' (2h/w) apply common solutions • 'Programming Lab' (2 hrs/week) hands-on training for project • Weekly hand-ins (every week) train for project (w/ feedback!) • MC-test sample questions (given early) train for MC-test Teacher-centric: • Lectures (2-3 hrs/week) with activation exercises { apply common solutions } { construct, implement, test, verify, define, apply } { construct, implement, relate } { analyze, compare } introduce fundamental concepts/problems/solutions (in terms of models & impl)

  36. Implementation Process • Process(course specific): ? 1) Think carefully about: overall goal of course (what are the stud. to learn?) 2)Operationalize these goals: and express them as intended learning outcomes alignment 3)Choosecarefully the form(s) of examination (~ intended learning outcomes) 4)Choosecarefully the form(s) of teaching (~ intended learning outcomes)

  37. Conclusions (pre vs. post) • Subjectively: • Constructive Alignment (!!!): • To the point that I bothered making a film about it :) • Own behavior changed: • From 'intuition' to conscious choices;awareness of alternatives and of consequences of choices (~ student learning) • My students' behavior changed (from my perspective): • More focusses on learning the objectives (esp. 'to relate') Disclaimer: (many factors involved that vary from-year-to-year) • Student background and prerequisites; • The "Susan/Robert ratio"; • Teacher's experience gain; ... ...and many more

  38. Objectively (I/III):(Questionnaire at end, 7-step scale) self-reported • Student satisfaction: • "slightly more satisfied"..or • "constructive alignment doesn't compromize student satisfaction" • Student proficiency: • More useful figures (~learning)! • However: I only havepost-alignment data :( • Thus: "inconclusive" :( Pre ('04-'05) Pre ('04+'05) Post ('06-'07) Post ('06+'07)

  39. Objectively (II/III):(Competences explicitly tested & trained) • Competences(tested and trained for): • Conclusion: • "Substantial SOLO-level increase" (~ good teaching)! • Much better projects (esp. 'modelimpl' relationship)!

  40. Objectively (III/III):(Qualitative data from 2006 eval) • Anonymous student in 2006 evaluation: Overall: “This course has been awesome! It took me a while to be able to think in models, but I saw the light along the way.” Teaching: “Lectures have been great, the theoretical exercise classes have been rewarding and the feedback has been immense and insightful” Exercises: “I did not have a lot of time to do the exercises, but they seemed relevant from week to week.” Project: “The mini project was a good and solid exercise in analyzing a problem, making a model and implementing it. A very good exercise!”

  41. 1 2 3 4 5 6 utline O • Introduction: • Background, Motivation, and Expectations • The Theory of Constructive Alignment: • FILM: “Teaching Teaching & Understanding Understanding” • From Theory to Practice: • “From content to competence” --- short (10’) break --- • Implementing Alignment (case study): • Implementing alignment in Teaching Computer Science • Computer Science Analysis: • Preliminary Analysis of DK experiences (~new grade scale) • Open discussion: • Q’n’A / open debate / discussion, …

  42. The New Danish Grade Scale Conversion (between EU countries): 7 steps: 4 steps 4 steps 8 steps 8 steps ECTS 10 steps 10 steps SCALE ... ... ... ... ... 21 steps 21 steps A, B, C, D, E, Fx, F • Problems (comparability ~ EU nations): • Information loss (10 steps  7 steps): • (13,11)  A; (9,8)  C; … • The “13” (“exception grade”); doesn’t exist in other scales! • Some places only access if you have top grade (~ 13) • …and a number of other motivations pigeon hole principle

  43. “The Danish 7 Step Scale” For an excellent performance which completelymeets the course objectives, with no or only a few insignificant weaknesses. 12 A Excellent For a very good performance which meets the course objectives, with only minor weaknesses 10 B Very good For a good performance which meets the course objectives but also displays some weaknesses 7 C Good Grade := Degree of realizationof course objectives! For a fair performance which adequately meets the course objectives but also displays several major weaknesses 4 D Fair 02 E Adequate For a sufficient performance which barely meets the course objectives 00 Fx Inadequate For an insufficient performance which does not meet the course objectives -3 F For a performance which is unacceptablein all respects Unacceptable

  44. Intended Learning Outcomes • Consequence: • Every course has to explicitly define…: Intended Learning Outcomes (!) :)

  45. Collect data... (1000 courses!) • Systematically collect data (i.e. competences) • Quantifiable via The SOLO Taxonomy: Note: Work in progress (with Bettina Dahl Søndergaard, STENO/AU)

  46. Analysis: ”Nature of Subjects” • Analyzing for diff.’s in ”nature of subjects”: • i.e., CSvs. Mathvs. Physicsvs. Biologyvs. Chemistryvs. Geology vs. Statistics vs. …) * *) Tool used forentering ILO’s Note: Work in progress (with Bettina Dahl Søndergaard, STENO/AU)

  47. Analysis: ”Progression” • Analyzing for ”progression”: • i.e., ”undergraduate” vs. ”graduate” courses * Note: Work in progress (with Bettina Dahl Søndergaard, STENO/AU)

  48. Top 15 Competences • Top 15 Competences: • Computer Science (at Aarhus University): * Note: Work in progress (with Bettina Dahl Søndergaard, STENO/AU)

  49. Statistics: Computer Science (DK) • Danish Universities (~ Computer Science) • (excl. AAU/Aalborg, DTU/Copenhagen, RUC/Roskilde): • (Note: much more systematic impl. processundertaken at IMADA/SDU and DAIMI/AU.) * Note: Work in progress (with Bettina Dahl Søndergaard, STENO/AU)

  50. …and Identify Potential Problems • E.g. course: ”Databases”(at RUC/Roskilde): • Note: almost entirely non-operational(!) • i.e. measure how?! • obtain knowledge aboutthe structure of database systems; • be familiar with design of databases by useof special notations like E/R and analysisthrough normalization; • get an overview of the most important database models and a detailed knowledge about the most important model - the relational model as well as the language SQL; • get an overview of database indexing and query processing; • obtain knowledge about application programming for DB systems. Familiar with ?! Note: Work in progress (with Bettina Dahl Søndergaard, STENO/AU)

More Related