Learning Goals Workshop Ako Aotearoa Project

1. Learning Goals WorkshopAko Aotearoa Project July 2010 Based upon the CWSEI Learning Goals Workshop

2. After this workshop you should be able to … • Explain the goal limitation set by the course. • Write goals that target content, skills, or attitudes; • Use Bloom’s taxonomy to write topic-level learning goals that address all six levels of learning; • Effectively link assessments to learning goals. Pre-workshop assignment: the purpose was to provide an opportunity for you to come already thinking about these issues.

3. Workshop plan of action • Introduction • Goal limitations from course background • Course level goals • Topic level learning goals & Bloom’s taxonomy • Assessment: building questions that inform both students and instructors. Practice and collaborate in working groups  No substitute for discussion with peers & iteration.  General & small group discussion, and revisions.

4. 1. Introduction Definition: • Articulate what students should be able to do after a course, a module, a lesson, etc. • Reflects what you want students to learn and at what level. Values of carefully designed goals: • Instructor: to better define, guide, & check your teaching. • Students: Define what they should be learning, and why. • Both: Essential for measuringwhat students are learning; Goals help build assessments which provide feedback to students and instructor.

5. 1. Introduction Wide range of possible goals: • memorizing terminology • complex problem solving skills • transferring ideas to new contexts • thinking like a professional / scientist • many others ….. Goals are NEVER perfect but ALWAYS valuable

6. Goals at any level should emphasize students • Teaching is commonly viewed as teacher-centered. • But … students must do their own learning; we can not do the learning for our students. • Good teaching involves helping students actively engage with new ideas, skills, and behaviours. Simply presenting new content is NOT “teaching”. • “We will design more effective courses if we focus on setting goals for what students do, rather than for the faculty member.” From http://serc.carleton.edu/NAGTWorkshops/coursedesign/tutorial/goals.html

7. Learning Goals—it’s hard to learn when you don’t know what to pay attention to http://viscog.beckman.illinois.edu/grafs/demos/15.html

8. Goal limitations Example: eosc221 • Introduction to mineralogy • Context is important for steering goals • 2nd year students, mostly pursuing a geology degree • Pre-requisite for many subsequent courses • Well defined requirements for learning • Student background • Minimal experience in geology • 1st year chemistry • Assessment- weekly labs, mid term lab exam, final lab exam, final written exam- short answers. Also Clickers, in class exercises

9. Activity 1: Goal limitations Compare homework- Discuss in your groups in turn-1. How your course context (Homework Q 1.) might influence your learning goals Time 5 mins per course Total 10 mins

10. Workshop plan of action • Introduction • Goal limitations from course background • Course level goals • Topic level learning goals & Bloom’s taxonomy • Assessment: building questions that inform both students and instructors.

11. 3. Course-Level learning goals Characteristics: • Minimal technical terminology (unlike topic level goals) • Should be informed by … • Pre- and post-requisites • Department goals and resources • Discipline context & constraints (professions etc) • Should articulate … • Attitudes / beliefs you hope students will gain • Concept areas and skill sets students should acquire • Or, identify what students will be able to do with the new concepts and skills. Specifics (details) may be implied.

12. To help focus goals design, 3 domains of learning CONCEPTS: Eg: analyze, explain, and predict the motion of objects in the world around you. SKILLS:Cognitive/Process skills: - Reasoning, Problem Solving, Evaluating, Critiquing… - Compare / contrast; reflect; strategize, justify, plan … Technical skills: - Computer programming, specific software, … - Laboratory skills, wetlab, instruments, tools, … - Library, research, writing, presenting, … ATTITUDES: Appreciate, Enjoy, Value - e.g. Observe our planet from a more enthusiastic and scientifically informed perspective. About nature of learning and doing: - e.g. Believe that learning and doing physics is more about reasoning and making sense, not memorizing. (There are many other “models” for learning domains.)

13. Learning domains CONCEPTS SKILLS ATTITUDES Eg: Goals for a 1st yr physics course include all 3 aspects of learning: One Lecture-scale Learning Goal could touch upon 3 course-level goals Course-level learning goals Recognize that the behavior of the world around you is not magical and mysterious, but rather can be understood and predicted using certain fundamental principles. Use knowledge about properties and motion of electric charges to predict and explain various aspects of electricity. Lecture-level learning goals Be able to design a fluorescent light bulb. Explain and justify the requirements on specifications of various basic electronic components.

14. Activity 2: course-level goals • In your groups- discuss course level goals • Use the characteristic in slide 11 to complement and improve each others goals. • Which learning domains are targeted? • Which goals do your prefer -Why? Time 5 mins to look over 5 mins to discuss together

15. Recap: Some aspects of course-level goals Characteristics: • Minimal technical terminology (unlike topic level goals). • Should be informed by … • Pre- and post-requisites • Department goals and resources • Discipline context & constraints (professions etc) • Articulate … • Attitudes / beliefs you hope students will gain. • Concept areas and skill sets students should acquire Benefits: • Help structure the course & guide methods of instruction. • Study guide – help students structure their time & effort. • Help instructor’s identify assumptions about prior learning. • Enhances transfer between instructors. (Instructors retain flexibility at the topic level)

16. Course-level or topic-level? General Overview Specific Outcomes oriented Course context: department, discipline, program Course goals Topic goals Pedagogy  Active learning Assessment What can students do?

17. 4. Topic-level goals Bloom's Taxonomy of Learning Factual Knowledge: remember and recall factual information Comprehension: demonstrate understanding of ideas, concepts Application: apply comprehension to unfamiliar situations Analysis: break down concepts into parts Synthesis: transform, combine ideas to create something new Evaluation: think critically about and defend a position See the Lord and Baviskar paper, and handout #2.

18. 4. Topic-level goals Bloom's Taxonomy of Learning – with verbs Factual Knowledge: remember and recall factual information Comprehension: demonstrate understanding of ideas, concepts Application: apply comprehension to unfamiliar situations Analysis: break down concepts into parts Synthesis: transform, combine ideas to create something new Evaluation: think critically about and defend a position Define, List, State, Label, Name, Describe Describe, Explain, Summarize, Interpret, Illustrate Apply, Demonstrate, Use, Compute, Solve, Predict, Construct, Modify Compare, Contrast, Categorize, Distinguish, Identify, Infer Develop, Create, Propose, Formulate, Design, Invent Judge, Appraise, Recommend, Justify, Defend, Criticize, Evaluate Higher level: Require deeper conceptual understanding

19. Practice thinking about Bloom’s taxonomy Examples of topic-level goals: What Bloom’s level are these? (from eosc221) • Describe the rock granite • Compare and contrast granite with basalt • Design a methodology that you could use to distinguish between granite and basalt. • Criticize your neighbors methodology for distinguishing between granite and basalt Are any “levels” better than others? “high level” “low level”

20. Developing topic level goals Handout #2: Useful Tips for writing effective learning goals. • Topic (or module) level goals define what students can do after learning. • Each should be consistent with at least one course-level goal. • Keep in mind learning “domains”: • CONCEPTS • SKILLS • ATTITUDES

21. Topic-level learning goals(2-3 per class period) Example: • Understand how a fluorescent light bulb works. Or ….. • Be able to design a fluorescent light bulb and to explain and justify the requirements on the various basic components.

22. Rewording- Biology Learning Goal Examples Human Genetics for non-majors Predict the probability of generating sperm and egg cells with specific chromosomal makeup. (Blooms level 3). Explain how these cells are produced (Blooms level 2). Reworded:

23. Increasing Blooms level Introduction to Molecular and Cell Biology Original goal Characteristics Describe how the process of extracting information from genetic material is regulated at each step of conversion of DNA to RNA to protein. Low level goal—explicitly encourages students to memorize the steps from DNA to protein Higher level goal—encourages student to think about how proteins work, how they are produced, and how they can be altered by mutations in DNA.

24. Topic-level learning goals(2-3 per class period) Check-list for creating topic-level learning goals(Handout #3) • Is goal expressed in terms of what the student will achieve? Does it identify what students will be able to do after the topic is covered? • Is the Bloom’s level of the goal aligned with your expectations for students’ learning … Is this what students will be able to do if they “understand” the topic at the level you want? • If you expect reasoning for “why”, does it convey that? • Could you expect a higher level goal? • Is it well-defined? Is it clear how you would test achievement? • Do chosen verbs have a clear meaning? • Is terminology familiar/common? If not, is the terminology a goal? Not every goal can achieve the following, but it is better if you can: • Is it relevant and useful to students? (e.g. connected to their everyday life, or does it represent a useful application of the ideas).

25. Activity 3 – part 1:Your learning goals • Working with a partner or in a group: Exchange topic-level goals • Use the checklist to “critique” these goals • Discuss. Time 5 mins to look over 5 mins to discuss together

26. Activity 3 – part 2: Refine your learning goal … • Suggest changes to improve clarity (address checklist items) • All to discuss one or two of these Time 5 mins to refine the goal 5 mins to discuss together

27. Break

28. Now: Testing achievement of learning goals; i.e., assessing progress Handout #4: Formative Assessment Questions Why: • Formative assessment = closing the feedback loop. i.e. Test / quiz/ work / etc informs instructor & student about knowledge gaps. • Know what students think now (before exam), and react to it ! When …. • Pre-test Post-test • Active learning = formative feedback • Evaluations (marked work). How …. Today’s focus is how to ask question which effectively assess specific learning goals.

29. Help with writing questions Bloom's Taxonomy of Learning – with verbs Factual Knowledge: remember and recall factual information Comprehension: demonstrate understanding of ideas, concepts Application: apply comprehension to unfamiliar situations Analysis: break down concepts into parts Synthesis: transform, combine ideas to create something new Evaluation: think critically about and defend a position Define, List, State, Label, Name, Describe Describe, Explain, Summarize, Interpret, Illustrate Apply, Demonstrate, Use, Compute, Solve, Predict, Construct, Modify Compare, Contrast, Categorize, Distinguish, Identify, Infer Develop, Create, Propose, Formulate, Design, Invent Judge, Appraise, Recommend, Justify, Defend, Criticize, Evaluate Higher level: Require deeper conceptual understanding

30. The Montillation of Traxolinewhen assessment goes astray It is very important that you learn about traxoline. Traxoline is a new form of zionter. It is montilled in Ceristanna. The Ceristannians gristerlate large quantities of fevon and then bracter it to quasel traxoline. Traxoline may well be one of our most lukized snezlaus in the future, because of our zionter lescelidge. • - - - Answer these questions in complete sentences. Be sure to use your best handwriting. 1. What is traxoline? 2. Where is traxoline montilled? 3. How is traxoline quaselled? 4. Why is it important to know about traxoline? [attributed to Judy Lanier] Are questions testing anything?

31. Activity 4: Back to the goals and assesments related to the topic goals you brought in … • Exchange goals and assessments. • Is the Blooms level appropriate, can you raise it ? • Keep in mind: Student perspective, the learning “domains” are you testing concepts, skills, attitude. • Avoid rote memorization strategies. Try: • “Troubleshooting” (something changes, what could cause this?) • “redesign …” • “compare and contrast …” scenarios

32. Implementing assessments (questions) Not just exams … • In class is important – why? • active learning • Feedback for students AND instructors • What are some options? (All can be solo / pairs / group work.) • Clickers • Coloured cards to raise (multiple choice questions) • 1 minute papers • Group (team) work with “instant” deliverables to enable compare & discuss • Pre-test quizzes (solo and/or groups) • Pre-test + Post-test to foster curiosity, then wrestle with using new concepts This is the goal of this semester to develop these assessments

33. Other ideas for implementation: • Ask question in exam or homework. • Ask students to write answer to a question on a sheet of paper and turn it in. • Ask students to write answer on own, then discuss with others, then write a revised answer. • Compare question responses before / after a learning activity. • Ask students to discuss question in groups, write a group answer. • Ask a multiple choice question and ask students to vote using their hands, colored cards, or clickers. • Students can answer on their own or discuss in groups. • Give credit for answering questions but don’t grade.

34. Research Assistant suggestions 1. To use UBC goals and your own suggestions to refine goals and improve their use in the classroom • 2. To use UBC assessments and your own suggestions to develop: • Formative in class clicker questions- • Formative in class 5 min activities • A multiple choice test for the end of the semester

35. Research assissant things to ask • Are goals presented and clear in the class ? • Can goals be improved upon- higher blooms level ? More diverse learning domains ? • Is it clear how these goals relate to course goals ? • Are any of the UBC goals appropriate ? • Are any of the UBC assessments appropriate ?

36. Recap 1 - Values of explicit learning goals: • Students: Define what they should be learning, and why. • Instructor: to better define, guide, & check your teaching. • Both: Essential for measuringwhat students are learning; i.e. goals help build assessments which provide feedback to students and instructor.

37. Recap 2 - Building linked course level and topic level goals • Student (NOT teacher) point of view. • Cognitive domains (concepts, skills, attitudes). • Use of Bloom’s taxonomy as a framework. Assessment is the key – target appropriate levels of learning. • Implementation: • Formative feedback is key to informing students (and instructors) about the degree of success at meeting goals of the course.

38. Thank you for your interest Homework  • Refine your learning goals (course-level and topic-level) developed in the pre-workshop assignment. • Look up “learning goals xxx” (xxx = your discipline).A wide range of resources are available from many reputable institutions.

39. Workshop evaluations: Were these learning goals useful? Achieved? • Distinguish between course and topic goals. • Convert a syllabus (list of topics) into learning goals that reflect what students should be able to do; • Write goals that target content, skills, or attitudes; • Use Bloom’s taxonomy to write topic-level learning goals that address all six levels of learning; • Effectively link assessments to learning goals.