Making Learning Visible Through the Scholarship of Teaching and Learning University of Alaska, Anchorage March 8-9, 2007

1. Making Learning VisibleThrough the Scholarship of Teaching and LearningUniversity of Alaska, AnchorageMarch 8-9, 2007 Randy Bass and Heidi Elmendorf Georgetown University

2. Baseline Reflection Exercise: 1. What do you see? 2. What do you know? 3. What don’t you know? What questions do you have? 4. If you were going to find out more, how would you start? In the library? On the Web? 5. What are you bringing from other learning experiences that helps you make sense of this artifact?

3. What do we know about the difference between novice and expert learners? Historical Thinking and Other Unnatural Acts by Sam Wineburg

4. Expertise and Adaptive Expertise • Qualities of Knowledge • organized • contextualized • retrievable • flexible • Awareness of knowledge • epistemological • multiple perspectives • assumptive • limited • Use of Knowledge • self-defined goals • exploration • re-examination Metacognitive Cognitive Affective

5. A Traditional View… NOVICE MIRACLE EXPERT product product

6. product product product With a Focus on Learning Processes NOVICE processes EXPERT processes LEARNING product Even with attention to active learning and developmental processes, there is a tendency to see processes as inevitably leading to a version of expert product. And students still seem to believe in the miracle…

7. But what does expertise look like when seen through the lens of traditional expert product? • Expert product is missing • the evidence of context and process • Formal and tidy • Loss of complexity and uncertainty • Focus on cognitive • Emphasis on knowledge

8. NOVICE processes EXPERT processes LEARNING product product product ≠ Is our problem then a conflation of the products of expertise with the products of a developmental process? Processes that are effective in developing expert thinking do not necessarily lead to the ability to produce expert product.

9. LEARNING processes NOVICE processes EXPERT processes LEARNING processes LEARNING processes products products products Instead we need to expect learning products appropriate to learning processes How can we better understand these intermediate processes? How might we design to foster and capture them?

10. Using the Scholarship of Teaching and Learning to make the most of this expansion of learning • “Reflecting on one’s practice is what practice is all about.” • Lee Shulman, President, Carnegie Foundation for the Advancement of Learning

11. Three Cases • Authentic questions and the criteria of excellence (Bernstein, Psychology) • Capturing the making of conversational biologists (Elmendorf, Biology) • Close Reading (Bass, English)

12. Case #1: Dan Bernstein, University of Kansas

13. Traditional Abstract Questions Dan Bernstein, Psychology, University of Kansas

14. Old: What were the reinforcing consequences in the Welsh, Bernstein & Luthans [restaurant] study? How were the consequences identified? New: Suppose you were asked to implement a motivational program in a business setting. It is a small production unit with 25 employees who can engage in the following activities: production planning, inventory delivery, direct production, packaging, and marketing. Based on your understanding of the study of the fast food restaurant, how would you proceed to improve the quality of the employees' work by using access to activities as a motivator. Your answer should include the assessment of the relative values of activities, the establishment of contingencies, and a simple design for evaluating the project. Finally, describe the costs and benefits of the program for the business and make a recommendation about whether or not it should be implemented. Changing to ‘authentic’ questions Dan Bernstein, Psychology, University of Kansas

15. Comparison of achievement after exam with new ‘authentic’ questions Dan Bernstein, Psychology, University of Kansas

16. The story of the seven weldsfromGrant Wiggins

17. Blackboard Discussion Board • Learners should recognize quality work • Posted generalized example problems • Posted another variation on the question • Each student was to write an answer • Also expected to comment on one answer Dan Bernstein, Psychology, University of Kansas

18. Learning With Full Range of Techniques Dan Bernstein, Psychology, University of Kansas

19. Documented and made public through a course portfolio • Alignment: • Goals • Activities & Assessments • Student Work Dan Bernstein, Psychology, University of Kansas

20. Case #2: Heidi Elmendorf, Georgetown University Conversational Biologists In the Lab Blackboard Discussion Forum Research Projects In the Classroom

21. “My basic thought was, yes, there are scientists and what they study is highly complicated stuff, ‘I can’t understand all those facts’.” “I have generally felt inadequate in previous science courses, as humanities subjects have always been more ‘my thing’.” There is also something about science that is scary. Maybe this is my own personal problem, based on my experience with the subject, and my previous perceptions about science – but science is an intimidating field to me.” “I have taken several science courses, and somehow managed to pass them through a combination of memorization and lots of studying, but am the first to admit that I have difficulty thoroughly understanding concepts.” Students’ Self-Perception:‘outside of their comfort zone’ 70% self-identify as having had a poor or deficient preparation in science or as being lacking in science ability. Heidi Elmendorf, Biology, Georgetown University

22. Dichotomous Views of Science Scientists: Public: • Inquiry • Uncertainty • Problem-solving • Creativity • Exciting • Promising • Facts • Certainty • Memorization • Rules • Intimidating • Dangerous Heidi Elmendorf, Biology, Georgetown University

23. Mandate for Pedagogical Innovation • Is there a role for non-scientists in science? • “It is vital that the public has a better working knowledge of the science and technology that defines our very existence on the planet.” Rita Colwell • Then what should they learn? • “We know full well that you will forget most of the facts that you have learned. But what we hope you will retain is the capacity to integrate ideas and exercise thoughtful judgments across many aspects of human endeavor.” Shirley Tilghman • How well do K-12 approaches address this need? • “Fully half the students who had not taken a course in biology did as well or better than 40% of the students who had taken such a course." Mullis and Jenkins, 1988 Heidi Elmendorf, Biology, Georgetown University

24. Layers of Course Structure:conversation and evidence Group Projects Reading & Blackboard Weekly Quizzes Class Lab / Teaching Heidi Elmendorf, Biology, Georgetown University

25. “Structuring” the Conversation • The assignment • Join in the conversation once a week • Roles? ‘summarizer’ ‘responder’ ‘confused one’ ‘criticizer’ ‘connector’ • I am invisible on-line • Follow-through • Starting point for class: • Reading print outs of on-line discussion • On-line discussion integrated into class • Worked into research papers • Credit for participation; ungraded Heidi Elmendorf, Biology, Georgetown University

26. How to Look at the Data?The Method… “Coding” • Close reading and re-reading of Blackboard conversations • Being selective & the mountain of evidence • Categorize comments… • Use color and notes to ‘code’ • Tabulate Heidi Elmendorf, Biology, Georgetown University

27. Emergent Patterns • expressing interest/enthusiasm • asking questions • answering questions • referring to text • noting science content • referring to scientific process • bringing in outside information (linking out) • referring to each other (building community) • taking intellectual risks Heidi Elmendorf, Biology, Georgetown

28. Fostering community Venturing ideas Referencing science as a process Patterns of on-line conversations • Giving summaries • Stating science Facts • Expressing naive interest

29. What Attributes of Learning Can On-line Conversations Foster? • Support initial encounters with information • Encourage slower and more responsive conversations • Permits students to revisit and reflect on conversations • Emphasize process over product • Build ‘safe’ intellectual communities

30. Case #3: Randy Bass, Georgetown University Questions emerge from observed tensions Evidence makes the claim more complicated Talks through why the theme is important

31. This led to an inquiry -Developed Pedagogies that allowed me to gather evidence of their reading processes. -Made conversation and thinking aloud an integral part of the course. Randy Bass, English, Georgetown

32. Using analog video and an online discussion board to transform a “close reading” exercise Step #1 Group think-aloud captured on videotape Step #2 “Transcription Outline” Step #3 Online paper: Close reading Of own tape Randy Bass, English, Georgetown

33. What are the component activities that someone has to do well to be successful at this? _____ _____ _____ _____ _____ _____ Where does this breakdown for students? What are the component obstacles? _____ _____ _____ _____ _____ _____ Breakdown of a Learning Activity Randy Bass, English, Georgetown

34. Learning Activity: Reading a literary text to generate interesting questions about context and complexity. • Level One: Reading • Level Two: Recognizing complexity • Level Three: Generating researchable questions Randy Bass, English, Georgetown

35. Obstacles • Level One: Reading • Novice close reading strategies • ______ • ______ • Questioning strategies: • ______ • ______ • Level Two: Recognizing complexity • Close rereading strategies: • ______ • ______ • Deferral of meaning: • ______ • ______ • Level Three: Generating researchable questions about context and meaning • Strategies for Inquiry • ______ • ______ What does it look like when each of these breaks down? Randy Bass, English, Georgetown

36. Learning Activity: Reading a literary text to generate interesting questions about context and complexity. • Level One: Reading • Novice close reading strategies (“schoolish”): • Questioning strategies: • Level Two: Recognizing complexity • Close rereading strategies: • Deferral of meaning: • Level Three: Generating researchable questions about context and meaning • Strategies for Inquiry Intermediate reading/analysis strategies for opening up possibilities and deferring meaning Randy Bass, English, Georgetown

37. “Oral midterms” • 45 minute oral midterm • Compress video and put on cd-rom • Comments linked to time codes, asking them to rewatch and reflect. Randy Bass, English, Georgetown

39. What kinds of findings has this work yielded? • New pedagogical strategies and course designs • e.g. “Think aloud” exercise • New ways to make use of online discussions • New framework: • “Learning Activity Breakdown” • Basis for collaborative inquiry? • A theory or concept about a particular way of reading. • “Protocol of Deferral”: Intermediate reading strategies Randy Bass, English, Georgetown

40. What are the three cases a case of? What works? Focuses on new activities to improve performance of summative assessments Tracking grades over time What’s possible? Focuses on new ungraded course element related to process Coding to track intellectual development in conversation What is? Created new course element in order to make process visible Close reading of think alouds and related student writing All three motivated by discrepancy between course design and long-term learning values

41. LEARNING processes NOVICE processes EXPERT processes LEARNING processes LEARNING processes products products products Exploring a range of learning products appropriate to learning processes “Thin slices” of online discussion or blog Interviews, think alouds, performance Classroom assessment techniques Reflections, justifications

43. The Teaching Commons “The teaching commons is a conceptual space in which communities of educators committed to inquiry and innovation come together to exchange ideas about teaching and learning and use them to meet the challenges of educating students for personal, professional, and civic life.” Pat Hutchings and Mary Taylor Huber, The Advancement of Learning: Building the Teaching Commons

44. Exploring Social Pedagogies:Using Collaborative Inquiry in SoTL • a set of strategies for creating educational environments in which learning occurs in the context of a community • added emphasis on activities that ask learners to represent knowledge for others

45. Exploring Social Pedagogies:Using Collaborative Inquiry in SoTL • Through the close examination of the evidence of student learning, this project : • will create a framework for designing better education experiences • where students develop flexible and integrative thinking • in communication-intensive contexts • Our purpose is to: • capture the essential assignment structures of social pedagogies • create a clear, commonsense schema • demonstrate parallels between pedagogical practices and assessment issues

46. 70 Faculty 21 Campuses Five years Faculty conduct sustained scholarship of teaching projects on technology and learning in the humanities.

48. A Digital Story

49. Digital Story from Cultural History

50. another response (faculty collaborator) “I was watching the digital story, which I’d seen before, and I wrote in my notes ‘digital book report’. I mean, there isn’t anything there that you wouldn’t have learned from a couple of hours in the Civil Rights Museum in Birmingham.”