Assessment = Learning

1. Assessment = Learning What evidence will you and your peers accept? Diane Ebert-May, Ph.D. Lyman Briggs School Botany and Plant Pathology Michigan State University

2. Innovations Active Learning Inquiry Change Students Faculty Institutions use in-depth value in-depth Outcomes Assessment Content/ Process Cooperative Learning Self-Efficacy toward toward necessitates necessitates by by by collaborate with supported by develop achieve drives e.g. e.g. e.g. e.g. Reasoning

3. Cooperative Groups • 4 students per group • Person A, B, C, D in each group • First - read question/think about answer individually • Discuss: A with B • C with D • Form group consensus

4. Q1 - Engagement • Imagine I invited you to visit and review my course Introductory Biology to help me decide if I was practicing “active learning.” What criteria and dimensions of teaching would you look for in my class? • Oral report from groups

5. Biological Literacy for All • Utilize process of scientific inquiry to think creatively and formulate questions about real-world problems • Effectively communicate an understanding of and links among biological principles and concepts to peers and others • Gain confidence in ability to write about, criticize and analyze concepts in biology

6. Biological Literacy for All (2) • Reason logically and critically to evaluate information • Develop positive attitudes about the relevance of biology to their lives • Demonstrate positive interdependence and individual accountability within cooperative groups

7. NSF- Undergraduate Course and Curriculum Development Grant • Is it possible to implement strategies for active, inquiry-based learning and cooperative group interaction in large courses? • Do these strategies promote more effective learning by more students? • 600 -700 students in class meetings • 450 students in laboratory

8. Compared Two Approaches to Teaching • Traditional • Experimental

9. Who Asks Questions • Traditional: Lab manual/faculty • Experimental: Students

10. Inquiry Lab/Class Meeting • Traditional: Confirmatory • Experimental: Inquiry

11. Functional Cooperative Groups • Traditional: Not Often • Experimental: Often

12. Writing/Reading to Learn • Traditional: Not Often • Experimental: Often

13. TA Professional Development • Traditional: Superficial • Experimental: In-depth, on-going

14. Assessment • Traditional: Tests • Experimental: Multiple Forms • Aligned with student outcomes

15. Learning Cycle Model in Class Meetings Cooperative Groups: 4 Students Engage 5 - 10 minutes Explore 10 - 15 minutes Explain 10 - 20 minutes Elaborate 10 - 15 minutes Evaluate 10 minutes

16. Experimental Design Covariates Pre-Test / Factor Scores Control Lectures Experimental / Control Lectures Experimental Lecture / LabControl / Experimental Labs Experimental Labs •Fall 1994 •Spring 1995 •Fall 1995 • Self-Efficacy • Self-Efficacy •Self-Efficacy • NABT • NABT •NABT • Process Skills I, II • Process Skills I, II • Process Skills I, II • TOSRA •TOSRA •TOSRA Independent Variables Intervention --- TA --- Gender --- Ethnicity --- Age (Lab, Lecture)

17. Q2 - Exploration • What evidence will you (and your peers) accept that indicates your students have learned and achieved the outcomes of your course? • Written group report

18. Self-Efficacy Subscales • Fac 1: Writing and critiquing; analytical skills (.92) • Fac 2: Generalizability to other science courses (.87) • Fac 3: Biological literacy (.88)

19. Self-Efficacy, Fall 94 4.4 4.2 4.0 Factor Scores* 3.8 PosFac 1 3.6 PosFac 2 3.4 PosFac 3 CON EXP Experimental Condition * p < .01 (n =338)

20. NABT Process, Fall 1994 14.2 14.0 13.8 13.6 13.4 13.2 13.0 Control Experimental Lecture p < .01 (n =356)

21. NABT Content • No difference in scores between traditional and control groups

22. Model with 700 students • Two Sections @ 350 students each • Taught back to back on schedule • T, Th - 75 minute periods • Team taught - 2 faculty as team partners in both sections

23. Number of Absences

24. Grade Distribution A 10% A 8% B 41% B 39% Number of Students C 30% C 29% D 12% D 11% F 13% F 8%

25. Assessment and Curriculum Design • Two sides of the Same Coin

26. Design Course Goals • Course - goals • Modules - objectives • Activities - objectives

27. Develop Performance Expectations • Descriptions of Student Performance • Providing evidence convincing to you, your colleagues, and significant others that students have achieved a course goal or module objective • Descriptions of the conditions under which the evidence is collected

28. Assessment • Data collection with a purpose

29. What data are collected? • Measures of students’ knowledge and abilities • Answers to items on short answer tests • Essays • Position Papers • Oral Presentations • Poster Presentations

30. Assessments Linked to Goals • Goals articulated with outcomes that are measurable or observable (actions)

31. Functions of Assessments (Hodson 1992) • summative - description of students’ level of attainment • formative - diagnostic feedback to students and instructor • evaluative - provide instructors feedback about the effectiveness of the curriculum experiences • educative - engage students in interesting, challenging, significant experiences to develop further insight and understanding

32. Development of Rubric • Assessment tasks linked to outcomes • Define performance standard for biologically literate student.

33. Categories • Name of category • Criteria within categories -- discriminate among qualities you determine important • Reach consensus - students involved

35. Engagement problem • E.O. Wilson states that insects and other land-dwelling arthropods are so important that if all were to disappear, humanity probably could not last more than a few months. Based upon what you know today, do you believe that this statement is true? Write an essay and explain your reasoning as logically and thoroughly as you can. Do not use any outside resources for your essay.

36. What is the evidence we accept that students have learned? • What do we accept as a reasoned argument? • What are the criteria we use for evaluation of the argument?

37. EVIDENCE (E) CONCLUSION (C) Specific information relied on to support a given conclusion (E1, E2,...,En). Assertions put forth for acceptance. “So... C” “Evidence indicates that... E1, E2,...En.” WARRANT (W) Statement justifying the move from Evidence to Conclusion “Because E1, E2,...En, then C.” Argument Structure

38. Why use argument structure in science? • Consistent with nature of science • Justifies conclusions to the learning community • Develops problem solving and critical thinking skills • Develops art of questioning

39. What about content? • Argument and inquiry drive understanding of content • Content drives argument and inquiry

40. What did we find? • Students seldom disagreed with the statement • Some misconceptions about content appeared • Students rarely used warrants to support evidence

41. What did we do?

42. Revised Question • The keynote speaker at a scientific meeting made the statement that “insects and other land-dwelling arthropods are so important that if all were to disappear, humanity probably could not last more that a few months.” This generated considerable discussion, both pro and con at the meeting. What do you think about this statement? Construct an argument to explain your views and to illustrate your reasoning.

43. Question: • When we introduce innovation into our classes, how are student evaluations affected?

44. Self-Evaluation • Students who internalize valued achievement targets so thoroughly as to be able to confidently and completely evaluate their own and each other’s work, almost automatically become better performers in their own right. • McMillan and Forsyth (1991)

46. Goal 2 - Effectively describe biological concepts to peers • “So much group work made me realize that understanding a concept and being able to communicate a concept are different things.” • “...I find myself explaining things I was unable to explain before.”

48. Goal 3 - Confidence in ability to write about, criticize, analyze concepts in biology “I always was good at biology (or at least got good grades in biology), but never really understood any CONNECTIONS. This class made it clear to me that everything is connected to each other. We were forced to write down how we UNDERSTOOD concepts, not simply to memorize parts and functions.”

50. 4 - Use process of scientific inquiry to think creatively and formulate questions... “...we did our position papers on two real world issues that we deal with every day. One paper we did was on the environment and the other was on breast cancer...This class really makes you think about how important scientists are in the world today. They try to formulated and answer questions that will help us survive in the future...”