Rubrics Cube: Development of Six Rubrics to Aid with Student Performance Assessment

1. Rubrics Cube: Development of Six Rubrics to Aid with Student Performance Assessment Marcy Kelly, Ph.D. Associate Professor and Assistant Chair of Biology and Health Sciences Pace University - NYC

2. UB Class of 1995

3. Overview • Biology 101 at Pace University • Biology 101 Discussion Group Program • Six Rubrics to Evaluate Student Learning and Performance 1. Student Participation 2. Concept Maps 3. Exam question/answer level of difficulty 4. Lab reports 5. Student presentations 6. Student surveys/interviews

4. Biology 101 at Pace University-NYC • Enrollment in Biology 101 at Pace University NYC typically exceeds 100 students • The course has three components: 1. Lecture (45% of final grade) 2. Laboratory (20 students per section, 40% of final grade) 3. Discussion Group (10 students per section, 15% of final grade). • The Discussion Group Program was developed in order to ensure an environment that promotes active learning outside of the lecture setting.

5. Biology 101 Discussion Group Program • Students registering for Biology 101, must co-register for a discussion group section • Each section is capped at 10 students • Each section is facilitated by a peer leader • During each session, the students work on problem sets (modules) that reflect the materials covered in lecture • The peer leaders evaluate their students’ performance based upon their level of participation and their execution of specific tasks

6. Biology 101 Discussion Group Peer Leaders • The peer leaders are undergraduate students that were selected based upon their past performance in Biology 101 • Peer Leaders must attend a full day training session prior to the beginning of the academic year • The training session focuses on: • Different student learning styles • Process – the answer is NOT important • Small group facilitation • Grading rubrics • During the academic year, I meet with the peer leaders on a weekly basis

7. A Discussion Group Session at Pace

8. Rubric 1: Participation in Discussion Group • The peer leaders are asked to be my eyes and ears during each discussion group session (I do visit occasionally but, with 10 or more sections a semester…) • They are asked to evaluate student participation using quantitative and qualitative methods 1. Quantitative – rank participation numerically compared to others in section 2. Qualitative – prepare a narrative of all comments made during session * The peer leaders are asked to do this immediately following each discussion group session.

9. What are the peer leaders looking for when they evaluate participation? • An excitement and willingness to learn/understand the material. • Students that are willing to try to answer the questions even if they’re not correct. • Students that help other students that might be struggling • Students that effectively work together in a group

10. ASMCUE Mock Discussion Group Session • Let’s try it! • Needed: 1. 5 volunteers that are not afraid to act out a part in front of their colleagues and peers! 2. An audience to evaluate the participation of each volunteer (that’s everyone else!)

13. Rubric 2: Concept Maps • The peer leaders are sometimes asked to evaluate specific tasks that their students perform – such as concept map development • Additionally, I have asked the students to design concept maps in lecture to test their knowledge/understanding • Finally, I have asked the students to design concept maps to assess the impact of the discussion group program on their learning for a pedagogical research study (JMBE 2007 8:13-21)

14. What is a concept map? • Concept mapping is a technique for visualizing the relationships between different concepts. A concept map is a diagram showing the relationships between concepts. Concepts are connected with labeled arrows, in a downward-branching hierarchical structure. The relationship between concepts is articulated in linking phrases, e.g., "gives rise to", "results in", "is required by," or "contributes to". • The technique of concept mapping was developed by Joseph D. Novak at Cornell University in the 1970s, as a way to increase meaningful learning in the sciences. - Wikipedia

15. Rainbow Bright Colors Fantasy Somewhere Over the Rainbow Prism Rain Clouds Pot of Gold Leprechauns Sun Desk Chair Lamp Computer Printer Paper Pencil Pen Stapler Push Pin Examples of Simple Concept Maps

16. Example of a More Complex Concept Map http://cmap.ihmc.us/Publications/ResearchPapers/TheoryCmaps/TheoryUnderlyingConceptMaps.htm, The Theory Underlying Concept Maps and How to Construct Them Joseph D. Novak & Alberto J. Cañas

17. Evaluation of Concept Maps • Did the student pick the right central concept? • Did the map that the student constructed make sense overall? • Number of propositions? Propositions – 2 or more concepts connected by a line with linking words or phrases to form a meaningful statement.) • Number of crosslinks? Crosslinks – links between concepts in different segments of the domains within a concept map.

18. Sample Concept Map Rubric • Accurate central concept – 10 points • Concept map makes sense overall – 5 points • Correct propositions – 1 point each • Correct crosslinks – 10 points each Total score/highest score in class x 100 = grade Alternatively, you could design a map that you find acceptable and grade your students’ maps against yours

19. ASMCUE Mock Concept Mapping Session • Let’s try it! • Needed: 1. Split classroom in ½ 2. Each half should then split into groups of no more than 5 individuals 3. One half of room work on concept map 1 and other half work on concept map 2 4. Once completed, have a group on the other side of the room “grade” your groups map

20. Concept Map Directions • General Directions a. Look over the appropriate list of terms and decide what the central concept is for the list. b. Develop a concept map for the list with the focus on explaining the central concept. • List for Concept Map 1:DNA double helix, single DNA strand, nucleotide, hydrogen bonding, DNA polymerase, leading strand, lagging strand, meiosis, 5’ end, 3’ end, A, C, T, G, complementary DNA strand. • List for Concept Map 2: DNA template, single ribonucleotides, RNA polymerase, promoters, initial RNA transcript, RNA splicing, mRNA, rRNA, tRNA, triplets, and codons.

21. Lowest point value Highest point value Rubric 3: Exam question/answer level of difficulty

22. From which category is each question below? - Analysis • Can the world exist without RNA? • Calcium efflux in the thick ascending Loop of Henle, is it real? • Though plants, fungi, and prokaryotes all have cell walls, we classify them under different taxonomic units. What is the basis for placing these organisms in different taxa? - Synthesis - Comprehension

23. From which category is each question below? • Prepare a critical review of the following primary literature citations… • What is the most common compound in the cell walls of Gram-positive bacteria? • Jams, jellies, preserves, honey, and other foodstuffs with a high sugar content hardly ever become contaminated by bacteria, even when the food containers are left open at room temperature. This is because bacteria that encounter such an environment _____. - Evaluation - Knowledge - Application

24. Rubric 4:Lab Reports • Use same question level of difficulty chart to determine points allotted to each component of the report 1. Abstract 2. Introduction 3. Materials and Methods 4. Results 5. Discussion 6. Citations - Comprehension - Synthesis - Knowledge - Analysis - Evaluation - Knowledge

25. Rubric 4:Lab Reports

26. Rubric 5: Student Presentations • Use same question level of difficulty chart to determine points allotted to each component of the presentation 1. Title Slide 2. Hypothesis/Objectives Slide(s) 3. Materials and Methods Slide(s) 4. Table/Figure that Student is Presenting 5. Summary of Results Slide(s) 6. Discussion/Conclusions Slide(s) 7. Presentation style/Overall organization 8. Questions from audience - Knowledge - Synthesis - Knowledge - Knowledge - Analysis - Evaluation - Synthesis - Analysis

27. Rubric 6:Student Surveys/Interviews • Student Surveys – Likert scale surveys are easiest to quantitate * Student Assessment of Learning Gains • Student Interviews/Comments – Qualitative assessment tool, code comments * Look for keywords (visual aids, interactions, reflection, relate material, focus, repetition, critical thinking) * Keywords that require looking into – interesting, exciting, learn, boring * Evaluate frequency of comments– how often are similar comments made?

28. Further Assistance with Rubric Development • Website Rubistar – http://rubistar.4teachers.org/index.php • Text Stevens, D.D. and Levi, A.J. 2005. Introduction to rubrics: an assessment tool to save grading time, convey effective feedback, and promote student learning. Stylus Publishers, Inc., Sterling VA

29. Thank you!!!! • University at Buffalo and Dr. Jerry Koudelka • ASM Scholars in Residence Program • Fall 2005 Biology 101 students and peer leaders • Dr. Richard Schlesinger and Biology faculty, Pace University