Classroom Assessment Techniques

1. Classroom Assessment Techniques • Teaching Day • August 23, 2006 • Nancy Hollins

2. Classroom Assessment Techniques (CATs) CATsare simple methods faculty can use to collect feedback, early and often, on how well students are learning what they are being taught. The purpose of classroom assessment is to provide faculty, and students, with the information and insights needed to improve teaching effectiveness and learning quality. It differs from tests and other forms of student grading in that it is generally not used to assign grades. Classroom assessment can be used as a type of classroom research; as such it should be viewed as a legitimate form of scholarship.

3. Example: Written work • Paraphrasing, summarizing, making connections, applying to new situations • Limit writing by limiting the time allowed for writing or ask a specific question to focus the writing or ask for one page only • Analysis: Sort written work into categories (adequate/inadequate) and count –What are the common misconceptions? What is the general level of understanding and is that level sufficient?

4. Example: Worksheets • Recall and list, categorize, identify pros and cons, outline, concept maps • Easily scanned and therefore very time efficient • Analysis: identify commonly missed items

5. Example: Problem Solving • Identify type of problem, critique solutions, generate solutions • Can be written work or based on a worksheet. Documented problem solving (identifying the steps taken) is more time intensive for the student but can take the form of a flowchart so assessment is efficient for the professor. • Analysis: Sort into adequate/inadequate - look for superficial thinking, lack of critical thinking, missing knowledge

6. Example: Self Assessment • Study skills, knowledge, ability to apply, attitudes • Journals, surveys, anonymous writings that are read publicly. • Analysis: Sort into adequate/inadequate – address those issues that are common e.g. misperceptions, biases, time issues with studying.

7. Example: Classroom Management • Assignments, class activities • Journal, survey, suggestion box, class student representatives • Analysis: Sort into types of problems – address those that are common.

8. Why should I use CATs?For faculty, frequent use of CATs can: • Provide feedback about learning and teaching when it is still possible to make changes. • Provide information about student learning with a much lower investment of time as compared to tests, papers, and other traditional means of grading. • Foster rapport with students. • Encourage the perspective that learning is a joint enterprise between faculty and student.

9. Why should I use CATs?For students, frequent use of CATs can: • Help students monitor their own learning. • Point out the need to alter study skills. • Help students learn new study strategies. • Provide concrete evidence to students that the instructor cares about their learning.

10. How do I use CATs? • Start small. As you read through the classroom assessment techniques described in the slides that follow, consider which one has the most potential for a course you are teaching. What do you think you might learn by using this technique? • Will you need to modify the basic procedure to suit your particular situation? How will you modify it? • Decide whether or not student performance should be anonymous. With anonymity you get more honest feedback. But sometimes, even 1% of the final grade will encourage students to take the CAT seriously. • As you design the activity, keep it simple. What do you want students to do? What kind of response do you think you will get? Don't ask for more data that you need or are willing to use.

11. How do I use CATs? • Introduce the CAT by letting students know how the information you collect will help them. • Once you have collected the students' responses, sort and analyze the data. For the most part, student responses will probably sort easily into a few general categories. • When you have analyzed the data, share at least some part of that analysis with your students. How will you use the information? How can the students use the information? • Finally, don't feel you have to rush into using these techniques in every class or during every session. Begin slowly in order to avoid the only real danger in classroom assessment--too much data and not enough time or experience to know what to do with it.

12. Internet Resources • Southern Illinois University http://www.siue.edu/~deder/assess/catmain.html • Honolulu Community College http://honolulu.hawaii.edu/intranet/committees/FacDevCom/guidebk/teachtip/assess-2.htm • Penn State http://www.schreyerinstitute.psu.edu/Resources/class_assessment.asp • Middle Tennessee State University http://www.mtsu.edu/~itconf/proceed99/Martin.htm • Iowa State University http://www.celt.iastate.edu/teaching/cat.html

13. Angelo, T.A. & Cross, P.K. (1993). Classroom Assessment Techniques (2nd ed.). San Francisco: Jossey-Bass.

14. Specific Techniques Teaching strategies and CATs are often very similar. The difference is that CATs are used specifically to generate data to be used to make changes in teaching – with the goal of improving learning. The analysis of the assessment results is not focused on individual student performance but rather on overall class performance.

15. Goal Ranking & Matching • This CAT is best done at or near the beginning of a course when adjustments to the syllabus are possible. • The professor begins by identifying the goals for the course, ranking those goals in order of importance. • Students then identify their goals for the course and rank them. • Analysis: List the professor’s goals in order of importance. Next to them, list students’ goals in order of importance. Do the lists match in content? Do they match in order of importance? Are there significant differences between professor and students and between student and other student? • The professor uses the data to generate a discussion. This discussion may involve the “selling” of goals to students – why is it important for them to achieve a particular goal deemed important by the professor? On the other hand, perhaps the professor can incorporate student goals into assignments, class activities. And, if some goals just can't be realized, an early, honest response can align expectations in a positive way in order to prevent disappointments.

16. Background Knowledge Learning Probe • Research suggests that one of the best predictors of student learning is what the student already knows before coming to class. Students bring a lot of internalized old knowledge with them. They try to place new knowledge into existing knowledge and, when it doesn't fit, it is often discarded (not learned). • This CAT uses a series of multiple choice questions to probe background knowledge. Only a few questions need to be used to sample students' previous knowledge. Probes are useful at the beginning of individual topics as well as whole courses. • Analysis: Like multiple choice exams, probes are easy to score. Common misconceptions can be identified and addressed. Common gaps in knowledge can be filled. How this remediation takes place is obviously determined by the constraints of the situation.

17. Misconception/Preconception Check • The misconception/preconception check is a variant of the background knowledge probe, but it focuses directly on those kinds of prior knowledge (or beliefs) that may actually hinder learning. This technique can be particularly useful in courses dealing with controversial or sensitive issues, or those in which students may have developed intuitive but inaccurate theories. • When preparing multiple choice questions for a misconception check, begin by asking yourself the following questions: What misconceptions might be commonplace among students who take this course? Which of these are most likely to interfere directly with learning for the course? How can I deal with these misconceptions once they are identified? • Analysis: As with the background probe, common misconceptions can be identified and addressed.

18. Minute Paper • This CAT is typically used at the end of a class session. Students are asked to write for one minute on a particular topic. The topic may be the most important point of a session or the most disturbing or the most surprising. The professor might choose a topic that seems to be most puzzling to the students. • At the end of the minute, students are asked to generate a question they still have about the topic. This requires students to integrate the topic into their existing knowledge base and identify what they still don’t know. This approach also provides an easier (because more private) way to ask questions. • This CAT requires students to organize a "chunk" of information about a particular topic. Analysis: Look for commonalities in conceptual understanding (or misunderstanding) as well as difficulty with organizing information into a brief written work. The questions themselves can be incorporated into a review during the next class session.

19. RSQC2 • RSQC2 stands for Recall, Summarize, Question, Connect, and Comment. This is a modification of the One Minute Paper. Students take two minutes to recall and list in rank order the most important ideas from a previous day's class. Then they take another two minutes to summarize those points in a single sentence in order to "chunk" the information. Next, students are asked to write one major question that they want answered. Finally, students identify a thread or theme to connect this material to the course's major goal. As an option, students may add a comment regarding their confidence in their understanding of the specific topic. • Analysis: Analysis is less time consuming if students are told to use the subheadings (recall, summarize, etc.). The professor then looks at each subsection to identify a) what important points were not recalled, b) what misconceptions were found in the summaries, and c) what relationships between ideas are missing or inaccurate. The confidence subsection can be viewed as a type of self-confidence survey – do students lack confidence and therefore need more practice? The questions can be answered at the next session.

20. Directed Paraphrase • Students are given five minutes to summarize a key idea that was presented during a class period (current or just past). • To paraphrase, students must demonstrate that they understand the idea well enough to write about it in their own words. • The directed part specifies the audience for whom the student is writing. Can the student choose language to match the needs of a specified audience? • Analysis: Sort the papers into two piles - adequate understanding vs. inadequate understanding. Determine whether misconceptions are individual or class-wide. Then sort the papers into adequate language vs. inadequate language. Determine whether or not the class as a whole has the ability to adjust language to a specific audience’s needs.

21. Double-Entry Journals: • In this CAT, students make notes on one side of a notebook and use the other side to comment or ask questions about their notes, react to or reflect on the topic. These notes can be either a) taken during a lecture or b) based on assigned course readings. • This approach can also be used to document problem solving. On the one side students can record a sample problem solution and on the other side they can record the steps they took to solve it. • Analysis: At the beginning of the next lecture session, the instructor can start the class with the students discussing a portion of the questions and comments. By collecting and scanning the entries, the professor can also identify any issues that need further teaching.

22. Word journal • Students are asked to summarize a specific topic into a single word, then write a short paragraph explaining the word selection. • Analysis: This CAT can be used to examine the depth of reading comprehensive, creativity in summarizing information, and skill at defending selection. Similar to a one-minute paper, this CAT focuses the students’ attention on the use and precision of language. The professor can provide feedback by reading examples in the next class and having the students discuss the response as adequate/inadequate.

23. Muddiest Point • In this CAT, students are asked to write down what was least clear to them. This requires the students to rate their own understanding across several topics and then reflect on why one particular topic should be selected as least understood. • Analysis: Determine those topics that seem “muddy” to a number of students. Decisions on action may include additional class time if a significant percentage of the students have the same “muddy” point. Tutorial sessions can be scheduled if the number is high but not high enough to devote more class time. Handouts might address those points that are “muddy” to only a few. • This CAT should be used infrequently. Focusing on muddiest points too often can result in an emphasis on the negative.

24. What’s the principle? • The professor Identifies several basic principles that students are expected to learn in the course. • The professor then finds or creates sample problems or short examples that illustrate each of these principles. Each example should illustrate only one principle. • Create a What's the Principle? form that includes a listing of the relevant principles and the list of specific examples or problems. Students are then asked to match the principles to the examples. • Analysis: Scan for those principles that are inaccurately identified by a number of students. These principles need reinforcement.

25. Characteristic Features • This CAT involves listing characteristics or properties of a item, concept or principle. These characteristics are those that help differentiate the topic from others. This assessment technique is particularly useful for seeing whether students are separating items or ideas that are easily confused. By selecting especially critical differentiators, a professor can both highlight and assess the students' use of analysis to help them characterize central concepts. • The student is asked to place a plus or minus sign in front of characteristics that do/do not belong to the topic. • Analysis: Scanning the results is easy and rapid so this CAT is suitable for use in large classes. Even simple tallies will reveal the extent to which students are paying more attention to some traits and less attention to others.

26. Memory Grid • Students fill in cells of a two-dimensional diagram for which the professor has provided labels. For example, in a music course, labels might consist of periods (Baroque, Classical) by countries (Germany, France, Britain); students enter composers in cells to demonstrate their ability to remember and classify key concepts. • Analysis: Tally the numbers of correct and incorrect responses in each cell. Analyze differences both between and among the cells. Look for patterns among the incorrect responses.

27. Categorization grid • This CAT is similar to a memory matrix. Students are given a grid containing 2 or 3 important categories and a scrambled list of terms, images, or equations. Students are asked to sort the list into the categories. • Analysis: This CAT can be used to discover how students categorize information and how well learners understand "what goes with what“. Because of the grid, scanning of answers is fairly easy and efficient.

28. Pro and con grid • Students are given a grid on which they list the pros/cons, costs/benefits, advantages/disadvantages of a solution to a particular problem. • Analysis: The grid format allows for quick scanning to determine the depth/breadth of students’ analyses and their capacity for objectivity and critical thinking.

29. Empty Outlines • This technique works when content is delivered by lecture. The professor asks the student to outline all or part of the lecture. The student can be asked to start from scratch or the instructor may provide a partial outline of the lecture. This technique measures a) student understanding of relationships within the content and b) the effectiveness of student listening. The first time the technique is used, the instructor may want to provide a partial rather than blank or “empty” outline. • Analysis: a review of the outlines should indicate whether or not the students are understanding connections between information. If results are mixed, the instructor may ask the students for feedback – did they not understand the concepts or could they not organize them? This technique has the added benefit of enhancing students’ use of the technique of studying through outlining.

30. Concept Map • A concept map is a two-dimensional, hierarchical node-link diagram that depicts the structure of knowledge within a discipline as viewed by a student, an instructor or an expert in a field or sub-field. The map is composed of concept labels, each enclosed in a box or oval; a series of labeled linking lines, and an inclusive, general-to-specific organization. • Analysis: By reading the map, an instructor can: a) gain insight into the way students view a scientific topic; b) examine the valid understandings and misconceptions students hold; and c) assess the structural complexity of the relationships students depict. • In addition to these applications in assessment, faculty have also used concept maps to organize their ideas in preparation for instruction, as a graphic organizer during class, and as a way to encourage students to reflect on their own knowledge and to work together and share their understandings in collaborative group settings.

31. Problem recognition tasks • The professor provides students with a range of problems from which they must identify the type of problem that each example represents. This can be done individually or a list of problems and types can be matched. • Analysis: This CAT can be used to examine how well students can identify problem types and match problems with possible solution methods. Specific types that are inaccurately identified by a number of students can be the focus of further course work.

32. Documented problems • Documented problems are similar to the common request to "show your work." By asking students to describe their reasoning, the professor can get a direct measure of students’ problem solving ability. • Documentation of a problem can be something as simple as a brief paragraph or two of what was done (and why) or as extensive as a line-by-line report of each step (as in a mathematical proof). Having students write out the reasoning behind each step of a problem gives the professor very detailed feedback about the students' skills and understanding. To make the task manageable, you might select a sample to scan. • Analysis: The professor categorizes the problem solving strategies, identifying those strategies that are least effective and most effective. Additional practice and/or discussion may be added to a course if a number of students use the least effective strategies. The professor can also put a solution on the board and ask the students to analyze it, discussing its effectiveness and offering alternatives.

33. “To Use or Not to Use" Analysis: • The professor prepares a sample problem or the description of a situation and recommends a solution. Students are requested to discuss reasons why the solution is or is not appropriate. The professor might also provide two differing solutions that students debate. Depending on the scope of the course, the final decision can be based strictly on a student’s personal choice, or it may include a more objective analysis of theory, cost/benefit, research support, etc. • Analysis: The professor categorizes the responses into the whys and why nots. If there are significant differences in the responses, the responses may be used to guide a class discussion or provide written feedback.

34. Application cards • After teaching about an important theory, principle, or procedure, the professor asks students to write down at least one real-world application for what they have just learned to determine how well they can transfer their learning. • Analysis: Read through the applications and categorize them according to their quality. Pick out a broad range of examples and present them to the class. Have the students categorize their quality, discussing what determines a good answer from an adequate or poor response.

35. Transfer & Apply • In this CAT, students are given a new situation to which they are asked to apply a theory or principle taught in class. Using a form with several questions can be helpful in focusing this writing and therefore making analysis more efficient. This exercise can be written individually, in pairs, or in small groups and can take anywhere from 5–15 minutes. • Analysis: The professor scans the written papers to determine the ability of students to transfer learning to new situations. Poor performance may indicate more practice with new situations is desirable to enhance learning.

36. A Self-Confidence Survey • This CAT helps to identify areas where students feel competent and where they do not. The professor identifies the skills needed for achieving specific learning objectives. These skills are then listed with students indicating on a rating scale how confident they feel in the performance of that skill. • Analysis: The professor identifies the level of confidence and matches it with his/her own perception of performance. Over-confident students may not feel a need to study. Under-confident students believe they "can't get this stuff“, becoming demotivated. Students are placed into small groups to discuss the results of the survey, The students themselves may discover and offer potential remedies for both situations. • The rating scale allows the professor to determine an “average” confidence level for a particular skill. This average in itself may identify areas needing further practice.

37. Assessing Group Effectiveness • This CAT asks group members individually and anonymously to describe the task assigned to a group, explain the organization they see as necessary to accomplish the task, and reflect on the individual talents and team effectiveness required to conclude the task successfully. The focus is on self-assessment – what does the student him/herself contribute to the group, not on assigning blame or voicing complaints. The assessment should be undertaken early --- within a couple weeks of the group's formation. Ideally, the completed assessment forms are circulated within the group, becoming the focus of constructive discussion. The discussion itself can be summarized by the group in a formal, written, one-page report to the professor. • Analysis: A professor's role in providing feedback might consist of meeting with the group and orally reinforcing points of agreement and identifying differences as well. Responsibility for adjusting behavior to increase effectiveness rests with the group, not with the professor. The entire process can be repeated later in the semester, especially if a group is having difficulty.

38. Student- generated test questions • Allow students to write test questions and model answers for specified topics, in a format consistent with course exams. This will give students the opportunity to evaluate the course topics, reflect on what they understand, and what are good test items. • Analysis: Evaluate the questions and use the goods ones as prompts for discussion. Evaluate the answers for commonly held misconceptions. You may also want to revise the questions and use them on the upcoming exam.

39. Exam Evaluations • Select a test that you use regularly and add a few questions at the end which ask students to evaluate how well the test measures their knowledge or skills. Use the feedback to make changes in the exam for future classes (if reasonable). • Analysis: Track student performance on the exam over time…did your changes make a difference.

40. Self Assessment • This CAT is an excellent way to make visible different styles of thinking and different attitudes on controversial topics...without embarrassing anyone individually. Students are informed that their written responses will be private. • The professor presents students with alternative ways of looking at a controversial issue and asks them to indicate, by writing on a 3x5 card, which perspective they hold and why. The responses are then swapped 3-4 times face down, thus allowing the overall results to be read publicly without compromising confidentiality. • Analysis: The professor counts the cards to determine the most commonly held perspective. The argument for adoption of this perspective is analyzed for critical thinking. Points needing clarification or further discussion are the focus of the next class. • This technique is most successful when the level of trust is high, middle to end of semester.

41. Personal Journals • Students are asked to keep journals that detail their thoughts about their learning. Content may include such items as awareness of multiple perspectives, reflection on the application of course material, confidence in performance of specific skills. The journal entries may be general or address a specific issue such as reflection on performance of an assignment. The students turn in the journals several times during the semester so the professor can give feedback, as well as chart development. Journals may or may not be anonymous. • Analysis:The journals will reveal the development of individual students, but as a CAT the journal can also reveal patterns of development in the students as a whole that can be addressed directly.

42. Assessment of Effective Study Time • Effective study can be thought of as a function of time multiplied by effort. A self Assessment of Effective Study Time can bring habits of effective study to the surface by focusing a student's attention on these two factors. • Students engage in this assessment by filling out a study log (when do they study, what, and how) for one week and then examining the log for trends. • A professor's participation in this assessment can involve individual consultation with students as well as reporting trends in the logs as a whole. Students might need help to see opportunities to change their habits and increase their study effectiveness. Students may also report significant time expenditures that suggest that their course (and work) workload is overly heavy or they are spinning their wheels using ineffective study methods.

43. Chain notes • Students pass around an envelope (or envelopes) on which the teacher has written one question about the class. When the envelope reaches a student he/she spends a moment to respond to the question and then places his/her response in the envelope. These questions might ask about such things as the like/dislike of the format of the class sessions, the clarity of a particular topic, or the perceived relevance of an assignment. • Analysis: Go through the student responses and categorize the data with the goal of detecting response patterns. Discuss the patterns of responses with students.

44. Class Journals • Students are asked to keep journals that detail their thoughts about the class. Content may include such items as organization of lecture material, usefulness of handouts, effectiveness of group work, or relevance of an assignment. Specific questions can be introduced throughout the semester to address such as their thoughts on a specific media or classroom activity. The journals should be anonymous so that honest feedback is obtained. One possible way to do this is to have the students place some sort of identifying phrase or password on the journal so they can retrieve it. The students turn in the journals several times so the professor can get frequent feedback. • Analysis:The journals may reveal consistency in the positive and negative aspects of the course.

45. Student Group Reps • Students in a course are asked to select several students or volunteers to meet as a small group with the professor on a regular basis. This group will discuss how the course is progressing, what they are learning, and make suggestions for improving the course. The volunteers are periodically given time in class to get feedback from the other students without the professor present. • Analysis: The group reps should also be involved in the analysis of the information they bring to the meetings and in identifying possible solutions to any problem. Some issues will be for your information, some should be addressed in class.

46. Suggestion Box • The professor may put a box near the classroom door and/or office door and ask students to leave notes about any class issue. • Analysis: Review and respond to these suggestions during the next class session.

47. Caveat! • Keep CATs simple. • When just starting out, limit the use of CATs to only one or two per course. • You have to analyze the results quickly in order for any remediation to be effective. • Don’t use a CAT when you can’t get to the analysis in a timely manner. • Document the results of using a CAT. This documentation will come in helpful when your teaching is reviewed by others as well as when you reflect on your teaching at a later date.