Using Rubrics to Analyze Critical Thinking Creating a Stackable set of skills

1. Using Rubrics to Analyze Critical ThinkingCreating a Stackable set of skills Dr. Hari Singh School of Business Humboldt State University

2. Using Rubrics for skills Discuss history and uses or rubrics. Show how rubrics can reduce a variety of problems. Creating a roadmap that develops consensus among faculty, reduce student complaints, and can be a basis for assessment. For critical thinking rubrics can be made progressively more challenging. Advance skills require not only evaluation of information but synthesis of knowledge, and development of advanced research skills.

3. What is a rubric? A rubric is a mechanism or tool for developing an evaluation scale across different dimensions. Most rubrics are analytical: they decompose a skill or subject into key dimensions or traits. Each component can be evaluated separately. Rubrics are criticized for resulting in excessive rigidity or inhibiting innovation. A rubric is a tool and it can obviously be misused. The important thing is to ensure that the tool is effective and flexible.

4. Some History of Rubrics In the early part of the twentieth century, rubrics were constructed to develop writing scales. Objective measurement of student writing on some scale that could by applied uniformly across classes and instructors. Top Down: Earlier rubrics were developed typically at universities and applied by school administrators. The goal was trying to ensure more consistency and sometimes resulted in excessive standardization.

5. Historical development of rubrics violated the “the law of distal diminishment”. • Any tool developed for effective teaching becomes less useful (potentially more harmful) if it originates far away from the class room. • Students and teachers should provide input on a continuous basis for the construction and modification of rubrics. • An iterative process: Experience from the classroom provides relevant feedback for developing a more nuanced rubric. Relevance of rubrics to Learning

6. Descriptive not prescriptive Rubric should not become rigid straight jackets that reduce flexibility and creativity. A rubric is merely a classifying device. The key issue is what is classified and what type of consistency is derived from it. The consistency should be about the performance thresholds of the student learning outcomes.

7. Flexibility is key Performance thresholds should allow flexibility in different pedagogical styles, different teaching delivery methods, and innovative ways encouraging student learning. Striving for consistency in evaluating student learning outcomes is fully consistent with allowing and encouraging different teaching styles and methods.

8. Existing problems Lack of consensus among faculty about the specific types of performance thresholds. Lack of consistency of academic standards for these performance thresholds across Classes. Inadequate transparency of grading students for different thresholds. Morestudent complaints because threshold performance level is not transparent and communicated clearly to students. Difficulty in assessing student learning outcomes (SLO) consistently.

9. Consensus on learning values Constructing a rubric encourages teachers to develop a consensus about the key learning goals of the course. As Callison (2000; P. 31) states, “Rubrics are texts that are the visible signs of agreed upon values. They cannot contain all the nuances of the evaluation community’s values, but they do contain the central expression of those values.” Facilitate communication between students, teachers and other stakeholders by providing a context for these discussions to take place. Callison, D. (2000). Rubrics. School Library Media Activities Monthly. 17 (2), 34-36, 42.

10. Transparency for students Popham (2003) points out that a well constructed rubric can help students to understand more clearly the expectations of instructors. The transparency that rubrics provide for ranking, ratings, and grades can make the teaching process more meaningful for students Provide a set of stackable skills that can be made progressively more difficult. Greater transparency will result in less complains by students . Popham, W.J. (2003). Test better, teach better: The instructional role of assessment. Alexandria, VA: Association for supervision and curriculum development.

11. Transparent & Progressive norms Brescianiet al. (2004) point out that well developed rubrics can serve as effective and transparent norms to bring expectations of faculty and students for a consistent vision. Increases consistency in class standards so that similar student expectations are applied across different classes of the same course.   Rubrics can be progressively benchmarked at higher levels. Progressively increase the challenge and difficulty of critical thinking in subsequent courses so that complex learning skills can be mastered. Rubrics can provide a transparent and clear roadmap for a ratcheting up the standards across a sequence of courses. Bresciani, M.J., Zelna, C.L., & Anderson, J.A. (2004) Assessing student learning and development: A Handbook for practitioners. National Association of Student personnel Administrators.

12. Facilitating assessment The consistency of learning goals and a detailed roadmap of how to assess the goals is a key issue. It reduces the subjective component of grading (Moskal, 2000). Rubrics help identify specific weak points to strengthen the overall chain of learning. Moskal, B.M. (2000) Scoring rubrics: What, when and how? Practical Assessment research and Evaluation, 7 (3).

13. How to Set up the process Time line should allow a pilot run. Find champions among faulty for classes where goals can be tested. Bad results don’t reflect on instructors. Typically assessment should be towards the end of the program. However, a pre-test at the onset of the program is useful. Form an Assessment Group which includes all disciplines\skill areas.

14. Insights about methods Students perceptionsabout learning are less reliable. Student outcomesof learning can be based on class assignments. Benchmarking learning thresholds is useful. Instructors, students, and assessors need a common consensus. A well constructed rubric is a good vehicle for achieving clarity and consensus among all groups.

15. Constructing Rubrics Identifying 5 or 6 dimensions to be tested (use student feedback) Deciding 5 thresholds for each dimension (use student feedback) How to scale each dimension? Examples of rubrics that work well Discussion among participants with rubric examples