Reflective Judgment

1. Reflective Judgment • Ryan Hargrove • Stephen Smith • Sharon Dotger

2. Navigation Key • Clicking on will return you to the table of contents. • If you want an example, just click on • Wait to click on until you are in the review activity. • If no icon is present, simply click the mouse to advance the tutorial. Example Return to Review

3. Table of Contents • Defining Ill-Structured Problems • Defining Reflective Judgment • 7 Stages of Reflective Judgment • Correlations to Other Constructs • Relevancy to Education • Opposing/Competing Theories • Review Activity • References

4. There have been frequent reports about the relationship between chemicals that are added to foods and the safety of these foods. Some studies indicate that such chemicals can cause cancer, making these foods unsafe to eat. Other studies, however, show that chemical additives are not harmful, and actually make foods containing them more safe to eat. What is your opinion on this issue? Think about this problem. Write down your thoughts, feelings, and opinions. Consider how you would decide who is right? What do you do with competing evidence? Formulate an argument supporting your case. Click here to go to the next screen.

5. The problem that was presented to you was an example of an ill-defined problem. An ill-defined problem is one that addresses complex issues and thus cannot easily be described in a concise, complete manner. Furthermore, competing factors may suggest several approaches to the problem, requiring careful analysis to determine the best approach. Did you consider any competing factors in the problem your were presented? Did you suggest that there may be several approaches to the problem? What kind of analysis went into determining the best approach, and ultimately your opinion?

6. This diagram illustrates several of the possible approaches to the food additive problem. Each of these approaches represent divergent thoughts, feelings, and opinions.By clicking on each approach you will be introduced to an example of how these groups might answer the food additives question differently. Medical Professionals Retailers Cancer patients Consumers Food Additives

7. Cancer patients Approach is based on the dangers of food additives to human health, particularly in relation to the development and treatment of cancer.

8. Approach is based on the belief that additives are essential for the preservation, quality, and taste of many of the foods we eat everyday. Retailers

9. Approach is based on past and current research suggesting that some additives have been found harmful, however precautionary measures are in place to prevent the likelihood of similar problems occurring again. Medical

10. Approach is based on critical inquiry, and a need to complete disclosure of information to the public. A belief that additives can be both beneficial and extremely dangerous, and therefore should be considered on a case by case basis Consumer

11. After reviewing the various approaches were you able to better understand the concept of ill-defined problems? This example was intended to clarify the competing factors that exist in ill-defined problems in order to demonstrate how levels of reflective judgment are measured. Keep this example in mind as we continue to explore the concept of reflective judgment. The following tutorial will go into detail about the stages of reflective judgment, its correlations to other constructs, its relevancy to education and the opposing and competing theories. Continue to next section

12. Defining Reflective Judgment • “…development of complex reasoning in late adolescents and adults, and how the epistemological assumptions people hold are related to the way they make judgments about controversial (ill-structured) issues” (King & Kitchener, 2004, p. 5). Return to Review

13. Unique Qualities • Unique from the previous models of intellectual judgment • Focuses on the relationship between epistemology and judgment • It is possible to hold epistemological positions that go beyond relativism, as described by Perry

14. Measuring Reflective Judgment • Reflective Judgment Interview (King & Kitchener, 1994) • Reasoning about Complex Issues Test (Wood, 2002) Continue to next section

15. Seven Stages of Reflective Judgment • Click on the stage number to learn about reasoning at that stage. You can also read about how individuals at the different stages would reason about the problem of chemical additives. 7 6 5 4 3 2 1

16. Stage One • Single concrete category for knowing. • Certain knowledge is gained by direct personal observation and needs no justification. Stage Two Example

17. Stage One Example • I hear it and believe it because I figure if it’s on the news, it’s got to be true or they wouldn’t put it on. King & Kitchener, 1994, p. 48 Stage Two

18. Stage Two • Two concrete stages of knowledge. • A person can know with certainty through direct observation or indirectly through an authority. • Authorities such as scientists, teachers, and religious leaders know the truth. • Evidence is not a criterion for establishing truthfulness. Stage Three Example

19. Stage Two Example • Well, some people believe that using chemical additives is wrong and that is what they want to believe. But I would never believe that way and nobody could talk me out of the way I believe because I believe the way my parents taught me. Stage Three

20. Stage Three • Several concrete categories of knowledge are interrelated. • Knowledge is assumed to be either absolutely certain or just temporarily uncertain. • Beliefs are justified according to the word of an authority in areas of certainty and according to what “feels right” in areas of uncertainty. • Evidence can neither be evaluated nor used to reason to conclusions. • Opinions and beliefs cannot be distinguished from factual evidence. Stage Four Example

21. Stage Three Example • I know there can be bias in scientific studies on both sides. Until they take the chemical additive off the market, no one is going to do an accurate, reliable study. After they take it off, somebody unbiased is going to do an accurate study. Then they will know. King & Kitchener, 1994, p. 56 Stage Four

22. Stage Four • Knowledge is understood as a single abstraction. • Knowledge is uncertain, and knowledge claims are assumed to be idiosyncratic to the individual. • Beliefs are justified by idiosyncratic uses of evidence and opinion. • Differences in points of view exist because of people’s upbringing or because they deliberately distort information. • Evidence is used in support of a point of view along with unsubstantiated opinion. Stage Five Example

23. Stage Four Example • I’d be more inclined to believe that chemical additives were harmful if they had proof. I don’t think we’ll ever know. People will come up with different interpretations because people differ. Who are you going to ask? There is no one you can trust. King & Kitchener, 1994, p. 60 Stage Five

24. Stage Five • Two or more abstract concepts of knowledge can be related. • Knowledge is seen as contextual and subjective. • Beliefs are justified by using the rules of inquiry for the appropriate contexts. • Evidence can be evaluated qualitatively; within a perspective, some evidence is stronger or more relevant than other evidence. Stage Six Example

25. Stage Five Example • I would want to find more data about chemical additives. I would see if I could find a scientific test that would lead to a clear conclusion. If I could back it up, I still wouldn’t know because there could be error in the test. I’d have a really hard time knowing the actuality of my results. King & Kitchener, 1994, p. 64 Stage Six

26. Stage Six • Abstract concepts of knowledge can be related. • Knowledge is actively constructed by comparing evidence and opinion on different sides of an issue; solutions are evaluated by personally endorsed criteria. • Evidence on points of view can be compared and evaluated as a basis for justification. Stage Seven Example

27. Stage Six Example • You can’t say, “you are stupid or wrong” to someone. But I think if you push them far enough, they too, would have to admit their argument is based on assumptions that are empirically falsifiable. It’s a problem that all claims carry with them. There are no absolutes. King & Kitchener, 1994, p. 69 Stage Seven

28. Stage Seven • Abstract concepts of knowledge are understood as a system. • The general principle is that knowledge is the outcome of the process of reasonable inquiry for constructing a well-informed understanding. • Active open-minded thinking where beliefs can be improved, encouraging openness to alternatives and counterevidence.

29. Stage Seven Example • While there are no absolutes, one argument or position may have more evidence than another. The key to making a decision is weighing the evidence in light of a predetermined set of criteria. Continue to next section

30. Correlations to other constructs • Educational Level • Age • Critical Thinking • Verbal Ability • Personality

31. Educational Level • Prereflective thinkers (stages 1, 2, and 3) are typically individuals with high school educations. • Quasi-reflective thinkers (stages 4 & 5) are typically found in upper-class undergraduates and masters students. • Reflective thinkers (stages 6 & 7) are found in advanced graduate students. Return to Review Correlations

32. Age • Since educational level typically increases with age, traditional students follow the correlation with educational level. • Non-traditional undergraduates display statistically similar judgments as traditional undergraduates • Adults who do not have post-secondary education display reflective skills after 65. Correlations

33. Critical Thinking • During the development of the model, results from the Reflective Judgment Interview were compared with results from the Watson-Glasser Critical Thinking Appraisal. • When WGCTA scores were controlled, reflective judgment scores continued to explain some of the variability in decision making. Correlations

34. Verbal Ability • The results of the two assessments for reflective judgment, the Reflective Judgment Interview and the Reasoning about Complex Issues Test, have both been compared with individual’s verbal ability. • When verbal ability is controlled, reflective judgment scores continue to explain a portion of the variance in reflective judgment. Correlations

35. Personality • When comparing the results of the Reflective Judgment Interview to the results of the Omnibus Personality Inventory, a few aspects of personality were found to be correlated with reflective judgment: • Thinking introversion • Response bias • Altruism • Autonomy • Complexity • Theoretical orientation Continue to next section Correlations

36. Relevancy to Education • Reflective judgment can be developed in students of all ages. • The reflective judgment model can help teachers design lessons to develop reflective judgment and to help them understand how to help their colleagues develop. To see suggestions for developing reflective judgment at stages two and three, click here. To move on to learning about opposing theories, click here.

37. Developing from Stage 2 • Acknowledge that decisions are harder when there are no right or wrong answers. • Acknowledge students’ feelings of anxiety when confronted with multiple perspectives. • Provide, clear, unambiguous directions for assignments. • Point out noted authorities who hold alternative points of view. King & Kitchener, 1994, p. 250 Stage 3

38. Developing from Stage 3 • Legitimize students’ struggle with feelings of confusion. • Model good use of evidence - present justification for both sides of an argument; distinguish inapplicable evidence from relevant evidence; explain the rationale behind one’s choice of appropriate authorities. • Provide detailed assignments and clear expectations. King & Kitchener, 1994, p. 251

39. Opposing Theories and Viewpoints • Epistemological Beliefs Model • Epistemic Metacognition Model • Contextual Model • Staged Metacognition • Epistemological Reflection Model • Women’s Ways of Knowing

40. Epistemological Beliefs ModelD. Schommer-Aikins • Multiple Developing Beliefs • Potentially Independent • Possibly Asynchronous • Staged Development Opposing Theories

41. Epistemic Metacognition ModelB. K. Hofer Opposing Theories

42. Contextual ModelElby & Hammer, 2004 • Argue that content and contextual knowledge is a significant part of reasoning about epistemic issues. • However, King & Kitchener have shown in their data that reflective judgment is not dependent on context. Opposing Theories Return to Review

43. Staged MetacogitionD. Kuhn, 2004 Opposing Theories

44. Epistemological ReflectionBaxter-Magolda • Development stages of knowing: • Absolute • Traditional • Independent • Contextual • http://www.iub.edu/~teaching/top30.html Opposing Theories

45. Women’s Ways of Knowing • Developed by Belenky • Considered to be received knowledge • http://www.iub.edu/~teaching/top30.html Opposing Theories

46. Review Question 1 • True or False? • Previous studies with the reflective judgment model have shown that as educational level increases, reflective judgment increases. TF

47. You are correct! WooHoo! Return to Q 1 NEXT

48. DOH! Return to Q 1 NEXT Review

49. Review Question 2 • Who is credited with the creation of the reflective judgment model? • William Perry & Philip Wood • John Dewey & Jean Piaget • Patricia King & Karen Kitchener • Mary Budd Rowe & Mary Atwater

50. You are correct! WooHoo! Return to Q 2 NEXT