Depth of Knowledge and Cognitive Demand

1. QualityCore Professional Development Depth of Knowledge and Cognitive Demand Day 1, 2–1

2. Essential Question • How can we know that our expectations of students are at a level of rigor that prepares them for the End-of-Course Assessment and for success in college or the workplace? Day 1, 2–2

3. Learning Objectives • By the conclusion of this segment, you will be able to: • Explain the differences between the cognitive levels of the depth-of-knowledge (DOK) framework • Analyze sample test items for depth of knowledge and compare those items across levels to differentiate the level of thinking required • Use the QualityCore Test Blueprint to review the percent of items found at each depth-of-knowledge level and discuss the classroom assessment implications of that distribution

4. What is Depth of Knowledge? • A framework for evaluating curriculum, standards, and assessments so they can be studied for alignment • Focuses on content and cognitive demand of test items, instructional strategies, and performance standards

5. Alignment • Items need to measure the content described in the ACT Course Standards, • And they need to measure them at the performance levels specified, • So that the intended curriculum (e.g., as specified in ACT Course Standards) aligns with enacted curriculum and assessed curriculum (e.g., end-of-course tests).

6. In English • An item measuring main idea may be at a low level of cognitive demand: • What happened in this story? • Or it can be at a high level of cognitive demand: • What is the theme of this passage?

7. The Depth-of-Knowledge Framework • First introduced in response to No Child Left Behind, where states were required to show that their standards, curriculum, and assessments were aligned with equivalent breadth and depth • Initially developed in collaboration with Council of Chief State School Officers (CCSSO) • Now widely used across the country

8. QualityCore and DOK 4 Levels • Level 1 measures Recall at a literal level. • Level 2 measures a Skill or Concept at an interpretive level. • Level 3 measures Strategic Thinking at an evaluative level. • Level 4 measures Extended Reasoning. • Webb (2002)

9. DOK Level 1Recall: Literal • Requires recall of information such as a fact, term, definition, or a simple procedure • Requires students to demonstrate a rote response or perform a simple procedure

10. DOK Level 1: English • Find facts and details • Identify figurative language • Label simple parts of speech • Recall elements and details of passage structure (e.g., plot, sequence, setting, character) • Use appropriate grammar, punctuation, capitalization, and spelling

11. Sample Questions Level 1 • Who is Hamlet’s mother? • What is the setting for the beginning of the play Hamlet? • What happens right after Laertes is fatally wounded?

12. Sample Level 1 ACT Course Standards • Persuasive Language and Logic • Locate important details and facts that support ideas, arguments, or inferences in increasingly challenging text . . . • Writing • Correctly spell commonly misspelled/confused words

13. Do You See Any Others? • What is another ACT Course Standard that is Level 1? • What kinds of questions could you ask in class discussion or on an exam to reflect the Level 1 standard you found?

14. English 9 Test Blueprint • 15–25% of the questions on the English 9 End-of-Course Assessment are Level 1. • http://qc.vantage.com/qualitycore/do/log

15. DOK Level 2 Skill/Concept: Interpretive • Requires mental processing beyond recall or reproducing an answer • Students must make some decisions about how to approach a problem • Cognitive demands are more complex than in Level 1

16. DOK Level 2: English • Use context clues to determine the meaning of a word • Predict a logical outcome based on information in the story • Summarize major events, ideas, and pertinent details in a text • Use organizational strategies to structure written work

17. Sample Questions Level 2 • What does the word vagabond mean in sentence 3 of the story? • What is likely to happen next, given the events described in the story? • What is the lesson of this fable? • What does the narrator reveal about her mother in the first paragraph? • Write a paragraph summarizing paragraphs 1–3 in the text.

18. Sample Level 2 ACT Course Standards • Apply knowledge of Greek, Latin, and Anglo-Saxon affixes, inflections, and roots to understand unfamiliar words . . . • Summarize and paraphrase information . . . identifying key ideas . . . • Distinguish between fact and opinion

19. Do You See Any Others? • What is another ACT Course Standard that is Level 2? • What kinds of questions could you ask to reflect the standard?

20. Test Blueprint for English 9 • 50–70% of the questions on the English 9 End-of-Course Assessment are Level 2.

21. DOK Level 3Strategic Thinking: Evaluative • Requires planning, thinking, explaining, justifying, using evidence, conjecturing, and postulating • Cognitive demands are complex and abstract

22. DOK Level 3: English • Describe author’s purpose and how it affects the interpretation of text • Analyze and describe the characteristics of a passage • Infer across an entire passage • Identify abstract themes • Analyze and synthesize information • Use voice appropriate to the purpose and audience • Edit writing to produce a logical progression of ideas

23. Sample Questions Level 3 • How does Edgar Allen Poe create a feeling of suspense in his poem “The Raven”? How does this differ from the way Simenon creates suspense in his story about Inspector Maigret? • What makes John a true tragic figure in this passage? • What does the author do to make you believe Mary’s story? • Rewrite this story from the point of view of the brother.

24. Sample Level 3 ACT Course Standards • Explain the effects of the author’s life upon his or her work (e.g., Alexander Solzhenitsyn’s experience in the gulag as reflected in his novel One Day in the Life of Ivan Denisovich) • Critique the treatment and scope of ideas from multiple sources on the same topic . . .

25. Do You See Any Others? • What is another ACT Course Standard that is Level 3? • What kinds of questions could you ask to reflect the standard?

26. Test Blueprint for English 9 • 15–25% of the questions on the English 9 End-of-Course Assessment are Level 3.

27. DOK Level 4 Extended Thinking • Requires complex reasoning, planning, developing, and thinking • Usually requires extended time • Requires higher-order thinking

28. Level 4 Reading • Students take information from at least one passage and are asked to apply this information to a new task. They may also be asked to develop hypotheses and perform complex analyses of the connections among texts. Some examples that represent but do not constitute all of Level 4 performance are: • Analyze and synthesize information from multiple sources. • Examine and explain alternative perspectives across a variety of sources. • Describe and illustrate how common themes are found across texts from different cultures.

29. Level 4 Writing • The standard at this level is a multiparagraph composition that demonstrates synthesis and analysis of complex ideas or themes. There is evidence of a deep awareness of purpose and audience. For example, informational papers include hypotheses and supporting evidence. Students are expected to create compositions that demonstrate a distinct voice and that stimulate the reader or listener to consider new perspectives on the addressed ideas and themes. An example that represents but does not constitute all of Level 4 performance is: • Write an analysis of two selections, identifying the common theme and generating a purpose that is appropriate for both.

30. Level 4 Assignments • What kinds of assignments and standards are Level 4?

31. DOK and Assessment • Find the sample test items in your Participant Handbook. • Fold over the page with the key for now. • Take the test and indicate the depth-of-knowledge level of each question. • Check your answers against the key.

32. How Can You Use This Framework? • What are three ways you can use the depth-of-knowledge framework to increase the rigor of your instruction, your assignments, and your assessments?