Faculty Development Workshop East Tennessee State University College of Nursing

1. Faculty Development WorkshopEast Tennessee State University College of Nursing Cognitive Code and Test Construction: Writing Questions that Promote Critical-Thinking Carolyn S. Merriman, RN, MS, APRN Associate Professor Fall 2017

2. At the end of this session the participants should be able to: • 1. Understand what is a cognitive code and how it can guide writing critical-thinking questions; • 2. The purpose of writing critical thinking questions in an academic setting; • 3. Describe the components of writing a critical-thinking question; • 4. Revise a test question to make it a higher level, critical thinking test item.

3. What is the purpose of tests at ETSU’s College of Nursing? Write down a few thoughts about this…

4. What is the purpose of tests in the College of Nursing? Test basic nursing knowledge? Prepare graduates for standardized tests: NCLEX-RN? Internal curriculum evaluation? Promote critical thinking in practice?

5. What is a Cognitive Code? It deals with knowledge and higher level thinking. Includes: 1.Acquisition of facts and information 2.Use of knowledge in practice 3.Higher level cognitive skills (Oermann & Gaberson, 2017)

6. Bloom’s TaxonomyOld and New

7. Bloom’s TaxonomyNewest Version • Remembering:Can the student recall or remember the information? define, duplicate, list, memorize, recall, repeat, reproduce, state • Understanding:Can the student explain ideas or concepts? classify, describe, discuss, explain, identify, locate, recognize, report, select, translate, paraphrase • Applying: Can the student use the information in a new way? choose, demonstrate, dramatize, employ, illustrate, interpret, operate, schedule, sketch, solve, use, write. • Analyzing: Can the student distinguish between the different parts? appraise, compare, contrast, criticize, differentiate, discriminate, distinguish, examine, experiment, question, test. • Evaluating: Can the student justify a stand or decision? appraise, argue, defend, judge, select, support, value, evaluate • Creating: Can the student create new product or point of view? assemble, construct, create, design, develop, formulate, write.

8. Using Cognitive Code Important to decide on the level of cognitive learning needed… Then develop outcomes and assessment methods for that level…(Oermann & Gaberson, 2017)

9. Revised Cognitive Code Classification of Cognitive Levels from NCSBN Bloom’s taxonomy for the cognitive domain is used as a basis for writing and coding items for the examination (Bloom, et al., 1956; Anderson & Krathwohl, 2001). Since the College of Nursing’s Cognitive Code Guidelines

10. College of Nursing’s Cognitive CodeGuidelines

11. Internal Curriculum Evaluation Process: • Writing critical thinking test items. • Measuring test items ability to discriminate between those who know the content and those who do not. • Designing valid exams • Measuring the reliability of exams.

12. Test construction is Multi-step: (Sax, 1997) 1. Identifying reason: Course Objectives 2. Determining best type of test items for evaluation of target audience 3. Creating a Test Blueprint

13. Test construction is Multi-step: • Identifying reason: Use course objectives and sub-objectives to guide test construction

14. Test construction is Multi-step: 2. Determining best type of test items for evaluation of target audience • What level are my students? • Beginners, Juniors, Seniors: Apply the Cognitive Code Guidelines

15. Test construction is Multi-step: 3. Test Blueprint: Content Validity • Use course objectives • Identify categories: Nursing Process, Client Needs Bloom’s Taxonomy, NCSBN, AACN, NLN, QSEN • Determine mastery questions in advance (90% or above)

16. Faculty Made Test Blue Print www.utexas.edu/provost/sacs/pdf/

17. What is so Important about a Test Blueprint? • Useful tool guiding the item writer so sufficient items developed at appropriate level. • Provides accountability for educational outcomes produces. • Can be used to inform students about the nature of the test and how they should prepare for it.

18. Skills needed to write critical thinking test items: 1. Sound clinical knowledge • Proper use of the English language, including grammar. • Ability to conduct a systematic exam analysis using item analysis data to evaluate worth of test items. • Editing ability to enhance test items’ proficiency in measuring students’ knowledge and problem solving ability.

19. Guidelines to Developing Critical-Thinking Exams How do I get started? What do I need to be a better critical- thinking test item writer? 3. What helpful hints will guide me?

20. How do I get started? A. Understand the effects of a sound curriculum design Instruction Objectives BALANCEEvaluation Outcomes

21. How do I get started? B. Gather resources for test ideas

22. How do I get started? B. Gather resources for test ideas Patient scenarios from clinical or practice Review books Test banks Case Studies

23. What do I need to be a better critical- thinking test item writer? Define Critical-Thinking

24. What do I need to be a better critical- thinking test item writer? Define Critical-Thinking “The process of analyzing and understanding how and why a certain conclusion was reached.” Morrison, Nibert, Flick, 2006

25. How can the importance of critical thinking skills be conveyed to students?

26. Making Connections with Students Early and often talk to students about the role of critical thinking tests in nursing school and their ability to use critical thinking in practice.

27. What do I need to be a better critical- thinking test item writer? Critical-Thinking Tests (Morrison, et.al. 2006) Rationale Application or above cognitive level Multilogical thinking High level of discrimination/plausible alternatives

28. Critical-Thinking Tests: 1. Rationale Rationales should include why an option is correct and why the distracters are incorrect. Makes the exam process a teaching tool.Test review: Learning experience, faculty friendly, and legally defensible.

29. Critical-Thinking Tests: 2.Application or Above Bloom’s Taxonomy: Cognitive Levels Remembering Understanding _____________________________ Applying Analyzing Evaluating Creating

30. Critical-Thinking Tests: Application or Above Application: Verbs Apply, Calculate, Classify, Develop, Examine, Generalize, Illustrate, Show, Solve, Use

31. Critical-Thinking Tests: 3. Multi-logical thinking Thinking that requires knowledge of more than one fact to logically and systematically apply concepts to a clinical problem (Morrison, et. al., 2006) Viewing problems from more than one point of view or frame of reference.

32. Critical-Thinking Tests: Multi-logical thinking What intervention is best in preventing increased intracranial pressure? (Analyzing level: Cognitive Code VS What is the definition of intracranial pressure? (Understanding level: Cognitive Code)

33. Critical-Thinking Tests: 4. High level of Discrimination Use key words that require more than regurgitation of facts. All options should be plausible, with one being more important than others.

34. Key words that demonstrate high level of discrimination Best Most Important First Highest Priority

35. The nurse knows that which assessment finding is characteristic of a client with Parkinson’s disease? A.Night blindness. B.     Pain in lower extremities. C.     Shuffling gait. D.     Incontinence.

36. The nurse knows that which assessment finding is characteristic of a client with Parkinson’s disease? A.Night blindness. B.     Pain in lower extremities. C.     Shuffling gait. D.     Incontinence. What Cognitive Code level is this question?

37. The nurse knows that which assessment finding is characteristic of a client with Parkinson’s disease? A.Night blindness. B.     Pain in lower extremities. C.     Shuffling gait. D.     Incontinence. Understanding: It is not Critical thinking. Not a high level of discrimination. Does not require multilogical thinking to answer.

38. The nurse is making a home visit to a 75-year-old male client who has had Parkinson’s disease for the past five years. Which finding has the greatest implication for this client’s care? • The client’s wife tells the nurse that the grandchildren have not been to visit for over a month. • The nurse notes that there are numerous throw rugs throughout the client’s home. • The client has a towel wrapped around his neck that the wife uses to wipe her husband’s face. • The client is sitting in an arm chair, and the nurse notes that he is gripping the arms of the chair.

39. Parkinson’s disease is characterized by a shuffling gait, and throw rugs throughout the home pose a safety hazard for this client (B). Visits from the grandchildren (A) may or may not be significant to this client, and the nurse should gather more information about the client’s feelings regarding such visits. Drooling is also characteristic of Parkinson’s disease, and a towel wrapped around the client’s neck (C) may be a good means of keeping the client dry, however, the nurse may need to counsel the family to be sure the client is dry and clean. Tremors are characteristic of Parkinson’s disease and grasping the arms of a chair (D) may help to control tremors in the hands and arms. Rationale

40. What Cognitive Code level is this question?

41. Cognitive Code Level: Applying • Rationale • Written at the application level • Requires a high level of discrimination to choose the right answer from among plausible choices • Requires multilogical thinking to answer---a shuffling gait (which puts the client at risk for falls) is a sign of Parkinson's’ disease, and must know what nursing care is most important for this client—promotion of safety. Throw rugs pose a safety hazard.

42. Testing Stems that Promote Critical Thinking • Which intervention is most important? • Which intervention, plan, assessment data is/are most critical to developing a plan of care? • Which intervention should be done first? • What action should the nurse take first? • Which intervention, plan, nursing action has the highest priority? • What response is best?

43. What do I need to be a better critical- thinking test item writer? Understand Terms and Use Specific Steps as a Guide Chapter 5 in Oermann and Gaberson Evaluating and Testing in Nursing Education

44. General Rules for Writing Test Items (Oermann & Gaberson, 2017) Make sure every item is measuring something important! Every item should have a correct answer. Use simple, clear, concise, precise, grammatically correct language. Avoid jargon, slang, or unnecessary abbreviations.

45. General Rules for Writing Test Items-con’t. (Oermann & Gaberson, 2017) Use positive wording. Don’t give irrelevant clues to the correct answer. Items should not depend on another. Eliminate extraneous information. Arrange for a critique and prepare more items than the blueprint specifies.

46. Helpful hints in test-writing? No proper names Eliminate ‘multiple’ multiples Non-sexist language Short and Sweet: Cross out “of the following” Delete scenarios Items cry out for Independence !

47. More helpful hints … Ask a single, clearly defined question Viable options Stem-Option Relationships No Freebies Person-first language Clear Option Ranges Logical order Vary correct answer

48. And Even More Helpful Hints… Eliminate “all of the above” and “none of the above” Rewrite any “all except” questions Ensure that alternatives do not overlap Style drug names uniformly Be aware of verbs and other writing issues

49. And the MOST IMPORTANT Helpful Hint… Abide by the Written Testing Policy and Cognitive Code Guidelines for ETSU College of Nursing!

50. Use of Statistics Important to use statistics to evaluate individual items and overall test reliability and validity.