Illinois Nurse Aide Competency Evaluation Program

1. Illinois Nurse Aide Competency Evaluation Program Test Item Development

2. Purpose • Every three years the INACE project works hand in hand with subject matter experts (instructors, CNA supervisors, and CNA’s) to assure that the Illinois Nurse Assistant/Aide Competency Evaluation Examination evaluates the knowledge, skills and attitudes necessary for a CNA to be successful in the daily performance of healthcare duties. • The CNAE-A is a natural resource for development of the most valid and reliable questions to add to the test bank for competency testing purposes. • This change in methodology will also provide ample opportunity for CNA Educators to be more active participants in the INACE process.

3. Goal • Develop at least one test question for each skill identified by the current Illinois Nurse Aide Task List Survey.

4. Task Analysis A Task Analysis is an intensive examination of how people perform work activities. This is especially helpful for ‘hard’ skills like CNAs perform. A task is a series of actions or behaviors that accomplishes a goal. The competency exam is based on 200 tasks that have been identified by CNA Instructors as tasks performed by CNAs in their daily work life. A TASK LIST SURVEY is performed biennially by the INACE project.

5. Bloom’s Taxonomy

6. Application Level Questions • Students must use judgment and critical thinking which requires them to apply knowledge, skills and attitudes to answer them correctly. These characteristics can be found in the Task Analysis Sheets provided.

7. Writing Application Level Questions • Typically a situation is described in a short paragraph with a problem posed in the stem of a question.

8. Anatomy of a Multiple Choice Question • Structure • Multiple-choice items consist of a stem and a set of alternatives. • The stem is the beginning part of the item that presents a problem to be solved, a question asked of the examinee, or an incomplete statement to be completed, as well as any other relevant information. • The alternatives are the possible answers that the examinee can choose from. • The correct answer is called the key. • The incorrect answers are called distracters.

9. Example When applying a cold compress to a resident’s body part for 20 minutes, how often must the nurse assistant check the area of the resident’s skin that is in direct contact with the cold compress?

10. General Guidelines • Keep questions simple and straight-forward • Item difficulty should reflect CNA student’s competency to perform the job • Acronyms should generally be spelled out (i.e. ROM-Range of Motion, EMS-Emergency Medical Services, SP-Standard Precautions) • Use complete sentences in the item stem when possible

11. General Guidelines • Focus on critical points for performing the job • Avoid special knowledge issues that only a few people know • Avoid addressing too detailed or non-critical points • Assess one piece of knowledge or skill per item • All questions should stand on their own • Randomize the position of correct responses

12. Avoid • Contractions (i.e. can’t, don’t, won’t) • Humor • Trick test items • Lengthy discussion in stem • Irrelevant material • Using negatives in response options (NOT, EXCEPT) • Using words like: Always, None, Never, or All

13. Writing the Stem • Before writing the stem, identify the one point (knowledge, skills, attitudes) to be tested by that item. • In general, the stem should not pose more than one problem, although the solution to that problem may require more than one step. • Phrase stems as clearly as possible because confusing questions can generate wrong answers from students who do understand the material. • Construct the stem to be either an incomplete statement or a direct question.

14. Writing the Stem • State the question as briefly as possible, avoiding wordiness and undue complexity. In higher-level questions, the stem will normally be longer than in lower-level questions, but you should still be brief. • State the question in positive form because students often misread negatively phrased questions. • Include any language in the stem that you would have to repeat in each answer option.

15. Incomplete Statement Example The nurse assistant has been instructed to collect a 24-hour urine specimen from a resident in a long-term care facility. The collected specimens should be stored:

16. Direct Question Example When washing hands, what action is done for about one minute?

17. Poor Example A nurse assistant who is coming on duty at a long-term care facility feels he is too busy to attend the end-of-shift report.

18. Corrected Example A nursing assistant who is coming on duty at a long-term care facility feels he is too busy to attend the end-of-shift report. It is important that the nurse assistant attend the meeting because:

19. Poor Example – NAT Never Does This Heat applications may have all of the following beneficial effects EXCEPT:

20. Wordy Example Marge Simpson is a CNA in a long-term care facility and has been assigned to a resident who has recently arrived. The elderly resident, Mrs. Green, is complaining that she is cold in her room. The nurse assistant should:

21. Corrected Example An elderly resident, Mrs. Green, is complaining that she is cold in her room. The nurse assistant should:

22. Item Response Issues Key and Distractor General Guidelines

23. Item Response Issues • There is only one correct answer (key) for each item and it is correct 100% of the time. • The purpose of distractors is to appear as plausible solutions to the problem for those students who have not achieved the knowledge, skills, attitudes being measured by the test item. • Conversely, the distractors must not appear as possible solutions for those students who have achieved the objective. • Distracters are: • Clearly not the correct answer when the best answer is selected • Possible to someone who does not possess the skill being measured • Incorrect answers should contain “likely” mistakes an unskilled person would make • Two or more distracters should not mean the same thing

24. Writing Responses • Basic rules for writing responses are: • Students should be able to select the right response without having to sort out complexities that have nothing to do with knowing the correct answer. • They should not be able to guess the correct answer from the way the responses are written.

25. Perfect Example When a nurse assistant is providing care to a terminally ill resident who is in the acceptance stage of the 5 stages of dying, the resident is most likely to:

26. Too Obvious Example The charge nurse instructs the nursing assistant to administer a hot soak to Mr. Green’s right foot. The primary purpose of the hot soak is to:

27. Writing Responses • Write the correct answer immediately after writing the stem and make sure it is unquestionably correct. • Write the distracters to match the correct response in length, complexity, phrasing and style, avoiding the use of such words as “always,” “none”, “never,” and “all.” • Distracters must be incorrect but possible. • If a recognizable key word appears in the correct answer, it should appear in some or all of the distractors as well. • Avoid composing alternatives with only fine distinctions between the answers. • Avoid using “all of the above” or both A&B as responses since these options make it possible for students to guess the correct answer with only partial knowledge.

28. Multiple Key Example HIV/AIDS may be transmitted by:

31. CONGRATULATIONS On behalf of the SIUC Illinois Nurse Assistant/Aide Testing Project and the Illinois Department of Public Health . . Thank you You made it through training!