Belal M. Hijji, RN. PhD, Arwa Owies, RN. PhD

1. An Examination of the Routine Blood Transfusion Knowledge Questionnaire Through Item Analysis Belal M. Hijji, RN. PhD, Arwa Owies, RN. PhD Faculty of Nursing, Philadelphia University, Jordan Presented in The second International Conference for Allied Health Care Professionals, Physicians, Physiotherapists, and Technicians, April 7-8 2010, Alexandria, Egypt

2. Contents • Background • Aim and objectives • Methods • Results • Summary of results and discussion • Conclusion • Acknowledgment • References

3. Background The second International Conference for Allied Health Care Professionals

4. The Routine Blood Transfusion Knowledge Questionnaire (RBTKQ) was developed (Hijji, 2007) as a measure to investigate nurses’ level of knowledge and practice of blood transfusion The RBTKQ includes has true-false items (three), multiple-choice (MC) (20), and multiple-response (MR) (10) For the purpose of this analysis, five sections of the RBTKQ that are relevant for inclusion: (A) is about issues related to patient preparation before blood bag collection, (B) is concerned with blood bag collection section, (C) is related to pre-transfusion initiation nursing activities, (D) addresses post-transfusion initiation activities, and (E) is related to complications of blood transfusion The second International Conference for Allied Health Care Professionals

5. Since its inception, the RBTKQ has not undergone any form of item analysis to evaluate its psychometric properties An examination of empirical item response data maybe appropriate in detecting item flaws (Crocker and Algina, 1986), and provides useful information about test-item quality (Oermann & Gaberson, 1998). The second International Conference for Allied Health Care Professionals

6. AIM The second International Conference for Allied Health Care Professionals

7. Principal Aim The main aim of this study was to determine the overall quality of the items and the test • Objectives: • A. Identify the items’ difficulty “p” and discrimination “D” levels • B. Investigating the performance of distractors • C. Find out whether the items discriminate between nurses whose scores place them in the top 30% of all nurses and those whose scores place them in the lower 30% of all nurses. This figure provides a stable index of differences between high and low ability groups (Crocker & Algina, 1986) The second International Conference for Allied Health Care Professionals

8. Methods The second International Conference for Allied Health Care Professionals

9. Data Collection Responses to the item were taken from the RBTKQ that was filled out by a random sample of 305 nurses (95.3%). Analysis was performed for p, D, and for the distractors. Because the p valueis unavailable with MR items (http://www.cat.ilstu.edu/services/opscan/opFAQStatQ.php), each one of these was broken down to a number of sub-items equal to the number of its options. Based on a nurse’s selection of an option, a label of “yes” or “no”, or “true” or “false” was assigned. The outcome of this procedure culminated in a total of 89 items for the whole questionnaire. The second International Conference for Allied Health Care Professionals

10. DataAnalysis scoring For most items, one point was awarded for the correct answer and 0 for the incorrect one. The maximum possible score was 57 (100%) points Scores were arranged in descending order from the highest to the lowest The number of nurses’ subpopulations was 91 for each group. The upper group’s scores ranged from 32 (56%) to 40 (70%); lower group from 8 (14%) to 27 (48%). The mean knowledge score for the sample is 29.6 (SD = 4.14). A passing score of 34 (60%) (one standard deviation above the mean) was set up. The second International Conference for Allied Health Care Professionals

11. For each item, data analysis details: • the p value (whole sample). The p value is set at 0.5 to 0.85 (Lin et al., 1999). • the p value of each option selected by both groups • the difference in proportions “D” between nurses’ subgroups selecting each option. A D value of 0.20 or above is appropriate (Crocker and Algina, 1986; Brown, 1983) for the correct option. • the number of omits The second International Conference for Allied Health Care Professionals

12. Results The second International Conference for Allied Health Care Professionals

13. Due to time restrictions, there will be partial presentation of the results The second International Conference for Allied Health Care Professionals

14. Table 1. Computations of item analysis indices for sections A and C The second International Conference for Allied Health Care Professionals [1] Difficulty index for the top and lower 30% of nurses [2] This is the difference between the proportions of nurses’ subpopulation. The h stands for high scorers; l for low scorers

15. D Table 2. Computations of item analysis indices for section D The second International Conference for Allied Health Care Professionals

16. Table 4. Computations of item analysis for sections A, B, and C The second International Conference for Allied Health Care Professionals [1] Omits were considered incorrect responses (Oermann & Gaberson, 1998). Here omits were applied to option D.

17. Table 4. (Continued…….) The second International Conference for Allied Health Care Professionals

18. Table 5. Computations of item analysis indices for sections D The second International Conference for Allied Health Care Professionals [1]Item not applicable to 22 nurses [2]Item not applicable to 76 nurses

19. Table 5. Computations of item analysis indices for sections E The second International Conference for Allied Health Care Professionals

20. Summary of Results and Discussion • Item difficulty • Eight items (27%) had p value of .50 to .85 (neither too difficult nor too easy) • Two items (6%) had p value of >.85 (easy) • 20 items (67%) had p value of < .50 (difficult). This could be, most probably, because nurses did not master the content • All distractors were selected by nurses and only a few of them, sometimes, omitted responding to some items. • Sixteen items (53%) had a D value of ≥ .20. This indicates that the items were especially easier for the top group and more difficult to the lower group. • The last two results are good indicators of the quality of item (Osterland, 1998, Oermann & Gaberson, 1998). Functional distractors decrease the chances that low ability nurses select the correct response by guessing which would introduce measurement error (Crocker & Algina, 1986) The second International Conference for Allied Health Care Professionals

21. Fourteen items (47%) had a D value of < .20. • The test was not designed to maximise variance among groups. In essence, D relies on a fundamental assumption, which is that examinees who exhibit mastery on the subject matter are presumed to be more likely to respond correctly to an item than low ability examinees (Osterland, 1998). In this study in fact, none of the nurses per se mastered the content of interest. • The differences between both groups for the distractor were, mostly, negative values. This indicates that fewer upper group nurses selected the distractor than did the lower group nurses. • This is another positive feature of items (Osterland, 1998) • However, the differences for some distractors in eight items were scant (≥ -.04). For two items, the differences were positive indicating that higher ability nurses selected distractors more often than low ability nurses. • In this case, Osterland (1998) recommends reviewing the options for possible improvement. The second International Conference for Allied Health Care Professionals

22. For a number of items (n = 13, 43%), nurses responses were widely scattered among the responses’ alternatives, and/ or the correct responses attracted less attention than the distractor(s). • This result may indicate that nurses could not understand the items or that the items covered unfamiliar content. Therefore, these items may need to be reviewed (Crocker & Algina, 1986). The second International Conference for Allied Health Care Professionals

23. Conclusion • This study has highlighted the need for: • Jordanian nurses to master the content related knowledge and practice of blood transfusion. • Examining the questionnaire items for possible improvement • However, no educator should retain, revise, or eliminate an item on the basis on item statistics alone (Oermann & Gaberson, 1998). The second International Conference for Allied Health Care Professionals

24. Acknowledgements The second International Conference for Allied Health Care Professionals

25. We are immensely grateful to the Ministry of Health, all nursing staff, and hospitals’ management. We express our gratitude to Philadelphia University Deanship of Scientific Research and Postgraduate Studies for funding the study. The second International Conference for Allied Health Care Professionals

26. References The second International Conference for Allied Health Care Professionals

27. Brown, F. (1983). Principles of educational and psychological testing (3rd ed.). New York: Holt, Rinehart and Winston. Crocker, L. and Algina, J. (1986). Introduction to classical & modern test theory. Belmont CA: Wadsworth. Hijji, B. (2007). Knowledge and practice of blood transfusion: A survey of nurses in Abu Dhabi, United Arab Emirates. Unpublished PhD thesis, the University of Ulster. Lin, L.; Tseng, H. and Wu, S. (1999). Item analysis of the registered nurse licensure exam taken by nurse candidates from vocational nursing high schools in Taiwan. Proc. Natl. Sci. counc. ROC (D). 9(1): 24-31. Oermann, M. and Gaberson, K. (1998). Evaluation and testing in nursing education. New York: Springer. Osterland, S. (1998). Constructing Test Items: Multiple-Choice, Constructed-Response, Perfromance, and Other Formats. 2nd edition. London: Kluwer. The second International Conference for Allied Health Care Professionals

28. Thank You Principal Investigator bhijji@philadelphia.edu.jo bhijji1@gmail.com The second International Conference for Allied Health Care Professionals