Development of a Self-Assessment Method for Patients to Evaluate Internet-based Health Information

1. Development of a Self-Assessment Method for Patients to Evaluate Internet-based Health Information

2. Josette Jones, RNc, Licentiate MIS, Licentiate Nursing, Doctoral Student School of Nursing University of Wisconsin - Madison Academic Advisor: Patricia Flatley Brennan, RN, Ph.D., FAAN, FACMI Josette Jones

3. Purpose of the study • Development of a Self-Assessment Method for patients to evaluate quality and relevance of health care oriented websites • Validation of the Stability and Consistency the Self-Assessment Method for website evaluation • the use of the tool • the conceptualization Josette Jones

4. Background and Significance • The Internet hosts a large number of accurate health-oriented websites with endless opportunities to inform, teach and connect patients. • Health care consumers need a way to judge the quality and relevance of the information provided on the Internet. Josette Jones

5. Background and Significance • Quality as perceived by the “general” users relates not only to accuracy of content but rather to presentation, perceived trust, clear credentials, and other markers that tend to give information 'weight'. • Patients will accept or use a WWW page when the information is perceived as relevant to his/her situation. Josette Jones

6. Helping patients determine the quality and relevance of health information found on the Internet is a key responsibility for clinicians who want to use network technologies to promote the health of patients and provide them with clinical service. Josette Jones

7. Development of a Self-Assessment Method for Evaluation of Websites

8. Study Design • Summary of criteria/guidelines for evaluating WebPages, published in journal articles and on-line publications • Categorization of the criteria through lexical and contextual analysis Josette Jones

9. Study Design • Comparison to the criteria to what the “general” user of the WWW perceives as quality and/or relevance. • Conceptualization of four criteria considered as indicative of quality and relevance as perceived by the “general” user of information on the Internet. Josette Jones

10. “Quality and Relevance” Criteria Josette Jones

11. The proposed Self-Assessment Method for Evaluating Websitesconsists of nine unrelated questions Josette Jones

12. Prompts to Patients • Match of interest • Clear and sufficient information • Novel info • Compelling case • Support from graphics and pictures • Ease of reading and understanding • Up-to-date information • Familiarity with publisher • Facilitate behavior change ? Josette Jones

13. Josette Jones

14. Josette Jones

15. Validation of the Stability and Consistency the Self-Assessment Method for Website Evaluation

16. Sample • 16 students from a small Midwestern university participated in the testing of the method • Female, Caucasian • Average age: 22 year • Medical/clinical knowledge: none to basic • Internet skills varying from none to good Josette Jones

17. Procedure • Participants were asked to list 3 topics related to health and to search the web for information on these topics • The test was repeated after 3 weeks Josette Jones

18. Testing the Use of the Self-Assessment Method • Identified a web site • Answered each item on the questionnaire with “yes” or “no”. Josette Jones

20. Validating the Conceptualized Criteria Each of the four criteria were validated for “quality and relevance” using a 4-point Likert scale. 1=not relevant 2=somewhat relevant 3=quite relevant 4=very relevant Josette Jones

21. “Quality and Relevance” Criteria Josette Jones

22. Josette Jones

23. Matching Perceptions of “Quality and Relevance” to Questionnaire Each questionnaire item was validated as useful to determine the “quality and relevance” using a 4-point Likert scale. 1=not relevant 2=somewhat relevant 3=quite relevant 4=very relevant Josette Jones

24. Perception of Quality and Relevance: Ranked by Modal Score - Frequency

25. Discussion The study suggests that: • Consumers evaluate quality and information against a simple set of indirect criteria • Design (Q5) and facilitation of behavior change (Q9) are rated less helpful in evaluating the quality and relevance of health related websites than the other questions. Josette Jones

26. Limitations of the study • Limited size of the convenience sample • Demographics of participants • No data available from patient sample. • No elicitation of unrecognized needs • No measurement of the number of steps it may take to locate the information Josette Jones

27. Acknowledgments • Dr. Patricia Flatley Brennan • The HeartCare team in Madison and Cleveland • University of Wisconsin, Eau Claire • Dr. M. Oleson - and -Gyda Bjornsdottir RN, MSN Josette Jones

28. Josette Jones, RNC, Licentiate Nursing, Licentiate MIS wouterjf@uwec.edu Academic Advisor: Patricia Flatley Brennan, RN, Ph.D., FAAN, FACMI • This study was supported by the HeartCare Project • funded through NIH grant R01-LM06249 Josette Jones