An Application of Audience Analysis in Web-based Health Information System Design

1. An Application of Audience Analysisin Web-based Health Information System Design Jennifer Turns, Ph.D. Acting Assistant Professor, Technical Communication Faculty Affiliate Program for Educational Transformation through Technology (PETTT) Center for Engineering Learning and Teaching (CELT) Acknowledgements: This work has been supported by the Program for Educational Transformation through Technology (PETTT). Many people have contributed to this work including Scott Macklin, Tracey Wagner, Aaron Louie, Brett Shelton, Kristina Liu, Alice Tanada, Jake Burghardt, Julianne Fondiller, Regina Yap, Ralph Warren, and Dr. Frederick Matsen.

2. Background- Approach – Results - Implications Today’s Goal • Provide a comprehensive account of the audience analysis for the “Arthritis Source” • Background • Approach • Results • Implications

3. Background- Approach – Results - Implications Arthritis Source • Developed in 1995 by Dr. Frederick Matsen • Focus on arthritis • Authorized information • User-centered information • Research test bed

4. Background- Approach – Results - Implications History of the Arthritis Source • Dynamic Content • Template-based content Embedded evaluations • Search EngineOptimization Question-based Navigation Creation Design ‘01 ‘00 ‘02 1995 Audience Analysis • Audience Analysis • Designing begins • Interview Analysis 1, n=20 • Survey analysis 2, n~400 • Survey Analysis 1, n~200

5. Background- Approach – Results - Implications Talk 1 – First Steps • Dynamic Content • Template-based content Embedded evaluations Question-based Navigation Creation ‘01 ‘00 ‘02 1995 • Audience Analysis • Designing begins • Interview Analysis 1, n=20 • Survey Analysis 1, n~200 • Survey analysis 2, n~400

6. Background- Approach – Results - Implications Talk 2 – Listening to Learners • Dynamic Content • Template-based content Embedded evaluations Question-based Navigation Creation ‘01 ‘00 ‘02 1995 • Audience Analysis • Designing begins • Interview Analysis 1, n=20 • Survey Analysis 1, n~200 • Survey analysis 2, n~400

7. Background- Approach – Results - Implications Talk 3 –Designing for Learners at Large • Dynamic Content • Template-based content Embedded evaluations Question-based Navigation Creation ‘01 ‘00 ‘02 1995 • Audience Analysis • Designing begins • Interview Analysis 1, n=20 • Survey analysis 2, n~400 • Survey Analysis 1, n~200

8. Background- Approach - Results - Implications Talk 4 – Audience Analysis • Dynamic Content • Template-based content Embedded evaluations Question-based Navigation Creation ‘01 ‘00 ‘02 1995 • Audience Analysis • Designing begins • Interview Analysis 1, n=20 • Survey analysis 2, n~400 • Survey Analysis 1, n~200

9. Background- Approach – Results - Implications Web-based Health Information • Significant use already exists • Opportunities • Challenges… • Site quality • Information quality • Findability of information • Evaluation Methods

10. Background- Approach – Results - Implications Focusing on Audience Analysis • Inform our own design and evaluation • Contribute to broader discussion • Design • Evaluations

11. What categories? Inform design Theoretical foundation Speak to team What methods? Empirical tradition Balance breadth and depth Acknowledge distributed users Background- Approach – Results - Implications Audience Analysis

12. Categories Roles Goals Knowledge Circumstances of Use Culture Ergonomics Theoretical Perspectives Technical Communication Reader Response Theory Cognitive Science Constuctivism Distributed Intelligence Situated cognition Socio-Cultural Theory Human Factors Background- Approach – Results - Implications Multidisciplinary Influences

13. Background- Approach – Results - Implications Defining Categories • Role – Dominant persona of users (job, affiliation) • Goals – Reason for the interaction • Knowledge – The extent and nature of prior relevant knowledge • Circumstances of Use – Setting, resources, strategy, timing • Culture – Group level beliefs, language, preferences • Ergonomics – Relevant perceptual & motor abilities, skills

14. Background- Approach – Results - Implications Method – Online Survey • Questions: Adaptive, ~25 questions • Participants: • Duration: 9/1/2000 – 7/2/2001 (10 months) • 472 respondents / 710 starts • Analyses1 – • Descriptive Statistics • Content Analysis • Qualitative Coding • Statistical Analysis • 1Acknowledgments: Tracey Wagner, Kristina Liu, Alice Tanada, Kristen Schuyler

15. About Visit Could you tell me about your visit or visits to the Arthritis Source? Could you tell me what you were trying to do when you visited the Arthritis Source? Did you benefit from your visit or visits to the Arthritis Source? What kind of information do you think other arthritis patients should know? About Knowledge of Condition Could you tell me what you think arthritis is in general? Could you tell me how RA/OA affects the body? Do you know what contributes to getting RA/OA? Do you know how RA/OA is diagnosed? If no, Do you remember what your doctor told you about your diagnosis? What is most difficult to understand about RA/OA? Background- Approach – Results - Implications Method - Phone Interview

16. Background- Approach – Results - Implications Phone Interview • Participants – 20 users (10 OA, 10 RA) • Analyses1 • Conceptions/misconceptions • Overarching Goals • Specific Information Needs • 1Acknowledgments: Tracey Wagner, Kristina Liu

17. Background- Approach – Results - Implications Mapping Data to Categories

18. Background- Approach – Results - Implications Results - Overview • Role • Goals • Knowledge • Circumstances of Use • Culture • Ergonomic

19. Users with many roles Cross-over roles “Person with arthritis” is too simplistic… Person with pain Person with condition that is not arthritis Person who is exploring whether they have arthritis Background- Approach – Results - Implications Roles n=462/472

20. Educational spread Misconceptions Low bone density is associated with Osteoarthritis Bone spurs cause arthritic pain Joint injections are helpful There is little you can do (and 3 others) Background- Approach – Results - Implications Knowledge n=399/472

21. “Why visiting” (n=433) Condition information (54%) Ways to minimize pain (7%) Seek clarification or a second opinion on diagnoses (5%) Desire to better control arthritis condition (2.5%) Preparing for surgery (5%) … Other Insights Social/emotional support (online survey analysis) Great variety of specific information needs(phone interview analysis) Ghost information needs (phone interview analysis) Background- Approach – Results - Implications Goals

22. International Use Use across regions Background- Approach – Results - Implications Culture Other 3% International 19% Rural 26% Urban 29% North America 81% Suburban 42% n=403/472 n=408/472

23. Coming from… One of many sources Background- Approach – Results - Implications Circumstances of Use n=433/472 Percentage n=372/472

24. Vision issues Motor issues Background- Approach – Results - Implications Ergonomics 18 different types of arthritis 21% over 60 Percentage Age n=408/472

25. Background- Approach – Results - Implications Future Audience Research • Still Analyzing • Persona / Profiles • Gain additional insight • Space of information needs • Circumstances of use • Cultural Issues • Implications? • #1 - Aggregating questions from varied of information sources. • #2 Studying use over time through user online journaling

26. Background- Approach – Results - Implications Design Implications • Accessibility • Adapting to knowledge levels • Supporting goals (finding/using content) • Addressing misconceptions • Support evolving content

27. Background- Approach – Results - Implications Evaluation Implications • Site quality • Vision • Cultural issues • Comprehensiveness • Information for user-centered perspective • Judging Site vs. Judging Impact • Multiple candidate types of “success”

28. Background- Approach – Results - Implications Concluding Remarks • Audience analysis of web-based health information • Arthritis Source Directions • Dynamically generated, template-based content • Question-driven navigation • Audience Analysis • Contributes to conversation on Internet health information • Case study for TC students/professionals • Challenges in audience analysis

29. Areas of Interest/Activity • Supporting Communication • Developing a website to document teaching challenges (NSF Proposal) • Studying design team use of communication tools (Ph.D. Work) • Studying the Processes of Communication • Empirical analysis of team communication behaviors (Opportunity) • The Interplay of Communication and Learning • Reflective Learner, Writing to learn, Scaffolding writing (Ph.D. Work) • Professional Practice in Technical Communication • Assessing how designers scope design problems (Extension) • Characterizing student ability to behave as reflective practitioners (Extension)

30. Web-based Health Information • Site quality • Owner credentials, update dates (Hoffman, 2000) • Quality of information • Comprehensiveness (e.g., Chen, 2000) • Accuracy (e.g., Chen, 2000) • Providing references (e.g., Hellawell, 2000) • Findability of information • Time required (e.g., Gotwald, 2000) • Getting to real questions (e.g., Lechner, 1996) • Need for evaluation methods • (e.g., Wu, 2000, Delamsthe, 2000, Charatan, 1999)