Listening to the Learners: A Case Study in Health Information Website Design

1. Listening to the Learners:A Case Study in Health Information Website Design Jennifer Turns Acting Assistant Professor Technical Communication Tracey Wagner Educational Technology Analyst Program for Educational Transformation Through Technology Acknowledgements: This work has been supported by the Program for Educational Transformation Through Technology (PETTT). Many people have contributed to this work including Scott Macklin, Aaron Louie, Kristen Shuyler, Brett Shelton, Kristina Liu, Alice Tanada, Jake Burghardt, Julianne Fondiller, Regina Yap, Ralph Warren, and Dr. Frederick Matsen.

2. Background- Case Study - Results – Implications - Conclusion Today’s Goal • Demonstrate how a characterization of users as learners can inform the design of an informational website. • Background • Case Study • Results • Implications • Conclusions

3. Examples: legal information cooking information home repairs and renovations Issues Type of content Organization of content Style of content Access to content Background- Case Study - Results – Implications - Conclusion Informational Website Design

4. Background- Case Study - Results – Implications - Conclusion Audience Analysis • “Know thy user” • Common dimensions include • Role – Dominant persona of users (job, affiliation) • Goals – Reason for the interaction • Knowledge – The extent and nature of prior relevant knowledge • Circumstances of Use – Setting, resources, strategy, timing • Culture – Group level beliefs, language, preferences • Ergonomics – Relevant perceptual & motor abilities, skills

5. Background- Case Study - Results – Implications - Conclusion When users are learners • Learner centered design (Soloway et al, 1994) • Key Challenges • Diversity: learners are diverse • Engagement: learners need support for engaging in learning • Growth: learning changes the learner

6. Background- Case Study - Results – Implications - Conclusion Design Relevance? • What is the nature of diversity, engagement, and growth in a specific domain and with a specific user populations? • How can we characterize users along these dimensions? • How do we design informational web sites to address these dimensions?

7. Background- Case Study - Results – Implications - Conclusion Arthritis Source • Origin: Developed in 1995 by Dr. Rick Matsen • Goal: Help learners-at-large satisfy their knowledge needs with authorized information. • Challenges: Expand scope and ensure effectiveness of Arthritis Source

8. Background- Case Study - Results – Implications - Conclusion Audience Analysis Methods • Challenge • Getting to actual users • Our methods include • Online Survey • Phone Interviews • Logfile Analysis

9. Users with many roles Many goals / questions Condition information (54%) Ways to minimize pain (7%) Seek clarification or a second opinion on diagnoses (5%) Preparing for surgery (5%) Desire to better control arthritis condition (2.5%) … Background- Case Study - Results – Implications - Conclusion Diversity n=462/472

10. Educational spread Misconceptions Bone spurs cause arthritic pain Joint injections are helpful Low bone density is associated with Osteoarthritis There is little you can do (and 3 others) Background- Case Study - Results – Implications - Conclusion Diversity n=399/472

11. International Use - 128 countriesin 2½ months Background- Case Study - Results – Implications - Conclusion Diversity

12. Background- Case Study - Results – Implications - Conclusion Engagement • Motivated learners: 48% from search engine • Personally engaged: 65% have arthritis • Heavily one-time visitors: 80% first visit • Moderate time on page: avg. visit ~ 7 mins. • Common misconceptions and questions suggest a need for longer engagement periods and deeper interaction with information

13. Background- Case Study - Results – Implications - Conclusion Growth • Windows into growth • Many statements of information needs • Users at different points in patient careers • Opportunities for growth • Efforts to articulate questions • Ongoing interest in exploring Growth

14. Background- Case Study - Results – Implications - Conclusion Design Challenges • Diversity • Support users with variety of questions/goals • Support users in different contexts • Support users with different knowledge • Engagement • Help users find relevant information quickly • Find ways to engage users in knowledge that might be useful to them (e.g., correcting misconceptions) • Growth • Support user concerns and knowledge over time

15. Background- Case Study - Results – Implications - Conclusion Design Direction – Content • Content via Templates • Common organization • Linked to questions • Ensures coverage • Links to • Diversity • Engagement • Growth

16. Background- Case Study - Results – Implications - Conclusion Design Direction – Search • Question-based Search • Start with question • Choose from “similar” questions • Enter content at point where question is answered. • Links to • Diversity • Engagement • Growth

17. Background- Case Study - Results – Implications - Conclusion What we are working on • Building on these strategies • Expanding content • Improving search • Evaluation Studies – Is it working? • User evaluation of content • “Longitudinal” Study - Impact of extended site interaction on knowledge and health outcomes

18. Background- Case Study - Results – Implications - Conclusion Moving in the Right Direction:Evidence of Growth? • Sequences of searches showing growth? • 8:26 am "my husbands leg bone below knee is splinering and knee has moved over it,is there a surgery to save his leg" • At 9:22 am "aproximately 6inches below knee cap the long bone is spintering is there a surgery that can cover this and help support the leg bone“ • 10:01am "can a femur tibia splintering of the bone be surgicially be repaired ?“

19. Background- Case Study - Results – Implications - Conclusion Concluding Remarks • Learner-Centered Design - Diversity, engagement, growth • Learner-Centered Design and Audience Analysis • One strategy for audience analysis • Complements other audience analysis approaches (e.g., roles, tasks) • Learner-Centered Design and Our Case Study • Variety of data can be organized via this framework • The dimensions are powerful organizer • Diversity is easy to capture • Engagement & growth are challenging to capture for web populations. • Using this framework informed our design

21. Background- Case Study - Results – Implications - Conclusion Method – Logfile Analysis • Analyze record of users interactions with site • Available data include • Entry/exit pages, pages viewed, time on page, country of origin, etc.

22. About Visit Could you tell me about your visit or visits to the Arthritis Source? Could you tell me what you were trying to do when you visited the Arthritis Source? Did you benefit from your visit or visits to the Arthritis Source? What kind of information do you think other arthritis patients should know? About Knowledge of Condition Could you tell me what you think arthritis is in general? Could you tell me how RA/OA affects the body? Do you know what contributes to getting RA/OA? Do you know how RA/OA is diagnosed? If no, Do you remember what your doctor told you about your diagnosis? What is most difficult to understand about RA/OA? Background- Approach – Results - Implications Method - Phone Interview

23. Background- Case Study - Results – Implications - Conclusion Method – Online Survey • Questions: Adaptive, ~25 questions • Participants • Duration: 9/1/2000 – 7/2/2001 (10 months) • 472 respondents / 710 starts • Analyses • Descriptive Statistics • Content Analysis • Qualitative Coding • Statistical Analysis

24. Background- Case Study - Results – Implications - Conclusion What we are working on • Supporting goals (finding/using content) • Support evolving content • Evaluating effects • Learning • Health outcomes • Use patterns • Various changes over time—growth

25. Background- Case Study - Results – Implications - Conclusion Method – Online Survey • Questions: Adaptive, ~25 questions • Demographics • Role • Goal • Successful during today’s visit • Participants • Duration: 9/1/2000 – 7/2/2001 (10 months) • 472 respondents / 710 starts

26. Background- Case Study - Results – Implications - Conclusion Method - Phone Interview • Questions: • About your visit (5): Could you tell me what you were trying to do when you visited the Arthritis Source? • Knowledge of condition (5): Could you tell me how RA/OA affects the body? • Participants – 20 users (10 OA, 10 RA) • Analyses • Conceptions/misconceptions • Overarching Goals • Specific Information Needs

27. Background- Case Study - Results – Implications - Conclusion Method – Logfile Analysis • Analyze record of users interactions with site • Available data include • Entry/exit pages, pages viewed, time on page, country of origin, etc. Sample Logfile Data: 2 requests cpe-65-172-233-177.sanbrunocable.com [29/Apr/2002:00:01:07 -0700] "GET /shoulder_elbow/shoulder/roughness/management/exercise/02/exercise06 HTTP/1.1" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Q312461)" cpe-65-172-233-177.sanbrunocable.com [29/Apr/2002:00:01:11 -0700] "GET /shoulder_elbow/shoulder/roughness/management/exercise/02/exercise05 HTTP/1.1" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Q312461)"