Evaluation of Hospital Portals Using Knowledge Management Mechanisms

1. Evaluation of Hospital Portals Using Knowledge Management Mechanisms Chei Sian Lee, Dion Hoe-Lian Goh, Alton Yeow-Kuan Chua Nanyang Technological University, Singapore

2. Research Background • Demand for hospital portals increases as more people are looking for health information online • Effective information storage and management to ensure that the contents on the hospital portals are timely, credible and accurate is a major responsibility of these hospitals • Review of past portal evaluation studies showed that there is little work done in establishing guidelines for hospitals in implementing KM mechanisms in portals

3. Research Objective • Evaluate various features used by hospital portals to facilitate KM and collaboration between hospitals and users • Focus on hospital portals from two geographical regions (i.e. North America and Asia).

4. Literature Review • KM provides the process to help both organizations and users to capture, store, organize and share the knowledge within and across communities effectively • KM mechanisms that will be effective in managing hospital portals • Knowledge Access • Knowledge Creation • Knowledge Transfer

5. Research Model Knowledge Access mechanism refers to the mechanisms through which the users get access to the portal and information on the portal. Knowledge Creation mechanism refers to the process of capturing users’ information such as demographics, preferences and behavioral behaviors and creates new knowledge that will benefit the portal providers and the user. Knowledge Transfer mechanism refers to the mechanisms that allow the portal providers to foster user-to-user and provider-to-user sharing of knowledge.

6. Evaluation Checklist Formulation • Our evaluation checklist consists of 52 items that were derived by reviewing the features available on the selected portals and with reference to past studies • These checklist items were grouped under sub-dimensions based on similar functionalities • Based on the three KM mechanisms in our research model, the following 17 sub-dimensions were formed.

7. Evaluation Checklist Formulation • Knowledge Access: • Access to Portal (AP): Portal appears on first page of at least one search engine • Query (Q): Portal supports queries using free-text search • Results (R): Portal allows results of queries to be further searched • Browsing (B): Portal allows browsing through glossary, sitemaps, etc • Information Customization by User (ICU): Portal allows information to be customized by users • Information Customization by Organization (ICO): Portal allows information to be customized by organizations • Accessibility (A): Portal allows visual and hearing impaired users to access • Information Presentation (IP): information presented in rich manner (eg. images, video, audio, flash)

8. Evaluation Checklist Formulation • Knowledge Creation: • Acquisition of User Information (ACUI): Portal has features to capture user information (eg. membership signup) • Feedback (FB): Portal is able to collect feedback from users (eg. comments, suggestions) • Domain Data Acquisition (DDA): Portal has features to acquire domain-specific data from users (eg. surveys or polls)

9. Evaluation Checklist Formulation • Knowledge Transfer: • Collaboration from Organization to User (COU): Portal has features such as “Ask an Expert” • Collaboration between Users (CBU): Portal allows users to interact (eg. discussion forums) • Synchronous Support (SS): Portal offers real-time collaboration (eg, chat) • Resource Sharing (RS): Portal shares additional resources such as links to external websites and contributions by other users. • User Support (US): Portal guides users through online tutorials, demo, hotline, on-screen help • Information Alert (IA): Portal delivers news and alerts to users through means such as newsletter, event calendar, email alert, RSS feed aggregator.

10. Data Collection • Sampled 20 hospital portals from North America and Asia • Portals that are not available in English were excluded

11. Portals Selection Criteria • appeared in the Best Hospitals 2005 survey conducted by US News & World Report, Best Health • recipients of prestigious awards such as the eHealthcare Leadership Awards • received high ratings from third-party agencies such as alexa.com • provided self-regulating policies and third party seals such as HON Code and TRUSTe

12. Selected Hospital Portals

13. Portal Evaluation Approach • Each portal was evaluated by two members of the research team. • For each portal, the evaluation results obtained were compared and any discrepancies were eliminated by combined re-assessment. Cohen’s Kappa results of test ranged from 0.77 to 1.0 and suggest a high degree of agreement among evaluators.

14. Results

15. Results (cont’d)

16. Results (cont’d)

17. Main Findings • Most hospital portals appear to have more features supporting knowledge access (KA) than knowledge creation (KC) and knowledge transfer (KT) • Among all features to support KA, access via Search engines (AP) is the most prevalent • The most common feature available to support KC is obtaining direct feedback from users (FB) • The most common feature available to support KT is resource sharing (SS)

18. Main Findings Two least common features among the portals are collecting domain-specific data (to support KC) and accessibility (to support knowledge access) Overall, North America hospital portals appeared to perform better than Asia hospital portals in terms of supporting all Knowledge mechanisms (KA, KC and KT)

19. Conclusion The checklist serves as a useful way to evaluate the effectiveness of a portal in supporting knowledge management objectives of the organization Results from this study can be used by organization to develop highly interactive portal to meet user expectations and enhance their knowledge management processes