AnnotatEd: A Social Navigation and Annotation Service for Web-based Educational Resources

1. AnnotatEd: A Social Navigation and Annotation Service for Web-based Educational Resources Rosta Farzan & Peter Brusilovsky Personalized Adaptive Web Systems (PAWS) Lab University of Pittsburgh

2. Introduction • Web is lacking important features • Supported in standalone hypertext systems • Important for demanding application such as web-based education • Advanced hypertext features • Content annotation • Allows user to annotate non-restricted web content • Guided Tour • Allows user to navigate a complex hyperspace along a predefined path • Adaptive Navigation Support • Helps user to select the most relevant link to navigate Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh

3. Problem • Integrating several functionalities in one system • Result > Sum of its parts • Our work • Integrating Annotation and Adaptive Navigation Support • Open-corpus web educational resources AnnotatEd Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh

4. Outline • Background • Web Annotation • Adaptive Navigation Support • Social Navigation Support • AnnotatEd • Implementation phases • Evaluation • Conclusion & Future Work Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh

5. Web Annotation • Group 1 • Developing advanced architecture and sophisticated but easy-to-use interface • Group 2 • Developing annotation systems for supporting collaboration over the web Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh

6. Web Annotation – Group 1 • Support content-aware annotation on arbitrary web pages • Gibeo • Add .gibeo.net to any web page • Highlighting any part of the text • Associating quality with highlighted text (important, wrong, cool, ..) • Shared annotations • Marginalia • JavaScript web annotation system • Highlighting any part of the text • Associating comments with the highlighted text • Public and private annotations Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh

7. Web Annotation – Group 2 • Annotea • Collaborative development of Semantic Web via shared annotation • Annotations are in the form of comments, notes, explanations, … • Web discussion in Microsoft Office • Collaborative annotation of any web page Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh

8. Adaptive Navigation Support • Classic adaptive navigation support • Guiding learners to appropriate resources • Requires manual annotation of the content • Not Well suited for the large volume of open-corpus documents • Social Navigation Support • Social navigation theory • “Moving toward cluster of people” (Dourish & Chalmers, 1994) • Making use of past learners’ interactions to guide new users • Relies on collective knowledge of community Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh

9. Social Navigation Support (SNS) • Traffic based SNS • “footprints” • Information about number of visits • Promotes links with higher number of visits • Modified CoWeb • Tracks how often a page is accessed or modified • Visualized the density of aggregated access over last 24 hours • Annotation based SNS • Employing users’ annotation • Promotes links to pages with users’ annotations • Educo • Allows learners to associate comments with a document • Keeps track of when comments are modified Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh

10. AnnotatEd • Social navigation and annotation services for browsing web educational resources • Accompanies the learners from page to page • Redirecting all the links through AnnotatEd server • Allows annotation in the form of free format comments, highlights • Offers Social Navigation Support Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh

11. SNS in AnnotatEd • Augmenting links inside the pages • Measuring the past activity of the group • Click stream • Time spent reading each page • Learners’ annotations • Visual icons • Group activity • Individual activity • Traffic based SNS • Background color: magnitude of group visiting activity • Human icon: magnitude of individual visiting activity • Darker color = higher number of visits • Annotation based SNS • Background color: magnitude of group annotation activity • Sticky note icon: presence of learners annotation • Sticky note color: magnitude of individual annotation activity • Implementation • Baseline, Phase II, Phase III Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh

12. Baseline Implementation • Complete tracking feature • Simple annotation interface • Private notes • Traffic based SNS • Links to pages with user annotation were augmented for the author of the annotation • Preliminary evaluation • 60% of students appreciated the ability to annotate • Interested in sharing annotation and seeing annotations of others • Type of learners’ annotations • Praise  37 notes • Problem 36 notes • General 34 notes Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh

13. Phase II Implementation • Enhanced interface • Highlighting feature • Public and private annotations • Typed annotations • Praise, Problem, General • Signed annotations • Motivating students • 90% of annotations were signed • Annotation based SNS • Type and density of learners’ annotation • Annotation temperature • Praise  warm • Problem  cold Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh

14. AnnotatEd – Phase II Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh

15. Phase III - Implementation • Interface problem in Phase II • 61% of general annotations were praise • Type box was too distant from save button • Problem type was used mostly to report problem to the teacher • Instead of negative point about the page • Bookmarking feature • Generating a report for annotated and bookmarked pages Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh

16. Evaluation • Classroom study • School of Information Science at University of Pittsburgh • 5 semesters • This paper focused on the recent studies (2 semesters) • Introductory C programming (undergrad course), Information Retrieval (graduate course) • System was introduced in the class by the instructor Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh

17. Hypotheses • Providing navigation support for “browsing” web educational resources is important • Browsing-based access is popular among students even with presence of other access options such as map based navigation, or search • Annotating links with SNS cues would affect students navigation decision • Pages with annotation-based SNS are important pages • Students would appreciate the annotation functionality of the system • Students would appreciate Annotation based SNS Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh

18. Browsing vs. Map vs. Search • AnnotatEd is connected to KnowledgeSea, a navigation map with search feature • Same educational material can be accessed through • Browsing, Navigation map, Search Browsing is at least as important as map-based navigation and more important than search Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh

19. Importance of Browsing • Access through browsing • Provides more context • Makes students more certain about the content of the page • Comparing exact pages accessed through browsing and map-based navigation • 62 pages • Desirability ~ time spent reading (TSR) the page Students spent significantly more time on pages accessed through browsing Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh

20. Effect of SNS on Link-Following • Comparing number of accesses to links with and without traffic or annotation based SNS Significantly more access on pages with SNS specially annotation-based SNS Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh

21. Importance of Pages with Students’ Annotation • Comparing average accesses to pages with and without students’ annotations before appearance of annotations Annotated Pages are Important Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh

22. Subjective Data Analysis • Students appreciate the annotation feature of AnnotatEd • Students appreciate annotation based SNS Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh

23. Conclusion • AnnotatEd provides social navigation support and annotation service • Our evaluation shows that • Browsing is an important information access method and provide context-rich information access method • Pages with students’ annotation are important • Traffic and annotation based SNS affect students navigation pattern and are appreciated by students Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh

24. Future Work • Evaluation with larger number of users • Improving annotation interface • Pen-based annotation • Will encourage higher number of annotations • Improving bookmarking feature • Employing bookmarks for providing social navigation support Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh

25. Thank You Rosta Farzan (rosta@cs.pitt.edu) & Peter Brusilovsky (peterb@pitt.edu) PAWS Lab – University of Pittsburgh