Design of metadata surrogates in search result interfaces of learning object repositories:

1. Design of metadata surrogates in search result interfaces of learning object repositories: Linear versus clustered metadata design Panos Balatsoukas Anne Morris Ann O’Brien

2. Contents • Definition of user-centred metadata • Evolution of metadata surrogate design • Aim and Objectives of the usability test • The META-LOR 1 prototype • Methodology • Results • Conclusions – Recommendations • Future Research

3. Metadata definitions • ‘Data about data’ • “Structured data about an object that supports functions associated with the designated object” – (Greenberg, 2005) • Learning object metadata: metadata used for the efficient description of learning objects and the effective support of educational-learning functions related to the described learning objects.

4. User-centred metadata ContentPresentation Learner Technologies Relevance Usability

5. Evolution of metadata presentation and content

6. Copied from: Marchionini et al (1993) http://hcil.cs.umd.edu/trs/93-10/93-10.html

13. Design of metadata surrogates • Metadata elements providing access or arranging access to the resource should follow content related elements such as title, abstract, subject heading or keywords. • Users prefer content related metadata for finding and identifying sources and technical and physical metadata for selecting and obtaining access to the resource. • There is a debate among researchers as to whether metadata surrogates should be displayed in list, tabular, dynamic or category-based format in search result interfaces. • It is suggested that abstracts should contain contextualised information relevant to users’ search query. • Metadata surrogates should not include only topical or subject related information.

14. Design of learning object metadata in search result interfaces • The need to include a description/abstract of the contents of the learning object in the metadata surrogate; • The use of user-centred metadata terminology and vocabularies; and • The use of clustered rather than linear and information cluttered learning object metadata surrogates.

15. Aims and objectives • To examine users’ interaction with two different learning object metadata surrogates: 1. a linear metadata surrogate interface, and 2. a clustered metadata surrogate interface. • The objectives of this study were: • To investigate the time needed by learners to identify a relevant learning object, using both interfaces; • To study the impact of task complexity on users’ interaction with both interfaces; and • To examine learners’ subjective satisfaction for both interfaces.

18. Pop up box Linear metadata surrogate

19. Pop up box Metadata categories Clustered metadata surrogate

21. Methodology 1 • Usability participants’ profile: • 12 postgraduate students in Information and Computer Science • Task List analysis and scenarios: • 3 tasks with varying degrees of complexity (Low, Medium and High complexity) • Error rate and Time • Observation (Think Aloud protocol) • Background and post test questionnaires • Post test interviews

22. Methodology 2 The three Tasks:

23. Results of the usability test

24. Differences in Time • participants performed the three tasks slightly faster using the clustered metadata surrogate interface. • Mean time of 314 secs in the Linear. • Mean time of 301 in the Clustered

25. Task complexity and Interface • There were no significant differences observed between task complexity and metadata interface design

26. Subjective satisfaction • Subjects were significantly more satisfied with the clustered metadata surrogate interface. • Mean overall satisfaction for clustered metadata surrogate = 7.8. • Mean overall satisfaction for the linear metadata surrogate interface = 6.3

27. Qualitative results (1) • Participants (n=10) liked the way metadata was presented in the clustered metadata surrogate interface: • Plausibility and engagement • Structure and organisation of information • Two participants preferred the linear interface (prior familiarisation; not meaningful metadata clustering).

28. Qualitative results (2) • Subjects liked the use of most of the general and technical category metadata (e.g. title, subject, description, format, identifier) • Few of the educational related metadata were perceived as useful (e.g. Audience, interactivity level, difficulty)

29. Qualitative results (3) • Subjects did not like the inclusion of many metadata elements and lengthy metadata surrogates. • Some participants (n=4) would like to select the metadata elements to be displayed in the surrogate. • Other metadata elements: • Relation metadata • People’s comments • The time it takes for a learning object to be downloaded/accessed • Accessibility needs • Information about the quality of learning objects

30. Conclusions - Recommendations • The provision for alternative displays of metadata surrogates, for example, both in linear and clustered forms. • The design of adaptive interfaces that present the content and format of metadata surrogates according to learners’ needs. • The use of pop up boxes for documenting and presenting the meaning of learning object metadata elements to users. • Need to extend the LOM standard with new metadata elements, such as, ‘the time it takes for a learning object to be downloaded’, ‘accessibility needs’ information, as well as ‘information about the ‘quality’ of a learning object.

31. Research in progress… • Usability assessment of three learning object repositories (MERLOT, ARIADNE Knowledge Pool and JORUM/UK). • Survey of students’ perceptions of the importance of learning object metadata elements. • User study on the criteria students employ to judge the relevance of learning objects. • Development of Heuristic evaluation checklist for the evaluation of metadata surrogates in search and search result interfaces. • Development of guidelines and recommendations for the design of learning object metadata schemas and Learning Object Repositories.