Yongwu Miao, Colin Tattersall, Judith Schoonenboom, Krassen Stevanov, Adelina Aleksieva-Petrova

1. Using Open Technical E-learning Standards and Service-orientation to Support New Forms of E-assessment Yongwu Miao, Colin Tattersall, Judith Schoonenboom, Krassen Stevanov, Adelina Aleksieva-Petrova Manchester, January 12, 2007

2. Overview • Introduction to New Forms of Assessment • Characteristics of New Forms of Assessment • A Standard-based Approach to Model New Forms of Assessment • Delivering New Forms of Assessment in Today’s Open Infrastructure • Comparison with Software Development Approaches to Support New Forms of Assessment • Conclusions and Future Work

3. New Forms of Assessment • Traditional forms of assessment multiple-choice, filling-in-blank, sliding, ranking, matching, …… • New forms of assessment (NFA) self-/peer assessment, 360 degree feedback, portfolio assessment, …… • Competence assessment is usually based on new forms of assessment

4. Characteristics of New Forms of Assessment • Involvement of multiple roles/users • Variety in task types • Complex process control • Exchange of information Explaining through a peer assessment example

5. A Peer Assessment Example • Students are instructed how to conduct this peer assessment • Each student selects a different paper and reads it • Each student then prepares a brief article (400-500 words) about their chosen paper • Pairs of students then exchange articles and review each other’s articles • Each student writes a response to the review, revise the article, and then submit documents including paper, original article, reviewed article with the reviewer’s comments, response and the final version of their article to the tutor • The tutor then marks on students’ exercises

6. A Peer Assessment Example learner1 tutor learner2 Involvement of multiple roles/users

7. A Peer Assessment Example learner1 tutor learner2 instruction instruction instruction selecting/reading paper1 selecting/reading paper2 writing article1 writing article2 reviewing article2 reviewing article1 responding review1 responding review2 final assessment1 final assessment2 Variety in task types

8. final assessment1 final assessment2 A Peer Assessment Example learner1 tutor learner2 instruction instruction instruction selecting/reading paper1 selecting/reading paper2 writing article1 writing article2 reviewing article2 reviewing article1 responding review1 responding review2 Complex process control

9. A Peer Assessment Example learner1 tutor learner2 instruction instruction instruction selecting/reading paper1 paper1 paper2 selecting/reading paper2 writing article1 article1 article2 writing article2 reviewing article2 review2 review1 reviewing article1 responding review1 response1 response2 responding review2 final assessment1 final assessment2 Exchange of information

10. A Standard-based Approach A combined use of IMS LD, IMS QTI, and assessment-specific services • Using IMS LD to model multi-role/user-involved assessment processes • Using IMS QTI to model traditional assessment tasks and using assessment-specific services to model special assessment tasks • Using IMS LD to model a learning flow with complicated process controls • Using IMS QTI, IMS LD, and assessment-specific services to model complicated dataflow As a consequence, a competence assessment process can be modeled as a unit of assessment, a specific unit of learning which contains assessment-specific elements (e.g., QTI documents and assessment-specific services

11. play method condition act expression action role-part component role activity environment property show/hide staff learner learning support assessment learning object service An extended IMS LD model notification activity-description internal external change-property assessor assessee item assessment_test assessment_item outcome A simplified IMS QTI model assessment-specific service QTI_type resource item_body response_processing interaction response learning design (or assessment design) unit of learning (or unit of assessment)

12. Delivering New Forms of Assessment Using today’s open technical e-learning infrastructure • Including IMS LD compatible engine like CopperCore • Including IMS LD compatible client like SLeD • Including IMS QTI compatible player like APIS • Including assessment-specific services like a concept-mapping tool and a simulation tool

13. Delivering New Forms of Assessment A screenshot of execution of peer assessment example

14. Comparison with Traditional Software Development Approaches Many software tools supporting new forms of assessment have been developed and used. In comparison with these tools, our approach has advantages in • Supporting for interoperability • Supporting for flexibility • Supporting for seamless integration with learning processes

15. Conclusions and Future Work Conclusions: • New forms of assessment can be modelled as units of assessment through a combined use of IMS LD, IMS QTI, and assessment-specific services • In comparison with traditional software development approaches, the standard-based approach fosters interoperability, flexibility, and seamless integration with learning activities Future Work: • developing a high-level assessment authoring tool which enables practitioners to define their own customized assessment models • Developing mechanisms to transform such assessment models into units of assessment automatically

16. Thanks for attentionQuestions?