Kirsi Silius kirsi.silius@tut.fi Anne-Maritta Tervakari anne.tervakari@tut.fi

1. A Multidisciplinary evaluation framework and tool for the evaluation of usability, pedagogical usability, accessibility and informational quality of web-based courses Kirsi Silius kirsi.silius@tut.fi Anne-Maritta Tervakari anne.tervakari@tut.fi Seppo Pohjolainen seppo.pohjolainen@tut.fi Tampere University of Technology (TUT) Finland

2. user group user group user group user group usability presentation of information reliability of information accessibility fluency of use quality of information instructional quality pedagogical usability benefits value added Viewpoints of evaluation experts experts experts

3. Organisation of teaching process Development of quality of teaching Value added Development of learners’ learning skills Social acceptability Utility Testing and development of educational ICT Support for organisation of the teaching process System acceptability Pedagogical usability Usefulness Support for learning, tutoring and scaffolding process and support for the achievement of learning objectives Usability Easy to learn Practical acceptability Cost Efficient to use Compatibility Easy to remember Reliability Few errors Etc. Support for the development of learning skills (interaction with other actors and growth of learners` autonomy Subjectively pleasing Usefulness of web-based learning environments based on Nielsen (1993), modified by Tervakari & Silius 2002-03 Nielsen, J. 1993. Usability Engineerings. Boston (MS.): Academic press.

4. The main phases of survey at TUT • Context analysis: to find significant background factors, which have strong impact to the instructional value of the certain web-based learning environment. • Assessment of implementation of web courses using multidisciplinary framework based on usability, pedagogical usability, accessibility and informational quality • Advantages and disadvantages of web-based learning in various contexts. What are the critical factors of quality in web-based learning? • Value added of web-based courses in various contexts.

5. Common usefulness society, culture etc. web-based course usefulness of certain web-based course in certain teaching and learning context common usefulness of web-based course evaluation of web-based course with the tool experts or evaluators context independent factors Usefulness in certain teaching and learning context organisation, science, subjects, actors etc.. value-added? teachers their personal contexts advantages? teaching and designing of web-based course usefulness from teacher’s viewpoint disadvantages? usefulness from learners’ viewpoint studying learners their personal contexts

6. Evaluation sections

7. Iterative evaluation resources evaluation by teachers and designers as well as questionnaires for real users with the tool analysis of data by teachers and designers web-based course design and/or development of web-based course training of designers and teachers for evaluating their web- based courses with the tool implementation of web-based course development of web-based course experts’ evaluation and questionnaires for real users feedback to teachers and designers analysis of data by experts

8. Purpose of this evaluation tool • This tool is not • for making ranking lists of web-based courses • for evaluate the validity or substantiveness of the content • This tool is to • help teachers to pay attention to properties of their web pages which might need some improvement

9. 1. 2. 3. Start of evaluation

10. Choose sections

11. Questions to be answered

12. Report 1: Results of the evaluation

13. Report 2: Guides and tips

14. Report 3: Own answers