Quality Assessment Studies in the area of E-Learning

1. Quality Assessment Studies in the area of E-Learning Symeon Retalis (retal@unipi.gr) University of Piraeus Department of Technology Education and Digital Systems Computer Supported Learning Engineering Lab [http://cosy.ted.unipi.gr]

2. Why do we care about the quality in e-learning? • E-learning, which • was once a poor and often unwelcome stepchild within the academic community, • is becoming increasingly more visible as a part of all educational levels.

3. Is distance education effective or not? • In his painstaking annotated bibliography The No Significant Difference Phenomenon (1999), Russell lists 355 sources dating back to 1928 that • state that there is no evidence of any kind that categorically proves that technology has any impact on learning – either positive or negative [http://www.nosignificantdifference.org]. • The quality assessment needs to see the true educational value of what’s on offer.

4. Key shortcomings of original research in quality assessment • Much of the research does not control for extraneous variables and therefore cannot show cause and effect. • The validity and reliability of the instruments used to measure student outcomes and attitudes are questionable. • Many studies do not adequately control for the feelings and attitudes of the students and faculty—what the educational research refers to as “reactive effects.” • It is not taken into consideration how the different learning styles of students relate to the use of particular technologies

5. An example: Findings in NCSL • [Pinelle & Gutwin, 2000] “A Review of Groupware Evaluations” • The main findings are that • almost one-third (1/3)of the groupware systems were not evaluated in any formal way, • that only about one-quarter (1/4) of the articles included evaluations in a real-world setting, • and that a wide variety of evaluation techniques are in use. • From the studies included in this survey, 41% of the articles that • included evaluations were of actual real world software implementations, but only 25% considered the software’s organizational and work impact. Their main conclusions from the review are that more attention must be paid to evaluating groupware systems and that there is room for additional evaluation techniques that are simple and low in cost.

6. TELL: Towards Effective network supported coLLaborative learning activities[http://www.softlab.ece.ntua.gr/research/research_projects/tell/] • The necessity of e-learning centers • supporting the integration of NT in education • The need for new methods & tools for instructional design • Design patterns • Validation of quality approaches in specific areas • Need for experimentation in real classroom environments The E-LEN projecthttp://www.tisip.no/E-LEN

7. There is no silver bullet • No “novel effect” anymore! • We need to examine what works and what does not in a complex e-learning environment with relation to the high order learning outcomes • Designing effective e-learning environments will always be hard! • there is no silver bullet

8. Quality Assessment Studies in the area of E-Learning Symeon Retalis (retal@unipi.gr) University of Piraeus Department of Technology Education and Digital Systems Computer Supported Learning Engineering Lab [http://cosy.ted.unipi.gr]