Authentic Assessment in Computer Supported Collaborative Learning Environments in Higher Education

1. Department of Technology Education and Digital Systems University of Piraeus Authentic Assessment in Computer Supported Collaborative Learning Environments in Higher Education Konstantinos Mathiopoulos Fotini Paraskeva University of Piraeus, Department of Technology Education and Digital Systems Konstantinos Mathiopoulos Fotini Paraskeva

2. Department of Technology Education and Digital Systems University of Piraeus Contents at a glance • Theoretical background of Authentic Assessment • Definitions of Authentic Assessment • Traditional Assessment VS Authentic Assessment • Advantages and disadvantages of Authentic Assessment • Authentic Assessment’s expectations • Research gap • Goal of the paper • Study on Authentic Assessment in Computer Supported Collaborative Learning Environments in Higher Education • Conclusions • Future research Konstantinos Mathiopoulos Fotini Paraskeva

3. Department of Technology Education and Digital Systems University of Piraeus Konstantinos Mathiopoulos Fotini Paraskeva

4. Department of Technology Education and Digital Systems University of Piraeus • Definitions of Authentic Assessment: • "A form of assessment in which students are asked to perform real-world tasks that demonstrate meaningful application of essential knowledge and skills "-- Jon Mueller 2008 • "...Engaging and worthy problems or questions of importance, in which students must use knowledge to fashion performances effectively and creatively. The tasks are either replicas of or analogous to the kinds of problems faced by adult citizens and consumers or professionals in the field." -- Grant Wiggins -- (Wiggins 1993, p229). • "Performance assessments call upon the examinee to demonstrate specific skills and competencies, that is, to apply the skills and knowledge they have mastered." -- Richard J. Stiggins -- (Stiggins, 1987, p. 34). Konstantinos Mathiopoulos Fotini Paraskeva

5. Department of Technology Education and Digital Systems University of Piraeus Traditional Assessment VS Authentic Assessment Multiple choice or other tests that are easily marked, often revealing only whether students can recognise, recall or “plug in” what was learned out of context. Linking learning and working by creating a correspondence between what is assessed in school and what students need to do in the workplace. Konstantinos Mathiopoulos Fotini Paraskeva

6. Department of Technology Education and Digital Systems University of Piraeus Mandernach, B. J. (2003). Incorporating Authentic Assessment. Retrieved 01-05-09, from Park University Konstantinos Mathiopoulos Fotini Paraskeva

7. Department of Technology Education and Digital Systems University of Piraeus Authentic Assessments in general are expected to: • stimulate students to learn more deeply • (Birenbaum, 1996; Dochy & McDowell, 1997; McDowell, 1995; Frederiksen, 1984) • stimulate students to develop professionally relevant skills and thinking processes used by professionals • (Gielen et al; Savery & Duffy, 1995) • motivate students to learn by showing the immediate relevance of that which is learnt for professional practice (Herrington & Herrington, 1998; Lizzio & Wilson, 2004a; Martens, Gulikers, & Bastiaens, 2004; McDowell) Konstantinos Mathiopoulos Fotini Paraskeva

8. Department of Technology Education and Digital Systems University of Piraeus Research gap: New ways of assessment are required so as to replace standardized tests and quizzes. It is important to examine how authentic assessment could be integrated in a web based environment in higher education. The impact of authentic assessment on student learning in a computer supported collaborative learning environment is an open hot issue in higher education. Konstantinos Mathiopoulos Fotini Paraskeva

9. Department of Technology Education and Digital Systems University of Piraeus Goal of the paper This paper is a report on findings of a study focused on examining perceptions of assessment authenticity and the influence on student learning in a computer supported collaborative learning environment at higher education level. A growing body of literature and research on new modes of assessment stresses that the effects of assessments on student learning should always be examined in the light of the whole learning environment along with student perception of the learning environment (Biggs, 1996; Birenbaum, 1996; Segers, Dierick, & Dochy, 2001; Struyven, 2005). Konstantinos Mathiopoulos Fotini Paraskeva

10. Department of Technology Education and Digital Systems University of Piraeus The method: Research Questions • Does authentic assessment change the way that students think about assessment in general? • Does implementation of authentic assessment improve study approach(deep/surface)? • Does authentic assessment promote reflective thinking? • Does authentic assessment increase motivation of learners in a computer supported collaborative learning environment? Konstantinos Mathiopoulos Fotini Paraskeva

11. Department of Technology Education and Digital Systems University of Piraeus The Method: The Participants Participants were thirty-six post graduate students who attend a two-year Master's Degree (M.Sc.) in a department of "Information and Digital Systems" (120 ECTS) in the area of e-Learning. Students had various educational and professional backgrounds and age range from 22 to 55 years. Data was collected during the course of Computer Mediated Communication and Collaboration. Konstantinos Mathiopoulos Fotini Paraskeva

12. Department of Technology Education and Digital Systems University of Piraeus The Method: The Instruments The moodle course management system was used (http://eptlab.unipi.gr), in order for students to be able to communicate through chat and forums, access documents and presentations, participate in workshops (project submission, self-assessment and peer assessment) and answer to questionnaires. Konstantinos Mathiopoulos Fotini Paraskeva

13. Department of Technology Education and Digital Systems University of Piraeus The Method: The Instruments • A 41- item questionnaire for perception of authenticity based on 5-dimensionsal framework for authenticity (Gulikers, Bastiaens, & Kirschner, 2004; 2006) and on perceptions of the purpose of formative computer based assessment through quiz items (Sandra T. Miller, 2009) • The 20-item deep and surface study approach questionnaire based on Revised-Study ProcessQuestionnaire - 2 Factors (R-SPQ-2F; Biggs, Kember, & Leung, 2002). • The 16-item reflective thinking questionnaire based on the Level of Reflective Thinking Questionnaire(Kember D. et al, 2000). • The 81-item motives, cognitive and meta-cognitive strategies questionnaire based on the Motivated Strategies for Learning Questionnaire (Pintrich, R. R., & DeGroot, E. V. (1990). Konstantinos Mathiopoulos Fotini Paraskeva

14. Department of Technology Education and Digital Systems University of Piraeus The Statistical Results of the Research Table1: Pearson r correlation analysis of dimensions of authentic assessment, perceptions of authentic assessment and study approach. Konstantinos Mathiopoulos Fotini Paraskeva

15. Department of Technology Education and Digital Systems University of Piraeus The Statistical Results of the Research Pearson r correlation analysis indicates that authentic assessment indicators have positive linear relationship with deep study approach. Statistically significant is the correlation between deep study approach and overall authenticity r(36) = .390 p< .05, Form /Result r(36) = .364 p< .05, Criteria Transparency r(36) = .348 p< .05, Perception Of Purpose r(36)= .401 p< .05 and Value of Authentic Assessment r(36)= .395 p< .05. On the contrary, authentic assessment indicators have negative linear correlation with surface study approach. Statistically significant factors are Perception Of Purpose r(36)= - .339 p< .05 and Value of Authentic Assessment r(36)= - .524 p< .01. Konstantinos Mathiopoulos Fotini Paraskeva

16. Department of Technology Education and Digital Systems University of Piraeus The Statistical Results of the Research There is positive linear correlation between Perception of Purpose of Authentic Assessment and the dimensions of authentic assessment. Overall authenticity r(36)=.737 p< .01, Physical Context r(36)=.709 p< .01, Form/Result r(36)=.766 p< .01, Criteria r(36)=.680 p< .01, Criteria Transparency r(36)=. 570 p< .01 apart from task factor due to low internal consistency Cronbach a=.436. There is positive linear correlation between Value of Authentic Assessment and the dimensions of authentic assessment. Overall authenticity r(36)=.699 p< .01, Physical Context r(36)=.761 p<.01, Form/Result r(36)=.796 p< .01, Criteria r(36)=.725 p< .01, Criteria Transparency r(36)=.580 p< .01. Perception of Purpose of Authentic Assessment and Value of Authentic Assessment are positive connected by r (36) = .795 p< .01. Konstantinos Mathiopoulos Fotini Paraskeva

17. Department of Technology Education and Digital Systems University of Piraeus The Statistical Results of the Research Table2: Pearson r correlation analysis of dimensions of authentic assessment, perceptions of authentic assessment and reflection. There is a positive connection between authentic assessment and reflection. Reflection is related with Criteria Transparency by r(36) = .349 p< . 05, with Perception of Purpose by r(36)= .489 p< .01 and with Value of Authentic Assessment by r(36)= .472 p< .01. Reflective thinking is positively correlated with Physical Context r(36) = .363 p< .05, Form /Result r(36) = .366 p< .05, Criteria r(36) = .330 p< .05, Criteria Transparency r(36) = .395 p< .05, Perception of Purpose r(36)= .480 p< .01 and Value of Authentic Assessment r(36)= .333 p< .05. Konstantinos Mathiopoulos Fotini Paraskeva

18. Department of Technology Education and Digital Systems University of Piraeus The Statistical Results of the Research Table3: Pearson r correlation analysis of perceptions of authentic assessment and motivation. Authentic assessment increases learners’ motivation. Goal Orientation is positively correlated with Perception of Purpose of authentic assessment r(36) = .362 p< . 05. Task value is positively correlated with Perception of Purpose of authentic assessment r(36) = .410 p< . 05 and Value of Authentic Assessment r (36) = .451 p<. 01. Self-efficacy for learning and performance is positively correlated with Value of Authentic Assessment r (36) = .345 p<. 05. Konstantinos Mathiopoulos Fotini Paraskeva

19. Department of Technology Education and Digital Systems University of Piraeus • Authentic assessment in computer supported collaborative learning environments is highly preferable and benefits not only the assessment procedure, but also the learning experience for learners. • According to perceptions of authentic assessment, for successful application, all dimensions (Overall Authenticity, Task, Physical Context, Form/ Result, Criteria, Criteria Transparency) should be taken into consideration. • Students perceived authentic assessment to be more valuable and efficient, when the purposes (instruction planning, self directed learning, collaboration, identification of strengths and weaknesses, self-assessment, peer assessment) were clear and understandable. Conclusions Konstantinos Mathiopoulos Fotini Paraskeva

20. Department of Technology Education and Digital Systems University of Piraeus • The value of authentic assessment (topic understanding, guidance of student learning, application of mastered skills and knowledge, fair assessment) was enhanced by a suitably designed e-learning environment, which implement all aspects of authentic assessment. • Moodle could be used in this way, since activities could be created or modified so as to become authentic. • This study revealed that an increased perception of assessment authenticity in a web based collaborative learning environment resulted in more deep learning. Conclusions Konstantinos Mathiopoulos Fotini Paraskeva

21. Department of Technology Education and Digital Systems University of Piraeus • Intrinsic motivation of learners was also augmented when goals of authentic assessment were considered more transparent. On the contrary, there was no sign of connection between extrinsic motivation and perceptions of authentic assessment. Authenticity did not seem to be imposed by the teacher or the curriculum and it became part of the learning procedure. • High perception of the value and the purpose of authentic assessment is also reflected in students’ increased appraisal of the task they perform and self-efficacy for learning and performance. • Authentic assessment may be part of the learning process and it may be better used as assessment for learning and not as assessment of learning. Conclusions Konstantinos Mathiopoulos Fotini Paraskeva

22. Department of Technology Education and Digital Systems University of Piraeus • Future research should: • focus on how curriculums may be changed so as to take advantage of the benefits of authentic assessment • orientate to the production or transformation of applications, in order to support authentic learning environments. • Examine ways of changing instructional design in Computer Supported Collaborative Learning Environments in Higher Education so as authentic assessment to be part of the ongoing learning process and not an extra assessment procedure. Future Research Konstantinos Mathiopoulos Fotini Paraskeva

23. Department of Technology Education and Digital Systems University of Piraeus • Herrington A. & Herrington J. (2006). Authentic learning environments in higher education. United States of America, Information Science Publishing • Herrington, J., & Oliver, R. (2000). An instructionaldesign framework for authentic learning environments. United States of America. Educational Technology Research and Development • Gulikers, J. T. M., Bastiaens, Th. J., & Kirschner, P. A. (2004). A five-dimensional framework for authentic assessment. Maastricht, Educational Technology Research & Development • . • Gulikers, J. T. M., Bastiaens, Th. J., & Kirschner, P. A. (2006). Authentic assessment, student and teacher perceptions: the practical value of the five-dimensional framework. Maastricht, Journal of Vocational Education and Training, • Gulikers, J. T. M., Bastiaens, Th. J., Kirschner, P. A., & Kester, L. (2006). Authenticity is in the Eye of the Beholder • Beliefs and perceptions of authentic assessment and the influence on student learning. Maastricht, Studies in Educational Evaluation. • Pintrich P., Smith D., Garcia T., McKeachie W. (1991). A Manual for the Use of the Motivated Strategies for Learning Questionnaire. Technical Report 91-B-004. The Regents of the University of Michigan. 1991 • Miller, T. (2009). Formative computer –based assessments: the potentials and pitfalls of two formative computer based assessments used in professional learning programs. Canada, Queen’s University. References Konstantinos Mathiopoulos Fotini Paraskeva

24. Department of Technology Education and Digital Systems University of Piraeus Biggs, J. B., Kember, D., & Leung, D. Y. P. (2001). The revised two-factor Study Process Questionnaire: R-SPQ-2F. British Journal of Educational Psychology Kember et al. (2000) Development of a questionnaire to measure the level of reflective thinking. Assessment & Evaluation in Higher Education. Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong Mandernach, B. J. (2003). Incorporating Authentic Assessment. Retrieved 01-05-09, Park University ,USA References Konstantinos Mathiopoulos Fotini Paraskeva

25. Department of Technology Education and Digital Systems University of Piraeus Thankyou Konstantinos Mathiopoulos Fotini Paraskeva