Assessment of Learning Outcomes in Engineering - an International Perspective

1. Assessment of Learning Outcomes in Engineering- an International Perspective Mona Dahms mona@plan.aau.dk UCPBL, Aalborg University

2. Presentation Outline • Curriculum Development and Assessment • An International Perspective • The Situation in Denmark aloe conference 2008

3. Definitions • Learning outcomes: ”sets of competences, expressing what the student will know, understand or be able to do after completion of a process of learning” (ECTS Guide 2005); describes the end product of learning • Assessment: Judgement of students’ performance, i.e. judgement of whether students have achieved the learning outcomes • Evaluation: Judgement of quality of study programs and activities aloe conference 2008

4. Study activity 1 Design of 1st sem. Study activity 2 Design of 2nd sem. Study activity 3 Competenceprofile of graduate Design of study program Design of 3rd sem. Study activity 4 Design of 4th sem. Study activity 5 Design of Nth sem. Curriculum Development (CD) – a sequential process aloe conference 2008

5. This is where assessment is carried out!!! CD – a top-down process Competence profile of graduate Evaluation of graduate competences Design of study program Evaluation of study program Design of semester program Evaluation of semester program Design of study activity Evaluation of study activity Teaching, learning and assessment

6. = learning outcomes The Cowan Curriculum Development model aloe conference 2008 Cowan, 2004

7. Learning/teaching Assessment Constructive alignment Learning outcomes “Alignment is about trust in the relationship between student and teacher” (Derick) aloe conference 2008

8. setting criteria (LOs) collecting evidence making judgement and possibly providing feed-back for improvement Example: The customer tastes the food served and ...... setting criteria collecting evidence making judgement and always providing feed-forward for further improvement Example: The chef tastes the food to be served and ......... What is assessment? Formative Summative

9. Assessment Important questions to consider are: • WHO sets criteria? • WHEN and WHERE is evidence prepared? • HOW is evidence presented? • WHO makes the judgement? • WHO provides feed-forward/feed-back? aloe conference 2008

10. Changing competences… …necessitates changes in assessment creating Higher level competences evaluating analysing applying Learning outcomes assessed Declared learning outcomes understanding Lower level competences remembering aloe conference 2008

11. Traditional assessment Understanding Good understanding Passed examination: 25 % Good understanding Failed examination: 0 % Pass Poor understanding Passed examination: 50 % !!!! Poor understanding Failed examination: 25 % Fail Exam Fail Pass Jakobsen and Rump, 1998 aloe conference 2008

12. The International Perspective • Malaysia (2006?) and • South Africa (1997) have both introduced • Outcomes Based Education (OBE) at all levels aloe conference 2008

13. Malaysian Qualification Framework (MQF) Learning Outcomes 1. knowledge of discipline areas 2. practical skills 3. social skills and responsibilities 4. value, attitude and professionalism 5. communication, leadership and team skills 6. problem solving and scientific skills 7. information management and life-long learning skills 8. managerial and entrepreneurial skills “The challenge to us is in drafting assignments, projects and exams as means of evaluating the MQF learning outcomes.” aloe conference 2008

14. South African Critical Cross-Field Learning Outcomes • Identify and solve problems and make decisions using critical and creative thinking • Work effectively with others as members of a team, group, organisation and community • Organise and manage themselves and their activities responsibly and effectively • Collect, analyse, organise and critically evaluate information • Communicate effectively using visual, symbolic, and/or language skills in various modes • Use science and technology effectively and critically showing responsibility towards the environments and the health of others • Demonstrate an understanding of the world as a set of related systems by recognising that problem-solving contexts do not exist in isolation aloe conference 2008

15. Achievements in SA ? • A study of critical thinking abilities of 1st year education students documented deficient critical thinking abilities • Possible reasons: Teaching practice; educational change; socio-cultural environment; language ability (Lombard & Grosser 2008) aloe conference 2008

16. aloe conference 2008

17. Skills aloe conference 2008 Kolmos and Holgaard 2007

18. Skills aloe conference 2008 Kolmos and Holgaard 2007

19. Skills aloe conference 2008 Kolmos and Holgaard 2007

20. Skills aloe conference 2008 Kolmos and Holgaard 2007

21. Statements on team based exams • “I dread a transition to an individual exam because I believe that this would result in a significant decline in the academic level. Besides I am of the opinion that the team based exam is the type of exam which best relate to the working environment which the students will be in later on.” (External examiner) • “I think it is important that the members of the group are tested in each others’ co-presence and in a collaborative process. It opens the possibility for partially evaluating their team play and co-operation, ability to stand out in a group, ability to follow up and to have an effect on a discourse.” (External examiner) aloe conference 2008 Kolmos and Holgaard 2007

22. Conclusion • Individual assessment does not test complex knowledge and collaborative competences (Kolmos and Holgaard 2007) aloe conference 2008

23. Thank you for your attention! Comments and questions are very welcome at: mona@plan.aau.dk