Learning Outcomes and Self-Assessments of Baccalaureus Students

1. Learning Outcomes and Self-Assessments of Baccalaureus Students Dr BožanaKnežević & Irena Brdar University of Rijeka Faculty of Maritime Studies IMLA 19 Conference Rijeka-Opatija, 28th September - 1st October 2011

2. Overview IMLA 19 Conference Rijeka-Opatija, 28th September - 1st October 2011

3. Background • The BSc Nautical Studies & Maritime Transport Technology is a three-year study programme. • Maritime English 1, a core course, 15 weeks, 4 contact hours a week & students are awarded 4 ECTS. • To demonstrate the four basic language skills: reading, writing, listening & speaking at the B level (“independent user”) of the Common European Framework of Reference for Languages (2002): • To read, organise & present a short written general or technical article. • To give/follow instructions; ask/answer & address questions; participate in a conversation on familiar topics, participate in discussions on professional topics; • To give opinions in study contexts such as classrooms, tutorials or consultation hours; • To understand lectures; take notes in lectures; write a short essay; • To write a short report developing arguments for & against; follow the main points in discussions about professional topics; guess unknown words from the context in which they are spoken. IMLA 19 Conference Rijeka-Opatija, 28th September - 1st October 2011

5. Aims & central questions IMLA 19 Conference Rijeka-Opatija, 28th September - 1st October 2011

6. Sample IMLA 19 Conference Rijeka-Opatija, 28th September - 1st October 2011

7. Instrument • a questionnaire with 22 statements, grouped into five categories (subscales): • reading (5 items) • writing (3 items) • speaking (6 items) • listening (4 items) • grammar (4 items) • a questionnaire with 45 statements, grouped into seven categories (subscales): • reading (6 items) • writing (10 items) • speaking (10 items) • listening (4 items) • grammar (4 items) • presenting (7 items) • reflective (4 items) 1st measurement – Time 1 (Jan 2010) 2nd measurement – Time 2 (Oct 2010) IMLA 19 Conference Rijeka-Opatija, 28th September - 1st October 2011

8. IMLA 19 Conference Rijeka-Opatija, 28th September - 1st October 2011

9. Mean assessments for subscales Time 1 - Jan 2010 Time 2 - Oct 2010 IMLA 19 Conference Rijeka-Opatija, 28th September - 1st October 2011

10. 1. How did the students’ self-perceived learning outcomes differ according to the individual language skill? Mostconfident about: • understanding the main points in a technical /general text, in the course books & the course reading text, the general meaning of a text. • listening skills i.e. abilities to understand lectures or informal conversations about familiar topics & to deduce unknown words from the context in which they were spoken. • their abilities to ask & answer questions. Time 1 Time 2 • The only marked increase in the perceived abilities after ME2: • the ability to participate in discussions on professional topics & to give opinion in study contexts such as classrooms, tutorials & consultation hours. • the ability to follow the main points in discussions about professional topics. Time 1 & Time 2 - comparison IMLA 19 Conference Rijeka-Opatija, 28th September - 1st October 2011

11. 2. Was there a correlation between various language skills? • e.g. • If the students have the ability to understand the main points in a technical/general text, then • they are able to understand lectures & informal conversations about familiar topics, then • they can follow the main points in discussions about professional topics & • they can deduce unknown words from contexts in which they are spoken. • If the students can understand the main points in a technical/general text, in the course books & course reading texts, we could then say that their awareness of grammar is low. Time 1 IMLA 19 Conference Rijeka-Opatija, 28th September - 1st October 2011

12. e.g. • If the students are confident about their speaking skills, then • their abilities to demonstrate effective planning of the presentation, • to present clear, detailed descriptions on a selected topic, • to use visuals to summarise what the students want to present & • to answer questions from the audience are also seen as important. Time 2 IMLA 19 Conference Rijeka-Opatija, 28th September - 1st October 2011

13. Conclusions • Most students perceive themselves to be well prepared, competent & confident users of English. • The major results of the study also demonstrate the relevance & appropriateness of self-assessment for students at the beginning of their career in higher education: self-assessment directs student further learning & the results can shed real light on the skills of the students in the course. • drawbacks (the data did not estimate the overall level of skills in the students, they only analysed skill levels in the questionnaires). IMLA 19 Conference Rijeka-Opatija, 28th September - 1st October 2011

14. Promoting metacognitive knowledge Students’ reflections (unedited selection) The central part of the students’ project is thus an ability to reflect on the process of one’s learning. • “As far as I understood it, this project was not only about collecting points for the final grade, but also to give something back to us, that something being fun, the understanding of honoring deadlines, respecting our classmates and teachers, and in the end, improve our use of the English language. Although feeling scared and not up to the task at first, eventually this became an interesting project. Working along with my partner we both had fun, and that I believe, along with the things mentioned above, is the purpose of this project.” • “The project was harder to complete than I thought it will be and it wasn't easy to deal with the deadlines but I am glad we managed to do everything on time. The hardest thing for me was the presentation part because I get nervous talking in front of so many people although these were my colleagues and I see them every day. It was also hard to keep the duration of the presentation under five minutes because it is hard to decide what to say and what not to, everything seems important. I think that the outcome of this project matches with my expectations I had before and I am glad because of it. I believe this project improved my speaking, reading and writing skills as I expected so I see no surprises there. Also I learned a lot about fire and fire-fighting which can always be useful on ships but also in everyday life. • To sum up, I had a good time doing this project and I wish there were more of these because it is much easier to learn and improve your knowledge in this way. If I had to do this all over again I would definitely try to study the guidelines that our professor gave us because they make the project so much easier to do. To conclude, I believe that this project has a lot to give to students who do it properly because with a certain amount of effort you will improve your language skills and that, I think, is what we are all here for.” • “This student project was a very good idea, but in real world, it was not so good. The idea of project was great – everyone has to work with someone he wants about some part of our new life, like navigators. My project partner was ok, he helped me and he had brought the material for the project. We had worked together about it and we think it went really good at the end. • The bad parts of project were people and implementation. Some people were too lazy to make it on time, some did not respect the rule of “5 minutes”, the laziness took too much time of studying time and etc. … • What would I like to change on our project? Maybe I would like that we have had little bit more practice with the presentation. We had not practiced a lot it, so we were not so good with it. The 5 minutes were enough for presentation. At first, we were protesting about 5 minutes, we thought that we would need more time, but in the end, we had noticed it was really enough.” IMLA 19 Conference Rijeka-Opatija, 28th September - 1st October 2011

15. Discussion questions • What are the implications of the measurement of learning outcomes for reforming the course programme & student assessment? • What level of attainment of outcomes is required to assure thequality of programme offerings? IMLA 19 Conference Rijeka-Opatija, 28th September - 1st October 2011