340 likes | 470 Views
Evaluating your project G oing beyond student satisfaction. Ivan Moore RAEng, Eng SC ivan@engsc.ac.uk. Evaluation. Why evaluate? Minimise risk Partnership with students Professional practice Publication. Developing teaching practice. Theory. Developing teaching practice.
E N D
Evaluating your projectGoing beyond student satisfaction Ivan Moore RAEng, Eng SC ivan@engsc.ac.uk
Evaluation • Why evaluate? • Minimise risk • Partnership with students • Professional practice • Publication
Developing teaching practice Theory
Developing teaching practice scholarship Theory
Developing teaching practice scholarship Previous experience Theory
Developing teaching practice scholarship Previous experience Theory Different contexts
Developing teaching practice scholarship Previous experience Theory Different contexts Teaching in practice
Developing teaching practice scholarship Previous experience Theory Different contexts Teaching in practice Outcome
Developing teaching practice scholarship Previous experience Theory Different contexts Teaching in practice Outcome Further scholarship Reflection, feedback
Principles of evaluation • An integral part of our teaching practice • An ongoing process, so that we learn from systematic reflection • Should be participatory • Should enable us to make appropriate modifications along the way • Should enable us to make judgments on specific sessions, but also to draw out wider implications
Purposes of evaluation • Mike Prosser • Quality Assurance • student satisfaction • the mean score is important as a measure of quality • Quality Enhancement • Student conceptions/how they experience the course • The deviation is important: more focused view
Purposes of evaluation • Mike Prosser • Is the learning environment/teaching approach having any influence on student conceptions/approaches? • A student experience survey is more important than a student satisfaction survey
Evaluating your development • Three approaches • Goal-oriented evaluation • Goal-free evaluation • Context-based evaluation • Several methods • Questionnaire • Focus groups • Structured interviews • frequent, large student numbers, superficial • summative, representative sample, rich information
Student learning as a journey The outcomes (or goals) The learning environment Starting point
Evaluating the learning journey The outcomes The learning environment Expectations and preparedness
3 Evaluating the outcomes • What skills have they developed? • intellectual • professional • inter-personal • What other attributes or attitudes have they developed? • confidence • problem-solving • self-awareness (metacognition) The learning environment The outcomes Expectations and preparedness
2 The learning environment • How are they experiencing the learning environment? • What opportunities is it providing? • What skills/attitudes do they feel they are developing? • What model or theory are you using? • motivation • sense of control • confidence The learning environment The outcomes Expectations and preparedness
1 Expectations and preparedness • Expectations • Standards - challenge • skills and attitudes they will develop • extent of collaboration and independence • role of the tutor • Preparedness • meeting demands • working collaboratively • asking questions • active learning • reading and analysing • finding information The learning environment The outcomes Expectations and preparedness
Goal oriented evaluation • Determine your goals • 4 or 5 at the most • For each goal, devise 4 or 5 statements to test achievement or progress • Scored 1-5
Deconstructing an example • Find the statements that focus on: 1 Team work 2 Research skills 3 Motivation
Deconstructing a goal • Can you identify any theory or model to help you with this? • Motivation • interest in the subject/task • clear goals, tasks and standards • support • feedback • sense of belonging and sharing • success - improvement
Designing an evaluation questionnaire • In your group • Quickly agree and write down some learning experience(s) and outcomes you would like for your students [goals] • For each goal • Devise 4 statements that will help to determine if the goal is being achieved • Record the goals and statements on a flip chart
Goal – free evaluation (Scriven) • What did the students get out of the experience? • skills • attitudes • enjoyment • self-awareness
Goal – free evaluation (Scriven) • The module has helped me develop my ability to work as part of a team • Is goal oriented. The goal is team working • The most useful thing I learned in this module was • Is goal free. • Why? • Illuminative evaluation
Goal-free evaluation • In your group • Devise up to 3 ‘questions’ that you might ask students to answer that might provide information on what they are experiencing • Record your questions • Discuss what information you might receive from these questions • Draw up your conclusions for feedback • Share our outcomes
Context-oriented evaluation • Where, when and how are the students gaining the experiences and outcomes you wish for them? • (may not be within your module) • Since the beginning of the year/module: What skills, if any, have you developed? What helped you to develop these skills? How have you changed the way you study? What prompted you to make these changes?
Context-oriented evaluation • Since the beginning of the year: • What skills, if any, have you developed? • What helped you to develop these skills? • How have you changed the way you study? • What prompted you to make these changes?
Other methods • Focus groups • Structured interviews • Continuous feedback • What did you find most difficult/confusing today? • What things did you find helped you learn last week? • What should I: • Start/stop/continue?
The three minute paper • What was the most useful or meaningful thing you learned during this session? • What question(s) remain uppermost in your mind as we end this session? • What was the ‘muddiest’ point in this session? • What would you like me to stop doing? • What would you like me to start doing? • What would you like me to continue doing?
Other instruments • Kolb’s Learning Style Inventory (LSI) • Honey and Mumford’s LSQ • Sternberg’s thinking styles • Felder’s Index of Learning Styles • Weinstein’s Learning and Study Strategies Inventory (LASSI) • Entwistle’s Approaches to Study Inventory
The Course experience questionnaire(P Ramsden) • Designed as a performance indicator • 24 statements relating to 5 aspects • 1 overall satisfaction statement • Research-based • Drawn from statements made by students in interviews • Students with positive responses take a deep approach
The five sub-scales • Good teaching • Clear goals and standards • Appropriate assessment • Appropriate workload • Generic skills
The five sub-scales • Good teaching 3, 7, 15, 17, 18, 20 • Clear goals and standards 1, 6, 13, 24 • Appropriate assessment 8, 12, 16, 19 • Appropriate workload 4, 14, 21, 23 • Generic skills 2, 5, 9, 10, 11, 22
From Ramsden CEQ • The module has helped me develop my ability to work as part of a group • I have usually had a clear idea of where I was going and what was expected of me • I have found this module interesting • The tutor motivated me to do my best work • Learning this way has helped sharpen my analytical skills • There has been more assessment of what I have memorised than of what I have understood • This module has helped me to develop the ability to plan my own work