Managing the Assessment Lifecycle: principles and practices in the first year

1. Managing the Assessment Lifecycle: principles and practices in the first year Prof Keithia Wilson Program Lead for the FYE, Griffith Health GIHE Senior Fellow for the FYE 2010 ALTC National Fellow for the FYE 2007 Australian University Teacher of the Year Prof Keithia Wilson ALTC National Fellow

2. Acknowledgment to Country In the Spirit of Reconciliation Following on from Sorry Day I would like to acknowledge & honour the Traditional Custodians of this land that our Logan campus is built on, the Yugambeh People, and pay respect to their Elders past & present Prof Keithia Wilson ALTC National Fellow

3. Overview • Principles & practices for enabling commencing students success with FY Assessment • Designing assessment • Preparing students to undertake assessment • Marking Assessment • Post-Assessment feedback process to enable & support learning 2. Strategies for academic recovery with at-risk commencing students Prof Keithia Wilson ALTC National Fellow

4. Overview • Current Context for the FYE • Purposes of FY Assessment & Student Motivations • The FY Assessment Lifecycle from the perspectives of – • Students and • Staff • Principles & Models for enhancing effective Practice with FY Assessment Prof Keithia Wilson ALTC National Fellow

5. Information Sources • The First Year Experience & Assessment literature generally • Findings from a recent ALTC Grant on First Year Assessment Practice (2009-2010) • Findings from a current ALTC National Fellowship on the FYE working with FY teaching teams (2010-2012) • Focus on the student voice & understanding the student experience of assessment Prof Keithia Wilson ALTC National Fellow

6. Step 1 – Context • How can we improve the assessment process and experience for commencing students? Understanding the current context for the FYE Prof Keithia Wilson ALTC National Fellow

7. Federal Gov’t reform agenda in Higher Education Aims to – • Widen student participation in Higher Education – A FAIR GO! • Increase the access of students from low SES & disadvantaged backgrounds to university (higher numbers - Target increase from 12% to 20% by 2020) • Increase the success of students from low SES & disadvantaged backgrounds at university (higher retention) • Moving from an elite model of HE (0-15% population participation) to a mass model of HE (16-50%) (Trow, 2004) • 25% participation in Australia (50% USA) • 1.3% of Indigenous Australians attend university (30% lower graduation rate) Prof Keithia Wilson ALTC National Fellow

8. Evolution of Approaches to FY Orientation & Engagement & the FYE • First Generation Strategies = Co-Curricular - A focus on designing FYO&E supplemental activities & strategies which are outside of the classroom • Second Generation Strategies = Curricular - A focus on enhancing FY curriculum design, pedagogy & assessment practices • Third Generation Strategies = Whole-of-Institution - A focus on an Institution wide approach to 1st & 2nd generation strategies, with practice standardisation & QA mechanisms for continuous improvement = Whole-of-School/Program - A focus on the strategic combination of 1st & 2nd generation strategies for a particular disciplinary context (School or Program) Prof Keithia Wilson ALTC National Fellow

9. Understanding the Current Context for the FYE Meta Model 1 - FY Transition Practice Student Diversity Student Transition * Course Design * Course Delivery * Course Assessment Prof Keithia Wilson ALTC National Fellow

10. Student diversity So what is important to understand about student diversity? • Defining student diversity • Assumed knowledge Prof Keithia Wilson ALTC National Fellow

11. What is Student Diversity? Traditional Students (TS) medium-high SES second generation higher entry levels full time on-campus Elite Model Non-Traditional Students (NTS) low SES first-in-family lower entry levels full-time & working not on-campus much Indigenous NESB International, refugees disability home care responsibilities from rural & remote settings Mass Model Prof Keithia Wilson ALTC National Fellow

12. What is assumed knowledge & capabilities? Academic Skills Information Literacy Computer Literacy Reading Skills Written Communication Numeracy Skills Critical thinking & analysis Independent Learning (self-regulation) viz. time on task, self-study, time management, uni-work-social life balance, successful student behaviour Prof Keithia Wilson ALTC National Fellow

13. What is assumed knowledge & capabilities? Academic/Cultural/Social Capital – the “Hidden Curriculum” Role Understanding student role expectations & appropriate & effective behaviour (rights & obligations) Success Reading the academic context to accurately determine performance requirements re studying & assessment Support Capacity for help-seeking without fear of negative labelling (dumb/stupid) Personal Identity Sense of belonging & personal fit with university (overcome the “outsider within” phenomenon – “A stranger in a foreign land”) Prof Keithia Wilson ALTC National Fellow

14. Griffith University context & student profile • Student enrolment of 40,000 • 70% of students are first-in-family at uni (NT) - FIF correlates with low SES & lower entry scores/Ops • 6th largest low SES student intake in Australia (16% overall, 30% Logan campus) • 3rd largest Indigenous student intake in Australia (600) • 30% International student enrolment (N’thn Europe, China, India, Indonesia, Middle East, Canada) • 10% students studying in distance mode • More of our students work in paid employment & longer hours in employment than the national average (NTS reality) Prof Keithia Wilson ALTC National Fellow

15. Are NTS capable of being successful at university? The national research evidence shows - The success rate (or tendency to pass their year’s subjects) of low SES students is 97% of the pass rates of their medium & high SES peers & has been stable over the last 5 years (Bradley et al, 2008:30) However, they require higher levels of support to succeed e.g.,financial assistance, academic support, mentoring & counselling services (Commonwealth of Australia, 2009) Prof Keithia Wilson ALTC National Fellow

16. Therefore....implications for assessment practice • Taking account of assumed entry level knowledge in unit content & assessment design & making this explicit to students • Building foundational academic skill development into unit content & assessment design • Developing commencing students skills as independent, self-regulating learners across the first year • Creating a unit & program level culture that respects & values diversity in all of its forms (age, gender, race etc.) Prof Keithia Wilson ALTC National Fellow

17. Student Transition So what is important to understand about Student Transition? • An evidence-based model for conceptualising student transition • Senses of success (Lizzio, 2006) • Predictors of student success Prof Keithia Wilson ALTC National Fellow

18. Meta-model 2: The ‘Five-Senses’ of Student Success (Lizzio, 2006) Sense of Capability Sense of Connectedness Sense of Student Identity Sense of Purpose Sense of Resourcefulness Prof Keithia Wilson ALTC National Fellow

19. What predicts commencing students’ satisfaction with their degree program? Sense of Purpose Strongly Enhances Enhances Sense of Capability Commencing Student Satisfaction Sense of Connection Enhances Good Teaching Enhances Perceived Effectiveness of Orientation Enhances Enhances Time on Task Prof Keithia Wilson ALTC National Fellow

20. What predicts commencing students’ academic outcomes? Academic Capital Low SES First in Family ESL Reduces Competing Demands Time in employment Time as carer Reduces Semester 1 GPA Task Engagement @ Uni Attendance at Orientation Time on task Strongly Enhances Prior Academic Achievement Entry Scores (OP) Enhances Prof Keithia Wilson ALTC National Fellow

21. What predicts commencing students’ retention? Academic Capital - Competing Demands - Semester 1 GPA Student Retention into yr 2 Task Engagement @ Uni + Prior Academic Achievement + Sense of Purpose + + + Student Satisfaction + Prof Keithia Wilson ALTC National Fellow

22. Therefore....Implications for practice • Curriculum strategies for developing time on task & self-regulation skills  assists assessment success • Strategies for enhancing sense of purpose in curriculum & assessment practices • Building academic & social capital through unit & assessment design throughout the FY • Effective Orientation Programs & early student engagement strategies to encourage attendance at Orientation Prof Keithia Wilson ALTC National Fellow

23. Step 2 – Assessment Principles & Practices • So...what do students say about their experience of assessment? A quick overview of some salient research evidence! Prof Keithia Wilson ALTC National Fellow

24. Assessment characteristics which positively influence student learning & engagement • perceived validity of assessment tasks & approaches • perceived extent to which the learning environment is empowering • perceived fairness of the learning environment, especially with assessment tasks & practices Prof Keithia Wilson ALTC National Fellow

25. Students’ perceptions of the validity of assessment tasks & approaches (Sambell et al., 1997) 3 sets of priorities - • Educational values – authentic/meaningful tasks, perceived to have long term benefits, applying knowledge • Educational processes – reasonable demands, encourages independence by making expectations clear • Educational consequences – rewards effort and breadth & depth in learning Prof Keithia Wilson ALTC National Fellow

26. Students’ meta themes in assessment(Savin-Baden , 2004) 2 forms of student disempowerment: • Unrewarded learning – the relationship between quantity of work & its weighting • Disabling assessment mechanisms including: • Processes – lack of information & inadequate feedback • Forms – assessment methods that do not fit with espoused forms of learning Prof Keithia Wilson ALTC National Fellow

27. Students’ perceptions of fairness(Lizzio, Wilson & Hadaway, 2008) Strongly influenced by – • Extent to which they feel personally respected by academic staff (convenors & sessional staff) in the learning & assessment process – relational culture • Adequacy of the information & support systems provided for them to “do their job” in relation to assessment Prof Keithia Wilson ALTC National Fellow

28. The Reality of Assessment for Commencing Students Student feedback from multiple sources indicates: • Strategic nature of Assessment for student engagement, success & retention • For many students assessment IS the learning • Assessment items which are too difficult, not meaningful, and not clearly explained are a key cause of student drop-out in the first semester • Well designed & implemented assessment is key to student engagement, learning & retention Prof Keithia Wilson ALTC National Fellow

29. Assessment Practice in the First Year So what is important to our understanding? • A model of purposes of FY assessment & student motivation with assessment • A model for management of the FY Assessment Lifecycle informed by : • Student perspective • Staff perspective • Principles of Good Practice Prof Keithia Wilson ALTC National Fellow

30. Purposes of FY Assessment • Diagnostic  • Transition enabling  • Motivating  • Formative  • Learning  • Summative/evaluative  • Terminal  • Meta-reflective  Assessment of student readiness Assessment to aid transition to & engagement with uni Assessment as stimulation for learning Assessment for learning Assessment as learning Assessment of learning Assessment of capability/mastery Assessment of learning process Prof Keithia Wilson ALTC National Fellow

31. Student view of ‘motivating assessment’ (Wilson & Lizzio, 2011) • Engaging Design Relevance (personal, academic, professional), intellectual challenge, teacher enthusiasm • Enabling Management Task clarity, task scaffolding, formative conversations with & support from staff • Teacher Authority Clear and firm expectations & standards, consequences explained, responsibility invoked Prof Keithia Wilson ALTC National Fellow

32. Meta-model 3: The first-year assessment lifecycle – student experience 6. Academic Outreach and Recovery Prof Keithia Wilson ALTC National Fellow

33. The first-year assessment lifecycle – staff experience Design Phase 6. Academic Outreach and Recovery Post-Assessment Phase Student Preparation Phase Assessment Phase Prof Keithia Wilson ALTC National Fellow

34. Principles of Effective Practice in First Year Assessment Good Practice Principles rely on: 1. A lifecycle approach to assessment (4 Phases) • Design phase • Student Preparation phase (pre-submission) • Assessment phase (marking) • Post-Assessment phase (feedback & feed-forward) 2. A system’s approach to assessment (3 Systems levels) • Individual Unit/Course/Subject level • First Year Program level – horizontal & vertical integration • Degree Program level Prof Keithia Wilson ALTC National Fellow

35. 1. Design Phase: Individual Unit/Subject Fit for purpose – Optimising student motivation & engagement by designing assessment tasks which are perceived to be relevant & meaningful to students & their learning (sense of purpose) Level of difficulty & complexity – Designing tasks with a conscious understanding of the assumed entry level knowledge & appropriate learning level of commencing students Progressive knowledge & skill development – Designing assessment tasks across a semester in such a way as to ensure effective, cumulative knowledge & skill development Variety – Ensuring a variety of different types of items both within a unit/subject, and across units /subjects in the same semester (FY program level planning) Prof Keithia Wilson ALTC National Fellow

36. 1. Design Phase: Individual Unit/Subject Early success -Optimising an experience of ‘early success’ for students to build academic & personal efficacy & confidence Formative Assessment - Emphasising early formative assessment designed to develop skill & confidence viz. Early – weeks 4-6 is optimal Smaller pieces for either no marks, or fewer marks (10-20%) to encourage recovery from possible failure (building hope) Speedy, quality feedback (peer or staff, individually or collectively) with feedback by weeks 7-8 Self-assessment of all written items by self-evaluating on the identified criteria (builds meta-competence) Prof Keithia Wilson ALTC National Fellow

37. 1. Design Phase: Individual Unit/ Subject Level • Written presentation of assessment tasks which present the task clearly and without ambiguity– • Start with the aim of the task • Present the sequence of steps involved in the task, from the simple to the complex, thus breaking down large tasks into manageable chunks • Refer students to the marking criteria for the task • Ensure that all available information is provided as much as possible in the one place Prof Keithia Wilson ALTC National Fellow

38. 1. Design Phase – Essays – a “special case” There are a range of data sources to indicate that essays in the first year are a source of great angst for both students and staff, and may be more complex than we (staff) think : • Professional Staff feedback • Academic Staff feedback • Student feedback • Quality of FY essays, especially in the first semester • Quality of student essays in the second & third years & beyond! • Wilson & Lizzio – Findings from ALTC Grant 2009-2010 Prof Keithia Wilson ALTC National Fellow

39. Assessment Tasks – exams & laboratory reports Engaging Assessment Design Deep Approach to Learning Assessment Outcomes: Student Grades Assessment Management & Support Assessment Task Efficacy Prof Keithia Wilson ALTC National Fellow

40. Assessment Tasks– Oral Presentation Engaging Assessment Design Deep Approach to Learning Assessment Outcome - Student Grades Assessment Management & Support Assessment Task Efficacy Prof Keithia Wilson ALTC National Fellow

41. Assessment Tasks - Essay Engaging Assessment Design Deep Approach to Learning Assessment Outcome - Student Grades Assessment Management & Support Assessment Task Efficacy Prof Keithia Wilson ALTC National Fellow

42. 1. Design Phase – Essays – a “special case” • Semester 1 – essay tasks that are: • Relatively short (e.g., 750-1000 words) • Relatively simple (e.g., summary/description & simple interpretative analysis such as answers to how or why questions) • Include limited research requirements (e.g., simple searches of readily obtainable information) • Semester 2 – essay tasks that are: • Longer (1,200-2,500 words) • Incorporate more complex analysis (e.g., critical analysis & evaluation tasks – compare & contrast) • Include more complex research requirements Prof Keithia Wilson ALTC National Fellow

43. 1. Effective FY Assessment Practice Principles – Design Phase: FY Program Level Workload distribution – Ensuring roughly equal workload between all units in a semester to assist development of time management skills Scheduling – coordination of submission dates to stagger the student workload Threshold/Difficult Units - Identify & front-load any threshold units in a semester to enable student success Variety of assessment types – Ensure variety across units in a semester Group Work – Coordinate any group-based assessment tasks across the FY Program & preferably reduce to 1 unit only if the assessment requires additional meetings outside of class Prof Keithia Wilson ALTC National Fellow

44. Meta-model 4: Progressive Enabling: How might we effectively manage the assessment process? Systems Awareness Help me by coordinating the experience Task Design Provide me with ‘fit for purpose’ tasks and roles Self-Management Help me by managing yourself Prof Keithia Wilson ALTC National Fellow

45. Reflective Task: Assessment Design Phase • How does this information apply to your FY context? • What are the opportunities for enhancing FY assessment design? • What are the challenges with enhancing FY assessment design? Prof Keithia Wilson ALTC National Fellow

46. 2. Student Preparation Phase: Individual Unit/Subject Level Motivating students – 3 Domains of Motivation • Articulating explicitly the academic relevance of each assessment task (the knowledge and skills that will be useful to students later in their course/degree); • Articulating explicitly the personal relevance of each assessment task (the knowledge and skills that will be useful for students in the future); • Articulating explicitly the professional relevance of each assessment task (the knowledge and skills that will be useful to students later in their career); Prof Keithia Wilson ALTC National Fellow

47. 2. Student Preparation Phase: Individual Unit/Subject Level Motivating students – 2 Types of Relevance • Articulating future positive relevance (personally & professionally) of the knowledge and skills to be gained from an assessment task to the course/degree/future employment; • Articulating future negative relevance of the knowledge and skills to be gained from an assessment task to the course/degree/future employment for students who may not engage sufficiently; Prof Keithia Wilson ALTC National Fellow

48. 2. Student Preparation Phase: Individual Unit/Subject Level Motivating students – Challenge • Articulating explicitly the intellectual challenge of each assessment task (the challenge offered to students to think and learn); • Articulating to students the investment of work required to be successful with the task (encourages development of self-regulatory behaviour); Staff Stance • Staff conveying to students their own personal enthusiasm for the task (staff engagement increase student engagement); • Being clear and well organised (assists anxiety management & increases performance ability). Prof Keithia Wilson ALTC National Fellow

49. 2. Student Preparation Phase: Individual Unit/Subject Level Management Strategies - • Scaffolding learning -Designing a process for scaffolding assessment preparation which prepares students for each assessment task: • Providing detailed goals, criteria, standards for each task to clarify what good performance is – viz. clear goals & standards • Providing opportunities for students to actively engage with, & potentially modify those goals, criteria, standards • Providing high & low quality examples of performance for each assessment task or a similar task • Providing targeted resources such as practice items, quizzes, mini-essay writing, step-by-step processes organised sequentially etc. • Providing multiple & regular opportunities for discussion of assessment tasks & requirements • Ensuring consistent information& resources are provided from all teaching staff (convenors & tutors) on assessment tasks Prof Keithia Wilson ALTC National Fellow

50. 2. Student Preparation Phase: First Year Program Level • Clear Goals and Standards – Ensuring all course convenors have detailed criteria for all assessment tasks in their units/courses • Responsive culture – Ensuring a responsive FY Program culture in all units/courses, including training of sessional staff  consistent messages about performance & success • Consistencyof referencing style – Providing a single referencing style only for FY students for their FY of study for all units/courses in their Program • Consistency of information storage by unit convenors for web-site information for all units (same folders) • Consistent terminology to describe same types of assessment tasks across a program e.g., critical reflection/critical analysis/essay/critique Prof Keithia Wilson ALTC National Fellow