Margaret Macleod Keithia Wilson Griffith University

1. Quality Improvement of First Year Assessment:A process and template for the review & enhancement of assessment design & management Margaret Macleod Keithia Wilson Griffith University

2. Acknowledgment to Country In the Spirit of Reconciliation Following on from Sorry Day I would like to acknowledge & honour the Traditional Custodians of the land we are meeting on today, Turrbal and the Jagera People, and pay respect to their Elders past & present And acknowledge the contribution of our First Nation Peoples to Higher Education & learning Keithia Wilson GU - May 2012

3. Introductions Margaret Macleod: • Currently the Griffith Health Group Assessment Consultant • 10 years previous experience as Learning Adviser at Griffith University, Logan campus Prof Kethia Wilson: • Program Director of the FYE, Griffith Health • ALTC National Fellow for the FYE (2010-2012)

4. Session Overview 1. Mini-input • Context for the intervention • Literature - Importance of assessment • Assessment review process • Assessment review template • Evaluation & Outcomes • Reflection & group discussion 3. Final summary of themes

5. Griffith University context • Large metropolitan university in Brisbane (1 of 7 in S-E Qld, & 1 of 4 in Brisbane) • Multi-campus - 5 campuses x 60 k corridor • Student enrolment of 40,000 • 70% of students are first-in-family at uni - FIF correlates with low SES & lower entry scores • 6th highest low SES student intake in Australia • 3rd highestIndigenous student intake in Australia • 25% International student enrolment • 10% students studying in distance mode Keithia Wilson GU - May 2012

6. Why assessment matters! There is very strong evidence in the literature to suggest – • Assessment is a central feature of curriculum • Assessment practices have a profound effect on student behavior (Entwistle, 1991) • Assessment is believed to have a more powerful influence on student behavior than teaching (Boud, 2012) • Assessment has also been identified by leading educators as one of the least sophisticated aspects of learning and teaching in higher education (James, 2010)

7. In Summary “While students can with difficulty escape from the effects of poor teaching, they cannot (by definition if they want to graduate) escape the effects of poor assessment”. (David Boud, 1995, p 35)

8. Effects of Poor Assessment The effects of poor assessment have been well documented in the literature (Rust, 2002) – • Student disengagement from learning, and • The development of negative attitudes towards learning

9. Impact of poor assessment on Non-Traditional students Anecdotal & observational data from Professional staff (e.g., Learning Advisors) indicates that poor assessment contributes to students, particularly non- traditional students, experience • a sense of personal inadequacy, • disempowerment, • perceived inability to negotiate the complexities of university culture, and • lower retention rates.

10. First Year Students’ Appraisals of Assessment Wilson & Lizzio (2012) ALTC Grant Commencing students evaluate their assessment tasks in terms of two factors: • The motivational value of the task, and • Its manageability, or the extent to which the task is scaffolded, with resources provided to enhance engagement and understanding.

11. Key Findings for FY Assessment • Students’ perceptions of the motivational value of assessment tasks are key in predicting student engagement, • Students’ perceptions of the effectiveness of the design of the assessment task are key in predicting their levels of motivation for the task, • Student evaluations of task manageability are key in predicting student’s sense of self-efficacy or confidence in undertaking the assessment task.

12. Dual Assessment Factors guiding the Review Process Thus, the dual factors of • Effective /appropriate task design • Combined with processes for managing the assessment experience and scaffolding student understanding of the assessment task are core to a positive, empowering and potentially successful student experience with assessment Reinforced by feedback from Professional staff, in particular, Learning Advisors

13. Common problems students experience with assessment • Difficult, ambiguous or complex tasks • Tasks that lack relevance to learning outcomes or ‘sense of purpose’ • Misleading or generic marking criteria • Inconsistencies in task information • Insufficient resources to help scaffold tasks • Inappropriately timed tasks

14. Review Process Overview Academic-professional staff partnership • Worked together to design and implement a process to review and enhance first year assessment, and work effectively with academic staff • Macro (FY Program) and micro (Individual tasks) level review and enhancement of first year assessment • Undertaken in two Schools in Griffith Health in 2010 Strategy: • Focus on student voice and understanding of student experience of assessment • Academic staff engagement • Design of template to evaluate written assessment tasks • Report, including recommendations and resource development • Change process combining FYE Team & Individual Convenor discussions for decision-making at FY Program & Course/Unit levels

15. The Review Process: Design of Templates • Macro level Template • Programmatic focus: • comparison of all assessment tasks re timing, type, variety, word length, weighting • Micro Level Template • Individual assessment task focus: • Assessment design • Scaffolding of tasks and targeted Resources Final report: Overall summary and detailed evaluation of written assessment in all first semester, first year courses in each School

16. Margaret Macleod, GU, 2011

17. The Review Process: Academic staff engagement • Ensuring academic support: Creation of FY leadership team, comprising Dean L&T (Health), HOS/DHOS, Degree Program Convenor/s, First Year Advisors, & FY Course/ Unit Convenors Creation of FY Enhancement/Management team, comprising FY Leadership Team + FY Course/Unit Convenors • Facilitating staff buy-in & engagement: Consultation process with FY staff prior to review to seek commitment & agreement (or otherwise) to the audit process 3. Conducting the Assessment Review: Accessing Course/unit outlines for analysis Preparation of Final Report • Facilitating the feedback process: Macro level: programmatic discussions with FY leadership teams to discuss report, share information & make FY Program decisions Micro level: Learning Adviser met with individual course convenors to assist in redesign of assessment & creation of resources to scaffold student engagement & learning

18. Assessment Review Outcomes:Macro level audits Program Level changes • Relocation of some more difficult courses & tasks to 2nd year (e.g., research methods) • Reduction of high stakes, highly weighted (e.g., 40 to 50%) first assessment tasks in favour of early, formative low stakes assessment tasks in most courses • Agreement in FYE Teams on total word lengths for semester 1 tasks (from 5-6,000 to 4,000 words); • Reduced standardised weightings for word length (e.g., 1,500 word range from 15 to 45%); • Reduction in number of group work tasks for assessment purposes & later submission dates (not before week 6) • Reduction in number of referencing styles (from 2/3 to 1) • Coordination of submission dates • Blackboard Audit resulting in consistent information storage

19. Assessment Review Outcomes: Micro level audits Individual assessment task changes: • Total redesign of some assessment tasks • Partial redesign of many tasks re task clarity, level of difficulty, variety, relation to course aims, marking criteria • Development of task specific resources to scaffold student learning

20. Assessment Review Evaluation Staff and student outcomes: evaluated using data from commencing students (Starting@Griffith Survey – week 7/semester 1) and feedback from teaching staff (end sem 1) • Student feedback: Increases in 2011 from 2010 in • student engagement between 5 & 7% • student satisfaction between 6 & 8% • student capability up by 10% • perceptions of good teaching between 11 & 14% • These were the highest in the university • Academic staff feedback: • Substantial reductions in student enquiries, student anxiety, & failure rates in threshold courses/units • Positive perceptions of process – requests that review process be continued into 2nd and 3rd years.