The University of Melbourne 11 May 2015

1. The University of Melbourne 11 May 2015 Not just another diagnostic test! Michael Jennings, Deb King (Melb), Adam Bridgeman (Syd), Liz Johnson (Deakin)

2. Today’s talk is brought to you by… • … the Australian Government’s Office of Learning and Teaching.

3. B.Engineering Numbers Table 1. Bachelor of Engineering first-year numbers (UQ enrolment data) Prereqs: 1991 - English, Maths 1 (Maths B), Maths 2 (Maths C), Physics, Chemistry 1995 onwards - English, Intermediate Maths (Maths B), Physics OR Chemistry Approx. 70% of students come in with both Inter. & Advanced Maths (B & C)

4. History of diagnostic testing at UQ • Leo Howard Test – 1972 to1994 • Paper test – 2007 • Online test – 2009 to present • Science diagnostic test – 2015

5. Current Engineering test – 2009 to present • UQ T&L grant – cross-discipline team of Engineering & Science (inc. Maths) • Background questions • F-10 maths & Senior Intermediate Maths • Physics • Chemistry

6. Current Engineering test – OLT grant • Led by USQ, involving UQ, UNE, UTS, and UNewcastle • http://getset.uqcloud.net • 20 maths questions • 18 MCQs, 2 enter the answer • Can’t remember & never seen before options 12 physics and 12 chemistry questions

7. Aim of the engineering test • Is NOT to tell students how much they don’t know / forgotten • Is to gauge current mathematical / science understanding • Is to tell students what maths/science is used in their courses so they can choose the appropriate courses – feedback matrix • Is to identify at-risk students

8. Current Science test – OLT extension grant – 2014/2015 • Led by UQ, involving Melbourne, Sydney and Deakin • https://getset.science.uq.edu.au • 19 maths/statistics questions • all MCQ • Can’t remember & never seen before options

9. Results – engineering test • Consistent over the years • Students who have done both Intermediate and Advanced Maths (Maths B & C) at school do better • Senior topics not done as well as junior ones • Differentiation and integration • Correlation between test results, ATAR (OP) and success in first-year engineering

10. Engineering students’ thoughts

12. Kavanagh, L., O’Moore, L, & Samuelowicz, K. (2009). Characterising the first year cohort knowledge. Proceedings of the AAEE 2009 conference. Burton, L., Dowling, D., Kavanagh, L., O’Moore, L, & Wilkes, J. (2012). Examining First Year Students’ Preparedness for Studying Engineering. Proceedings of the AAEE 2012 conference.

13. BSc student’s feedbackSemester 1, 2015

14. 2009

15. 2012 Results

16. 2013

17. Feedback matrix • https://getset.science.uq.edu.au

18. OLT Extension Grant 2014-2015 • Melbourne, Sydney, Deakin science, maths and business programmes • UQ Science • Macquarie • Southern Cross, Charles Sturt, UWS, Curtin, Griffith, USQ, UTS, UNSW

19. GetSet 2 – what’s new? • Core questions AND options CORE Option 1 Option 2 Option 3

20. You want GetSet? You got it! • However, you need: • TIME • $$ • UQ Science 2015 • $43,000 grant – $8000 per discipline

21. What maths is in your course(s)? • What do you want to know? • Procedural vs conceptual understanding. • With what do you think students struggle? • When are you going to do the test? • What are you going to do with the info from the test? • Post-test? End of first-year test? Major test? • QS in Science – www.qsinscience.com.au

22. Acknowledgements • The 2009 online diagnostic test was funded by a The University of Queensland Teaching and Learning grant. The project team was A/Prof Lydia Kavanagh (Faculty of Engineering, Architecture and Information Technology), Dr Liza O’Moore (School of Civil Engineering), Professor Peter Halley (School of Chemical Engineering), Professor Paul Lant (School of Chemical Engineering), Professor Peter Adams (Faculty of Science), Michael Jennings (School of Mathematics and Physics), Professor Lawrie Gahan (School of Chemistry & Molecular Biosciences) and Dr Anton Raynor (School of Mathematics and Physics), . • The OLT funded project involved USQ, UQ, UNE, UTS and UNewcastle. http://www.olt.gov.au/project-get-set-success-using-online-self-assessments-motivate-first-year-engineering-students-engag • The feedback matrix was designed by A/Prof Lydia Kavanagh (Faculty of Engineering, Architecture and Information Technology), Dr Liza O’Moore (School of Civil Engineering).

23. References • Burton, Lorelle, Dowling, David, Kavanagh, Lydia, O'Moore, Liza & Wilkes, Janelle (2012). Examining first year students' preparedness for studying engineering. In: Llewellyn Mann and Scott Daniel, AAEE 2012 Conference Proceedings. 23rd Annual Conference of the Australasian Association of Engineering Education (AAEE 2012), Melbourne, Australia, (1020-1028). 3 - 5 December 2012. • Jennings, M. (2011). The transition from high school to university: The University of Queensland perspective. In: John Hannah, Mike Thomas and Louise Sheryn, Te Ara Mokoroa: The Long Abiding Path of Knowledge - Proceedings of Volcanic Delta 2011. Volcanic Delta 2011: The Eighth Southern Hemisphere Conference on Teaching and Learning Undergraduate Mathematics and Statistics, Rotorua, New Zealand, (139-149). 27 November - 2 December 2011. • Kavanagh, L., O’Moore, L., & Samuelowicz, K. (2009). Characterising the first year cohort knowledge. 2009 Conference Proceedings. Annual Conference of the Australasian Association of Engineering Education (AAEE 2009).