Peter Morris & Shane Dowdall Dept. Computing & Mathematics, Dundalk Institute of Technology

1. Student self-assessment and independent learning in a 1st year mathematics module: identifying and responding to needs and challenges through technology Peter Morris & Shane Dowdall Dept. Computing & Mathematics, Dundalk Institute of Technology

2. Background • Where the project started…

3. Background • Where the project started… Image credit: http://www.promathtutoring.com/regents.html

4. Background • Where the project started… SIF 2 ERA ‘Repositioning Learner Assessment Project’ Research Grants 2009

5. Educational Context • B.Sc. in Computing • Level 7 • Common 1st Year • Approx 100 students • Mathematics for Computing 1 • Semester 1 module • Lectures & Tutorials • No Computers!

6. Student Cohort • 67% from Leaving Certificate • 1% > 350 points • 11% > 300 points • 7% < 200 points

7. Student Cohort • 16% Mature Students • 7% International • 7% Further Education

8. Issues • Minimum entry requirements (Leaving Certificate Maths): • B2 grade at Foundation Level • D3 at Ordinary level • E at Higher Level. • Mature students not studied maths for many years • Students entering from further education have no maths requirements • Minimum requirements vs. lecturer’s requirements

9. Aim The aim of this initiative was to improve mathematical fundamentals, for a diverse 1st year student cohort, through the introduction of software that incorporates self-directed online tutorials and assessments for learning.

10. Key Objectives • Inform students of their initial maths competency. • Promote independent learning. • Offer flexible timing of assessments. • Enable students to be assessed quickly. • Build students’ confidence in mathematics. • Improve engagement in mathematics. • Analyse the effect of initiative on students.

11. Literature • Assessment for Learning Petty(2006) • “the most powerful single moderator that enhances achievement is feedback” Hattie (1999) • Benefit of using computers in teaching mathematics • for students who “lack foundation in basic mathematics” Bennett, F. (1999) • Multimedia Learning • Multiple Representation Principle • Contiguity Principle • Mayer & Moreno (1998) • Tutorials & Tests • Alessi & Trollip (2001)

12. Literature • Variables influencing mathematics achievement Pajeres (1996) • Cognitive ability, socio-economic status, prior mathematics achievement… • Mathematical self-efficacy the student’s self-belief in their ability to solve mathematical problems • Ayotola & Adedeji (2009), Hackett (1985), Lent & Hackett (1987) • Mathematical Anxiety occurs when maths “is performed under timed, high-stakes conditions”, Ashcraft & Moore (2009) • Hoffman (2010), Aiken (1970), Ashcraft (2002, 2005)

13. What did we do? • Introduced mathematics software Understanding Mathematics • Visual Tutorials & Cumulative Assessments • Algebra • Fractions • Powers • 2-hour weekly supervised computer sessions • 20% of assessment mark

14. Diagnostic Test 1 & Survey 1 • Inform students of their initial competency • Surveyed them on their response (40 responses) • About half were “surprised” with their result • About ¼ expected to do BETTER • About ¼ expected to do WORSE • Students did not have an accurate perception of their own self-efficacy • 62.5% result acted “as an incentive” to engage with initiative • 0% were not incentivised by their result

15. Test 1 – Sample Questions

16. Test 1 – Sample Questions

17. Supervised Sessions • Wednesday 3pm-5pm • Initially two computer labs (about 40 PCs) • Introduce students to software • Students complete tutorials & assessments • Formal assessments: • Students indicate they want to take assessment • Supervised exam setting • Marks are recorded by supervisor

18. The Software

19. The Software

20. The Software

21. The Software

22. The Software

23. The Software

24. The Software

25. Evaluation: Diagnostic Tests • Diagnostic Test 1: before initiative • Diagnostic Test 2: after initiative • Both tests were similar: • 36 Questions • Paper based • Topics covered by software • No attempt mark • 48 students completed both Tests

26. Evaluation: Diagnostic Tests • For 48 students who sat both tests • Average mark changed from 20.1 to 24.52 marks • 12% increase

27. Evaluation: Tests

28. Evaluation: Survey 2 • Last 3 weeks of initiative • Gauge perceived changes in • Mathematical Self-Efficacy • Mathematical Anxiety • Mathematical Competency • Get students’ response to software

29. Evaluation: Survey 2 Are you less fearful of maths? 63% Yes Has your confidence in doing maths improved? 70% Yes

30. Survey 2: Comments Helping build confidence: “I as a mature student who had been out of school over 25 years found it very good.It allowed me to practice and gain in confidence.” “i would advice people to use as it gives you a confidence boost and reminds you of the maths you may have forgotten from school.” “…good for reminding old stuff from the school” “i thought it was very useful because i hadn't done maths in appox8 years…”

31. Survey 2: Comments Mixed comments for “weak” students: “its a good system to cherry pick the parts of each subject you are weak on and brush up on them.” “it should not be relied upon to teach students who are already weak at maths” “…favors those already good with maths as they fly through it and takes up a good lot of free time for others not so good.”

32. Survey 2: Comments On setting 20% of module’s assessment mark for initiative: “Don't force students to sit this subject by marking them on it. It is unfair if they have no choice but to sit the classes. They are extra classes meant to help those who don't know how or want to get better at maths. Forcing those who are already competent enough makes things [annoying].” “idont like those classes doin them jst because of 20% [otherwise] wouldn't be there”

33. Reflections & Findings • Decoupling of fundamentals from module content • Positive effect on programme in general • Particularly strong effect on mature students • Time consuming for supervisors • Recording of assessments • Providing guidance in using software

34. Reflections & Findings • Poor attendance • Delay in installation • Timetabling issues • Supervised session not on timetable • Competing with clubs & societies • Long gaps in timetable • Split of group into two 1-hour slots • Only 40% of students had undertaken any assessments by week 9 (of 12) • Some did all 3 assessments in one session

35. Issues & Recommendations • Compatibility of software • Issue with IE6 • Use of Virtual Machine • Early Installation • Prepare introductory material • Build confidence from start • Spend more time profiling students • Accessibility of Software • Limited access to the labs • No access at home • Must be considered!

36. Issues & Recommendations • Consider time allocation for supervisor • Video tutorials for introduction to software • Automating capture of results • Identify students who’ll benefit • Is the initiative worth running for all students? • Can you quickly assess students already competent? • Other benefits, e.g. independent learning

37. Future Considerations • Use of IT Learning Centre ONLY for the delivery of the software • Changing class contact hours for module to include practical lab classes • Other degree programs, e.g. engineering • Access to 3rd level courses • Life-long learning & Schools • Institute-wide induction programmes • Pre-entry skills tests (literacy & numeracy)

38. Questions • 12% increase for group as a whole • 38.5% to 66.5% - Group B’s improvement • Mature students gained in confidence • Improvements in mathematical self-efficacy & anxiety Thank you for listening!

39. Evaluation: Diagnostic Tests