Research into the effectiveness of learning through ICT for people with basic skills needs

1. Research into the effectiveness of learning through ICT for people with basic skills needs Dr Harvey Mellar, Dr Maria Kambouri, Professor Alison Wolf, Tina Goodwin, Violet Windsor Pavlos Koulouris, Annette Hayton

2. Reports Ufi, the DfES and the Basic Skills Agency have been involved in a research project looking at the effectiveness of ICT as a learning tool for people with basic skills needs. The project is now complete and has resulted in the production of a guide for learning centres (Using ICT to develop literacy and numeracy: A guide for learning centres working with adult learners) as well as a research report. • http://www.ufi.com/partners/hubandlearning/Ictnumeracy.htm

3. Overview • Introduction • How is ICT being used? • Is the use of ICT in learning attractive? • How effective is ICT in promoting learning? • What are the factors that engender successful learning with ICT? • How can tutors be trained to use ICT effectively? • Conclusions

4. Introduction • Moser Report: "At the heart of improved quality in delivery and materials must be increased use of Information and Communication Technologies (ICT) to improve basic skills" • Brooks, G. et al (2001) Progress in Adult Literacy: Do learners learn? London: Basic Skills Agency.

5. Methodology • 11 centres selected • Interviews: • 71 managers and tutors • 124 learners. • Pre and post testing • numeracy and literacy • two and a half months between pre and post • 70 learners completed the post tests

6. How is ICT being used? • Changing nature of literacy and numeracy • Accessing and using ICT • CD-ROMS and learning packages • Word-processing and office software • Email • WWW • Producing multimedia presentations

7. Incorporating ICT into teaching • Using materials • Student autonomy and the role of the tutor

8. Is the use of ICT in learning attractive? • Increased number of learners

9. Characteristics of learners • 68% male, 32% female • NFER study - 45% male, 55% female • learndirect learners on basic skills courses -40% male 60% female • 46% of post-interview sample have access to IT at home (36% of pre-interview sample)

10. Characteristics of learners - level

11. Reading levels

12. Characteristics of learners - age Mean age 32

13. Is the use of ICT motivating? • 92% of learners thought so • Motivating aspects of ICT use • self esteem • keeping up with the changing times • different from previous learning experiences • rewards and accreditation • games aspects • Some demotivating aspects

14. Using ICT • Starting to use ICT • Changing experience of ICT use, the web is: • a source of information for older learners • a source of entertainment for younger learners

15. How effective is ICT in promoting learning? • Learning • interviews with learners • tests • Retention • Monitoring and communication • Computer based assessment

16. Learner interviews • 64% thought that ICT had had a significant impact on their learning • 45% expressed strong reliance on the tutoring arrangements • 20% expressed their willingness or desire for more independent activity and saw the computer as an aid to this

17. Learner interviews

18. Comments • Meeting their expectations as regards literacy, but not numeracy • Making gains in ICT skills even when these had not been one of their initial learning goals • Expectations about preparation for specific areas of work were often not being met

19. ICT or Literacy? • Occasionally there was some evidence of tension between the use of ICT and the learning of literacy and numeracy

20. Changing aspirations • 26% indicated that their work aspirations had altered so as to include a desire for a greater involvement with ICT in their work

21. A story John is 35, and has a 7 year old daughter, he used to work in the motor industry but is now unemployed. He has used his girl friend’s computer to produce some fliers for an event he was organising and to play some games. He is aware of his difficulties with literacy and numeracy. He had tried going to a learning centre in a college before but he drifted away quite quickly. He decided to give it another try. The college put him on to a computer based assessment program. – he had not expected to use a computer and he quite liked the fact that he was using it. He felt he was learning how to use computers as well as doing the work he needed to do.

22. Over the next couple of months he worked through a number of computer based learning units. During this time he is going home and talking with his daughter and girl-friend about what he is doing with computers and comparing it with their experiences of using computers in school and work. Then he moves on to using the World Wide Web and email in projects - he enjoys this because it further extents his use of computers. He is starting to feel like an expert and is helping other learners in the centre with their use of ICT. He starts to talk about taking a course about the internet, he moves on.

23. Testing • Average time between pre and post-test was two and a half months • Sample sizes were small and results are indicative only

24. Numeracy • No significant difference in average score • Two further specific hypotheses were tested: • that ICT use might reinforce place value/arithmetical operations in particular • that doing literacy as well as numeracy might improve understanding and interpretation of graphs and figures. No significant differences were found

25. Literacy: Entry level 1/2 • Very high standard deviations for change scores, large and erratic changes • Reading - no significant differences found • Writing - a small but not statistically significant improvement in grammar

26. Reading: Entry level 2/3 Change: mean 1.97, SD 4.05 Significant p<0.05 Equivalent to an improvement of about 7 percentage points.

27. Writing: Entry level 2/3 • Small increase in word totals • Some improvement in spelling and grammar • Noticeable increase in complexity of writing, as measured by number of clauses

28. Effect size • EFFECT.XLS

29. Literacy: Level 1/2 • Reading - no significant changes • Free writing - slight improvement in grammatical accuracy • Rewriting - definite improvement in grammar, spelling and punctuation (but not statistically significant)

30. EAL • Reading - no significant change • Writing • a definite increase in number of words used for middle band (Entry 2/3) • no consistent pattern of change in word length, complexity or spelling • some improvement in grammar for the lower band (Entry 1/2) • slightly more spelling mistakes

31. Retention • The retention rate between pre and post literacy tests was 64% • NFER study had a retention rate of 57% for the reading and 54% for the writing test

32. Monitoring and communication • Little effective use of ICT for monitoring, and no integration with Educational Management Information Systems • Positive response by both learners and tutors to the communication possibilities opened up by ICT

33. Computer based assessment • Learners - positive (though could be time consuming) • Tutors - divided opinions

34. What are the factors that engender successful learning with ICT? • Pedagogic factors • assessment • learning styles • integration in teaching • learners’ views on ICT and learning • tutors’ views on ICT and learning • Organisational factors

35. Learners’ views on ICT and learning • ICT helps you to learn • ICT helps you to concentrate • ICT gives you feedback • ICT enables you to do your work more quickly • ICT enables you to produce better presented work

36. Tutors’ views on ICT and learning -1 • ICT provides a wide range of multimedia resources • ICT enables the tutor to individualise materials • ICT enables concentration • ICT provides practice

37. Tutors’ views on ICT and learning - 2 • ICT removes the need for handwriting • ICT supports literacy skills • ICT supports speaking and listening • ICT supports collaborative work • ICT encourages independence

38. Organisational factors • Tutors • Accessibility • Physical organisation • College culture • Technology management • Planning

39. Case studies of centres • College: vocational focus • College: communication workshop: • Community based class • Outreach project • Prison

40. College: vocational focus • Strong commitment by management, and strategic college level planning • Specific vocational focus. Intensive courses • Full time, well trained tutors, working in teams, weekly meetings • Ready access to high quality computer suites • Basic skills suite integrated in college facilities • Tutors have strong relationship to technology staff. Involvement of staff in planning. Technical support understands needs of basic skills learners.

41. How can tutors be trained to use ICT effectively? • Do generic trainers need special skills to provide support to people with basic skills needs? • Do basic skills tutors have the necessary skills to make effective use of ICT in their work? • ICT training for tutors of basic skills

42. Conclusions - 1 • ICT has a crucial role in enticing people into learning • There is some evidence for the impact of ICT use on literacy (but not numeracy) • For most learners the role of the tutor is very important, but there is a sizeable group for whom greater autonomy is promised by the use of ICT • ICT use in basic skills provision has not yet reached the critical threshold in terms of access, development of pedagogy, or tutor training for really effective implementation

43. Conclusions - 2 • There is a need for tutor training for basic skills tutors: • basic ICT skills • specific pedagogic approaches to integrating ICT into basic skills provision • the impact that ICT might have on the nature of learning

44. Further research • The impact of learndirect products • Intervention studies • Observational studies • Development of a principled approach to instructional design. • Reluctance of some learers to use ICT in their learning • Reluctance of some tutors to use ICT