In-depth case studies of students’ use of technology to support assessment

1. In-depth case studies of students’ use of technology to support assessment Gráinne Conole Institute of Educational Technology g.c.conole@open.ac.uk Maarten de Laat, Jonathan Darby, Theresa Dillon CAA conference, 5th July 2006

2. LXP project • Aim • To distil out subject discipline issues in using e-learning: • Collecting data on students’ experiences of using e-learning • describing the students’ personal background & learning context • drawing out learner beliefs and e-learning strategies How do learners engage with and experience e-learning? What is their perception of e-learning? What do e-learners do when they are learning with technology? What strategies do e-learners use and what is effective? How is e-learning integrated? How does e-learning relate to and contribute to the whole learning experience? How do learners manage to fit e-learning around their traditional learning activities?

3. Sample • HE Academy subject centres • Medicine, Dentistry and Veterinary Sciences • Economics • Information and Computer Sciences • Languages and linguistics • Data collection • Online survey: • To gain a wider understanding of learner XP • around particular artefacts and select effective • e-learners for case studies • Case studies of individual learners: • Describing e-learning activities carried out by the learner • Exploring e-learner context and background

4. Survey Economics: 134 Languages: 84 Medicine: 35 Computing: 170 Other: 16 Unknown: 8 Delete: 44 Total: 491 Audio diaries Economics:    3 Languages:     47 Medicine:    16 Computing: 19 Total: 85 Interviews Economics:    2 Languages:     2 Medicine:       4 Computing: 4 Total: 12 Data

5. Survey and audio diaries I use email to communicate with everyone, especially lecturers; arranging meetings, asking questions about work and queries over assignments etc I write all my assignments using Word and to sort through the information I find, make notes of what I still need to do and spell check my emails that I'm sending to lecturers. Search engines are used to find news articles I use them to find out information for assignments, and also to help me clarify my notes on each subject area that I study. Instant messaging is used to discuss issues with friends if a topic is not understood Use it to manage my data and thoughts... endnote for referencing spss - stats software - easier Data entry, producing stats The first thing i do when given any piece of word is type it into a search engine! this gives me the opportunity to see how different people interpret the title. from there i can focus on one main idea and use the electronic resources to support my initial findings or indeed rule them out. e-mail is always vital with communicating with different mediums. teachers for support

6. Survey and audio diaries My PDA is useful for reading things when I'm on the move. The use of the internet has replaced many aspects of my studying before I used the internet. I generally start by using the internet for research and then move to books etc when I have the background info I use my laptop to store data and type my course works. The MP3 player serves as a storage media used to save most of my assignments, electronic journals and articles, while I use MS word application to type most of my course works. The electronic library gives me access to books, journals and articles all of which are important for my study Instant messenger and skype for communicating with students, powerpoint for laying out slides and revision notes. Do not use any graphical packages for my studies but do use them for extra-curricular activities

7. Technology essential and integral Use of Internet to find information and for communication Online material to refer to and reflect Technology integral across work and social life Sharing with peers Interviews “I think that technology is becoming more and more important for my studies” “I think is technology also provides a means for me to communicate with tutors and the other students and when we are doing this research training course in the university we have this blackboard and the tutors most of them not all of them some tutors put their notes on blackboard and we can always check back and we can download it before the course started so we can get ourselves familiar with the material and then when we come to the discussion of the course we feel it just make more sense the topic makes more sense to us”

8. Subtle and sophisticated uses Internet as first port of call, difficulty of using paper-based journals Care in selecting appropriate resources; using time online effectively Interviews “If I can get something online I will go for that first because it is always good you can get electronic form to keep in your computer and you can always trace back but for paper – and also you can print out if you want you can read it first and then if it is useful you can print out but if something is not so relevant you can just leave it” “[Communication] through email because I think everyone most other people check their email quite often and so that’s the yeh and also this is the sort of communication media is less interruptive for me because its not like you phone somebody and you are not sure if she or he or she is convenient or not at that moment but you email and he can always pick up when he got time”

9. Opportunities to interact globally: with peers and experts Technology integral to learning Interviews “And you like through this media [Video Conferencing] you bring people together and because sometimes seeing people you can really boost kind of motivation thing because you know who you are talking to and you get more human interaction with it” “Yes sometimes I contact with the tutor [via mobile phone] cos like some tutors busy you have no other way to find them so mobile phone is the only way” “I remember this one day just something wrong with my computer with my internet connection the whole day because I couldn’t get internet so I think finally I sat down and I got a lot of work done so I think that might be one thing one of the caution you have to be careful”

10. Value of online resources Increased use of technology in recent years Easy to use and find information Communicating with friends, peers and tutors Interviews “Once I came here and I am doing my master and phd I started using the technology very very much and it was very helpful and easy and for collecting data and for receiving information” “I have attended more than four online conferences during the last three years and the last one it was an e-learning conference seminar … it was very helpful and the research of their website was the powerpoints which would be downloaded

11. Information Communication Perceptions Environment Uses

12. Perceptions/expectations All comfortable with using technologies See it as integral to their learning Sophisticated and varied use of different communication tools for different purposes Access to up to date and relevant information and resources vital Don’t see technology as anything special – another tool to support their learning Mismatch between institutions perceptions of student use of technology and actual use

13. Perceptions/expections Use of technology at school Computer as central learning tool • Critically aware of the pros and cons of e-learning • Use it where its appropriate – ‘vote with their feet’ • Still value face to face interaction with tutor • Use pen and paper where appropriate • Combination of print and screen based material Aware of varied and relevant resources for their subject area Sophisticated and focused users

14. Information Content Changing nature of content (more available, lower intrinsic value, higher interactivity and presentational standard) Cost and value Materials and information freely available on the Internet Perceived worth and value of material Presentation Use of multimedia to improve presentation expectations of good quality content & material Evaluation New skills needed in terms of assessing worth of content over the internet and ensuring work is that of the student

15. Communication Communication Using tools in a variety of ways to communicate – with friends, family, peers and tutors Use of the internet to access expert knowledge (indirect form of communication) Expectation of being able to communicate with anyone about anything when they want to Collaboration New forms of collaboration possible both with peers and via new ‘smart’ and adaptive technologies – distributed cognition and shared enterprise with tools

16. Environment Media Changing media – increase of USB pens, ipods, mps players, integrated phones, better screen displays for reading Motivation Gaming generation, used to highly, engaging and entertaining environment Interactivity Evidence of a shift from passive to interactive interactions across all aspects of their learning Near ubiquitous Many now have their own PCs and wireless internet access – becoming accustomed to being able to access information or contact people on demand, anywhere

17. Uses Integrated Use of tools in a combination of ways to suit individual needs Evidence of mixing and matching, comfortable with switching between media, sites, tools, content, etc. Provides more flexibility in terms of undertaking learning anytime, anywhere Pervasive Extensive use of technologies to produce find, manage and produce content Used for all aspects of their study Niche adaptive, utilitarian use Increased evidence of use of free self-assessment quizzes to test knowledge, utlising their community of peers – to share resources, ask for help, peer assess Personalised Adapting to suit their own needs Use of computer, internet and books simultaneously Annotation and adaptation of materials

18. Uses Management Sophisticated at finding and managing information (searching and structuring) Use of PC as central learning tool Used to having easy access to information (for travel, entertainment etc) and therefore have an expectation of the same for their courses Transferability Transfer of practices to the learning context: MSN chat, Amazon, ebay, Skype Time Concept of ‘time’ changing – both in terms of expectation of information and results on demand Fragmentation of the learning timetable Changing work patterns New working practices using an integrated range of tools Use of tools is changing the way they gather, use and create knowledge Value and perceived intrinsic worth of knowledge is different Skills – shift from lower to higher levels of Blooms taxonomy

19. Issues for assessment What to asses Devaluing of content and increased interactivity, raises the question of the value of assesses knowledge rather than critical awareness, as well as raising issues about how we can assess process rather than product How to assess Changing practices – multi-tasking and integrated use of tools, resources and community for a variety of purpose Who is being assessed Authentication – of material submitted (plagiarism) and exams undertaken Outcomes What constitutes a valued educational outcome? What is its value and purpose?