I-ACT Institute for Applied Creative Thinking Staffordshire University

1. Dr Geoff Walton Senior Researcher I-ACT Institute for Applied Creative Thinking Staffordshire University Dr Mark Hepworth Senior Lecturer Department of Information Science Loughborough University

2. Fostering higher order cognition to enhance undergraduates’ information literacy

3. Plan • Introduction - Geoff’s motivation for doing PhD • Research project • Intention to build a picture to illustrate what students did during an information literacy learning interventions in an attempt to reveal what their thoughts and feelings were for each stage • Implications for models of information behaviour • Implications for models of information literacy • New assessment tool for measuring levels of information discernment • Concluding remarks – A theory and formulae for further research?

4. Introduction • Geoff’s original motivation to prove the ‘correctness’ of information literacy as a model and ‘expose’ VLEs as the ‘emperor’s new clothes’ • Having completed PhD his view has changed somewhat! • Geoff’s own personal PhD journey • ‘proof’’ and ‘correctness’ are problematic • IL models don’t seem to be sensitive to the cognitive and affective processes that impinge on becoming information literate • VLEs, or more generally e-learning, offer very promising pedagogical opportunities

5. Research project – Research question and hypothesis • How do the psychological states associated with information behaviour and thinking help explain the learning processes in an information literacy blended learning and teaching intervention? • Students who participate in online social network learning (OSNL) will demonstrate: (1) a greater degree of knowledge about e-resources (2) a greater ability to evaluate information than those who do not receive this intervention

6. Research project - Methodology • Participants • Sport & Exercise first year undergraduates in first year core module Effective Learning, Information and Communication Skills (ELICS) • Three tutor groups who received the following delivery: • A. Face-to-face workshop plus interactive web tools plus online social network learning (n=12) • B. Face-to-face workshop plus interactive web tools (n=11) • C. Face-to-face workshop only (n=12)

7. Research project – data gathering instruments • Focus-group interviews (Group A) • Questionnaire (Group A) • Post-diagnostic test (All groups) • Assessment - portfolio (All groups) • Annotated bibliography • Reflective practice statements • Reflective essay

8. Research project – coding framework for qualitative data • Codes based on: • Bloom’s taxonomy • Hepworth’s information behaviour model • Moseley et al

9. Research project – face-to-face workshop • Overall context – Roles and norms • students studying at university and bring some prior knowledge with them • Task – Problem-based scenario part of portfolio assignment • Examples in the one hour ‘hands-on’ activity-based workshop mirrored the problem-based scenario for the assessment (see p1 of handout)

10. Research project – face-to-face workshop • Behaviour • Interaction with source • Source character • “looking at the library catalogue”, “search the library” • Source behaviour • “you can look for a certain subject [on the catalogue]” • “you [the tutor] showed us what to do”

11. Sometimes you don’t know the book exists, so you put in a little bit of what your are looking for and you get the matches The keyword we did the searches for Application Comprehension metacognition I have realised that these sources contain information that can help me to develop while studying at university Style state Affective state Being able to find out the books was really interesting You can get involved in the sessions yourselves more hands-on, it was active

12. Face-to-face workshop • Knowledge • Factual knowledge • “it [the library catalogue] allowed me to see what books were available and where I could find them” • Process knowledge • “you don’t necessarily need to know what book you are looking for [on the catalogue], you can look for a certain subject” • New behaviour • “I [now] use the online library to search for e-books and books”

13. Online social network learning • Task - to find out how to evaluate web-pages by discussing the issue online • Used a questioning approach for example, • So, how would you decide what makes a good quality web page? • Structure of process (see p2-3) • Behaviour • Interaction with sources • peers, tutors, Berkeley website on evaluating information, the Internet Detective, and the tutor-summary for example, “we had to feedback on each others. I remember I was commenting on his [posting], he was commenting on mine” • See examples of a postings on p3-4

14. “check the author or who it was created by” , “evaluate the web page by looking at other pages with the same topic and compare”, “look at the last time when the web page was updated” Allowed me to see what other people thought of the web site, the way they had evaluated it, not just myself. [...] You got to see what you was (sic) missing out or something you hadn’t looked at, so they could bring up the points saying may be look at at this.” Analysis Questioning metacognition Style state Affective state somebody commenting on your evaluation could highlight things you’d overlooked [...] you always think your own work is perfect, sometimes it’s a bit of an eye opener when somebody says you should have done this, gets you thinking about it I think it was quite interesting gets you involved as well and that side of it was quite fun - as opposed to being lectured to” [The tutor summary] gave the whole group a bit of recognition [...] you read through what other people thought of URLs and took advice from other people not just the lecturer’s, it worked really well, it is a good way of reflecting what you’ve done

15. Online social network learning • New knowledge • Both in terms of postings to the discussion and the tutor summary and final summary handout – synthesised output from discussion (see p3-6) • Transition to a feeling of less uncertainty • “It makes you aware, a little bit more aware of what web sites are more useful to you than others and there are quite a lot of web sites on line and you don’t want to be writing stuff in your assignments that’s not true. [Before] I didn’t know what the things at the end like .ac. org meant […].” • (Changed) Behaviour • “I have used [the evaluation criteria] actually, since we did it for essays and stuff, since we did these things in Effective learning it actually alerted me to what to look for when looking for a good web site and what to steer clear of.”

16. Commercial break • All and more in the book! • Practical examples of learning and teaching interventions underpinned by theory • Face-to-face learning • Online learning • Based on empirical research • Also, Journal of Documentation article in 2011

17. Assessment • Task • Answer the problem-based scenario see p1 • Behaviour • Found six resources, two books, two journal articles and two web pages, evaluated them and then wrote about it • Written reflective statement

18. I know about scope, audience, timeliness, scholarly vs. popular, authority documentation and objectivity. I have learnt how to judge how good a book or a journal is against this criterion. Application Evaluation I would evaluate a web page by; looking at the ULR address Synthesis Analysis It [URL] informs you that it is from a popular and reliable source, the BBC Evidence via assessed work transcripts see examples on p7 k

19. Assessment (Reflective statement) • New knowledge • “I have acquired new knowledge on the Library Catalogue, Swetswise, e-journals and the Web. I now know how to look for E-journals and E-books on Swetswise and E-brary, something I did not know how to do before.” • Task completion • See assessed work transcript on p7 • Changed behaviour • “I will incorporate this new knowledge and skills in the future by using these skills when completing a new task e.g. I can use the online library to search for e-books and books.“

20. Assessment (Reflective Statement) • Richness of written language used to express reflections on evaluating information very different between groups

21. Other evidence (quantitative data) • Assessment (annotated bibliography) • One-way ANOVA statistical test • Variety of evaluation criteria • Experimental group used greater variety of evaluation criteria than either groups B or C, but not statistically significant • However, large effect size (using Eta squared test) found – therefore, if this part of the study was carried out again with nineteen subjects per condition it would have produced statistically significant results • Frequency of evaluation criteria • Again, similar result to variety measure • However, large effect size - to gain a statistically significant result in a future study would require between twenty and twenty five subjects per condition.

22. Other evidence (quantitative data) • Post-delivery diagnostic test – 14 multi-choice questions on the library catalogue, e-journals, referencing and evaluating web pages (analysed using one way ANOVA statistical test) • Test scores between Group A (experimental group) and Group C (control group) are significantly different at p < .025, t= 2.66 , Degrees of Freedom (df)= 22 (within groups).

23. Implications for models of information behaviour • a number of cognitive states (knowledge and application) were evident at the face-to-face stage; • the higher cognitive states of analysis, synthesis and evaluation appeared to be most evident for those students who experienced the OSML process; • Three stages in behaviour transition – existing, new and changed • Affective and style states also appeared to be critical factor and the importance of task in maintaining a positive affective and style state leading to new knowledge; • Task completion via the assessed work indicated: • changed behaviour and implied that assessment is an essential component in facilitating deep learning and enabling students to become information literate • the complex inter-relation of these factors in a learning and teaching intervention.

24. Hepworth’s (2004) IB model Key: 1= associated with 2= associated with 3= interaction with 4= has impact on 5= may resolve situation and help complete task(s) Hepworth (2004, p705) Key: 1= associated with 2= associated with 3= interaction with 4= has impact on 5= may resolve situation and help complete task(s) Hepworth (2004, p705)

25. Implications for models of information behaviour (new model) Walton, G & Hepworth, M. (2011, p470)

26. Implications for models of information literacy • They are too rigid and over simplified • The step-by-step approach indicated by some models doesn’t necessarily reflect how the person becomes information literate • They don’t seem to reflect (or harness) the social nature of learning • SCONUL Seven Pillars’ (1999) did not recognise that potentially any student can synthesise information and produce new knowledge, this changed in redesigned model (2011) • Don’t take into account the affective dimension • ‘Grand narratives’ which aren’t sensitive to context

27. Concluding remarks – A theory (see p9) • Becoming information literate appears to be about an individual completing a task in a given context. This context leads to the interaction with sources (e.g., databases, e-journals, books, e-books, peer and tutors etc) and in so doing brings about the interplay of an individual’s behavioural, cognitive, metacognitive and affective states. It is this interplay which determines the level of new knowledge learnt (or produced or both) and the degree of changed behaviour (i.e., level of information literacy) exhibited.

28. Three Spheres of information literacy Becoming information literate takes place in a wider social context determined by roles, norms and tasks Find/ access/ locate Spheres can occur in no particular order Evaluate/ discern Use/ communicate/ produce Each sphere triggers its own set of behavioural, cognitive, metacognitive and affective states

29. Mathematical formulae to focus further research? Key As = Affective state B = Behaviour Bc = Changed behaviour Cana = Cognitive state, analysis Capp = Cognitive state, application Ccomp = Cognitive state, comprehension Ceval = Cognitive state, evaluation Cq = Cognitive questioning state Csyn = Cognitive state, synthesis Kx = Knowledge state Sb = Source behaviour Sc = Source character ΣK = Final knowledge state Mcog = Metacognitive state Ss = Style state • HERE

30. References • Bloom, B. S., Engelhart, D., Furst, E. J., Krathwohl, D. A. and Hill, W. H. (1956). Taxonomy of educational objectives: the classification of educational goals: handbook 1: cognitive domain. New York: David McKay Company Inc. • Hepworth, M. (2004). A framework for understanding user requirements for an information service: defining the needs of informal carers. Journal of the American Society of Information Science and Technology, 55 (8), pp695-708. • Hepworth, M. & Walton, G. (2009). Teaching information literacy for inquiry-based learning. Oxford: Chandos. • Moseley, D. Baumfield, V., Higgins, S., Lin, M., Newton, D., Robson, S., Elliot, J. and Gregson, M. (2004). Thinking skills frameworks for post-16 learners: an evaluation. a research report for the Learning & Skills Research Centre. Trowbridge: Cromwell Press. • Walton, G. & Hepworth, M. (2011). A longitudinal study of changes in learners’ cognitive states during and following an information literacy teaching intervention.  Journal of Documentation, 67(3), 449-479.

31. Selected publications • Walton, G. (2010). From online discourse to online social networking, the e-learning Holy Grail?. In Parkes, D. and Walton, G. (eds.). Web 2.0 and libraries: impacts, technologies and trends. Oxford: Chandos, pp33-65. • Walton, G., Barker, J, Hepworth, M. and Stephens, D. (2007a). Using online collaborative learning to enhance information literacy delivery in a Level 1 module: an evaluation, Journal of Information Literacy, 1 (1), pp13-30. [Online] http://jil.lboro.ac.uk/ojs/index.php/JIL/article/view/RA-V1-I1-2007-2/3 (accessed 17 June 2011). • Walton, G., and Barker, J., Hepworth, M. and Stephens, D. (2007b). Facilitating information literacy teaching and learning in a level 1 sport and exercise module by means of collaborative online and reflective learning. In Andretta, S. (Ed.) Change and Challenge: Information Literacy for the 21st Century. Adelaide: Auslib Press, pp169-202.

32. Dr Geoff Walton Senior Researcher I-ACT Institute for Applied Creative Thinking Staffordshire University Dr Mark Hepworth Senior Lecturer Department of Information Science Loughborough University