"A tool for developing information evaluation skills in a Web2.0 environment"

1. "A tool for developing information evaluation skills in a Web2.0 environment" Mike Leigh, Lucy Mathers and Kaye Towlson De Montfort University, ml@dmu.ac.uk, lmathers@dmu.ac.uk, kbt@dmu.ac.uk HEA ICS Workshop 18th November 2009

2. Session Outline • Project Background • Module Information • Methodology – Action Research Approach • Data Collection Methods • Information Source Evaluation Matrix • Results • Future Work • Conclusions

3. Project Background • Background • Student (mis)perception of their information evaluation skills • Previous study results • Observed wiki/blog postings • Existing online tutorials largely ignored • Intute Internet Detective, OU Safari, QUICK • Why? • Filling the skills gap

4. Module Information • Participants from three modules: • TECH2015 - Multimedia Animation Production 2 • INFO2404 - Object-Oriented Systems Analysis & Design • COMP5205 - Systems Analysis & Design • Similarities: • Coursework requires student to evaluate materials from a range of sources • Information synthesis and posting on Blogs/Wikis

5. Methodology – Action Research Approach Adapted from Allen (2001)

6. Data Collection Methods – Focus Groups • Questionnaire administered before and after the students’ use of the evaluation matrix (results in Appendix 1 - paper). • Completed evaluation matrices from the focus group sessions. • Feedback on the use of the evaluation matrix:- • Please write two statements outlining what you found useful in the use of the Evaluation Matrix. • Please write two ways in which the Evaluation Matrix could be improved. • Student postings available on the module wikis and blogs will be used in subsequent phases of this research programme.

7. Information Source Evaluation Matrix • Information evaluation requires the researcher to ask the 5Ws (questions) of the information source in relation to the task in hand: • Who? – is the author • What? – is the relevance of the points being made in the source • Where? – the context for the points being made in the source • When? – was the source published • Why? – the author’s apparent purpose / reason for writing the article.

8. Information Source Evaluation Matrix(Leigh, Mathers & Towlson, 2009)

9. How to use the Information Source Evaluation Matrix • Matrix covers the core questions • Use one sheet per information source • Remember to identify the task for which you want to determine whether the source will be useful • Remember to write down the source reference

10. How to use the Information Source Evaluation Matrix • Who - is the author? • How well known is the author in the field - may require a bit of detective work! • What - is the relevance of the points made? • How closely does the information in the source match the requirements of your task? • Where? - context for the points made • How closely does the author’s context match the requirements of your task?

11. How to use the Information Source Evaluation Matrix • When - was the source published? • How up-to-date is the information, is it still relevant to your argument? • Why? - the author’s purpose / reason for writing the article • How biased is the information? Is it opinion, good practice, fact?

12. How to use the Information Source Evaluation Matrix • Scoring is designed to help you • evaluate the usefulness of a particularsource • compare the usefulness of different sources for a particular task • If a source has a low score, it does not mean it cannot be useful… • … but it will help you … • Use and present the information from a source in an informed (academic) way

13. Results - perceptions • Increased confidence in ability to evaluate information • Less effort put into wiki-based assignments than reports with parity of marks • Less reliance on key words after use of matrix • Still preference for sources that agree with own perspective – scope for learning enhancement • Quality of journal papers acknowledged but also their specialist nature • Small change away from reliance on books • Recognition of value of commercial websites

14. Results – use of matrix • Overall, students perceive improvement of their evaluation skills as a result of using the matrix • Students found matrix helpful in evaluating material and easy to use • Matrix increased awareness of evaluation criteria • Matrix facilitates comparison of sources

15. Future Work • HEA ICS subject centre funding • Evaluate ISEM utilisation • diverse needs • different academic disciplines. • Modify ISEM - available online • User-chosen variable weightings • Free text area (for page numbers, comments, etc.) • Enhancement of appearance? • Online tutorials to support matrix use

16. Conclusions • Information evaluation skills gap identified & means for addressing it (matrix) developed • Raise awareness of evaluation criteria • Compare sources • Easy to use regardless of current skill level • Tool can be deployed in traditional, online and blended environments

17. References • M. Leigh, E. L. Mathers, and K. B. Towlson, (2009) “Using face-to-face sessions and focus groups to develop online support to enhance student information evaluation skills in VLE learning communities”. SOLSTICE Conference, It’s all in the blend? Edge Hill University 4th June 2009 • E. L. Mathers, and M. Leigh (2008) “Facilitators and barriers to developing learning communities”. The Higher Education Annual Conference: Transforming the student experience, Harrogate 1 – 3 July 2008 • ALLEN, W.J., 2001. The role of action research in environmental management. In: Working together for environmental management: the role of information sharing and collaborative learning. Ph.D. (Development Studies), Massey University. Available at: http://learningforsustainability.net/research/thesis/thesis_ch3.htmlaccessed 23/2/09.