Julia McMorrow School of Geography University of Manchester

1. VLEs for fieldwork preparation, assessment and evaluation:the Dark Peak website, WebCTquizzes and surveys Julia McMorrow School of Geography University of Manchester GEES LTSN, Virtual Learning Environments, Birmingham, 18 Nov 2003

2. Structure_____________________________ 1. Dark Peak website Preparing for fieldwork 2. WebCT quiz Formative assessment 3. WebCT surveys Pre- and post-fieldwork course evaluation 4. Conclusion Advantages and disadvantages of VLEs for fieldwork preparation, formative assessment and course evaluation

3. 1. Dark Peak website______________________________________________________ Purpose: • To prepare first year students for a one day physical Geography field course in the south Pennines: - Give students a ‘feel’ for the physical landscape at various spatial scales - Introduce or reinforce knowledge of the topics and techniques to be encountered in the field - Provide a resource for reflective follow-up work • University resource: - Graphics-rich resource for interdisciplinary learning and teaching • Public resource: - Student recruitment - Showcasing work of Upland Environment Research Unit (UpERU)

4. The Dark Peak web site http://www.art.man.ac.uk/Geog/fieldwork/ ________________________________________

5. The Virtual Tour Linear structure + map with hotlinks for direct access to the 7 ‘stops’

6. VT page

7. Click to enlarge and annotate image

8. Theme page

9. Using the website with WebCT______________________________________________________ Pre-fieldwork lab class 1. Dark Peak website Virtual Tour (VT) and Theme Pages studied alongside… 2. WebCT Quiz for formative assessment of learning from the website 3. WebCT Survey to evaluate website before going into the field Post-fieldwork individual use • WebCT Survey to evaluate the field course and retrospective views on website • Reflective learning; re-take Quiz, use website for follow-up work

10. 2. WebCT quiz______________________________ • Aims • To encourage students to prepare for fieldwork by engaging with the Dark Peak website • To allow students to test their understanding of topics covered in the website; formative assessment • Logistics • Compulsory: must pass to gain course credits • No time limit or limit on the number of attempts • Accessible within specified dates

11. WebCT quiz instructions http://webct.man.ac.uk

12. Visual and text prompt to the relevant web page Fieldwork Quiz: question formats 1 Multiple choice, least correct statement

13. Answer log updated as answers are saved Multiple choice, match letters to labels Fieldwork Quiz: question formats 2 Multiple choice, correct combination SENDA compliance issues

14. WebCT quiz: Providing online feedback • Students receive score and feedback for each answer as soon as the quiz is submitted • Links to relevant web page in General Feedback at end of each question • Reflective: able to view earlier submissions

15. Individual % marks and global mean for Quiz Automated marking: ‘Submission’ and ‘Report’ to track progress and compare results For each student: number of attempts, score, time taken

16. ‘Summary’ to compare results by question • For each question • Quartiles • Mean score • Standard deviation • Allows discriminatory power of questions to be evaluated

17. Columns: response summary for each multiple choice question ‘Detail’ gives responses by student and by question Rows: individual student responses to each question

18. 3. WebCT Surveys________________________________ Survey 1: Pre-fieldwork evaluation • To evaluate the website prior to fieldwork Survey 2: Post-fieldwork evaluation • To re-evaluate the website after fieldwork • To evaluate the field course • Format • Likert scale and free text • Visual prompts • Not marked, but, like the quiz, both must be submitted online to gain the course credits

19. Free text question: Post-fieldwork survey, ‘What was the most useful feature of the web resource in preparing you for the field visit?’ WebCT surveys: response format Multiple choice Likert scale

20. 4. Conclusion: on the minus sideDisadvantages of WebCT quizzes and surveysfor assessment and evaluation ____________________________________________________ • Time required to for tutor to learn WebCTand prepare content • Registration issues • Scores merged for duplicated surnames • No analysis of free text responses • Lower submission rates for Survey than for paper questionnaires collected on bus or in a lecture • SENDA compliance; designing for accessibility

21. On the plus side: Advantages of WebCT quizzes and surveysfor assessment and evaluation _____________________________________________________ For the tutor • Graphics easily incorporated • Ability to incorporate feedback • Instant, automated marking of multiple choice questions • Tracking submissions • Evaluating questions: detailed breakdown of results by question For the student • Distributed: access from any internet connection • Self-paced, reflective learning: no time limit, access to earlier attempts

22. Acknowledgements • Curriculum Innovation Fund, University of Manchester and UMIST for grant to develop Dark Peak website • Cassandra Worman, co-author of Dark Peak website • Colleagues at Manchester, especially Martin Evans, Kate Richardson, Chris Perkins and Joe McQuaker for contributions to website content • Graham Bowden, School of Geography Cartography Unit for website maintenance • Anita Gill, Roger Hewitt, Faculty of Arts, for WebCT support