What’s the problem with feedback?

1. School of Social Sciences and Law Feed-forward, supporting the writing process for undergraduate researchers:The case of GEOverse an undergraduate research journal in Geography.Dr Helen Walkington

2. What’s the problem with feedback? “Feedback on my work has been prompt” “Feedback on my work has helped me clarify things I did not understand” • NSS results show that students are less positive about assessment and feedback on their assignments than other aspects of the SLE (HEA, 2008) • QAA subject reviews identify assessment practices as one of the weakest features of Higher Education (Rust et al. 2005). • Written feedback as a monologue (Nicol, 2010) Assessment of learning vs Assessment for learning

3. Students conceive of quality feedback as a dialogic process or cycle rather than a single event (Beaumont et al., 2008).

4. Why the dissatisfaction? • HE focus : develop self directed learning skills The solution for mass HE? • A dialogic feedback approach for HE can scaffold self-directed learning for students by engaging them in self assessment and peer assessment activities (e.g. Nicol, 2010). • The production of feedback is more cognitively demanding than just receiving it: the construction of feedback is likely to heighten significantly the level of student engagement, analysis and reflection with feedback processes

5. A social constructivist approach … • to the assessment process (Rust et al. 2005), where students ‘co-construct’ knowledge via dialogue with each other and their teacher as part of an academic community of practice (Pask 1975, Vygotsky 1978, Lave & Wenger 1991). • Feed forward • In self-aware systems or systems which are capable of learning: anticipating future conditions and using "learned" information as a basis for "planning" (e.g. disaster planning strategies).

6. The Undergraduate Research Experience • Research cycle • Mind the gap!

7. Completing the research cycle • Could the publication process complete the cycle? • What counts as ‘publication’? (Walkington & Jenkins, 2008) • Common thread = feed-forward

8. Undergraduate journals • CUR 45+ e-journals in the US • UK journals: Bioscience Horizons, Plymouth Student Scientist, Reinvention, Diffusion, Geoversity, GEOverse www.geoverse.brookes.ac.uk

9. (Walkington, 2008)

10. Impact of writing articles on SLE 1. Sense of achievement “With the possibility of actual publication, it invoked a sense of pride and so I felt that the quality of my finished article was higher than usual.” 2. Understanding “It was also good to really understand what I had researched last term.” 3. The creative process “I enjoyed the freedom of opinions and sources used, it was very interesting following up a previous field study and expanding on it.”

11. Impact of writing articles on SLE 4. Sense of ownership “[I enjoyed] the challenge of writing an article that comes from my own experiences.” 5. A more critical approach to sources “you have the knowledge to be critical and the work is more personal so you have a different outlook on other journals researched.” 6. Synthesis of skills “It has allowed me to bring together all the skills I have been trying to learn throughout the university period.”

12. The experiences of students who had their work published. What’s new? • CV material, Masters places, job interviews • Sending link to friends / family / other students • More emphasis on applying constructive criticism • “critical evaluation of any piece of work, my own or others” • “trust the advice of others” • Feedback that improved my work. Desire to talk through feedback rather than relying on technology, desire to clarify feedback, desire to open up the communication process • Motivation to publish more articles

13. Ownership Understanding Creativity CV Academic recognition Further communication / dialogue Motivation to publish more From student to author Writing Publication Achievement Applying constructive criticism Critical evaluation

14. Another culture shift from receiving feedback to actioning feedforward: “Initially I found it quite hard as I felt the reviewers brought to light the mainly negative aspects of the article, and had overlooked the hard work that had been put in. Once I realised that their job was to provide suggestions to improve the article and not to point out all the positive elements I was able to view the comments in a positive light. I also really appreciated the accuracy of the feedback once I had completed the article as it really made my work stronger and of a higher standard” Quality feedback ‘It is more constructive as it is a working document’

15. Collaborative review • “So we had a dialogue within the page as well as directly editing the page and then we also used the comments section as a kind of asynchronous discussion about our comments on the general quality or otherwise of the paper.” • ‘It is useful to see how someone else reviewed the same paper. There were areas of agreement, but also areas where each reviewer picked up something different. I think working together is a useful process’. • ‘I did not fully understand the idea of working with other reviewers. Although it worked well in terms of being able to have a chat about the paper, ultimately everyone has their own opinion and I wanted to be able to say what I thought, without having to agree with the views of another.’

16. Postgraduate student experience - new role as reviewers. • “I’d never reviewed anything before for a journal, but I’ve done marking of undergraduate essays… and getting a little bit annoyed, ‘cos you don’t get to give much feedback. You know, you can make general comments on the essays, because time is restricted, whatever you have to say. It is satisfying that you make more feedback, make more constructive comments. I’m probably more constructive in my comments now which may help if I do things like marking or more reviews in the future” • “The wiki made the actual writing of the review easier, we had two or three drafts of the review”

17. Mutual desire for more dialogue For authors: • The process was quite smooth, the only thing that may have improved it for me would have been the opportunity to talk through the feedback in a meeting with an editor, as e-mail communication can sometimes be limited • I felt it quite challenging as the remarks on my work were via e-mail, so I could not directly speak to the editors about concerns and questions that I had, or to clarify their comments For reviewers: • I think perhaps there should be a way in which anonymously the author could then get back to the reviewers either through a wiki… for feedback. For example, if we made a comment and they were not sure what I was talking about there was no feedback from the reviewer to the author, it always goes through the editor, which I didn’t think was necessary. It would have been much easier if they’d gone back to the reviewer • I wanted feedback on my first review from an editor

18. Key finding Students require structured experiences to ‘scaffold’ their learning, so that they have the confidence to publish their work. This has led to: • Curriculum development with the writing of journal articles being embedded in modules as an assessment task and the teaching of skills in writing for publication; • Redesign of research pathway within geography, API; • Plans for RADAR to include the first dedicated digital repository for undergraduate research; • Extracurricular opportunities to develop dialogic skills e.g. undergraduate research conferences.

19. Conclusion CLOSE THE RESEARCH GAP through ‘publication’ Collaborative writing space (wiki) can support first time reviewers Authors shift from seeing articles as products to a process of 2 way communication, online spaces could support this through e.g. wiki’s for authors and reviewers ‘dialogic feedback’. For the future: • Further development of Collaborative authoring space; • Further opportunities to develop dialogic feedback using the e-journal

20. Acknowledgements Funding: • HEA, GEES Subject Centre • Reinvention Centre Fellowship • Undergraduate authors who have submitted their work • Postgraduate members of the Editorial Advisory Boards • Academic members of the Editorial Board • Web & Wiki developer