Non-traditional assessment in ‘traditional’ subjects Problem Solving

1. Non-traditional assessment in ‘traditional’ subjects Problem Solving

2. Moving beyond the ‘gold standard’ • The use of non-traditional forms of assessment is accompanied by a wide range of issues and potential problems both for students and staff • These problems are especially pronounced in disciplines such as the humanities where ‘traditional’ forms of assessment are all the more culturally and institutionally embedded • This short presentation will outline some of these issues, invite student feedback and set up questions for discussion …

3. Moving beyond the ‘gold standard’ • Personal experience of non-traditional forms of assessment relatively limited • Not until I arrived at Roehampton in 2010 that I worked with programmes such as Mahara or with forms of assessment such as e-portfolios and blogs • Even now, majority of my modules assessed by essay or exam • Exceptions are Research, Reasoning and Rhetoric (e-portfolio) and Imperial City: London in the C19 (blogs) • However, becoming more prevalent – abandoned essay component of Imperial City in favour of blogs for 2011-12

4. Moving beyond the ‘gold standard’ • Despite alternative modes of assessment e.g. MCQs, oral presentations, source analyses, etc., essays still seen by many within the humanities as the ‘gold standard’ of assessment • Students acculturated into formal essay-writing through secondary education – forms the backbone of A-Level coursework and exams • Much of the rest of their degree-level assessment also takes this, or similar, form

5. Moving beyond the ‘gold standard’ Many reasons why essays occupy such an central place within the humanities curriculum: • It is how wewere taught – patterns of assessment have a tendency to replicate themselves through emulation • Academic often concerned to maintain intellectual ‘standards’ and resist ‘slippage’ or ‘dumbing down’ – often resistant to innovation/novelty • Structured, formal essay the medium which most closely corresponds to the standard forms of humanities scholarship, i.e. monograph and journal article

6. Moving beyond the ‘gold standard’ • Links to issues of employability and transferable skills (Ted) • What are we training students for and what skills are we seeking to impart? • Does the structured, formal essay offer the most appropriate means of developing and assessing those skills? • What is it that other forms of assessment can offer? • How can we ease the transition from one mode of assessment to another, both for students and staff?

7. Issues for students Students face a variety of challenges and problems when asked to compose an e-portfolio, blog or other form of non-traditional assessment. Here are just a few examples, we may hear of more later: • Lack of familiarity • Most students (especially first years) have not been required to undertake this kind of work before • Many are therefore uncertain as to what is required of them – ‘Am I supposed to have a question?’; ‘What kinds of sources can I consult?’, ‘Can I use internet resources?’, ‘Can I use pictures/video?’, etc. • In RRR we provided extensive and continuous support to ease them through this process • Even so, while the majority of students kept up to speed, some remain slightly confused …

8. Issues for students Example: • One student wanted to use this image of Luis Figo playing for Real Madrid against Barcelona in the 2002 El Clásicofor their RRR project • “I’ve basically decided to use this picture because this really highlights the rivalry/hatred between the two sets of fans especially because of the pigs head thrown which can be seen in the background. I was also wondering that I’ve found it difficult to find historians who use modern sources or any literature on that” • Interesting idea, but image perhaps too modern to do justice to the topic and student clearly unaware of how to proceed …

9. Issues for students • Tendency towards informality • Working outside of the formal structures of an essay can be liberating • RRR portfolios really encouraged students to indulge their particular interests, to be adventurous, to advance their own opinions and to explore their chosen topic in considerable depth • However, lack of formal structures and relative familiarity of Mahara as a ‘social media’ space could sometimes lead to a certain complacency when it came to sloppy grammar, spelling, etc., not least because Mahara does not have an in-built spell-checker • Moreover, the relative freedom given to students to ‘indulge’ their interests could lead to the posting of material which was not always relevant or of an appropriate academic standard …

10. Issues for students Example: The Hollywood Effect

11. Issues for students • Lack of focus • Without structuring influence of an essay question, some students did not know how to advance, or indeed to constrain, their analysis • In RRR we made a concerted effort to encourage them to develop research questions which would enable them to do this • Encouraged them to focus on a specific object and draw out wider themes • However, in some cases the research questions were either not sufficiently linked to the source itself or were far to broad and unwieldy • Can happen in essays too, but novelty of medium and lack of familiarity perhaps more likely to encourage analytical disconnect …

12. Issues for students Example: Using a source to answer one of the most vexed questions in American history

13. Issues for students • Mahara and social media • Mahara a ‘poor (wo)man’s Facebook’ • Replicates some of the features of social media e.g. ‘friends’, ‘profiles’, etc. • However, by no means as intuitive or as well-designed as other social media programmes – doesn’t allow students to communicate or interact as much as they might do • ‘Friendship’ aspect of Mahara perplexing and largely irrelevant • Students used to dealing with much more intuitive systems – hardly encourages them to embrace the programme

14. Issues for students • Employability and technical competency • Mahara not a commercially employed programme • Does not have much relevance outside of academia • Would it be better to use platforms such as Wordpress, Blogger, etc.? • However, Mahara has value in that it is a closed system, unlike other blooging tools • Raises the question of what we are teaching: technical proficiency, analytical skills or both?

15. Issues for staff • Of course, staff should be better prepared and have a greater sense of what they think such forms of non-traditional assessment should achieve but, even so, there are issues to address • Lack of familiarity • As with students, staff may also not be used to such forms of assessment. • Essays relatively familiar (and sequential) forms to mark – sometimes hard to know where to start with an e-portfolio • Hard to know how much latitude to allow students in terms of informality, etc. • Have to adjust approach without lowering standards …

16. Issues for staff • Reframing expectations • Many academics find it very hard to move away from notion that essay remains the ‘gold standard’ • Easy to see e-portfolio’s as an adjunct, but not necessarily an alternative, to traditional forms of assessment • Perception that it is an ‘easy’ form of assessment or an example of ‘dumbing down’ hard to shake off • Requires one to move imaginatively outside of the confines of academic culture and understand that we are not just seeking to produce more people like ourselves who will write books and articles …

17. Issues for staff • Maintaining academic rigour • At the same time, however, lecturers have a duty to impart core intellectual competencies as well as practical skills • Purpose of RRR portfolio’s is to allow students to develop their own interests while also encouraging them to think in a critical and informed way about their chosen source • Have to provide feedback in a way which pushes students towards developing these academic skills without dampening their enthusiasm

18. Issues for staff • Technical issues • Managing communication –Mahara has a system of messaging distinct even from that of moodle – how to manage increase in workload on top of emails, phone calls and office visits from students? • Providing feedback – Can only place feedback directly on the student’s e-portfolio if they have formatted it appropriately – otherwise have to send a private message – a plethora of different ‘views’ which can prove confusing • Hard to keep record of feedback and correspondence

19. Issues for staff • General impressions • Some great work has come out of the RRR portfolios but also some rather weak work • Allows the best students to excel, to indulge their passion for a particular period/theme and to craft and design an public-facing document while also developing core analytical and interpretative skills • However, can also allow weaker students to coast and to put in the minimum of effort • Any different from other methods of assessment? • Important to encourage students to take pride in their ‘view’

20. Student experiences • What problems/challenges did students encounter? • How did they seek to overcome those problems? • More general thoughts on the experience …

21. Questions for discussion • How might we address some of the problems discussed in this presentation? • What other problems might you envisage with such non-traditional forms of assessment? • What is the e-portfolio/blog designed to achieve and how can we ensure that it achieves that object? • How do such forms of assessment relate to the changing landscape of higher education in the UK?