Creating confident, independent future researchers through research methods assessment innovation

1. Creating confident, independent future researchers through research methods assessment innovation HEA Social Sciences Conference 23-24 May 2013 Liverpool Dr Helen Williams School of Politics & IR University of Nottingham

2. Outline • Background with research methods teaching • Teaching formats tried • Assessment formats tried • Successes and failures of assessment formats • General problems • Student feedback • Quo vadis?

3. Teaching formats tried • 1-hour lecture, 1-hour seminar • Advantages: exposure to secondary literature using different research methods • Disadvantages: low engagement, no hands-on understanding • 1-hour lecture, 1-hour workshop • Advantages: hands-on learning • Disadvantages: lack of contextual understanding and understanding of secondary literature • Trial next year: 1-hour lecture, 1-hour seminar, 1-hour workshop

4. Forms of assessment tried • Exams • Multiple choice • Essays: principles of research design • Analysis of statistical outputs • Essays (Example 1) • ‘What is social science?’; ontology, epistemology, methodology • Presentation & report on topic of choice • Critical reviews (Examples 2 & 3) • Evaluate and compare the methodologies of one/two articles • Prescriptive projects (Examples 4 & 6) • Quantitative: perform X, Y and Z; write up findings • Qualitative: carry out X type of analysis • Creative projects (Examples 5 & 7) • Students pick research question and data • Quantitative-only, qualitative-only

5. Exams Cons Pros • Learning for the test • Parroting textbooks • Not particularly engaging • Low enthusiasm • Requires broader revision • Easy way to check comprehension of key elements • Avoids assessment-related attendance drops

6. Essays Cons Pros • Unfit for purpose • Heavy theoretical emphasis • No interaction with real-world research • Shallow, narrow understanding • Low engagement & enthusiasm • High absenteeism • Straightforward to mark • No extra tutor time required

7. Critical reviews Cons Pros • Students encounter sanitised research process • Requires a higher level of methodological understanding than students have • Difficult to foster creativity; markers looking for a particular answer • Assessment-related absenteeism • Increases critical thinking • Students encounter real research outputs • Students exposed to on-going academic arguments • Requires some extra tutor preparation

8. Prescriptive projects Cons Pros • Boring for students & markers • Students only experience part of the research cycle • Less room for reflection • Hands-on learning • Provides clear structure • Students engage with real data • Good for tutors with less experience

9. Creative projects Cons Pros • Requires greater guidance & flexibility from tutors • More time-consuming than classic essays & exams • Can be biased against political theory-oriented students • Doesn’t fit with ‘normal’ assessment regimes & word counts • Long-term retention of learning • Deeper understanding • More likely to undertake empirical dissertation • Greater engagement & enthusiasm • Critical engagement with data • Employable skills

10. Outcomes • Increased student engagement • Deeper understanding of methodology • Increased student engagement • Students starting to recommend module to friends • Deeper understanding • Greater confidence • Increased original empirical research for dissertation students • Greater employability

11. General problems • Staffing: time, expertise • Avoidance of teaching such modules • Dominance of one approach over others • Lack of communication between staff running lectures and seminar tutors • Room configurations: group size, room shape, computer & software requirements • Timetabling conflicts • High levels of absenteeism • Clear correlation between attendance and passing module • Wide variation amongst student abilities • Institutional resistance to non-traditional assessment methods, e.g. marked attendance & weekly tasks

12. Student feedback: negative • Not interesting/didn’t see the point of the module • Lack of connection between lecture content and seminars/workshops • Lack of connection to the rest of the degree • Extreme lack of confidence with assessments • Desire for examples of what’s expected • Underestimation of the amount of work required: initial perception that the module is easy, then unexpectedly difficult assessment

13. Student feedback: positive • The more extra resources, especially electronic, the better • How-to for using software, performing data analysis • PowerPoint slides, Word documents, practice exercises • Enjoyed hands-on activities • Felt better prepared for dissertations • More positive feedback in third year than at end of second year

14. Addressing student feedback • Consistent, explicit repetition of the point of the module, its application to the rest of the degree and to employability • Need to emphasise hands-on learning • Lectures, if used, need to be crafted carefully to mesh with small-group teaching • Students, especially women, need far more support than other modules

15. Quo vadis? • Increase in contact hours to 1 lecture, 1 seminar & 1 workshop per week • Change in assessment methods • Greater creativity • Reflective, hands-on learning • Mixed-methods projects • Production of qualitative and quantitative materials • How-to guides • Formative self-marking assessments • Group work • Summative assessments

16. Quo vadis? Questions to ponder • How can we institutionalise good methods training that is less reliant on individual expertise and enthusiasm? • How can we increase institutional appetite for supporting methods training? • How can we increase the connection between methods training & rest of the degree without an increase in resources?