Expanding your Expertise as a Social Science Researcher

1. Expanding your Expertise as a Social Science Researcher

2. Workshop Aims • Explore social science research ‘expertise’, including expert thinking, drawing on the literature • Examine how research expertise is learned, accelerated, and possibilities for your further development • Engage in a collaborative research design task • Plan a strategy for accelerating the further development of your research expertise

3. Programme

4. How you might benefit fromtoday’s workshop • More informed about the problematic nature of research expertise and how far its development can be accelerated • Reviewed and started planning how to sustain or reinvigorate your research activity • Extended your network of potential collaborators and ‘critical friends’ • Informed your thinking about how you can support others in developing their research 4

5. Ethical ground-rules • Sharing work experiences, aspirations and feelings entails sensitive information • But candour within a respectful, supportive, confidential context facilitates learning • So in this workshop, please commit yourself to following these ‘Chatham House’ rules: • - speak openly • - listen well • - take away ideas… • - but respect others’ intellectual property • and respect confidentiality

6. Session 1 (Aim 1) • The problematic nature of research expertise, including expert thinking

7. Starting assumptions • Today’s academics are affected by significant work intensification • It is only worth dwelling on what individuals have some chance of changing • Choices carry prices: where do you want to locate yourself on the expertise continuum? ‘Expertise is not an endpoint, it is a continuum… studies will allow us to frame…meaningful opportunities for advancing the development of talent…for ever-expanding numbers of individuals’ (Sosniak2006:300) 7

8. Research & trialling that informed the workshop • Draws on research training project (2010-14) within ESRC Researcher Development Initiative • Profs Alison Wray, Mike Wallace, Cardiff University • Reviewed generic expertise literature • Interviewed 31 very experienced researchers from different social science disciplines and countries • Trialled single and multi-disciplinary workshop designs, content, tasks • Outcome – resource for supporting development of mid-career researchers, based on project findings 8

9. Research interviews • Successful, very experienced researchers* from Business & Management, Applied Linguistics, other social sciences * Editors of top journals, past presidents/chairs of nationalassociations (e.g. US Academy of Management), plenaryspeakers at association conferences, recommendedby others, internationalreputation • Interviews of up to 90 minutes, or equivalent 11-question electronic schedule Other: Mainland Europe, China, Australia, New Zealand 9

10. Interview question areas • What is social science research expertise, including expert thinking? • For you, in your discipline, what constitutes research expertise and how do you know an expert? What makes you one? • How do you approach your research and how has that changed during your career? • How did you learn the skills of expert thinking? • Training? Mentors? Environment? Strategy? • How do you pass these skills on to others? • Trainer? Mentor? Advice? 10 45-E

11. Taking stock: individual review • On your own, please complete Exercise I in your handout (pages 1-3) • Afterwards, reflect on the implications of your review for your aspirations as a social science researcher 45-S 11 00-E

12. Informing your thinking through evidence from other contexts • The expertise literature has many limitations, there’s little research on social science researcher development, but some ideas might be useful • You can judge how relevant evidence from expertise literature and research interviews is to your situation, and so whether it has applicability • Allow for the possibility that even seemingly irrelevant observations could turn out to be informative • We will be discussing which (if any) of the features mentioned is unexpected, so note down your thoughts as we proceed 00-S 12

13. Some limitations of expertise literature …so who decides who has expertise? Expertise is a social construct… ..but we can’t directly measure social science research expertise – it’s not like winning chess championships Most research on expertise is psychological: controlled situations and externally validated measures (e.g. wins, scores)… The research supposes BOTH generic features of expertise, AND that expertise is not transferable across domains. Being a good athlete doesn’t mean you’re a good researcher It’s not clear how much we can generalize across domains 13

14. What is expertise? 1. A general definition of expertise (Schraw 2006:259), includes expert thinking • Large, integrated knowledge base • Sophisticated mental models that guide problem solving and critical thinking • Highly automated procedural and monitoring skills An expert is ‘up’ on the topic at hand, familiar with the range and breadth of the field, knowledgeable about ancillary fields... [Expert researchers] come up with the most amazing clever ideas. They see something out there which once they see it you think, ‘My gosh, why didn’t I think of that before’? Somebody who can write about [their research] really well and somebody who can stand up and talk about this in a very persuasive and clear way 14

15. What is expertise? 2. Identifying expertise (Ericsson 2006) • Defined in terms of individual capabilities acquired through experience: assumes no-one born an expert • Characteristics distinguishing expertise • Social (reputation, as attributed by others) • Length of experience in the domain • Consistent high-level performance of tasks within domain • this domain • Social (reputation, as attributed by others) Not only has a knowledge of alternatives but can go into depths about each alternative, can articulate the theory…of those alternatives. Has an ability to argue each perspective It’s not an objective quality. It requires a certain amount of recognition from the community for you to be defined as an expert 15

16. What is expertise? 3. No objective standards for many important domains of expertise (Shanteau 1988:205-6): ‘…external standards frequently do not exist for many real-world problems…Indeed, experts are essential in precisely those domains where there are no right answers.’ • Social and task-performance criteria are continually debated in the social sciences, since knowledge of the social world, its measurement, and authority to determine criteria are all contested • So no cross-social science consensus on the nature of research expertise, or criteria for measuring it Some experts may be stronger in theory and others in methodology, an ideal expert should blend the knowledge and understanding of both 16

17. What is expertise? 4. Expertise is both individual and social (Hoffman 1996:94): ‘Knowledge—in a sense—must exist inside heads. Where else could it reside?…when the expert carpenter leaves the workshop, something does leave with him. Could you or I use the tools to build, say, a china cabinet? However, knowledge—in a sense—is an attribution that resides in social groups. How else could it be developed, taught, or standardized? How could someone be regarded as an expert if her judgments are not followed in the decisions made by other people?’ • Social science researchers come to know a lot and do creative things with it, through long experience Expert is really not what is in one’s head, it’s how others’ heads judge this person 17

18. What is social science research expertise? 5. Aspects of expertise may differ between disciplines, fields or specialisms, for example: 18

19. How do experts think? • A style of skilful conceptualizing and arguing that is appropriate for a knowledge domain. Ochsestudied Nobel Prize winners (1990:259): ‘We must not forget that these subjects [interviewees] referred specifically to domain relevant thinking styles taught by masters in that field.’ • In the social sciences expert thinking styles reflect contestation about social knowledge • So expert thinking styles are NOT directly transferable to all knowledge domains A good question here means one that others in the field would…immediately recognize as important to be answered…Good means it’s answerable in principle or even in practice with what we know today 19

20. How do experts think? • Pursuing different intellectual purposes is appropriate in the social sciences • Expert researchers may pursue depth or breadth of understanding • , The ‘splitters’ are those who are trying to set up their work as unique, as different, as separate from all of these other approaches that are wrong. The ‘lumpers’, on the other hand, are trying to integrate, they compare and contrast and see how their approach compares with others and then make…comparative assessments between these alternatives… ‘Splitters’…are certainly experts. But they tend to be more narrow, they tend to be deeper and narrower, whereas the ‘lumpers’ are perhaps shallower and broader. And I think that is a trade-off 20

21. Who has the authority to label someone an ‘expert’ researcher? I know I’m operating most of the time very much at the edge of my capabilities Any decent project that I’ve done has involved me totally rethinking where I’m going at least ten times I think there is a danger in regarding people as experts. Anyone who eventually gets that title usually realizes that they have enormous gaps in their knowledge. One of the problems is that that unofficial title blinds people to the weaknesses in the research of ‘experts’. Just because an expert says or does it, does not mean that it is correct. Once experts start believing in their own expertness, we are in deep trouble 35-E 21

22. Discussion • Do any of these features surprise you – if so, why? • Which features give you most pause for thought about the trajectory towards greater research expertise that you might experience or expect? 35-S 45-E 22

23. Session 2 (Aim 2): Learning research expertise, and scope for accelerating development

24. How is expertise learned? 1. Self monitoring, building-up metacognition (awareness of one’s cognitive processes) in a domain (Feltovich et al 2006:57) ‘The development of expertise is largely a matter of amassing considerable skills, knowledge, and mechanisms that monitor and control cognitive processes to perform a delimited set of tasks efficiently and effectively. Experts restructure, reorganize, and refine their representation of knowledge and procedures for efficient application to their work-a-day environments…experts certainly know more, but they also know differently.’ 24

25. How is expertise learned? • Social scientists learn by monitoring their thinking, building up a reflexive, self-questioning approach to enquiry • Novices may not ‘know what they don’t know’, gradually building a large stock of knowledge and awareness of inter-linkages When I was a PhD student, I tried to think about questions in…the more conventional way, which is to be more linear. And to say, ”Well, this research…produced these findings, and if we extrapolate that, then we should find this…” [But now] I won’t be excited unless I can see how it fits into a bigger picture of some sort 25

26. How is expertise learned? 2. Building-up ‘chunking’. Experts remember and recall information in larger, more complex, inter-related ‘chunks’ than novices Chase & Simon’s chess-playing experiments: Better players ‘encode the position into larger perceptual chunks, each consisting of a familiar subconfiguration of pieces. Pieces within a single chunk are bound by relations of mutual defense, proximity, attack over small distances, and common color and type’. (1973:80) • Social scientists learn to group types of theory (e.g. realist), method (e.g. qualitative), study (e.g. social class), stock criticism (e.g. of opposing theories) and responses to them… 26

27. How is expertise learned? 3. Building-up flexible restructuring. Expert knowledge becomes organized in more abstract ways Chi et al (1982: XX) studied physics students: Novices showed an ‘inability to infer further knowledge from the literal cues in the problem statement. In contrast, these inferences necessarily are generated in the context of the relevant knowledge structures that experts possess’. • Expert knowledge reflects principles or patterns, not just the aggregate of concrete ‘facts’ about social phenomena, theories, methods 27

28. How is expertise learned? • Expert knowledge comes to includes dynamic (repeatedly modified) abstract mental models, including cause-effect schemata When I was a graduate student I imagined that all one needs to be a researcher is...to know the literature and have some methodological expertise, and that’s basically it Roughly every decade I discovered something in terms of, say, a method[ological] approach that I thought was more meaningful and helpful than the previous one. This at the same time increased my uncertainty as to how I understood what was going on 35-E 28

29. How is expertise learned? 4. Declarative to procedural knowledge. Consciously learned knowledge and skills gradually become tacit with repeated practice Expertise model (Dreyfus & Dreyfus 2005:787): ‘With enough experience…the brain of the expert gradually decomposes this class of situations into subclasses, each of which requires a specific response. This allows the immediate intuitive situational response that is characteristic of expertise.’ • With increasing experience, social scientists can draw intuitively on tacit knowledge of social phenomena, theories, methods, analytical tactics, research design options… 00-S 29

30. How is expertise learned? 5. Developing the ability to sustain motivation through increasing self-knowledge • Early setbacks are inevitable due to naivety… • But more awareness of motivators, strengths and weaknesses, enables focusing on rewarding work My first paper I submitted…I can look at that paper now and think that they were very kind in their reviews. But I was devastated. I thought my career was over. I sent in this thing that was totally inappropriate…I just did a small version of my dissertation, and it was totally wrong because I wasn’t in the conversation I've learned that I'm better at conceptual theorizing than at empirical research. In the beginning I was doing lots of empirical work…but I wasn't creative at it, and it appeared, at least to me, that I enjoyed and was better at trying to speculate about the big picture 00-S 30

31. Accelerating expert development? 1. Avoiding the ‘automaticity’ plateau. Actively self-monitoring habitual thinking and skilful practice, making continual adjustments to improve (Ericsson 2006b:691): …individuals who eventually reach very high levels do not simply accumulate more routine experience of domain-related activities, but extend their active skill-building period for years or even decades. • Fast learners consciously avoid continuing to act within the ‘comfort zone’ of their existing expertise • They continually push themselves outside their comfort zone by seeking novel opportunities, experiences, knowledge, ways of thinking…

32. Accelerating expert development? 2. Deliberate practice. Regular, short bursts of concentrated practice with feedback – Ericsson’s (2006b:694) musicians research: The expert performers and their teachers identify specific goals for improving particular aspects of performance and design training activities that allow the performer to gradually refine performance with feedback and opportunities for repetition (deliberate practice). The performers will gradually acquire mechanisms that increase their ability to control, self-monitor, and evaluate their performance…and thus gain independence from the feedback of their teachers. • Researchers seek support of expert colleagues who keep pushing them beyond their comfort zone

33. Accelerating expert development? • Most ‘deliberate practice’ in social science research involves real (not practice) tasks I see anonymous peer-review as central in the expertise-building enterprise One or two colleagues were very useful in developing my research skills largely through discussion and through co-operative critiquing of research designs. Their main help to me came through their pointing out the weaknesses of my designs and suggesting other possibilities. I have also learned a lot through listening to others describe and justify their research designs

34. Accelerating expert development? 3. Coaching/self-coaching. Practice tasks with formative feedback (Ericsson 2006b:698-9): …more-accomplished individuals in the domain, such as professional coaches and teachers, will always play an essential role in guiding the sequencing of practice activities for future experts in a safe and effective manner. Research on self-regulated learning…has documented effective study methods that are related to superior academic performance, especially in high schools. • Social scientists may actively seek collaboration with experts in their field so as to learn from them • Expert researchers may offer practice and feedback support to less experienced researchers

35. Accelerating expert development? 4. The research environment. Harnessing opportunities, supporting other researchers I learned a great deal from the ‘big cheeses’ from politics and policy from the UK and other countries [who] used to come in. And everybody would go down to coffee and tea to the common room every day, twice a day usually, because you never knew who was going to be there. There would always be somebody interesting …hanging out in my office and being able to chat to people when needed, but they chat to each other and we have meetings where we talk about shared issues…it’s that kind of community which is absolutely paramount See yourself as a citizen in your context where you do your work. You cannot do your work in isolation, so you’re dependent on the environment that surrounds you. So the first thing is, you’re not entitled to anything…you have to make the environment work for you

36. Accelerating expert development? 5. Dovetailing teaching with research. Making opportunities to inform research via teaching and supervision …teaching research methods… just forces you to be very clear and present [information] to others...You’re selecting work to illustrate the methods and you’re evaluating it with your students, so that means you need to work towards that deeper understanding where you can evaluate something …a lot of my learning came through having to write about what I was reading… Whenever I teach a new course my immediate goal is to write a book that covers the ideas in that course...I find that through having to write I need to clarify my ideas, and that sends me back to critical reading and thinking about what I have read

37. Taking stock: development opportunities • On your own, please complete Exercise 2 in your handout (page 4) • Afterwards, reflect on how you might: a) modify your learning pattern to accelerate the development of your research expertise b) compensate for any past lack of opportunities (or ones you didn’t take) c) maximize opportunities you now have 45-S 37 00-E

38. Discussion • What range of opportunities is there for mid-career researchers to accelerate the development of their expertise? • What range of opportunities can mid-career researchers create for yourselves, or initiate in order to get support? t 35-S 45-E 38

39. Session 3 (Aim 3) • Collaborative expert thinking in practice

40. What is interactional expertise? Interactional expertise means being able to contribute to collaborative research by ‘talking the talk’ – knowing just enough about others’ specialist expertise to work productively with them.Collins (2007:615) claims: There is an important kind of specialist expertise, called ‘interactional expertise’ that turns on fluency in the language of the domain rather than hands-on experience; it is acquired more through immersion in the discourse of the hands-on experts than through participation in their characteristic practices. • Large-scale research to tackle complex social issues requires experts with complementary specialismsto collaborate • So research expertise for mid-career social scientists may extend to interactional expertise needed for collaboration 40

41. If you just put half a dozen smart people in a room together you don’t have to have too much of an agenda for something good to come out of it. As long as people obey certain social rules, such as respecting each other, being flexible and letting ideas flow together You have to be willing to accept ideas of other persons, back up, and let others take the lead as well as you You have conversations on things you couldn’t imagine on your own...at the end of the day when you’re drinking that bottle of wine and you’re working on the debrief of the [jointly conducted] interviews, you wish you could take the white tablecloth of the restaurant with you because you’ve written on it 41

42. Group task (handout p5) • Create the initial outline for a research project that harnesses aspects of all group members’ expertise 2. Prepare your plenary report (<5 minutes) • Focus, questions, framing, data collection &analysis, potential answers, extra collaborator? 30-S 00-E 42

43. Session 4 (Aim 4) Implications for personal strategies

44. Tips from successful social science researchers interviewed • Seek training if you need it • Put the hours in and do things properly • Be streetwise about your career needs • Be focused, but alert to new opportunities • Try to work in a strong research environment • Learn to manage the trauma of reviewer feedback • Do something interesting and important • Put yourself out there and disseminate your work • Plan for the long term • Get a good mentor Which resonate most with you, and why? 44

45. Some possibilities for self-help Working with others: • Co-author with someone more experienced • Develop your ‘interactional expertise’ – ability to think, talk and contribute beyond the boundaries of your expertise to shared research activities • Listen to others’ points of view and create new, jointly constructed knowledge • Organize a research seminar • Engage with policymakers and practitioners about their concerns relating to your area of interest 45

46. Some possibilities for self-help Working alone: • Extend your comfort zone (knowledge, methods) • Talk ideas through and write to help your thinking • Target a tougher journal • Develop a grant proposal • Address reviewer feedback constructively Harnessing other responsibilities: • Use your teaching and supervision to extend your own research-related learning • Use your administrative roles to improve the research environment for everyone 46

47. Planning your personal strategy One way is to work incrementally: • Clarify your desired trajectory • Formulate key goals for moving along it • Decide what you can do immediately, over the next few months, the coming year • Review your strategy both regularly and when opportunities or hindrances arise… • On your own, please complete Exercise 3 of your handout (page 6) 30-S 45-E 47

48. Sharing take-away tips (If you are comfortable doing so) • Please identify one idea you are thinking of following up on that you are willing to share 45-S 00-E 48

49. Evaluation 45-S 00-E 49

50. References Chase, W & Simon, H (1973) Perception in chess, Cognitive Psychology 4: 55-81 Chi , M (2006) Two approaches to the study of experts’ characteristics. In Ericsson et al, 21-30 Chi, M, Glaser, R & Rees, E (1982) Expertise in problem-solving, in Stemberg, R (ed) Advances in the Psychology of Human Intelligence, Vol. 1, Hillsdale, NJ: Erlbaum, 7-75 Ericsson, K A (2006a) An introduction to Cambridge handbook of expertise and expert performance: its development, organization, and content. In Ericsson et al, 3-19 Ericsson, KA (2006b) The influence of experience and deliberate practice on the development of superior expert performance. In Ericsson et al, 683-703 Ericsson, KA, Charness, N, Feltovich, P & Hoffman, R (eds) (2006) The Cambridge Handbook of Expertise and Performance Cambridge: CUP Ochse, R (1990) Before the Gates of Excellence: The Determinants of Creative Genius Cambridge: CUP Schraw, G (2006) Knowledge: structures and process, in Alexander, P and Winne, P (eds) Handbook of Educational Psychology (2ndedn Mahwah, NJ: Erlbaum 245-263 Sosniak, L (2006) Retrospective interviews in the study of expertise and expert performance. In Ericsson et al, 287-301 50