What is the Imitation Game?

1. What is the Imitation Game? Robert Evans and Harry Collins Centre for the Study of Knowledge, Expertise and Science (KES) Cardiff School of Social Sciences http://www.cardiff.ac.uk/socsi/expertise

2. Overview • Theory and Background • Different Kinds of Expertise • Turing Test • Imitation Games in Practice • Software and data • Identify and Chance conditions • Qualitative and Quantitative Results • Comparative Research • Cross-cultural and longitudinal research

3. Understanding societies http://bmrc.berkeley.edu/courseware/cs298/spring99/w9/slides/sld006.htm

4. Three Kinds of Expertise • Contributory expertise: enables those who have acquired it to contribute linguistically and practically to the community through the expertise is sustained. The most common usage of the word ‘expert’. • Interactional Expertise: expertise in the language of a specialism in the absence of expertise in its practice. Like contributory expertise, it requires the tacit-knowledge acquired by immersion in a form-of-life (i.e. socialisation). It enables individuals to talk as if they had contributory expertise even though they lack practical or craft skills. • No Expertise: not even the ability to talk intelligently about a given domain or topic Collins and Evans (2002)

5. Turing Tests and Imitation Games In the Turing Test, the judge must decide which is the computer and which is the human. In the Imitation Game, the judge must decide which participant shares their social group.

6. Male pretending to be female Female judge setting questions Female answering naturally Modern Imitation Game R1‘once a week’ R2‘not very often, when they need doing’ R2 is female ‘because I expected the man to believe women are more regulated in their beauty regime than they actually are How often do you pluck your eyebrows?

7. Judge compares answers Judge’s verdict Judge’s confidence Judge’s reason Judge’s question Respondents Answers

8. Automatically recorded data • Software records • Questions and answers • analysis of topics and assumptions used by judges (i.e. what is distinctive about own culture) • Guesses and confidences • what discriminates and what does not (i.e. what is shared with others and what is not) • Judge’s reasons • cultural norms and variation (i.e. what counts as a ‘mistake’ and what as normal variation)

9. Researcher generated data • Researchers needed to record • Problems with method and software (e.g. how to use, do participants understand role etc) • Strategies used by judges and pretenders (e.g. what is hypothesis being tested by judge, how does pretender ‘know’ the answer) • Effort and determination (e.g. judge’s motives and fears)

10. Imitation Games with the Blind Identify Condition Chance Condition Judge is sighted Target expertise is ‘being sighted’ Blind participant has to pretend to be sighted Prediction: Blind person will succeed in pretending • Judge is blind • Target expertise is ‘being blind’ • Sighted participant has to pretend to be blind • Prediction: Judge will succeed in identifying participants

11. Sighted dialogs

12. Judges’ reasons • Question One • The second person is not black and white and you do not usually lose your sight over-night, so the fact they mention being registered suggest that they are blind. If the first one was blind they would normally say how they became blind if it was sudden. (level 2) • Question Two • I have both white stick and dog but would never use both at same time. Therefore if I was responding I would say something like I use a guide dog predominantly but sometimes use a white stick -- but if you are blind you would call it a cane normally. Also, number 2 was much less black and white. It's always grades of blindness. (level 4)

13. Blind dialogs

14. Blind dialogs

15. Judges’ reasons • R1 is the blind person • ‘I think respondent 1 gives himself away when he discusses the human judgments on the flight of a tennis ball.’ • ‘I cannot believe a sighted person saying that Hawk-eye does not alter the viewing.’ • R2 is the blind person • ‘The Hawk-Eye questions reveal some quite specific information that I don’t think was published in audio media. Also, the story wasn’t that important that I’d expect it to be picked up by the audio news services provided to the blind.’ • ‘person 2 seems really unfamiliar with hawk-eye, given that they say they watch Wimbledon’

16. Interpreting Results The Identification Ratio measures the excess of correct identifications

17. Comparisons across topics Mean IR for ID condition approx 0.6 Mean IR for Chance condition approx zero

18. IG and Comparative Research • Social Change and Social Difference • How to standardise data collection? • How to reflect local variations? • Imitation Game as ‘ethno-method’ • Participants’ discourse is always contemporary • Judge as proxy ethnographer • Participants as proxy stranger • Participant errors like breaching experiments • Imitation Game as secondary data • Identification ratios by time, place and topic • Qualitative data based on topics, reasons, errors

19. Imitation Games on Gender • Hypotheses • Malestream culture: hegemonic masculinity implies asymmetrical results • 1960s and all that: gender roles and identities more flexible so knowledge widely shared • Age as confounding factor • Younger people have less interaction • The 1960s are history?

20. Research Design • Research Methods module project • Recruit students and heterosexual, cohabiting parents • Phase One • Students: 29 games (15 F, 14 M) • Parents: 25 games (12 F, 13 M) • Phase Two • Students: 70 games (40 F, 30 M) • Parents: 70 games (50 F, 20 M)

21. Results by gender Phase One and Two, students Approximates identify condition for both groups, female judges appear more successful than male judges but not statistically significant (p = 0.37) Phase One and Two, students Approximates chance condition for both groups, no statistically significant difference between genders (p = 0.26)

22. Results by age Phase One only Clear difference between groups; young judges more successful than older judges (p = 0.031) Phase Two only Clear difference between groups; young judges more successful than older judges (p = 0.007)

23. Conclusions for gender • Differences by gender • No statistically significant differences in sample as a whole • Some differences in topics chosen to ‘represent’ and test cultures • Clear differences by age • Older participants much better at pretending (i.e. have more interactional expertise)I • Impossible to distinguish between ‘1960s’ hypothesis and the ‘learning’ hypothesis without longitudinal study

24. Conclusions • Interactional expertise • Useful concept for explaining social phenomena • Imitation Game • Combines qualitative and quantitative approaches • Opportunities for comparative research within cultures, across cultures, over time • Want to know more • http://www.cardiff.ac.uk/socsi/expertise

25. References and reading • Collins, Harry and Robert Evans (2002) ‘The Third Wave of Science Studies: Studies of Expertise and Experience’, Social Studies of Sciences, 32 (2): 235-96. • Collins, Harry and Robert Evans (2007) Rethinking Expertise, Chicago, IL: The University of Chicago Press. • Collins, Harry, Robert Evans, Rodrigo Ribeiro and Martin Hall (2006), ‘Experiments with Interactional Expertise, Studies In History and Philosophy of Science, Volume 37, No. 4 (Dec 2006), pp. 656-674. • Evans, Robert and Harry Collins (forthcoming, 2010) ‘Interactional Expertise and the Imitation Game’ in Michael Gorman (ed) Trading Zones and Interactional Expertise: Creating New Kinds of Collaboration, Chicago, IL: MIT Press