Quantitative Methods for Coding and Analyzing Metaphors in Linguistics, Psychology, and Discourse

1. Coding and analysing metaphors within linguistic, psychology and discourse perspectives: The use of quantitative methods Joep Cornelissen and Mario Kafouros, Leeds University Business School, May 2006

2. Presentation structure • Three studies (‘linguistic’, ‘psychology’ and ‘discourse’) within organisation theory • Analytical steps around (1) identification, (2) inference and (3) analysis (systematicity/conventionality/use) of metaphors across corpus of language • Choices for methods • Discussion: the potential of quantitative methods and measures

3. Context • ‘Organisations’ cannot be directly represented or experienced as single objects or entities (Sandelands & Srivatsan, 1993; Weick, 1989)  metaphorically represented as organism, machine, (open) system, container etc. • Theoretical debates about how metaphors work and shape theorising about organisations • Very little empirical research on how metaphors are developed, selected and retained, and impact and shape the field • Studies funded by the Economic and Social Research Council (ESRC) (RES-000-22-0791)

4. Study 1: ‘Linguistic’ study • “why are certain metaphors selected and retained over time?” • A cognitive linguistic analysis of conceptual metaphors in organisation theory 1989-2003 • hypothesised that four psychological principles (the between-domains distance principle, the within-domains similarity principle, the concreteness principle and the relational principle) determine the aptness of a conceptual metaphor, and its subsequent adoption and continued use over time • Data from academic publications 1989-2003

6. Adjectives ‘organisational’ and ‘corporate’ as target terms into the Topic (title, abstract or keywords) field tag • (1) metaphor focus identification, (2) metaphorical mapping and categorization, (3) metaphor analysis • Ad 1: two coders and use of dictionary • Ad 2: deductive and inductive, measured inter-rater agreement (K-statistic) on categorization and on coding of aptness principles • Ad 3: measured frequency (total counts) and mention (spread over years) of metaphors over time, correlated with aptness scores

7. The kappa-coefficient (Carletta 1996) measures pair-wise agreement among coders making category judgments, correcting for expected chance agreement. Good quality categorization of discourse phenomena normally yields a K of about .80 (Carletta 1996). • Correlations between both the sum scores of aptness of conceptual metaphors and their frequency over time, and the scores of conceptual metaphors on each of the four principles and their frequency over time • Correlations between both the sum scores of aptness of conceptual metaphors and the scores of conceptual metaphors on each of the four principles, and their simple mention over time

11. Study 2: ‘Psychological’ study • “What is the (psychological) impact of metaphors in organisation theory?” • The consequences or impact of a metaphor including an ‘explicatory’ form of cognitive change whereby a metaphor facilitates learning or leads to conceptual clarification and a ‘generative’ type of cognitive change with a metaphor leading to conceptual advances and insights that were inconceivable before • Six central metaphors-in-use: organizational improvisation as jazz, organizational identity, organizational behavior as theatre, organizational learning, organization as chaos and organization as evolution. • 250 participants (management and organization scholars) rated these metaphors on a number of scales

12. Participants were also asked to rate the effects of the metaphor in question; whether the metaphor had provided a language to communicate about a topic, facilitated learning/led to conceptual clarification, or led to conceptual advances. • P1: The higher the within-domains similarity of a metaphor, the higher the ‘generative’ and ‘explicatory’ consequences of a metaphor. • P2: The higher the between-domains distance of a metaphor, the higher the ‘generative’ consequences of a metaphor and the lower the ‘explicatory’ consequences. • P3: The higher the comprehensibility of a metaphor, the higher the ‘generative’ and ‘explicatory’ consequences of a metaphor.

18. Study 3: ‘Discourse’ study • Basic question in organisation theory – how people give meaning to events, work, environment within and outside of the organization • Massive literature on managerial and organisational cognition: schemata, cognitive maps, learning, scripts, mental models etc. • Most constructs strictly ‘cognitivist’ and ‘reproductive’ - emphasizing how the contents of cognition reflect, distort or otherwise mirror the world (e.g. computational metaphor) • Sensemaking as ‘productive’ account of meaning-making:how acts of cognition impose not only structure but also direction on experience (both in retrospective and prospective sense)

19. Sensemaking accounts, constituted through metaphors, link language (a discursive account of an organization as, for example, a machine), thinking (thoughts and ideas about what it means to see an organization as a machine) and action (acting as if an organization was a machine) • Need for systematic identification and analysis of metaphors in sensemaking/discourse processes

20. Discourse approach: (1) metaphor focus identification, (2) metaphorical mapping, and (3) metaphor analysis • Research context: corporate communications professionals within 6 organisations in the UK, asked to discuss their own experiences within a ‘critical incident’ • First step: Interview data transcribed and analysed manually • Second step: use of a second coder for the categorisation (metaphor mapping) • Third step: cluster analysis (metaphor analysis)

21. Preliminary results • BOC: failed takeover by Airliquide 6 years ago  “it was a time to re-engage the business (INTER-PERSONAL RELATIONSHIPS); we pointed out (VISIBILITY/VISUAL REPRESENTATION) to people that we were in control again, we had a new CEO who came (MOVEMENT) with a new message of growth and control, we organised brain-storm meetings to foster a can-do culture and to pre-empt employees bombarding (PHYSICAL AGGRESSION) corporate managers with questions”

22. BNFL: accident within the Thorpe plant  “we needed to bring the message across (TRANSFER (OF OBJECTS) to local community, media and council as quickly as possible, we announced (DELIVER NEWS AS OFFICIAL MESSENGER) this first to the local media, when we talked to the local community we translated (LANGUAGE) it for them and explained how we saw the situation, we also told them that we weren’t yet in the possession of all the facts (POSSESSION AND TRANSFER OF OBJECTS), we consciously did not release information (PHYSICAL CONTROL OF OBJECTS) to the national press, in part so as to pre-empt negative news coverage”

23. Metaphor mapping: both deductive and inductive, K-statistic to measure inter-rater agreement on categorization of metaphorical foci into categories (source domains) • Metaphor analysis: measure frequency of particular metaphorical expressions and the frequency of source domains across the entire dataset, and in relation to samples related to a particular critical incident (as the sensemaking context), a speaker, and the organization involved. • Metaphor analysis: measure distribution of metaphors by sentence and consecutive ‘communications’ or ‘utterances’, as the unit of talk between the interviewer and the interviewee (Pollio & Barlow, 1975)  (cumulative) frequencies, distribution

24. Discussion and implications • Quantitative methods at analytical points, and dependent on theoretical assumptions and research questions • Metaphor identification: use of quantitative measures not an issue in psychological research, but reliability and categorization measures (K-statistic, Perreault and Leigh statistic) important to linguistic and discourse research

25. Metaphor mapping and categorization: again kappa-statistic helpful for discourse and linguistic research • Metaphor analysis: measures dependent on research questions and dependent-independent variables (aptness, time)  frequencies, correlations, distribution measures (poisson, cumulative frequencies, estimate functions)