Content Analysis for New Media: Rethinking the Paradigm

1. Content Analysis for New Media: Rethinking the Paradigm Susan C. Herring School of Library and Information Science Indiana University, Bloomington

2. Content Analysis • Systematic analysis of the manifest content of communication • What people do, rather than what they say they do • Product-focused • Naturally-occurring data • Data may be from historical sources • Traditionally mass communication, publicly available • Primarily quantitative

3. Content Analysis (cont.) • Advantages • External & internal validity • Unobtrusive • Low cost • Limitations • Doesn’t get at intentions, internal states • Can be decontextualized • Can be reductionist

4. Discourse Analysis • Analysis of patterns of meaning and structure in units of language larger than the sentence • What people do, rather than what they say they do • Product-focused • Naturally-occurring data • Data may be from historical sources • Public and private communication, including conversation • May be qualitative (interpretive) or quantitative

5. New Media • The World Wide Web • Personal home pages, blogs, wikis, etc. • Computer-Mediated Communication (CMC) • Email, discussion boards, chat, Instant Messaging, SMS, 3-D virtual worlds, etc. • Convergent Media • Social network sites, MMOGs, YouTube, iTV SMS, etc.

6. “New” Characteristics of New Media Communication • Multimodal • Hyperlinked • Textual conversation (“typed speech”) • ALSO: persistent, plentiful, large scale, dynamically changing, etc.

7. A conservative approach • New media CA research should seek to reproduce the methods of traditional CA research (McMillan, 2000) • The 5 Steps of CA (cf. Krippendorf, 1980) • The researcher formulates a research question and/or hypotheses based on previous theory • The researcher selects a (random) sample • Categories are defined for coding • Coders are trained, code the content, and the reliability of their coding is checked • The data collected during the coding process are analyzed and interpreted.

8. Why use established methods? • Availability • No need to reinvent the wheel • Reliability • “tried and true” • Sophistication • Refined through successive applications • Interpretability • Enable comparison with previous research • Associated theoretical frameworks • Guide assumptions, research questions

9. Problems with conservative approach • Not fully representative of CA • Not adhered to in all “classical” research (cf. Bauer, 2000) • Inadequate to explain some new media content • E.g., linking, multimodality, textual conversation • Difficult to apply strictly to new media content • E.g., problems with random sampling • Assumes comparability of old and new media • E.g., units of analysis • Assumes research questions are the same for old and new media • But exploratory studies need not be framed through existing theory

10. A liberal approach • New media CA research should adapt and expand the methods of traditional CA research • Adapt traditional concepts and practices • Innovate new techniques • Integrate other paradigms

11. Other paradigms: Computer-Mediated Discourse Analysis • A set of empirical micro-methods for analyzing online communication • Adapts linguistic (especially discourse analysis) methods • Patterns resulting from analysis may be quantified and visually represented • Typically applied to text-based CMC (Herring, 2004) “Language-focused content analysis” (Herring, 2004)

12. Other paradigms: Social Network Analysis • A sociological approach to analyzing relationships and ties between social actors • Quantitative, based on graph theory • Patterns resulting from analysis are often visually represented • Used to analyze patterns of interaction and linking on the Web (Foot, et al., 2003; Kleinberg, 1999; Wellman, 2001) A form of content analysis?

13. Depends on definition of “content” • Thematic meanings present in text or images • Various types of information "contained" in new media documents, including themes, structures, features, links and exchanges

14. Need to adapt and innovate • Units of analysis • Web • Page (Crowston & Williams, 2000) • Homepage (Bates & Lu, 1997) • Site (Warnick, 1998) • Sphere (Schneider & Foot, 2004) • CMC • Message (cf. turn; Condon & Cech, 2001) • Session (cf. conversation) • Data collection • Problems with random sampling • Web • CMC • Coding categories • Pre-defined vs. emergent • Ethical prescriptions • Etc.

15. Examples of sampling challenges • Online news sites • Web pornography • Social network sites

16. Online news sites (Kutz & Herring, 2005) • Research questions • How often does news content change? • What changes? • Sampling requirements • Representative news content • BUT: many sites, much content, content of different types • Exhaustive set of changes over a representative time period • BUT: how often is often enough?

17. Research sites • CNN • BBC • English-language version of Aljazeera

18. Sampling procedure • Top news stories from each site • Sampled automatically at 1-minute intervals over a 3-week period • iterative process

19. Units of analysis • Top news story • headline • teaser article (blurb) • image and image description

20. Resulting sample • 2,244 changes to the text (title, blurb) and images of top news stories • CNN: 369 • BBC: 257 • Aljazeera: 1,618 Classified and coded all changes; assessed inter-rater reliability

21. Coding interface

22. Findings • Top story changes every 6-7 hours • Associated images change every 3-4 hours on CNN and BBC; every 2 hours on Aljazeera • 51% of changes add no new information • Formatting, spelling, grammar corrections • Stylistic polish • Rhetorical/ideological rewording

23. Web pornography (Herring & Martinson, in prep.) • Research questions • How diverse is the content of Web pornography? • How much control do consumers have over what they view? • Sampling requirements • Representative content • BUT: vast amounts • Images that consumers with varied preferences would choose to view • BUT: difficult to observe directly

24. Research site • SexTracker (www.sextracker.com) • Largest sex portal on the Web • 5th largest advertiser on the Web • Aims to satisfy all consumer preferences

26. Sampling procedure • Hypothetical navigational trajectories of porn consumers with varying preferences • Professional (amateur, non-amateur) • Sexual orientation (gay, lesbian, hetero) • Explicit (hardcore, softcore)

27. Sample navigational trajectory SexTracker home Hetero Softcore 1st site on list (1st ad on 1st site) back/close to SexTracker home

28. Resulting sample • 10 trajectories • 85 links (4-18 per trajectory) • 188 pages (83 site content + 105 ads) • 960 images (536 site content + 414 ads) + SexTracker portal (18 ads) Coded content of all images; Assessed inter-rater reliability

29. Findings • Amateur trajectories contain mostly professional images, less diverse representations of sexual preference and fewer hardcore images than professional trajectories • Gay trajectories contain mostly male images, but lesbian trajectories contain more male images and more hardcore heterosexual images than hetero trajectories • Softcore trajectories contain more hardcore images and fewer ads than hardcore trajectories

30. CMC on social network sites (Das & Herring, in prep.) • Research questions • To what extent do users interact textually on SNSs? • How coherent are message exchanges on SNSs? • Sampling requirements • Overall picture: who is interacting with whom, how often? • BUT: each participant is multiply connected to others; millions of users • Both “sides” of the conversation • BUT: sender and receiver’s messages not in the same place

31. Research site • Orkut • online social network service run by Google • 8th most-visited site in the world; 2nd most-visited in India • aim is to “provide an online meeting place for people to socialize, make new acquaintances, and find others who share their interests” • users can interact via profile linking, communities, “scrapbooks”

32. An orkut scrapbook

33. Sample • ‘Bengali’ community, April 2007 • Coded & visualized relationships between messages using Dynamic Topic Analysis (Herring & Kurtz, 2006); 10% of sample coded together

34. Preliminary findings • Message exchange is not very coherent • Many scraps receive no response • Scraps that respond to other scraps are often narrowly or tenuously (rather than optimally) related • Exchanges are more coherent when pseudo-synchronous

36. An IM conversation

37. Risks of expanding the CA paradigm • Analyses may not be comparable between old and new media, or across researchers • Exploratory approaches may sacrifice methodological precision and interpretability of research results • Innovative analyses may not initially appear representative or reproducible

38. BUT… • Techniques tend to become systematized and best practices refined over time • New knowledge created from innovative approaches can inform future techniques, methods, theories • CA will ultimately be more powerful for having integrated innovative responses to new media phenomena

39. Conclusion • Need to balance validity and interpretability of established methods with responsiveness to new data and research questions

40. Questions? herring@indiana.edu