Content Analysis in Journalism: Techniques and Applications

1. Journalism 614:Content Analysis

2. Content Analysis • Study of a recorded human communication • The coding of communication for the presence of certain traits, categories, or meanings • Analysis can relate the occurrence of coded content with other factors, such as features of the producer, effects on the receiver, etc. • Applied to the study of books, magazines, papers, transcripts, web pages, songs, speeches, postings,, statements, utterances, etc.

3. Topics of Content Analysis • Well suited for communication research • Critical for answering the classic question… Who says what, to whom, and with what effect? • Yet content differences do not equal effects • Must move beyond simple content studies to relate coded features to antecedents or consequences • Relating content features to one another is also meaningful - i.e., what political issues are featured in coverage of each presidential candidate?

4. Sampling in Content Analysis • Since you can rarely observe all content, must sample from available content for coding pool • Units of analysis may differ from units of observation • Observe story content to analyze newspaper differences • Sample selection depends largely on unit of analysis • Example, if studying differences between authors, the unit of observation may be books, pages, paragraphs, or sentences • Need to be clear about unit of analysis before planning sampling strategy to avoid problems later

5. Questions in Sample Generation • Must establish the universe to be sampled from • Ex - Content analysis of television violence by network • Which TV stations should you observe? • How many days will you observe them? • During which hours will you observe television? • You always make assumptions; be upfront about them • We will code the five major networks — ABC, CBS, NBC, FOX, CW — for a “random week” during the month of December from 7 PM to 10 PM, each day selected. • May be Random, Systematic, Stratified, etc.

6. Example for your exercise/project • First: Search for news stories containing relevant keywords during your specified time period from specified sources • Ex. You find 1816 stories mentioning “global warming” or “climate change” in the New York Times from 1/1/96 - 12/31/15 • Second: Randomly or Systematically sample a sufficient number of stories from the full pool of stories • Ex. Say you want 300 stories, you need approximately every 6th story (i.e., 1816/300 ≈ 6). So you pick a random number start number between 1 and 6 and then choose every 6th story. • Or you can use an online random number generator to pick 300 random numbers for you (www.random.org). • Third: Content code the 300 stories for relevant features.

7. Coding in Content Analysis • Coding is the heart of content analysis • Process of converting raw data into a standardized form • Classify content in relation to a conceptual framework • Ex. Emotionality, Partisan Bias, Source Attribution, etc. • Must carefully conceptualize coding categories • Relevant concepts and relevant categories within concepts • Manifest (visible) / Latent (underlying meaning) • How big a leap between observation and inference • The more manifest, the more reliable - ex. word count, date • The more latent, the more interesting - ex. meaning, frames

8. Data Management • End product of coding in usually numerical • Distinguish units of analysis and observation • Establish the base for coding (i.e., proportion) • Understand the limits of your coding system

9. Additional Considerations • Problems with coding long periods (100 years) • Imposing modern standards on the past • Coding TV content is technically difficult • Vanderbilt Univ. archive of network news coverage • Transcripts from Lexis/Nexis and “Closed Captions” • Emergence of computer-aided techniques • VBPro, Diction, LIWC, Wordstat, and others • Dictionary-based, customizable, syntactical • Latest is machine learning, unsupervised (topic modeling) or supervised (classify using training set)

10. Strengths and Weaknesses • Easy to undertake - no staff, no special equipment • Easy to correct errors - go back and recode • Allows for the study of dynamic processes - time • Unobtrusive - no effects on subject of study Yet…. • Limited to recorded communication - much is lost • Limited in terms of claims you can make

11. Content Coding Exercise • Must code text with a high degree of agreement • Reliability between coders is a key criteria • Can we agree on coding climate change stories? • Three content features: Frame, Certainty, IPCC • Frame - Science (emphasis on scientific findings) or Politics (emphasis on policy debate) - forced choice • Certainty - How certain is the threat of human-made global warming? (low, medium, high) • Does it mention the IPCC - Intergovernmental Panel on Climate Change? (yes or no)

12. Content Coding Example #1 “Some of the world's most distinctive and biologically diverse climate regions – from South America's Andes Mountains to southern and eastern Africa and the US Southwest – may be drastically altered by century's end, endangering plant and animal life there, if there are no curbs on greenhouse gas emissions. This according to a new climate-modeling report issued March 26. The researchers built their forecast on data contained in a massive study being published in installments this year by the Intergovernmental Panel on Climate Change.”

13. Content Coding Example #2 “Most climatologists agree that the earth's temperature has increased about a degree over the last century. The debate is how much of it is due to mankind's activity. Britain's Channel 4 television has just produced "The Great Global Warming Swindle," a documentary that devastates most of the claims made by the environmentalist movement. The scientists interviewed include top climatologists from MIT and other prestigious universities around the world. The documentary hasn't aired in the U.S., but it's available on the YouTube.”

14. Content Coding Example #3 “U.S. researchers report that many of the world's current climates may disappear if current global warming trends continue, while climates unlike any seen today would be created, increasing the risk of extinctions and other ecological events. Professor John Williams, University of Wisconsin, Madison, and colleagues forecast the risk of novel or disappearing climates by the year 2100, using global climate models and greenhouse gas emission scenarios from the recent assessment by the Intergovernmental Panel on Climate Change (IPCC).”

15. Content Coding Example #4 “To opponents of global warming, it must have seemed like a perfect storm: Al Gore pocketed an Oscar for his doomsday climate documentary, An Inconvenient Truth. The International Panel on Climate Change (IPCC) issued a report with dire warnings about CO2 emissions. Even George Bush offered conciliatory talk about reducing carbon pollutants. But then an unwelcome squall appeared on the horizon: The documentary The Great Global Warming Swindle. Director Martin Durkin's film combines interviews with distinguished scientists, a sober narrative, and damning graphs and statistics to challenge the advocacy politics of global warming.”

16. Content Coding Example #5 “Australia will spend $200 million ($161 million U.S) to reduce the clearing of forests in Asia, Prime Minister John Howard said. The funds to support sustainable logging would help curb greenhouse gas emissions, Howard told reporters in Canberra. His Liberal-National government, trailing the main opposition Labor Party in opinion polls, is tackling climate change as it heads to an election this year.The government has refused to ratify the Kyoto Protocol that sets a timetable for cutting greenhouse gas emissions, saying it would hurt economic growth.”

17. Content Coding Example #6 “While it is incontestable that the atmosphere is warming with possible dire consequences, it is not agreed on by all climatologists and scientists that man is the sole cause of this phenomenon. Many alternatives have been advanced to the theory that burning fossil fuels is the culprit. These claims include solar radiation, changing ocean currents, the effect of clouds, and cosmic rays from outer space. Among scientists who remain skeptical of the popular CO2 theory is Professor David Bromich of Ohio University. Bromich has stated, “The climate models I have seen are inconsistent with the evidence that we have for the past 50 years from Antarctica.”