The Performance of Distributed News Aggregators

1. The Performance ofDistributed News Aggregators Wolf Richter & Tobias Escher Oxford Internet Institute, University of Oxford re:publica 2 April 2008

2. Coming up • Introduction to Netizen Journalism as an alternative to “Traditional media” • Taxonomy of news aggregators • Our methodology to assess performance • Our Findings • Remaining issues

3. Calling at… • Introduction to Netizen Journalism as an alternative to “Traditional media” • Taxonomy of news aggregators • Our methodology to assess performance • Our findings • Remaining issues

4. Critique of “traditional” or “mainstream” media • Mass media effects on public • opinion • behaviour • Gatekeeping • Corruption • Decline in quality • “Overall, our research suggests that 60% of press articles and 34% of broadcast stories come wholly or mainly from one of these ‘pre-packaged’ sources.” Lewis et al (2008)

5. source: Amazon The Internet as a Counter Public, Netizen Journalism as the 5th Estate • Netizen-journalism comes with big promises • Reliable fact-checking • Self-selection / become your own editor • Empowerment of formerly passive audiences • Ordinary citizens (Hindman) • “The blogosphere is smashing the old media monopoly and giving individuals power in the marketplace of ideas.” (Hugh Hewitt Blog 2005) • High profile cases have demonstrated the power of Netizen-journalism as a 5th estate • Reutersgate • Rathersgate • Salman Pax • US senator Trent Lott

6. News selection • News publication New media environment empowers users at every stage of the news production process • News production process • Empower-ing effect • Every user can gather and publish “news” on the Web • Every user can search and aggregate news from an abundance of content available on the Web • Criticism • Dominance of “old elites” in the blogosphere • Most publications taken from traditional media or trivial content • “Echo Chamber” amplifies existing views and narrows perspectives Source: OII

7. source: Schmidt et al, 2006 What evidence do we have to fact-check the promises of the participatory media environment? • A-list bloggers = traditional elites • “With only one exception, all of the bloggers in our census [75 A-list bloggers] were either educational elites, business elites, technical elites, or traditional journalists. (Hindman, forth.) • Major part of content about “what I had for lunch” • “76% of bloggers say a reason they blog is to document their personal experiences and share them with others” Lenhart,Pew (2006) • Partisanship / inaccuracy

8. user-generated content Netizenjournalism news aggregation performance? What do we actually look at?

9. Calling at… • Introduction to Netizen Journalism as an alternative to the “traditional media” • Taxonomy of news aggregators • Our methodology to assess performance • Our findings • Remaining issues

10. A simple taxonomy of news aggregators

11. Calling at… • Introduction to Netizen Journalism as an alternative to “Traditional media” • Taxonomy of news aggregators • Our methodology to assess performance • Our findings • Remaining issues

12. Measures of performance for news aggregators • Appropriate quality measures for analysis depend on • Function • Target audience • Genre (news, comment) • Publication cycle • Medium (print, television) • Journalistic self-perception

13. Types of indicators for performance performance Held / Ruß-Mohl (2000)

14. PEJ study found Distributed News Aggregators to be transitory and tech-heavy Project for the Excellence in Journalism (2007) • study compared distributed news aggregators (Digg News, del.icio.us, and Reddit) with aggregators of “traditional news” like Yahoo! News • user-driven sites with no clear agenda • “In short, the user-news agenda, at least in this one-week snapshot, was more diverse, yet also more fragmented and transitory than that of the mainstream news media […]” • “What were the favorite stories on the user-driven sites? For the most part, there were no dominant ones. The only story with any real traction was the release of the Apple iPhone, and that was just on one site (it accounted for 16% of the stories on Digg that week). Otherwise, users put forth a mix of diverse and unconnected news events from day to day. “ • user-driven sites more tech-heavy • “In a week when the mainstream press was focused on Iraq and the debate over immigration, the three leading user-news sites—Reddit, Digg and Del.icio.us—were more focusedon stories like the release of Apple’s new iphoneand that Nintendo had surpassed Sony in net worth, according to the study.”

15. How do we measure ‘Performance’? • Timeliness and continuity chosen as performance measures: • Timeliness – how often and when do new articles appear in the feed • Continuity – do different articles deal with the same topic • Rationale • Distributed News aggregators praised for flexible reaction to current news events – no blackout of events due to frequency • Blogosphere criticized for punctual and arbitrary focus of attention – no set “agenda” no collective memory to pursue topics • User sites criticized as being tech-heavy. Therefore distinguish World News and Technology News • World News: major events, emphasis on politics (e.g. US elections, war in Iraq)

16. Observed news sources

17. Data collection process Every 3 hours31st January – 19th March

18. Calling at… • Introduction to Netizen Journalism as an alternative to “Traditional media” • Taxonomy of news aggregators • Our methodology to assess performance • Our findings • Remaining issues

19. Weekend Weekend Weekend Weekend Weekend Traditional aggregators Distributed aggregators World news: Traditional media show higher percentage of new articles per day Percentage of new “World news” articles per day

20. Weekend Weekend Weekend Weekend Weekend Tech news: Digg outperforms “traditional” media, which still seem to do a decent job… Percentage of new “Tech news” articles per day

21. Weekend Weekend Weekend Weekend Weekend … but lack even further behind if you compare the absolute numbers of articles Number of new “Tech news” articles per day

22. Weekend Weekend Weekend Weekend Weekend In absolute numbers the output of “Traditional media” is higher for World News Number of new “World news” articles per day

23. Night Night Night Night Night Timeliness: World News show clearly visible update cycles throughout the day Percentage of new “World News” articles per cycle (3h) Weekend fatigue

24. Timeliness: In Tech News more continuous update activity visible at Digg and Slashdot Percentage of new “Tech News” articles per cycle (3h)

25. World news – lifespan of articles World News Tech News

26. Summary findings: Timeliness • Traditional news show stronger performance (Percentage and absolute number of new articles) for World News and still fair performance for Tech News in comparison to distributed aggregators • Both traditional as well as distributed aggregators have clearly defined update cycles • Digg and Slashdot have more continous update patterns for their focus area technology • Weekends and nights are marked by lower activity for both paradigms • OhMyNews and Global Voices not competitive for World News and Tech News in timeliness of news production • In their respective focus areas (World News and Tech News) BBC/IHT and Digg/Slashdot show remarkably similar patterns:Critical vs subcritical mass?

27. Publish, then filter – the example of Digg filter for all content filter for technology content

28. Digg: Lifespan of articlessubmitted vs. filtered every hour:~4 new articles every hour:~200 new articles every hour:~1 new articles every hour:~100 new articles

29. Summary findings: Digg’s timeliness • On average, 55% of stories are replaced on Digg’s Front page (All popular) per update cycle. • Wu & Huberman (2007): “This characteristic time is consistent with the fact that a story usually lives on the front page for a period between 1 and 2 hours.” • However, some articles survive longer as during a day not all articles are replaced. • Community applies tougher filter to submitted technology news than to world news • 2% of articles make it from upcoming to popular for World News • 1% of articles make it from upcoming to popular for technology news • For Tech News, Digg produces a more stable “agenda” than for World News. Claims that distributed aggregators do not have an agenda should be corrected by looking at self-selected focus area of community

30. Continuity • PEJ study: user-news agenda (…)more fragmented and transitory than that of the mainstream news media • collection process • download articles • Yahoo Term extractor to reduce to important key phrases • further reduction by filtering stopwords & stemming They say representatives of Mr Mugabe, military chiefs and the opposition have held meetings chaired by South Africa since Saturday's elections. The sources say Mr Mugabe is to give an address to the nation but urge caution until the announcement has been made. Zimbabwe and South Africa's governments denied any negotiations or a deal. But the US embassy in the capital Harare said Mr Mugabe was expected to address the people of Zimbabwe shortly. Claim and counter-claim Western diplomats had been summoned to a meeting with the government, the embassy told the BBC. president mugage zimbabwe election political parties .... mugabe zimbabwe elect politic part ....

31. Continuity: Share of articles per key phrase

32. Summary Continuity • World News • “Distributed”: more fragmented & narrower agenda • “Traditional”: less fragmented, more connected & broader agenda • For Tech News • Digg and Slashdot show best performance in this area • BBC and IHT show average performance • overall: performance of traditional models not reached by new distributed models

33. Entre-preneurs • Divert traffic from traditional news sources by designing new products and service catering to the needs of Netizens; monetize attention through ads • Bloggers • Bloggers gain more credibility through the “peer review” process and get wider exposure to the public • Traditional news reporters • Get access to new topics and voices previously unheard of. Blog search delivers wide variety of opinions, but so far little access to original voices Who captures the benefits created by the new Participatory Media Environment? • Benefits • Users • Get access to news specifically catering to their interests Source: OII

34. Calling at… • Introduction to Netizen Journalism as an alternative to the “traditional media” • Taxonomy of news aggregators • Our methodology to assess performance • Our findings • Remaining issues

35. Remaining issues • OhMyNews International not as successful as original Korean version • Hard to find a representative for “the Blogosphere” • Measurement of continuity needs improvement: • Better keyword extraction • Better matching algorithm • Manual check

36. THANK YOU! • Questions now => ASK! • Questions later => WRITE! • Wolf.Richter@oii.ox.ac.uk • Tobias.Escher@oii.ox.ac.uk • and read our blogs ;) • http://people.oii.ox.ac.uk/escher/ • http://people.oii.ox.ac.uk/richter/

37. THE END

38. References • Hindman, M. S. (forthcoming). Voice, Equality, and the Internet. • Held, B. and S. Ruß-Mohl (2000). Qualität durch Kommunikation. Qualität durch Kommunikation sichern. Vom Qualitätsmanagement zur Qualitätskultur. Erfahungsberichte aus Industrie, Dienstleistung und Medienwirtschaft. B. Held and S. Ruß-Mohl. Frankfurt am Main. • Lewis, J., A. Williams, et al. (2008). The Quality & Independence of British Journalism. MediaWise Report, Cardiff University. • Lenhart, A. and S. Fox (2006). Bloggers: A portrait of the internet's new storytellers. Washington D.C, Pew Internet & American Life Project. • Project for Excellence in Journalism (2007). The Latest News Headlines—Your Vote Counts, Project for Excellence in Journalism: Understanding News in the Information Age. • Schmidt, J. (2007). Online Mainstream Media and Weblogs: Competing or Complementary Publics. Boundaries of Free Speech? Jerusalem, Israel. • Schmidt, J., M. Paetzolt, et al. (2006). Stabilität und Dynamik von Weblog-Praktiken? Ergebnisse der Nachbefragung zur “Wie ich blogge?!”-Umfrage. Berichte der Forschungsstelle „Neue Kommunikationsmedien. Bamberg, Germany.

39. Feeds