GDELT: A Global Catalog of Human Society

1. GDELT: A Global Catalog of Human Society Kalev Leetaru Yahoo! Fellow in Residence Georgetown University http://www.kalevleetaru.com/ GDELT Team: Kalev Leetaru (Georgetown), Philip Schrodt (Parus Analytical Systems), Patrick Brandt (UTD)

8. Our Digital World • 1/3 global population online • As many cell phones as people on earth • Facebook alone has: • 240 billion photographs (35% of all online photos) • 1 billion members with 1 trillion connections

9. Every Year • 6.1 trillion text messages • 2.2 trillion cell minutes in the US alone • 107 trillion emails • 1.6 million days worth of video uploaded to YouTube

10. Every Day • 2.5 billion new items added to Facebook • 300 million photos posted to Facebook • 500TB of new data about society’s innermost thoughts posted to Facebook • As many words posted to Twitter every day as the entire New York Times in the last half-century • 100 billion+ social media actions taken

11. Every Minute • 600 new websites created • 204 million emails sent • 700,000 shares on Facebook • 200,000 photos posted to Facebook • 277,000 tweets sent

16. The Rise of Web News

18. GDELT:“A Realtime Social Sciences Earth Observatory” Goal: process all media worldwide into a single global catalog of behavior and beliefs in realtime Team: Kalev Leetaru (Georgetown), Philip Schrodt (Parus Analytical Systems), Patrick Brandt (UTD)

19. What is GDELT? • Physical Event Catalog: quarter-billion events in 300+ categories for all countries 1979-present • Emotional/Thematic Catalog: georeferenced catalog of thousands of emotions and themes worldwide over the same period to contextualize events (food riot vs political protest) and global reaction

20. What is GDELT? • Monitor all available news media worldwide (experimenting with citizen and social media) • Translate 79 languages in realtime into English (Google Translate for Research award) • Identify every mention of 300+ categories of events worldwide into a massive quarter-billion-record catalog of human society • Identify latent narrative dimensions (emotion and thematic undercurrents) (GDELT 2.0) • Biographical profiles (GDELT 2.0) • Network dynamics (GDELT 2.0)

21. GDELT Behavior + Beliefs

22. What is GDELT? • Collection of quarter-billion georeferenced dyadic “event” records of “X did Y to Z” across the world in 300+ category CAMEO format • Material Conflict, Material Cooperation, Verbal Conflict, Verbal Cooperation • Provides physical behavior signal to study against beliefs/narrative media signals • Essentially takes a news report about a riot and puts it on a map with all of the associated details • “Twenty former soldiers were arrested for violent anti-government protests in front of the courthouse in Abuja, Nigeria yesterday.” => Violent Protest+Arrest, Abuja Courthouse (Nigeria), 20 Soldiers • Open academic equivalent to US DOD ICEWS

23. Sources • All international news coverage from AfricaNews, Agence France Presse, Associated Press Online, Associated Press Worldstream, Associated Press, Christian Science Monitor, Facts on File, Foreign Broadcast Information Service, New York Times, United Press International, and the Washington Post. • BBC Monitoring - global broadcast+print translated material. • Google News - live realtime stream of all major international, national, regional, and local web-accessible news media. • Google Translate for Research award to translate global news in realtime.

24. Covers all countries globally • Covers a quarter-century: 1979 to present • Daily updates every day, 365 days a year • Based on cross-section of all major international, national, regional, local, and hyper-local news sources, both print and broadcast, from nearly every corner of the globe, in both English and vernacular • 58 fields capture all available detail about event and actors • Ten fields capture significant detail about each actor, including role and type • All records georeferenced to the city or landmark as recorded in the article • Sophisticated geographic pipeline disambiguates and affiliates geography with actors • Separate geographic information for location of event and for both actors, including GNS and GNIS identifiers • All records include ethnic and religious affiliation of both actors as provided in the text • Even captures ambiguous events in conflict zones ("unidentified gunmen stormed the mosque and killed 20 civilians") • Specialized filtering and linguistic rewriting filters considerably enhance TABARI's accuracy • Wide array of media and emotion-based "importance" indicators for each event • Nearly a quarter-billion event records • 100% open, unclassified, and available for unlimited use and redistribution

25. John Beieler – Global 2013 Protests

26. GDELT 1979-2013 vs NASA Night Lights

27. Rolf Fredheim – Russia Network

28. Jay Yonamine – Afghanistan

29. New Scientist Magazine - Syria

30. John Beieler – Egypt 2013

31. John Beieler – 1979-2013 Protests

32. Interactive Data-Scale Exploration • Currently piloting Google’s Big Query database service with complete GDELT database. • Live interactive complex queries across all quarter-billion records return in just 1-6s. • Explore global patterns across quarter-century in realtime.

33. GDELT Behavior + Beliefs Not just “what happened” but “how did people feel about it”

34. Egypt

35. Hosni Mubarak

38. Event Data for Strategic Forecasting Kalev Leetaru Yahoo! Fellow in Residence Georgetown University Kalev.leetaru5@gmail.com http://www.kalevleetaru.com/ GDELT Team: Kalev Leetaru (Georgetown), Philip Schrodt (Parus Analytical Systems), Patrick Brandt (UTD)