Reporting Data in the Social Media World Panel Title: Practical Applications for Social Media in Survey Research

1. Reporting Data in the Social Media World Panel Title: Practical Applications for Social Media in Survey Research Casey Langer Tesfaye American Institute of Physics

2. It has always been difficult to control the trajectory of published data • Can we leverage social media to increase • Awareness of trajectory? • Accessibility of findings? • Accuracy of interpretations? Problem

3. Twitter search stream • Women + physics • September + October 2013 Proposed solution- topical case study At the Solvay Conference on Physics in 1927, the only woman in attendance was Marie Curie (bottom row, third from left).Source:http://ontd-political.livejournal.com/10806258.html#ixzz30UyWOvOz

5. Twitter search stream • Women + physics • September + October 2013 • Find references to survey data • Look for patterns Proposed solution- topical case study

6. Twitter search stream • Women + physics • September + October 2013 • Find references to survey data • Look for patterns • Goal: practical, data-driven recommendations Proposed solution- topical case study

7. Corey S. Powell ?@coreyspowell Oct 10 Men and women really do communicate differently...according to an analysis of old Enron emails. https://medium.com/the-physics-arxiv-blog/e7dbd9d6a518 … View summary Reply Retweet Favorite More Olivier DA COSTA ?@olivierdacosta Oct 10 Why Are There Still So Few Women in Science? ¦ NYT - Meg Urry, professor of physics and astronomy at Yale http://nyti.ms/18ROWK9 View summary Reply Retweet Favorite More Physics ?@Physics360 Oct 10 Women: Life Sciences v. the Lifeless Sciences: My current Taki's article reflects upon an impassioned New York... http://bit.ly/19jPruC Expand Reply Retweet Favorite More James Holloway ?@grasshapa Oct 9 “Data Mining Reveals the Emotional Differences in Emails Written by Men and Women” by @arxivblog https://medium.com/the-physics-arxiv-blog/e7dbd9d6a518 … View summary Reply Retweet Favorite More Matthew Feickert ?@HEPfeickert Oct 9 For those applying to grad school in physics, you should seriously consider a department's Female Friendliness. | http://www.aps.org/programs/women/female-friendly/index.cfm … Expand Reply Retweet Favorite More

8. Twitter data needs a framework or structure • Generalizing conclusions is tricky territory • Completeness • Representativeness • BUT there is much to be learned Working with Twitter data

9. First goal: transform the data into a usable form Data Structure

10. Dataset had 1894 lines, which collapsed down to 251 tweets • 66 no url • 185 with url Some basic numbers

11. Isolate relevant tweets • Some don’t reference survey data or relevant articles Groom the data

12. I only want references to pieces that report survey data • but how? Grooming text data is challenging

13. Some Tweets are easy to remove

14. Some are harder to judge

15. In a larger scale data set, manual investigations are impossible • Start with a sample, develop a strategy, hone and measure its effectiveness Automating is the way to go

16. Structure the data

17. 40% of the tweets containing links led to survey data (n=74) • Only 6 of those tweets linked to original sources Some metrics

18. How to group Tweets or sources that belong together? Looking for a starting point

19. Many different ways to look at links • Link in Tweet (lowest # of dupes) • Full link of referred url • Autolink from url Link duplication

20. By Autotext: 52 of 74 Tweets were in duplicate sets Link duplication

21. Largest set: 25 Tweets

22. Date range of Tweets October 3-20 NY Times article

23. The Viral Myth and Justin Bieber Effect Source: http://www.technologyreview.com/view/512271/researchers-peek-at-the-structure-of-the-viral-internet/ Original paper: http://research.microsoft.com/pubs/176494/diffusion.pdf

24. Second largest set: 13 Tweets

25. Let’s Talk about Autotext

27. Text comes from: • Autotext • Title • photo caption • article content • from the user themselves Tweets come from a variety of sources

28. Strategies include • Summarizing the story • Personalizing the story • Pulling out key points • Commentary about the article Tweets use a variety of strategies

30. And yet only one Tweet used the autolink:

31. None of the autotext included stats

33. Easy to spot large scale misinterpretations • But how to catch small? • What do misinterpretations look like? • Case study? Corpus? • Comment streams or a variety of searches can provide more data Accuracy

34. Tweeters are independent content curators • We can’t control what they say or how they say it or even necessarily be aware of all of it • But maybe we can afford the kind of talk we want People of Twitter

35. Twitter behavior patterned, but far less streamlined than we might have assumed In summary

36. What are the key metrics?* • Timeline of Tweets • Twitter handles (counts, networks) • Source of text In summary

37. Consider your goals: • To be credited with analysis • To spread correct information • Make sharing easy • Link to specific article, not a dynamic page stream • Prime headlines, captions, autotext, and links for sharing • Be aware of key players Some guidelines

38. Monitor Tweets over time • Set up API & web crawler • Generate and check custom metrics (e.g. dashboard) • Store whole articles and compare text to Tweet • Isolate & compare citations of stats? Potential areas for future work

39. Network structure of dissemination for stories • Find key nodes in your network Potential areas for future work

40. Email me: Casey Langer Tesfaye: clanger@aip.org Questions? Comments? Suggestions?

41. Extra slides start here Really? There’s more? I’m not amused.

42. Can we know our networks? What kinds of searches can or should we monitor and how? How can we set up our research results in the most shareable way? Questions

43. In the Fall of 2013 the persistently low proportion of women in physics became a hot topic of discussion. This discussion was in part fueled by a series of reports that grew additional lives once they were published, echoing among individual blog posts, news sites such as the Huffington Post and the New York Times, and reverberating into scientific magazines. The published pieces generated countless comments and tweets among readers and interested parties. What follows is an analysis of the interpretations, misinterpretations and trajectories of published findings, anchored in the “women physics” keyword search on Twitter. This goal of this analysis is to generate helpful, data-driven guidelines for researchers who are now forced into a social media space and hoping to stem misinformation and misreports of their results. Talk abstract

45. Research questions can evolve from the data • Timeline of dissemination • Network of dissemination • Use of autotext • Original sources? • Correct interpretations? Research Questions

46. Sharelines

47. Only one account tweeted links to survey data more than once Network analysis?

48. http://computationalculture.net/article/the-algorithmization-of-the-hyperlinkhttp://computationalculture.net/article/the-algorithmization-of-the-hyperlink Read this: