1 / 32

Trends in Social Network

Trends in Social Network. M. Tech Project Presentation . Guides : Amitabha Bagchi Maya Ramanath. By : Pranay Agarwal 2008CS50220. Introduction Why twitter ? Filter model for tweets Social Graph construction Evolving graphs for topics Results and Conclusion Future work. Outline.

siusan
Download Presentation

Trends in Social Network

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Trends in Social Network M. Tech Project Presentation Guides : AmitabhaBagchi Maya Ramanath By : PranayAgarwal 2008CS50220

  2. Introduction • Why twitter ? • Filter model for tweets • Social Graph construction • Evolving graphs for topics • Results and Conclusion • Future work Outline

  3. 2.4 B Online • Facebook 1.2 B • Twitter 200 M [1] Age old query : “what are people talking about” ? Introduction : Trends

  4. Search Engines • News Media • Online Contents • E-Commerce

  5. 200 Millions users, 400 Millions tweets everyday. [2] • All major news sources, Govt. offices etc. • Fast and real time • Data exposed via Twitter API. Why Twitter

  6. Needle in Haystack

  7. Model Features S(t) =Σ ( Wi∗ Fi(t)) S (t) = score of tweet t Wi = Weight of feature I Fi(t) =feature i value for tweet t Filter Model for tweets

  8. Too short tweets (<= 2 words) are always “chat” tweets. • 40 % of the “chat” tweets had one of the stop words while only 2% of the “informative” tweets had it. Stop words = (I, me, mine, you, yoursetc.) Two Phase model

  9. Goal : “Enhance Timesense(Yahoo! proprietary) capabilities using twitter.” Tasks : • Marcelo Filter model • Trend prediction using Social Graph. Yahoo! Proposal

  10. Evolving graph for a topic The hypothesis was that during evolution of graph, its structure and topology gives rise to patterns, which could act as distinct features to distinguish “trending” topic from “non-trending” topic. • Collecting data from Twitter API. • Storing and processing Social Graph. • Topic wise clusters of tweets. Spatio Temporal Analysis of Topic Popularity in Twitter

  11. Wrote a python library to communicate with the API. • Collecting friends and followers relations • Several instances of nodes making calls to Twitter API under normal rate limit.[3] • Frequent outages in the API service causing further delay and blocking. • Resolve first for “Good” users, who will be involved in creating and sharing of “informative” tweets. Twitter API

  12. Node : Each node is a user, which also contains several other details of user profiles. We also label users for which we have resolved all the relations. • Edge : An directed edge edge(u → v) represents user “u” follows user “v” stored in Adjacency List. • Index : We need to make several queries where given a user details we want to get it’s all followers and friends. To make this query fast and efficient we indexed the graph by a unique key “uid” as user id of all user nodes. This “uid” is same as the twitter user id, which is already present in the tweet object. Graph Database Neo4j [4]

  13. 30 Millions nodes and 60 Millions edges. • Our graph is only 10% of the whole twitter graph. Validation • Almost 95% of the top celebs present • Around 60% of the users of the second set present in our graph This is a very strong indication that our graph mostly contains “active” and “good” users while there could be significant fraction of twitter users as “inactive”

  14. “Timesense”, a Yahoo! Proprietary service, which gives list oftopic search queries along with “buzz” score which indicates the “trendiness” • The search queries returned by Timesense are not clustered together, which means different search queries related to same event is given as different queries mark appelhoustonmlb draft mark appel and pat appel mark appel 2013 mlb draft mark appel contract what high school did mark appel go to mark appel major stanfordmark appelstanford baseball baseball player mark appel of stanford Tweets Clustering

  15. Users in twitter use different variants of the same topic • We implemented a Bi-gram matching algorithm to cluster together search queries like these. • One pass of all the public tweets and fetch the tweet if it contains any of the bi-gram terms pair in it. N gram Matching

  16. “Topics that are going to become very popular witness intense discussion within communities at first. When the level of intensity rises then the users who bridge communities enter the discussion in a big way causing a merging of what were earlier disjoint discussions.” Evolving Graphs

  17. The vertex set of Gt0comprises the users V0who tweet about t on window 0 (the edge set is empty) • The vertex set Vit of Gti is the set of all users who have tweeted on a topic in windows 0 through i • An edge(u ← v) is added to Eit if u ∈ V (Current set) and v has tweeted about t on window i Window = 30 Minutes. Algorithm

  18. Experiments

  19. High Trend : First row, Low Trend : Second Row

  20. High Trends

  21. Low Trends

  22. Largest component size increases for all the topics. But the increase in the size is much more significant in case of topic 1 and 2 • Topics 1 and 2 contain most of its nodes in the largest • low ratio c1/c2 for topics 2 and 3 shows that there are many small independent clusters of communities discussing among themselves without leading to a large component component.

  23. Users who bridge communities enter the discussion • Bridge users serve as a barometer of the topics rising popularity External edges = {(u → v) : u ∈ S, v ∈ V \ S} Total Edges = |{(u → v) : u ∈ S}| φ(S) = External Edges / Total edges Conductance

  24. Collecting tweets clusters for these topics using above Algorithm • Resolving all relations for all the authors in these tweets. Resolve Missing Edges

  25. First row : low, Second row: high Trend

  26. Low Trends

  27. High Trends

  28. Table shows the higher conversion of external to internal edges, in case of trending topics, which means more behavior influence and spreading to followers in case of trends. • Largest connected component contains around 15 % and 35 % of all users in case of topic 1 and 2 respectively, while in case of topic 3 it is 80 % and > 90% in case of topic 4.This strongly supports the hypothesis that in case of trending topics, users form large connected community.

  29. Resolve all relations ? • May be NOT.. 0.6 Drop 0.1 Drop Limitations

  30. Identify good “sensors” or users • Resolve as many relations possible • Better topic detection and clustering from tweets. • Efficient Graph processing data Structure • Inter relation of topics Future Work

  31. Thanks

  32. [1]http://www.internetworldstats.com/stats.htm • [2] Twitter official blog • [3]  Twitter api. 2013. • [4] neo4j http://www.neo4j.org. 2013. • S. Ardon, A. Bagchi, A. Mahanti, A. Ruhela, A. Seth, R. M. Tripathy, and S. Triukose. Spatio-temporal analysis of topic popularity in twitter. CoRR, abs/1111.2904, 2011. References

More Related