Expertise Networks in Online Communities: Structure and Algorithms

1. Expertise Networks in Online Communities: Structure andAlgorithms Jun Zhang Mark S. Ackerman Lada Adamic University of Michigan WWW 2007, May 8–12, 2007, Banff, Alberta, Canada.

2. Outline • Automatically identify users with expertise. • Analysis of the java forum • Test various network based ranking algorithms such as HITS and PageRank • Use simulations rules to evaluate how other alogorithms perform on Java Forum. • Evaluate performance in communities with different characteristics.

3. Expertise Finder – Systems that help to find others with appropriate expertise to answer a question. Current Expertise finders – Modern Information retrieval techniques. Represent as term vector, match expertise queries using standard IR techniques. Problem : Reflect if a person knows about a topic but does not distinguish person’s relative expertise levels. Solution – Use network based ranking algorithm + content analysis. Introduction

4. Expertise Network • Usually have discussion thread structure • Not a network focused on social relationships • User replies because of interest in content. • CEN – Community Expertise Network – Distribution of expertise along with network responses • Structural Prestige – Closely related. Receiving more positive choices is prestigious.

5. Empirical Study – Java Forum • People come to ask questions. • 87 sub forums with large diversity of users. • 333,314 messages in 49,888 threads. • 13,739 nodes and 55,761 edges. • Used human raters and selected 135 users – omitting users postings less than 10 times.

6. Characterizing the Network • Bow-tie Structure analysis • Degree Distribution – To capture Level of interaction. • Scale Free - Highly uneven distribution of participation. • Degree Correlations • Indegree – how many people a given user helps. • Does not provide users’ own tendency to provide help- Eg. Only reply to newbies or talk to similar expertise level people. • For Each asker-replier count indegree of replier vs asker.

7. Expertise Ranking Algorithms • Simple Statistical Measure • Answers lot = knows the topic well. • Spammers – inflammatory or disruptive posts. • Handling Problem • Users’ relevance feedback. • AnswerNum – No of questions answered. • Also count no of users a user helped. • Shows broader or greater expertise.

8. Z- Score Measures • Replying many = High Expertise • Asking many = lacks expertise on topics • Z – Score Combines both q + a. • Measure how different from a random user • Post answers with p = 0.5 so n*p =n/2 replies • Std Dev. Sqrt ( n*p*(1-p) = Sqrt(n) / 2 • Asks and answers ~= 0, Answer more +

9. Expertise Rank Algorithm • Problem in Counting no posts • user answered 100 newbie questions ranked equally expert as 100 advanced users’ ques. • Adopt method similar to PageRank. • Intuition B<-A and C<- B .C’s Expertise boosted. • C(Ui) – Total no of users helping U1 • d – Damping factor was set to 0.85 • Could also be weighted including WiA – No times i was helped by A • In this study, weighting does not improve the accuracy.

10. Evaluations • 2 raters- Java Programming experts. • Five Levels of Expertise Rating.

11. Statistical Metrics • Frequently used correlation measures • Spearmans rho : Does not handle weak ordering(i.e. Multiple items in ranking such that neither item is preferred over the other). • Kendall’s Tau : Gives equal weight to any interchange of equal distance, no matter where it occurs. Eg between 1 & 2, 101 &102 • TopK :Calculates Kendall’s Tau only for highest 20 ranks

12. Performance of Various Algorithms in different statistical metrics.

13. Simulations • The Need for it • Understanding the human dynamics that shape an online community. • This will help select appropriate algorithm for communities where dynamics different from the Java Forum. • 2 Models - Best Preferred and Just Better Network

14. Best Preferred Network • Many experts answered others’ questions and seldom asked questions. • Very much similar to the Java Forum. P of replying increases exponentially with expertise level difference between 2 users

15. Just Better Network • Eg. Within an Organisation, experts may be under time constraints. Choose to answer only questions makes best use of their expertise. • Users having slightly better level of expertise answers. • U’s probability of answering a’s question

16. Contd… • Users make best use of their time • They are more selective in answering. • ExpertiseRank propagates expertise score from newbies to intermediate users who answer their question. • From them to experts. • In General ExpertiseRank outperforms others.

17. Network generated from both the models.

18. Summary & Future Work • Structural Information can be used to evaluate expertise network in online setting. • Relative expertise could be found using social network-based algorithms. • These algorithms did nearly as well as human raters. • In Future, Combine content information – to differentiate specific knowledge and structural information.

19. THANK YOU !!!