1 / 32

Crowdsourcing Insights with Opinion Space

Crowdsourcing Insights with Opinion Space. Ken Goldberg , IEOR, School of Information, EECS , UC Berkeley. “We’re moving from an Information Age to an Opinion Age.” - Warren Sack, UCSC. Motivation. Goals Engage community Understand community Solicit input

livvy
Download Presentation

Crowdsourcing Insights with Opinion Space

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Crowdsourcing Insights with Opinion Space Ken Goldberg, IEOR, School of Information, EECS, UC Berkeley

  2. “We’re moving from an Information Age to an Opinion Age.” - Warren Sack, UCSC

  3. Motivation Goals • Engage community • Understand community • Solicit input • Understand the distribution of viewpoints • Discover insightful comments Goals of Community Members • Understand relationship to other community members • Participate, express ideas, and be heard • Encounter a diversity of viewpoints

  4. Motivation Classic approach: surveys, polls Drawbacks: limited samples, slow, doesn’t increase engagement Modern approach: online forums, comment lists Drawbacks: data deluge, cyberpolarization, hard to discover insights

  5. Approach: Visualization

  6. Approach: Level the Playing Field

  7. Approach: Wisdom of Crowds

  8. Related Work: Visualization Clockwise, starting from top left: Morningside Analytics, MusicBox, Starry Night

  9. Related Work: Politics Clockwise, starting from top left: EU Profiler, Poligraph, The How Progressive Are You? quiz

  10. Related Work: Opinion Sharing • Polling & Opinion Mining • Fishkin, 1991: deliberative polling • Dahlgren, 2005: Internet & the public sphere • Berinsky, 1999: understanding public opinion • Pang & Lee, 2008: sentiment analysis • Increasing Participation • Bishop, 2007: theoretical framework • Brandtzaeg & Heim: user study • Ludford et al, 2004: uniqueness & group dissimilarity

  11. Related Work: Info Filtering • K. Goldberg et al, 2001: Eigentaste • E. Bitton, 2009: spatial model • Polikar, 2006: ensemble learning

  12. Opinion Space: Live Demonstration

  13. Six 50-minute Learning Object Modules, preparation materials, slides for in-class lectures, discussion ideas, hand-on activities, and homework assignments.

  14. To try it: google: “opinion space” contact us: http://goldberg.berkeley.edu

  15. Dimensionality Reduction low variance projection maximal variance projection

  16. Dimensionality Reduction Principal Component Analysis (PCA) • Assumes independence and linearity • Minimizes squared error • Scalable: compute position of new user in constant time

  17. Canonical Correlation Analysis z • 2-view PCA • Assume: • Each data point has a latent low-dim canonical representation z • Observetwo different representations of each data point (e.g. numerical ratings and text) • Learn MLEs for low-rank projections A and B • Equivalently, pick projection that maximizes correlation between views x y Graphical model for CCA x = Az + ε y = Bz + ε z = A-1x = B-1y

  18. CCA on Opinion Space z • Each user is a data point • xi = user i’s responses to propositions • yi = vector representation of textual comment • Run CCA to find A and B, use A-1 to find 2D representation • Position of users reflects rating vector and textual response • Ignores ratings that are not correlated with text, and vice versa • Given text, can predict ratings (using B) x y Graphical model for CCA x = Az + ε y = Bz + ε z = A-1x = B-1y

  19. Multidimensional Scaling • Goal: rearrange objects in low dim space so as to reproduce distances in higher dim • Strategy: Rearrange & compare solns, maximizing goodness of fit: • Can use any kind of similarity function • Pros • Data need not be normal, relationships need not be linear • Tends to yield fewer factors than FA • Con: slow, not scalable j δij i j dij i

  20. Kernel-based Nonlinear PCA • Intuition: in general, can’t linearly separate n points in d < n dim, but can almost always do so in d ≥ n dim • Method: compute covariance matrix after transforming data into higher dim space • Kernel trick used to improve complexity • If Φ is the identity, Kernel PCA = PCA

  21. Kernel-based Nonlinear PCA Input data KPCA output with Gaussian kernel • Pro: Good for finding clusters with arbitrary shape • Cons: Need to choose appropriate kernel (no unique solution); does not preserve distance relationships

  22. Stochastic Neighbor Embedding • Converts Euclidean dists to conditional probabilities • pj|i = Pr(xi would pick xj as its neighbor | neighbors picked according to their density under a Gaussian centered at xi) • Compute similar prob qj|i in lower dim space • Goal: minimize mismatch between pj|i and qj|i: • Cons: tends to crowd points in center of map; difficult to optimize

  23. Metavid

  24. Six 50-minute Learning Object Modules, preparation materials, slides for in-class lectures, discussion ideas, hand-on activities, and homework assignments.

  25. Opinion Space: Crowdsourcing Insights Scalability: n Participants, n Viewpoints n2 Peer to Peer Reviews Viewpoints are k-Dimensional Dim. Reduction: 2D Map of Affinity/Similarity Insight vs. Agreement: Nonlinear Scoring Ken Goldberg, UC Berkeley Alec Ross, U.S. State Dept

  26. Opinion Space Wisdom of Crowds: Insights are Rare Scalable, Self-Organizing, Spatial Interface Visualize Diversity of Viewpoints Incorporate Position into Scoring Metrics Ken Goldberg UC Berkeley

More Related