Using Large -Scale Argumentation to Enable Open Innovation

1. Using Large-Scale Argumentation to Enable Open Innovation Mark Klein

2. Open Innovation: A Powerful Emerging CI Tool

3. A Promising Approach for Complex Challenges “No matter who you are, most of the smartest people work for someone else” (Joy’s Law) Complex societal problems inherently involve many stakeholders and areas of expertise, and require innovative out-of-the-box solutions Open innovation can be a good match for this: • Idea synergy • The long tail (marginals • Many hands • Many eyes • Wisdom of the crowds • Volunteer ethic

4. Challenges with Open Innovation at Scale “Upon this gifted age, in its dark hour, Rains from the sky a meteoric shower Of facts…they lie unquestioned, uncombined. Wisdom enough to leech us of our ill Is daily spun; but there exists no loom To weave it into fabric…” Edna St Vincent Millay. Huntsman, What Quarry? (1939)

5. Challenges with Open Innovation at Scale • High harvesting costs: • redundancy, due to ignorance or manipulation • signal-to-noise: only ~10% of ideas are worth consideration • Google project10tothe100 needed 3,000 volunteers & 9 months to filter/sort ideas • Open for Questions was shut down due to overload • IBM flew 100 executives to New York for a week to filter Web Jam ideas • Shallow ideas: lots of rudimentary ideas • solo ideas w/o collaborative refinement • tough to track and reward IP – “I didn’t see he had the same idea” • Unsystematic • No mechanism to ensure coverage of critical issues/ideas • Shallow evaluations: • quick personal impressions, not customer criteria-based • little/no analysis and fact-checking • Rating dysfunctions • With large corpuses, users tend to rate only the highest-rated ideas, creating feedback loops => rating lock • Process opaqueness • “It’s tough to steer when you can’t see where you’re going”

6. Argument Mapping Can Help! Planeta.com (5/1/08) had a 13-page discussion on carbon offsetting pros and cons

7. Argument Mapping Can Help! • No redundancy: can ensure each unique point appears just once • Better coverage: easier to find gaps that need to be filled • Deeper ideas: crowd builds issue/idea trees instead of standalones • Credit tracking: non-redundancy makes idea precedence clear

8. Open Issues: Argument Mapping at Scale • Structure • How do we keep large growing maps well-structured? • Content • How do we foster good (creative, detailed) solution ideas? • How do we rapidly prune the top 10% of ideas? • How do we ensure good (well-founded, complete) evaluations? • Process • How do authors know where they can best contribute, in large maps? • How do managers know what process problems need intervention? • Outcome • How do customers know which content is ready to ‘harvest’?

9. Overview of Approach

10. High-Speed Idea Pruning: Bag of Stars • A key trick: assign posts to users in a way that maximizes value e.g. • to posts whose value is uncertain • to calibrate ratings across multiple rater pools • ...

11. Rate to highlight worthy posts Certify well-structured posts - a coaching role Structure Moderators Unbundle – break your thoughts into points that each contain just one issue, idea, or argument. Locate – search the argument map to see where your point(s) belong Enter – If it’s a new point, create a new post, else refine existing post. The live-and-let-live rule: only edit a post to strengthen it The honest broker rule: remain strictly content-neutral Only certified posts can be viewed by readers

12. Extended IBIS to Support Design Problems • Added criteria to IBIS so we can track evaluation completeness: • Augment issues with list of criteria successful solutions should satisfy • Annotate arguments with the criteria they address Issue: what actions can help avoid global warming? criterion: low-cost criterion: effective criterion: few negative side-effects Idea: reduce the energy-intensity of products and services Pro: typically has positive ROI (addresses: low-cost)

13. Recombinant Design • Map defines a multi-dimensional design space • Issues = design dimensions • Complete solutions are packages (combinations) of ideas • Benefits • Enables systematic exploration of design space • Fosters creativity by recombination • Challenges • Creates very large search spaces due to exponential combinatorics

14. selected best Nonlinear Negotiation optimum U(A2) optimum U(A1) Contracts How find pareto-optimal solution packages in very large search spaces with self-interested stakeholders? Standard (linear) negotiation protocols fare poorly in such contexts New protocols, often counter-intuitive, are needed

15. Metrics & Attention Mediation A free open-source web service that deliberation platforms can call to calculate metrics for their maps A growing library of metrics, defined based on systematic analysis of deliberation needs and challenges A language for defining “alerts” that notify participants of metrics signal that they need to know about

16. Identifying Metrics:process-goal-exception analysis

17. ImplementingMetrics Data mining (dimensionality reduction , hierarchical clustering ...) Belief propagation (e.g. Bayesian) Network topology (includes social network) analysis Statistical analysis Graph-based query languages (e.g. SPARQL) Time series analysis A little semantics, in abundance, allows us to define metrics of unprecedented power compared to those for social media

18. Maturity Metric: Network analysis versus

19. Groupthink Metric: Gini coefficient

20. Groundedness Metric: belief propagation

21. Balkanization Metric: SVD & Clustering • Apply dimensionality reduction to idea and argument ratings => ”bias space” • each point represents a user • each axis represents a ”bias" (i.e. set of post with correlated ratings) • Balkanization = distinct clusters in bias space

22. Attention Mediation: A Keystone Capability • Guide ideation towards gaps (maturity) • Guide ideation towards pareto front (negotiation) • Guide evaluation to where it is most needed (efficiency) • Guide managers to areas that need intervention • Guide customers to the good stuff

23. Harnessing the Collective Intelligence of the Research Community http://deliberatorium.mit.edu/brainstorm-signup