Crowdsourcing for Business: An Emerging Paradigm

1. Crowdsourcing for Business:An Emerging Paradigm Shourya RoyArea Manager, Human ComputationXerox Research Centre India, Bangaloreshourya.roy@xerox.com Workshop on Social Computing, IIT Kharagpur5th Oct, 2012

2. Crowdsourcing : What is it? • The act of taking a task traditionally performed by an employee or contractor, and outsourcing it to an undefined, generally large group of people, in the form of an open call • Digitization, image labeling, user studies, machine translation evaluation, logo design, EDA simulation, innovation contests, . . .

3. Make progress towards deciphering this handwriting Put words which you are unsure about in parenthesis Handwriting Recognition Problem* Instructions to Crowd Many tasks are easy/feasible/doable for humans, but difficult/challenging/impossible for computer programs – Xerox Confidential –

4. Examples (1/5)

5. Examples (2/5)

6. Examples (3/5)

7. Examples (4/5)

8. It has been Existing Humans were the first “computers,” computers, used for math computations 9 examples of crowdsourcing, before ‘crowdsourcing’ existed : http://bit.ly/mXFdRp

9. Internet and Mobile Have Made it More Common and Promising – Xerox Confidential –

10. Increasing Activities and Popularity Increasing Popularity as Depicted by Google Trends Crowdsourcing on Google Scholar Over the Last Few Years • “2M contributors who does more than 4PY of work on an average day!” -- CEO

11. Changing Demographics

12. What is the Problem • Given a computational problem, design a solution using human computers and automated computers – Xerox Confidential –

13. Why is it Different? Main doer is Human (and not Machines as in Assembly Lines) Human in the loop (and not Guinea Pigs) The outcome is determined by an algorithm (and not the natural dynamics of the crowd) Humans are actively computing(not merely carrier of sensors)

14. Where is Research? • Quality Estimation and Assurance (Redundancy and voting; Gold data; joint estimation of worker quality and task difficulty; Symbiosis with Machine Learning) • Complex Tasks (No discrete answer; Exploration and exploitation; crowd workflows;) • Task Design (Optimize cost, quality and time; infinite completion time; Real time) • Incentive and Motivation (Payment vs. non-payment; Optimal payment; Payment and quality; ) • Market Design (Reputation Mechanism; Monitoring and feedback; Task Discovery; • Behavioral Aspects (Noisy behaviour; Non-reproducible;) – Xerox Confidential –

15. An Emerging Research Field

16. An Interdisciplinary Research Field

17. That’s Alright – but Xerox!!?

18. We have transformed… 2009 Technology-led ~25% Services 2011 Services-led ~50% Services $15.2 billion ~$23 billion Revenue $132 billion $500 billion + Market Opportunity Document Outsourcing Business Process Outsourcing Information Tech Outsourcing Document Outsourcing Services Leadership In … into the world’s leading enterprise for Business Process and Document Management

19. Xerox Revenue by Business Segment* * http://www.fastcompany.com/magazine/161/ursula-burns-xerox

20. Is Crowdsourcing a Viable Alternative to Outsourcing? • Outsourcing is • Focus on the core business while partnering with 3rd party vendors to tackle the non-core operations • Tasks requiring human intelligence and skills • Data and process migration by smart use of technology • Heavily human intensive; typically with the help of computing technologies • Large distributed workforce enabled by technology executing tiny pieces of work requiring human intelligence

21. Data Entry by Crowd • We started by considering a typical outsourced process (Data Entry) • Objective is to understand a process in detail and identify implications for crowdsourcing • Digitisation of insurance forms and medical records for US based insurance companies • Typing in, validation/ correction of information from scanned forms • Outsourced, distributed process

22. Features that make Form Digitization process amenable to crowd sourcing • Relatively low skill data entry work, known as ‘key what you see’ • Already an outsourced process requiring a low level of interactivity between sequential steps • Strong workflow tool to manage work, which flows through a series of system and human steps • Between sites • Between sequential tasks • Between agents (given their known skill set)

23. Findings from Work-Practice Study (1/2)

24. Findings from Work-Practice Study (2/2) • Workplace Ecology : Data security is physical, technical & social • Crowdsourcing: lose physical and social enforcement, reduced control of workforce. Need technical solutions. • Skills and Knowledge1)‘key what you see’ data entry actually involves extensive rule set. 2) Form difficulty is situational. 3) Non-standard means non-standard. • Crowdsourcing: Situational-based incentives and supporting learning • Being a Corporate Employee Pay alone not enough to achieve SLA. Agents made accountable. • Crowdsourcing: reduced accountability could increase rejections of difficult work. • Making the Workflow Work:Push model of work • Crowdsourcing: Pull model of work raises coordination and completion issues. • Collaborative Working: Work is not collaborative at workflow level; but it is at claim level (floorwalkers & colleagues). • Crowdsourcing: building collaboration in? Pull models of supervision?

25. Conclusion • Crowdsourcing is an emerging Research area • It requires expertise and research competencies from a number disciplines • Crowdsourcing can be applied in various domains to solve problems in a more effective manner • Finally, a large fraction of the crowd comes from India • Focused research and technologies will be highly relevant

26. References • TurKit: Tools for Iterative Tasks on Mechanical Turk; Greg Little, Lydia B. Chilton, Robert C. Miller, and Max Goldman • Matt Lease Tutorial • Soylent – A cr • Fold.it – S. Cooper et. al