The What and How of Measuring Electronically Mediated Work:

1. The What and How of Measuring Electronically Mediated Work: A Review of Different Countries’ and Private Organizations’ Experiences Anne E. Polivka July 2019

2. BLS

3. Outline of Talk • Discuss what is trying to be measured • Why may want a particular measure • How measurements have been made • Compare various estimates • Concluding thoughts

4. What Included in Review • Electronically Mediated Employment/Platform Work/Digital Platform Work • Will be using terms interchangeably • Excluding sharing economy, collaborative platform work • Where there is no remuneration for work or lending of an asset • i.e. Couchsurfing, Zipcar, 1000Tools, Freecycle • Discussing from the worker’s perspective not business’s

5. Included in Review • Early “modeled” estimates • Government administrative data • Private company data • Household survey data covering • Private sector, academic, government sponsored, and national statistical offices’ surveys • 24 Surveys

6. What To Measure

7. What is Platform Work? • Concepts of platform work differ • No agreed upon definition • What is included varies along many dimensions

8. General Conceptual “Definitions” • EU describes the main features of platform work as: • Paid work organized through platforms • Three parties involved: platform, client, worker • Aim is to conduct specific task or solve specific problem • Form of outsourcing/contracting out • Break-down of ‘jobs’ into tasks • On-demand services

9. General Conceptual “Definitions” • BLS describes electronically mediated work as arrangements where workers • Use a company’s website or mobile app to connect to clients or customers and obtain short jobs, projects or tasks • Are paid by or through the company that owns the website or mobile app • May do short jobs, project or tasks in person or online • Chooses when and whether to work

10. General Conceptual “Definitions” • U.S. Commerce Department describes digital matching companies as: • Using information technology (IT systems), typically via web-based platforms such as “apps” or internet-enabled devices to facilitate peer-to-peer transactions • Relying on user-based rating systems for quality control • Allowing individuals providing services via platforms flexibility in deciding their typical hours of work • Relying on workers using own assets and tools

11. General Conceptual “Definitions” • Scandinavian countries describe platform work as: • A triangular relationship between the platform company, those who provide labour and those who purchase labour services.

12. General Conceptual “Definitions” • Non-official (personal) working definition of digital platform work is at a minimum: • Paid work where an app or website (digital platform) plays an integral role in helping to connect workers to customers ….. • Want to exclude • Job search for a longer term job e.g. CareerBuilder or Monster.com • Employment placement agencies • Social media e.g. LinkedIn

13. In Determining What to Measure • There are additional questions have to address when deciding what to measure • Identified 8 questions need to be aware of • Although researchers/designers may not always be intentional in these decisions • Could be implicit in what data decide to use or formulation of question wording

14. Does location of where work is done matter? • Platform work can be done at a customer’s physical location (in-person), or virtually (completely on-line) • For some purposes may not want to make a distinction • For some purposes may want to measure both, but separately • For other may want to restrict to in-person work

15. Does the type of customer matter? • Want to restrict the measure to peer-to-peer transactions? • Want to include peer-to-business transactions, but make a distinction between peer-to-peer and peer-to-business? • Don’t care about the type of customer?

16. How much labor services does the transaction need to involve? Transactions done through digital platforms occur along a labor service-capital spectrum Only labor services Labor using assets Selling goods made Renting asset, some maintenance service Selling own used goods Providing financial capital

17. How much labor services does the transaction need to involve? • Transactions done through digital platforms occur along a labor service-capital spectrum • Only services provided – e.g. baby sitting, dog walking • Services provided using personal asset – e.g. ride-hailing, repair tasks • Selling new goods produced or obtained for sale - e.g. making handcrafted items or buying antiques • Renting asset, maintenance service may be involved - e.g. renting spare bedroom or vacation home • Selling own used or surplus goods • Providing financial capital –e.g. providing start up capital

18. How much control does the platform company need to exhibit? Transactions done through digital platforms can have various levels of platform company involvement in the customer, client interaction Minimal Workers posting on bulleting boards Intermediate Worker posts profile including wage Customer selects worker Company can facilitate payment and background checks Intermediate Company maintains lists of tasks on site Worker chooses task Payment trough site Customers rating of work on site Extensive Worker posts profile including wage Company algorithmically matches First to accept gets work Payment via app Customers rate via app Very extensive Company algorithmically matches Company determines wage and price Customers rate via site Worker can only turn down a few assignments and must maintain minimum rating

19. How much control does the platform company need to exhibit ? • Minimal • Workers post ads on electronic bulletin board • Customers contact workers themselves (by phone or electronically) • Workers set own price (wage) • Workers can turn down customers • Workers determine how and when to complete work • Customers pay workers directly -E.g. Craig’s list, Angie’s list

20. How much control does the platform company need to exhibit? • Intermediate • Company recruits workers for provision of specific type of service • Workers post a profile of themselves including rate of pay will accept • Customers search profiles and select a worker • Company can facilitate payment through website • Company can maintain workers’ background checks -E.g. Care.com Sittercity.com

21. How much control does the platform company need to exhibit? • Intermediate • Company maintains list of customers’ tasks workers can review and select to complete on website • Payment is through the app or website • Customers’ rating of workers can be posted on the platform -E.g. Amazon Mechanical Turk

22. How much control does the platform company need to exhibit? • Extensive • Workers set up a profile including their schedule and rate of pay will accept • Companies algorithmically match customers’ requests and workers based on workers profile • First worker to accept obtains assignment • Payment is through the app • Workers can be rated via the app -E.g. Taskrabbit

23. How much control does the platform company need to exhibit? • Very Extensive • Company algorithmically matches worker and customer when worker has app on • Company determines worker’s rate of pay and customer’s price • Customers pay through the company app • Customers can rate worker via the app • Workers can only turn down a limited number of assignments and must maintain high rating -E.g. Uber

24. Over what time period does the activity need to have occurred? • The reference period in which the activity could have occurred can be wide or narrow • Ever (or unspecified) • In the last year • Last 6 months • Last month • Last week

25. How intensively did a person have to undertake the activity? • This question related to reference period, but it is a distinct question • How often did a person do the activity within the reference period • Only once at some point • Only once in the last year • Once a month in the last year • Several times a month in the last year • Weekly • How many hours within a time period

26. Does the proportion of income earned from the activities matter? • Should the measure only include those who earn more than a threshold amount of their income from the activities? • If so • What is the threshold - 25%, 50%? • Note that having a threshold will skew the measure toward lower income people • Should the threshold be for all activities combined or separate categories, platforms

27. How much commitment to working and availability of work is there? • Is it sufficient for a person to be registered with a platform or does the person have to have worked on the platform? • Does it matter how long people have been registered with a platform and when they last worked, if ever, through the platform? • Do workers desire more work and/or have experienced difficulties obtaining all the platform work they desire?

28. How much commitment to working and availability of work is there?

29. How much commitment to working and availability of work is there? • In Norway only 1.3% of registered Upwork freelancers did an assignment every month • Australian Survey • 14.3% of respondents who participated in digital platform work more than 12 months ago stated despite attempting, ‘they didn’t get any work’ • ILO Survey • 58% of respondents reported availability of tasks was insufficient and an additional 17% said that the did not find enough well-paying tasks

30. Why Want A Measure

31. What question/issue is being addressed • Digitalization of the economy and/or “complete” income measures • Use broad measure (labor services, rental, selling, capital investment), may not need distinction by type of activity, but do need to have obtained income through platform (just registering insufficient) • Number of workers involved • May want to restrict to labor services, or at least make distinctions by type of activity

32. What question/issue is being addressed • Application of national labor laws, health and safety regulations, or local market effects • Restrict to in-person labor services, considerable platform control, may want to condition on number of hours worked • International labor markets or means of economic development for developing countries • Restrict to online tasks, may want to include financing platforms

33. What question/issue is being addressed • Evaluate importance as a form of employment • Restrict to labor services, use reference period similar to labor force surveys (e.g. 1 week) • Workers’ payment rights • May want to restrict to online and/or companies with intermediated amounts of involvement or greater • Effects on specific occupations or industries • Ask about specific activities (i.e. ride-hailing, property rental)

34. How Measurements Have Been Made

35. Included in Review • Early “modeled” estimates • Private company data • Government administrative data • Household survey data covering • Private sector, academic, government sponsored, and national statistical offices surveys • 24 Surveys • All approaches have some drawbacks

36. Early “Modeled” Estimates • Combine data from various sources to obtain indirect measure • Estimate for a small subgroup • Web-scrape data for “all” groups (subgroup and others) • Scale estimate for subgroup using Web-scraped data

37. Early “Modeled” Estimates (cont.) • For example Harris and Krueger (US), De Groen and Maselli (EU) • Estimate of the number of Uber drivers • Google trend estimates of Google searchers for 26 platforms and Uber • Scale the number of Uber drivers by the ratio of search estimates for 26 platforms to Uber searches x #Uber workers

38. Early “Modeled” Estimates (cont.) • Involve strong assumptions • Largely superseded by direct estimates

39. Private Company Data • Bank account data • For example JP Morgan Chase study • Sample of 39 million U.S. Chase checking accounts • Track payments through 128 platforms (number in 2018, fewer in earlier years) • Estimate how many families participated in online platform economy • Have estimates for current month, 1stquarter, and in prior year

40. Private Company Data (continued) • Cannot identify payments through 3rd parties such as PayPal • Can be influenced by changes in payment process (i.e. switches to debit cards) • Not universal coverage, particularly not geographically • Chase does not have a branch in 23 states including VA and DC. Tax data show DC 4th highest proportion employed in platform work (2.5% 2016) and VA 7th highest by state (1.14%) • Based on specifically identified platforms • Unit of measurement families • Data not available to general researchers

41. Private Company Data • Web-scraping postings for projects/tasks • For example Online Labour Index (constructed by Oxford Internet Institute) • Track all projects/tasks posted on 5 largest English language labour platforms • Represents at least 70% of the market traffic • Measures utilization of labor platforms across countries and occupations • Not an estimate of filled “orders” or measure of workers, but can follow trends • Since 2017 observe work activities on 4 major platforms –still not a measure of proportion of employed

42. Government Administrative Data • Tax filing or tax information forms • For example analysis of 1099-K forms in the U.S. • 1099-K forms are “information” forms sent to people by companies if a person received payment via a credit card • Informational only, not a tax bill • Researchers screen forms based on names of platform companies and “unduplicate” so only count individuals once even if receive multiple forms

43. Government Administrative Data • 1099-K Analysis/data • Country specific, not generalizable internationally • Data not available to researchers in general • Does not have rich auxillarydata that surveys often have • Restricted to platforms identified by researchers • Collins, Garin, Jackson, Koustas, Payne • National analysis 50 important labor platforms • Bernhardt, Prohofsky, Rothstein • California, 20 platforms

44. Government Administrative Data • Tax forms • Sent for tax collection purposes, not research • Effected by changes in laws and rules • Platform companies prior to 2016 were being cautious; since, due to a tax ruling, only sending 1099-K’s to those with $20,000 or 200+ transactions • California may make W-2 workers • Interpretation of rules can vary by state • VT, MA, $1 earned must send state 1099

45. Government Administrative Data • Tax filings • Often subject to minimum income thresholds • Concerns in EU that Value Added threshold not meet by many platform workers • Sensitive to cyclical variation • During economic downturns less likely to exceed • Compliance can be less than complete • Changes over time and sensitive to economic conditions • Effected by changes in laws and rules • Study found starting to issue 1099-K, increased reporting by effect group more than 20%

46. Government Administrative Data • Registration information • More of a European data source • Can vary across time, countries, and industries • For example, Denmark trying to install seats in Uber cars to measure ridership

47. Household Survey Data • There will always be a role for household survey data • Challenge is conveying to respondents what is meant by a digital platform • Type of sample will influence estimates • Sample probably more important than mode of data collection embedded in sample, but mode of collection can also influence

48. Household Surveys–Common Understanding of Platform Work • Surveys have taken 4 broad approaches to conveying what is meant by a digital platform and electronically mediated work • Use term “digital platform” or other technical terms • Ask about specific platforms by name • Ask about specific activities done on platforms • Describe and ask about attributes of platform work

49. Understanding term Platform WorkUsing Term Directly or Technical Terms • ONS Pilot, Norwegian 1st Pilot, Sweden, Eurobarometer, Rand American Life Panel (Katz and Krueger for missed work), Finnish LFS Pilot • E.g. Sweden Have you ever tried during the past year to get an assignment via digital platforms(e.g. Uber, Urb-it, Taskrunner, Offerta etc.)?

50. Understanding term Platform WorkUsing Term Directly or Technical Terms • E.g. Eurobarometer A collaborative platform is an internet based tool that enables transactions between people providing and using a service. They can be used for a wide range of services, from renting accommodation and car sharing to small household jobs.* Have you ever provided services on these platforms? * Asked about knowledge and use of first