Safety and Humans in the Loop: Public Policy Values in AI Systems

1. Safety and Humans in the Loop: Public Policy Values in AI Systems Prof Robert Merges UC Berkeley BCLT Annual Symposium April 4, 2019

2. Three key issues • Humans in the "do loop": designing code that takes account of people. Case study - moral decisions in autonomous vehicle software: encoding "trolley problem" decisions in AV systems. • Taking care of people in AI systems: congestion, safety and liability in AV and drone systems. • Jobs for people: exploring a requirement for a human host/override driver in autonomous vehicles.

3. Trolley problem • Controversies of coded solutions • Best practice/safe harbor?

8. Basic Trolley Problem: Moral Responsibility for Death

9. Cultural Variation? • “We found that Chinese participants were less willing to sacrifice one person to save five others, and less likely to consider such an action to be right.” – Natalie Gold et al., Cultural differences in responses to real-life and hypothetical trolley problem, Judgment and Decision Making, Vol. 9, No. 1, January 2014, pp. 65-76.

10. Buddhist Monks and the Trolley Problem

11. AV Tech and Trolley Problem • Like a philosophy article, your decision must be encoded (and so can be critiqued) • Unlike a philosophy article, your decision has real-world consequences!

12. Issue 2: Safety and Liability • Need for access to “black box” – march down the safety curve • Chance for state experimentation: federal structure of auto regulation

13. US Federal Policy • National Highway Traffic Safety Administration, Dept. of Transportation • Sets national safety standards • Leaves other areas to state governments

14. US Federal Government Policy • States retain their traditional responsibilities for – • vehicle licensing and registration, • traffic laws and enforcement, • and motor vehicle insurance and liability regimes.

15. Safety/Liability: Alabama Example • Alabama SB 2018: Defines a truck platoon: • “A group of individual commercial trucks traveling in a unified manner at electronically coordinated speeds at following distances that are closer than would be reasonable and prudent without the electronic coordination.”

16. Truck Platoon

17. Liability implications • Exempts the trailing trucks in a truck platoon from the state’s “following too closely” vehicle regulations if the truck platoon is engaged in electronic brake coordination and any other requirement imposed by the Department of Transportation by rule.

18. Airplane Travel Learning Curve: 1946-2009

20. Moving down the “safety curve” • Data available at https://www.pbs.org/wgbh/pages/frontline/flyingcheap/safety/howsafe.html. • Curve continues: See International Civil Aviation Organization, ICAO Safety Report, 2017 edition, available at https://www.icao.int/safety/Documents/ICAO_SR_2017_18072017.pdf (showing drop in fatalities per departure from 2012 to 2017). • Laurie F. Beck, Ann M. Dellinger, and Mary E. O'Neil, Motor Vehicle Crash Injury Rates by Mode of Travel, United States: Using Exposure-Based Methods to Quantify Differences, 166 American Journal of Epidemiology 212-218 (2007), at 213 (overall fatality rate in the US for 1999-2003 was 10.4 deaths per 100 million person trips).

21. Second generation issues • Effects of AV technology on employment patterns • Work displacement and job creation

23. Surveying the trucking industry recently in “The Future of Work: Robots, AI, and Automation,” Darrell West, a scholar at Brookings, warns that full adoption of driverless vehicles “would put at least 2.5 million drivers out of work.” Assessing the full range of employment, West observes that Robots, autonomous vehicles, virtual reality, artificial intelligence, machine learning, drones and the Internet of Things are moving ahead rapidly and transforming the way businesses operate and how people earn their livelihoods. For millions who work in occupations like food service, retail sales and truck driving, machines are replacing their jobs. Levy argues that if the use of industrial robots on assembly and production lines continues to grow at the current pace,

24. The stock of robots in the U.S. will be 105,000 higher in 2024 than in 2014. If I conservatively assume that each robot replaces two assemblers and fabricators, the 105,000 additional robots would result in 210,000 fewer assembler and fabricator jobs in 2024 than otherwise would have been the case. Where do these displaced workers look for a way to make a living? Levy’s answer is: farther down the ladder: AI’s near-term effect will not be mass unemployment but occupational polarization resulting in a slowly growing number of persons moving from mid-skilled jobs into lower wage work into such fields as food preparation and serving, building and grounds cleaning and maintenance and personal care and services.

25. Hidden Technological Determinism • Because technology makes all this possible, it will inevitably occur • Don’t stand in the way of progress • Two options: defeatism or Ludditism

26. Is this inevitable? • No! • Technical systems are “socially determined” • Example: No Supersonic Transport in the US • The structure of technical systems can reflect “human values” – if we want it to

28. “[The person] that hath a trade, hath an estate” – B Franklin