The change on Moral Perception in relation to increased autonomous systems

1. The change on Moral Perception in relation to increased autonomous systems Experimental Proposals by Tjerk de Greef & Alex Leveringhaus

2. What is moral perception? • Denotes the discernment (the ability to judge well) of morally relevant issues in particular situations • Moral perception informs our moral reasoning, which is the deliberation of what is the right thing to do • In international law and just war theory, moral perception plays a key role: • Accurate perception of conflict situation & knowledge of relevant regulation/principles. Ability to apply correct principles to particular combat situation. • Determine what course of action is necessary. • Determine whether a target is a legitimate target/combatant (in other words: operator must weight discrimination, necessity and proportionality for sound targets)

3. Increased Autonomous Systems, like drones, are… Perfect for the dull, dangerous, dirty, and dodgy tasks BUT Potentially induce risks in moral perception, decision making, and action taking THEREFORE Experimental studies required

4. Questions? • Do we fully understand the psychological effects on remote operators of conducting war at a distance? • What are the psychological effects if we extend the concept of ‘war at a distance’ to reduced interaction (i.e., increased autonomy) with drones?

5. Characterization of Increased autonomous systems • Distance from the battlefield -> due to controlling a targeting system from a large distance -> decreased risk for the operator • Distance from the battlefield -> reduction of battlefield stressors -> leads to bettermoral judgments • Increased surveillance capabilities-> due to more sensors • Increased surveillance capabilities -> due to better sensors • Increased surveillance endurance-> due to increased flying time (up to 36 hours) • (2) + (3) + (4) -> reduced collateral damage due to improved recognition of patterns/improved situation awareness • (2) + (3) + (4) introduced a bonding risk with object/actors in the area of interest

6. Drones vs. Traditional Targeting Systems • Distance from the battlefield -> due to controlling a targeting system from a large distance -> decreased risk for the operator • Distance from the battlefield -> reduction of battlefield stressors -> leads to better moral judgments • Increased surveillance capabilities-> due to more sensors • Increased surveillance capabilities -> due to better sensors • Increased surveillance endurance-> due to increased flying time (up to 36 hours) • (2) + (3) + (4) -> reduced collateral damage due to improved recognition of patterns/improved situation awareness • (2) + (3) + (4) introduced a bonding risk with object/actors in the area of interest

7. However, drones have downsides • Soldiers are removed from the horrors of war and see the enemy not as humans but as blips on a screen, there is very real danger of losing the deterrent that such horrors provide (cf. game mentality) • Are operators becoming trigger happy with remote controlled armaments, situated as they are in complete safety, distant from the conflict zone (cf. incorrect judgment of necessity principle) • Not so precise as predicted leading to increased collateral damage

8. Assumptions • Focus in the usage of targeting systems in bello and not in the justice in the declaration of war (jus ad bellum)

9. Research Question in experiment • How is moral perception affected by tele-operating a UAV? • Specific judgements: necessity, proportionality, discrimination.

10. Two separate experiments • Distance in manipulated -> studying moral perception • Drone Endurance & Sensor Quality/type are manipulated -> studying quality of attack and bounding with object/actors • Preferably using same environment

11. Experiment • While tele-operating a system, systematically manipulate • Proximity • In the field with direct sight • At a large distance with no direct sight • Stressors/unexpected situations (or some other intervening variable) • Many • Little • Type of Task • Surveillance & Reconnaissance • Engagement • Using naïve (non-professional) subjects (Delft University)

12. What we measure, for example • Experienced stress • Experienced risk • # of engagements • After each engagement, freeze and question about necessity, proportionality and discrimination • After each engagement, check whether order/action fits RoE/LoAC • Ask for rationale for specific actions