In need of a model for complexity assessment of highly automated human machine systems

1. In need of a model for complexityassessment of highlyautomated human machine systems Fredrik Barchéus, Pernilla Ulfvengren, Johan Rignér

2. Content • Goal of research • Purpose of paper • Future ATM, context • Theory; Human Factors, automation and complexity • Models of complexity • Conclusion and discussion

3. Goal of our research • Apply and integrateknowledge of human factorsintodevelopment and continuousimprovement of critical systems in order to improve overall system performance.

4. Purpose of this paper • Initial work, problem definition and ideas for further research • Link multiple areas of research • Design and system development • Human Factors and UserCentered System Design • Automation and complex systems • Models of complexity • Identify potential criteria for human factorsspecific to operating in highlyautomated and complex systems. • Explorefuture research needed to develop a model for complexityassessment in these systems

5. Future ATM system – Multiple goals • Commercial pressure • Demand for increasedcapacity • Descreasecosts • Decreaseenvironmentalimpact • Increaseefficiency • Increasesafety • Deployment of new technologies • Shift of paradigm: airspacebased to trajectorybased • High level automation • Integrated systems

6. Effects and consequences of future system on human operators and system performance • New operationalrules, new tasks, new roles • Increased automation • ”Ironies of automation” • Less operator understanding/predictability of operationalprocesses • Change of responsibilitiesamong human roles: • air traffic controllers, pilots, groundhandlers etc. • Variouslevels of automation in new and old parts in operating system. • Actors with different technologies in joint systems

7. Multiple goals, trade-offs in design • Good design – managingtrade-offs • To evaluate your design choices • There is no perfect solution • You need to know the trade-offs. • The perfectcar: • DC3 aircraft – madeaviationavailable to public • Not best on anysingle parameter • Best trade-offsbetween speed, comfort, price, size etc.

8. Design and system development process • Design requirements: • Precise, limited design requirementsspecification • Bothenabler and blocker of designing for operability with full functionality. • Timing: identifyingneeds for improvement • Easier and cheaper to changeearly in design phase. • An alternative is to addrestrictiveuserinstructions • In IT-systems design: • Insufficient or faulty initial requirements. • Customermay not define or evenknowwhat design requirements that willfulfilloperationalrequirements

9. Human Factors and User-Centered System Design • Needs driven, context and operationalfocus • Front-endanalysis with user in earlyfocus • Operator analysisdifficult, toounspecified design requirements. • Still not alwaysapplied from the start of system development. HF remains an add-on in design of human-machine systems.

10. Design requirements Operationalrequirements Beyond normal operations, complexity Userrequirements Tech.requirements Userinvolvement and testing ”As few as possible”

11. Automation and complexity • Automation • Enablecost-effective systems • Enablesafer systems • Affect work environment, content, tasks and procedures • Imperfect automation leads to complacency and mistrust from operators • Full automation in part hindered by insufficient data • The human remains in the system as a backup • ”Not all that could be automatedshould be automated” • Levels Of Automation, LOA

12. Levels of automation • 10 the computer doeseverything • . • . • . • . • 5 the computer acquires information, suggest one solution and waits for the human to execute • . • . • . • 1 the human doeseverything

13. Levels of automation Information acquisition Informationanalysis Decisionselection Action implementation 10 . . . . 5 . . . 1 10 . . . . 5 . . . 1 10 . . . . 5 . . . 1 10 . . . . 5 . . . 1

14. The Swiss cheesemodel of accidentcausation Activeerrors Latent conditions Operator error Managerialdecisions Training

15. Swiss cheesemodel Decision to automate Tightlycoupledautomated systems

16. Complexity of technological systems CDM Automation SWIM ASAS Tight Dams Nuclear plant ? Aircraft ATM Coupling Assembly-lineproduction Mostmanufacturing Loose Universities Linear Complex Interactions

17. Quantifyinginteractions • ASAS Free Flight V=6 V=3 V=12 Controller separation Pilot separation Pilot separation (System)

18. Complexityassessment of changedresponsibilities Controller  Pilot Task migration Default case Pilot  Controller Emergentcognitivefunctions

19. Layeredmodel for System of Systems δ γ β α δ γ β α δ γ β α δ γ β α Organisation Team Individual Operations Economics Resources Policy aircraft, crew, engineers… Development, ATM, airline… Regulations, SOP’s…

20. Example: Multitude of equipment and procedures • Aircraft descent from a pilot’sperspective • Airspeed mode • Vertical speed mode • FMS mode • Manufacturer and airlineeconomicprofilediffers • Trajectorydiffersbetween different modes

21. Example: Change of communication routes Aircraft Aircraft Pilot Cabin Pilot Cabin Catering Catering Before After

22. Example: Change of responsibility and procedures • ASAS applications • ASAS Self separation • ASAS Separation • ASAS Spacing Self separation Pilot Separation Spacing Controller Default

23. Validity of models and methods • Simplifications • Cover only sub-systems under certain conditions • Use of domain knowledge • Simulation models need valid basic assumptions • Purpose of automation model(s)? • Training, procedure design • Limited wider modeling applicability

24. Future system - Again • Multitude of equipment and procedures • Moreinteractions – highercomplexity • Change of communication routes • Changedinteractions – complexity? • Change of responsibilites and procedures • Changedinteractions – complexity? • Highlyautomated • Tightlycoupled – errorpropagation • Significant system integration • More/lesscomplexity?

25. Conclusion and discussion • Complexity and automation highly intertwined in the context of SESAR and the future European ATM system • Paradigmatic change – viability of old methods and models • Old systems remain in new context – new interactions need new reassessments • Mixed system functionality and equipage – full system assessment • Lack of “full context” – careful use of domain expert knowledge • Collaboration between ComplexWorld and HALA!

26. Thank you! Questions? barcheus@kth.se KTH Royal Institute of Technology Stockholm, Sweden