How much is enough? Requisite modelling for socio-technical problems

1. How much is enough?Requisite modelling for socio-technical problems Graham Mathieson, Dstl Fellow Kathryn Hynd UK Defence Science and Technology Laboratory Presentation to 21 ISMOR, Sept. 2004

2. Structure of presentation • Background and aims • Things we know about socio-technical systems • NCO Domains & current modelling • Case Study 1 – Combat ID • Case Study 2 – Network Effects C2 • Building practical requisite models • Conclusions

3. Background and aims • Increasing awareness of socio-technical issues • Military OR challenged to deal with wider range of factors • Current modelling (both conceptual and practical) has relatively narrow coverage, largely limited to physical and informational domains (with a little of the cognitive) • Presentation addresses the validity of modelling to support OR and seeks to describe a ‘requisite’ model • Main focus has been to identify how much of the cognitive and social domains need to be added

4. Requisite Non-requisite Some definitions and declarations • Models are abstract representations • descriptive/explanatory • conceptual/practical • Requisite (adj)made necessary by particular circumstances(Concise Oxford) • Requisite model is minimum that is fit for purpose • Requisite model: • Contains all critical factors which may determine the study conclusions (e.g. factors significantly affecting option rank ordering) • Can be defined in relation to an isolatable sub-problem (i.e. one in which a sub-set of factors are not too dependent on others) • Non-requisite, by implication, means not fit for purpose • Using non-requisite models carries risks. When is the risk too high?

5. Resources People Technology Processes Structure Culture Things we know about socio-technical systems Organisation size correlates with formality of interactions Degraded comms system performance can lead to improved information service Technical system performance can critically depend upon human social phenomena People process information according to their culture, experience, expectation, emotional state Humans naturally use very little of the information available to them People create and use informal structures, which can be more influential than the formal ones Even in safety-critical organisations, people do not consistently follow formal processes It takes years to change an organisational culture without wholesale re-staffing Structures emerge in organisations despite the intentions of the people involved

6. Social domain Cognitive domain Information domain Physical domain NCO Domains & current modelling Current coverage of OR models Maturity of source models in each domain

7. Case Study 1 – Combat ID • ‘The process of combining situational awareness, target identification, specific tactics, techniques and procedures to increase operational effectiveness of weapon systems and reduce the incidence of casualties caused by friendly fire’ SA CID TID TTPs

8. Case Study 1a - Combat ID • Operation Provide Comfort, 14th April 1994 • 2 US army Black Hawk helicopters take off to pick up 16 members of UN humanitarian relief operation • 2 USAF F-15s enforcing No Fly Zone report 2 low flying, slow moving radar contacts to USAF AWACS, which reports no known contacts in area • F15s fail to obtain IFF responses from contacts. They conduct a visual ID pass, and identify the helicopters as Iraqi Hinds. They shoot down both Black Hawks, killing all passengers and crew • Example of a breakdown in Combat ID. Provides a method for identifying key factors in the criteria for the requisite model?

9. Case Study 1a – Factors First time AWACS crew had flown together Social domain Helicopter, AWACS and fighter communities socially isolated Roles and responsibilities in AWACS crew confused At time of VID, F15 pilots expected contacts to beIraqi Cognitive domain Feature emphasised in recognition training similar for Black Hawks/Hinds Black Hawk visually identified as a Hind Information domain Black Hawks not on Air Tasking Order F15s briefed only on potential threats, not on friendly helicopters Physical domain Black Hawks on different RF frequency to other air missions in area

10. Combat ID - the requisite model • This incident shows us social and cognitive factors were key (in fact dominant) in this breakdown of combat ID • Implies that a modelling approach that doesn’t incorporate these factors is not requisite • Borne out by recent operational experience that showed the worth of ‘social’ interventions - liaison officers • Also supported by literature on human error, and accident analysis in complex civilian systems.

11. Case Study 2 – Network Effects C2 • C2 concepts for Network Effects • Based on work in progress by NATO SAS-050 • NATO Force under UN mandate undertaking an amphibious landing to restore sovereignty of Keswonian territory in the face of a Tetlovian invasion • Conflict includes mixture of conventional joint and combined military assault, surveillance and border security, and restoring/maintaining security for humanitarian relief operations.

12. Re-planning Mission negotiation Tasking Tasking Tasking Tasking Recce Effects generated Recce Strike Manoeuvre An intelligence failure has led to a ‘surprise’ incursion by Tetlovian forces on the eastern border, threatening to over-run Keswonian positions. First indication to Joint Force comes from a Liaison Officer with Keswonian Regt. Help! ‘Traditional’ C2 Concept Keswonians Time Liaison Officer JFHQ WOC CAOC Division Brigade

13. Tasking Tasking Recce Effects generated Recce Strike Manoeuvre An intelligence failure has led to a ‘surprise’ incursion by Tetlovian forces on the eastern border, threatening to over-run Keswonian positions. First indication to Joint Force comes from a Liaison Officer with Keswonian Regt. ‘Functional’ C2 Concept Keswonians Liaison Officer JFHQ Reporting Time Reporting WOC CAOC Division Brigade Reporting Reporting

14. Social domain Cognitive domain Information domain Physical domain Case Study 2 – Factors (sample) Behavioural predictability Cultural homogeneity Teamwork experience Motivational level Interpersonal and categorical trust Task knowledge and competence Availability of own force capability information Richness and reach of information networks Level/type of co-location Persistence of teaming

15. Case Study 2 – Requisite model • The Network Effects case study could not be effectively analysed using only physical and informational variables • Socialisation factors affecting willingness of team member to work together creatively were at least as important as information sharing and collaboration tools • The variables critical for assessing C2 options were roughly balanced across the four domains (PICS)

16. Building practical requisite models • UK is seeking to develop a demonstration of requisite modelling of C2, which includes social, cultural and organisational variables. • Synthesis of a wide range of scientific theory is needed, covering social (including organisational) and cognitive theories and constructs, to complement conventional informational and physically-based modelling. • Proposed architecture balances breadth and depth, as well as being sensitive to danger of too much complexity.

17. Social domain Social domain Socialising Cognitive domain Organising Organisation domain Teamworking Cognitive domain Information domain Processesing Information domain Informing Physical domain Physical domain Acting Developing a modelling architecture NCO Domains (PICS) Expanded domain construct (PICOS) Modelling architecture (active/interactive)

18. Taskwork (the ‘production’ process) Changing Scenario Changing Characteristics Task evolution Process as Task sequences I/P O/P

19. Teamworking The ‘Frame’ represents the Agent’s set of knowledge, beliefs, expectations, goals, understandings, etc Agent • Physical interactions • Sharing environment • Communications • Sharing resource Abstract representation of processes involved in achieving ‘synchronised’ or ‘shared’ understanding (using idea of overlapping ‘Frames’) Frame Frame Frame ‘Synchronisation’ Agent Agent

20. ‘Working together’ Task ‘team’ Gross Social processes influencing knowledge in agent and team Frames Agent Agent Gross Structure processes influencing agent relationships and links to/between resources ‘Being together’ Emerging meta-model HQ Process as Task transition network Team knowledge determines whether team is forming, storming, norming, or performing Teamwork/Taskwork ‘balance’ affects task performance/efficiency Teamworking processes influencing knowledge in agent and team Frames

21. Conclusions • ‘Requisite’ models represent all critical factors (i.e. those which could significantly shape study conclusions). • Modelling sub-sets of the problem is only requisite if the factors modelled are relatively isolated from others. • The risks of not using requisite models are high. • The default model (for C2 problems at least) should assume a balance of variables from all NCO domains as a start point. • The practical construction of requisite models is possible, both in terms of underlying domain knowledge and modelling capability. • The proposed modelling architecture is capable of hosting variables from all domains – thus making a good basis for requisite modelling.