Front-End Operations Research

1. Front-End Operations Research Neville J Curtis Land Operations Division Defence Science and Technology Organisation neville.curtis@dsto.defence.gov.au

2. Outline of the presentation • The wide scope of OR • What do I mean by OR • What do I mean by the front end • Where the degrees of freedom are • Why we have to be rigorous • Some ways of working at the front end • Some examples of what we’ve learnt • Words • Pictures • No formulae • And very few numbers!

3. Jobs that an Operations Researcher may do • Provide an exact number to solve something • Provide a best fit of inputs to do a job • Do a quantitative comparison of some similar bits of kit to do a well defined role • Compare competing options of procedure, organisation or acquisition • Look at an existing system to find options for change caused by new desired outcomes, environments or technologies • Influence the debate on a future change • Provide analytical scrutiny for any of the items above

4. Puzzles, problems and messes Harder OR Softer OR

5. The Puzzle, Problem and Mess Spectrum- what do we do if we have a Mess? Turn it into a problem Work in the mess space Harder OR Softer OR

6. Why Front-End OR? • Because we need: • To understand, as far as we can, the issues, so that: • We can usefully explore the system, so that: • We can propose bright ideas, so that: • Decision makers have credible, auditable and transparent material, so that: • Longer term programs are set in the right direction with less risk downstream • And we don’t want: with disastrous long term consequences…..

7. What do we need to get things going? • When no one has given us an optimisation puzzle or three options to compare, and: • We don’t know much about the system under study • We don’t know much about measures or ways of looking at it • We don’t know how things are linked (trade-offs, antagonisms, synergisms) • We don’t know what matters and what doesn’t • We don’t know what the clients and owners are worried about • We don’t know what can be changed • We don’t know what data there is (or if it matters) • We don’t know how important the “length of the string” is • We don’t know how the system will respond to changes in context • And we only know that there is a desire to improve a system, and of course: • We don’t know what “improve” means • These questions, of course, tell us what we need to do!

8. Light-hearted diversions • We tend to see two types of response to the list on the previous vugraph: • The pessimist – throws hands up in horror, desperately looks for previous work as a reference, realises that there is no logical way of doing it, and asks to do something else • The optimist – greets task with glee, sees lack of bounds and constraints as an opportunity, relishes being able to be creative, becomes very enthusiastic • Outrageous (inflammatory) generalisation: • Soft OR is better suited to Myers-Briggs NT preferences (physical scientists) – possibilities and analysis • Hard OR is better suited to Myers-Briggs ST preferences (mathematicians) – facts and analysis

9. This presentation • Will not tell you how to do a specific Front-End study • Instead I’ll give some ideas of how it could be done through examples: • How do we progress soldier modernisation? • How do we address counter terrorism? • How do we guide the future Army? • These have differing degrees of previous shaping, ranging from making sure that you’ve included all that needs to considered (soldier modernisation) to a blank sheet (counterterrorism) • The common feature is that they are all areas where we had to do a preliminary (front end) OR investigation • I will present in terms of models and some manipulations • We are now getting to the stage where we have “standard” methods in our tool box

10. Front End Operations Research • I will explore and structure issues that are poorly defined or understoodthrough using my scientific skills, backed by numeracy, literacy and communication so that subsequent studies are much better focussed and in the long run, decisions on ways forward are made with less risk • ie I will do P, by means of Q to achieve R

11. Some ideas to get started at the Front-End • Model the system, include: • Context • Actions (verbs) and influences • System components (nouns) and interactions • Axes of interest • Notions of measures (may be hierarchical) • While remembering that models are useful not necessarily right and that models could be diagrams, tables, pictures, spreadsheets, agent based simulations…… • Manipulate the models so that: • We have a “fit for purpose” feel for what’s going on when things are done under different contexts or vignettes • We know where things can change (ie go wrong or be made better) • We can come up with “bright ideas” as options for change • We can say something useful about these options • We can advise decision makers about possible ways forward (ie not the way)

12. Example of Front-End studies for Soldier Modernisation • Context – Army’s role is changing: technology is becoming smaller, cheaper, less likely to break and users are more technology savvy. We want to look at modernising the dismounted soldier. This is a “Problem” (but it turned out that it needed some work to refine it better..) . • Outcome (Concept) – I will advise decisions makers on what sort of things need to be looked at (equipment, doctrine, integration, useability, testing and evaluation),by means of doing a thorough front end study including understanding what the dismounted soldiers do now, how they could do things in the future and how technological insertion and acquisition may be justified to decision makers so that dismounted combatants will be better equipped for success in modern warfare • To put it another way – we want to have sufficient understanding of the system so that we can discover good ways forward with our partners (coach role) and provide credible evidence to the fund owners (judge role) that soldier modernisation should go forward • NB fine details of exactly what and how many to buy come further down the track and are not part of a front-end study

13. Our first question • What does the infantry do and how will it be improved by the use of this new technology? • This is where the work was needed…

14. What did soldier modernisation teach us (1)?- understand the whole picture Another soldier system This visual model reminds us to consider the entire system, not just the soldier and his kit. Eg what is the effect of the physical environment on the equipment? NB it has changed in the last ten years: now fight in the urban – this brings new physical environments and civilians into the mix Soldier interactions: equipment, physical environment, civilians, red forces, other soldiers…

15. Activities (what) Patrol Assault Defend Observation post Check point Tactical relocation Escort ….. (NB more than 7) Core Skills (how) Engagement Information Collection Sustainment Movement Decision Making Communication Protection (NB the magic 7) What did soldier modernisation teach us (2)?- We need to classify actions, but a single list isn’t enough Big question: what do soldiers do when on an operation? Approach: create a model with two axes NNB these all have meaning to the decision makers and the end users (front end OR is not a black box)

16. Specific versus generic:a degree of freedom • Activities are: • Situation specific (attack that hill, with that force, that equipment, that enemy, those environmental conditions), that • Can be wargamed and this gives you quantitative results • Can give “statistical” results, but it’s only one point • Can be broken down into steps • Can be investigated in depth to find the critical part • Can be perturbed to learn a little more • And if you do enough in-depth analyses you can get some insights about the overall landscape • Core skills are: • Generic elements used to a greater or lesser extent in the activities, that • Are less amenable to quantification • Provide a different sort of insight • Can be modelled in terms of contribution to the overall goal (the “ility” issue) • Provide common threads on issues like equipment/technology insertion

17. What did soldier modernisation teach us (3)?- use both axes to tease out all the issues • A Bunch of Guys Sitting around a Table (BOGSAT)… • A seminar wargame of thinking about the issues that are likely to occur with the first round of soldier modernisation equipment • For a stage in an activity go through all the prompts (skills) and ask what can go wrong (pessimistic analysis): • Technology not good enough • Environment against you • Blue force constraints • Red force issues (logical and lateral thinking!) • Come up with technology products (or other approaches) that could improve things • Group by core skill • We now have an audit trail to the needs/requirements of the second round of soldier modernisation

18. Same thing on a diagram Skills Activities

19. Example results • Patrol plus Information Collection: • Issue - can’t see through walls • Ideas – talk to the building’s inhabitants, penetrating radar • Assault plus Engagement: • Issue – civilians present • Ideas – write rules of engagement, less than lethal weapons • Check point plus Protection: • Issue – soldiers vulnerable • Ideas – new procedures, remote checking • Tactical relocation plus Sustainment: • Issue – equipment weighs too much • Ideas – motorised forces, modular equipment • Note the sequence: • Is there a better procedure? • Can you improve things by a better organisational structure? • If all else fails, is there a technology fix? • KEY MESSAGES– the “bright ideas” now have an audit trail and have some weight as “answers” that decision makers will fund. If you group them up by core skill then you can start to say something justifiable about priority. In the longer term, the concept of operations will lead to a technical specification (eg what sort of walls do you want to look through?) The protocol also means that you have less chance of missing things, or being charged with being selective…

20. What did soldier modernisation teach us (4)?We need to think about measures better • Consider a hierarchy • Lowest level (engineering Measure of Performance) • eg ballistic dispersion (measurable) • Now do something with the kit (call this a Measure of Effectiveness) • eg kill probability (comprises measures of performance plus..) • Now think about combining some things to do a purposeful activity (call this a Measure of Outcome) • Eg was the assault successful? eg “This assault is a success if we take the hill within a set time, use less than 40% of resources, lose less than 25% of our force and we can repel a counter attack of….” – combines measures of effectiveness plus… • But the general doesn’t care about the ballistic dispersion (and it may not matter anyway), he cares about a Measure of Capability • Eg How good is the force equipped with the new kit? • So who’s asking the question and what does the decision maker need to know? • Intrinsically one can see how we’re going from a hard OR approach to a soft OR one as we go down the page.

21. Example of Front-End studies for Counterterrorism • Context – the World has changed and high level planners need guidance about what to do in a new area • Context rephrased – decisions makers need to do something, but don’t know what (“a Mess”) • Outcome – some bright ideas on some initiatives that could be taken and some insights about the relative merits of options • Approach – try and understand the system, so that we know what matters, what is vulnerable, what can be fixed etc so that we have a framework for action

22. Intervention 1 Intervention 2 Intervention 3 What did working on counterterrorism teach us (1)?Influence diagrams are models that help give a common understanding of the system, and are a source of options when we manipulate them

23. Event 4 Event 2 Event 3 Event 1 What did working on counterterrorism teach us (2)?Look at both the specific and the generic • The specific gives us a sequence of events that we can examine to show vulnerabilities • The generic gives us insights of what things are important • Influence diagrams are very useful

24. A Specific Case – “the Terrorists’ Year” • Identify all the possible steps that red takes in planning, preparing, executing and exploiting a terrorism event • What can blue do for each of these? Applicable to any “Purposeful Human Activity” System

25. What did working on counterterrorism teach us (3)?When we draw influence diagrams, nodal analysis is useful This gives us some rules to follow when we manipulate the models (this is a step below systems dynamics) Look for these as priority items:

26. Examples of Front end studies for the future Army • Context: there will be a future Army which almost certainly will not look like the current Army (“a Mess”) • Outcome: what advice can we give to the Army on future force structure development so that sound options are presented to the funding committees and good decisions are made? • Scientific challenges • How do we make sure that the contexts are properly considered? • How do you make sure that concepts are sound and well explored? • How do you model the Army? – how do we define what an Army does? • How do you include S&T in the models? • How do you look for “better things” rather than “same things better” (ie replace what we’ve got)?

27. What did the future army work teach us (1)?This model shows that we should be advising on the development process not handing over a solution There is no single future! So ID what you’d need for each future And keep an eye on key indicators But it helps if you can describe what we do now

28. Practical implications • You never reach the future (the Army after Next) therefore it can’t be seen as a project to achieve • The future pulls you (“backcasts”) as the light on the horizon • Some intermediate forces already exist: • Legacy items • Things are that planned to be bought • New intermediate forces will be acquired as you understand the trajectory of change • You have to sell the notion of “the road to get there” (wherever there is) as the OR product, not a rigid project (a professional challenge) • “messes are managed not solved”

29. What did the future army work teach us (2)?Present in the real world, work in the model world Present results for the real world But do your work in the model world Explore verbs and only go to nouns to see if your bright idea can be made to work. Work in the soft OR area and then dive deep into specific vignettes for learnings (could be hard OR) - complementarity

30. What did the future Army work teach us (3)?When we model the core skill interactions we demonstrate how complex the Army is Any change will perturb the system…

31. What did the future Army work teach us (4)?We can model the core skills by influence diagrams- but they can be hard to read! This is one way of expressing a core skill Eg Blue positioning Usage rate Targetting capability Weapon capability Eg Range Accuracy Terminal effect

32. What did the future Army work teach us (5)?We can do things in a semi quantitative manner to assist our narrative arguments • Eg rate our current Technology Based Variables (TBV) • 3 = high capability, 2= medium, 1 = low • Give a weighting based on the nodal analysis and other reasons • Have a pay off matrix that links changes to other TBVs • Can derive an indicative “system value” in terms of blue capability or blue minus red • Then insert a change (eg technology) • What happens to the indicative “system value”? • And why? • Example: hybrid engines: • Better Movement & Sustainment (and thus Information Collection) ie use the weight saving in the form of extended range (option 1) • Or better Engagement and Protection ie replace fuel weight with something else (option 2) • Option 1 looks better. “The raw scores for blue system value are 3.6 to 2.8 while blue minus red differentials are 4.5 to 2.9 – this is supported by the following narrative comment…..”

33. What did the future army work teach us (6)?morphological analysis can be used to develop options • Model Armies can be constructed from balances of the core skills: consider a set of 7 core skills – Engagement, Information collection, Communication, Mobility, Sustainment, Protection and Decision making (axes of interest) • Give each a rating of 1 (poor) to 3 (good) and come up with some feasible combinations of these • NB can’t all be good (expense), can’t all be poor (unpalatable), some aren’t possible (Field Anomaly Relaxation) so we’ll have a limited set. Which of these represent reasonable options? • ie “at the touch of a button” we can generate concepts (sum subscripts = 14) • Eg: E2I2C2M2S2P2D2 the all round force option, (baseline) • E3I2C2M3S1P1D2 the rapid moving strike force option, • E2I2C1M1S3P3D2 the attrition force option, or • E2I3C2M1S1P2D3 the information rich force option etc • Agent based models can be tied to the core skills: • Code in the characteristics learned from the core skills • In reverse, look at military history and express these in terms of core skills • Play them off (for combat…)

34. Results (MANA study) • Comments: in itself this is not a bad result: • We can reason why it occurs and learn something • It tells us that there will be competing issues to consider

35. Data collection First position – self (subjective) Second position – other (speculative) Third position - fly on the wall (objective)

36. How do I know that my Front EndStudy is good? • If it was a “hard” study we’d get peer review to check the maths • “Soft” therefore we need to consider “fit for purpose” – or was “due analytical diligence” followed? • a code of best practice for soft OR • 6 rights to consider: • Is the context right? • Do we understand the system right? • Can we explore changes to the system right? • Is this the right way to find options? • Are the measures right? • Is this the right way of doing the study?

37. Some comparisons of the Front-End to studies on more developed issues • The models can be deceptively simple (this is an art) • We don’t try to capture everything (too hard or not possible) • Everything can be challenged (show due diligence) • Numbers are indicative not straitjackets (treat with caution as there are probably subjective weightings, only some of the issue will be looked at it and we don’t understand it all anyway) • Nothing can be logically deduced (creativity) • No front end study is right (but can be useful) • Give the same job to two different people and you’ll get two different approaches (but similar insights) • Even though there’s no formal definition of a mathematical objective function there will be something in the decision maker’s mind (help them) • Soft OR can be hard to do (the paradox)

38. The Front End as part of the DSTO/UNSW@ADFA MSc program • Compulsory courses (common to all DSTO MSc): • System Engineering • Research Methods • Compulsory foundation courses: • Operations Research • Problem Structuring • Quantitative OR • Foundation courses (2 of): • Analysis of military systems • Capability Options Analysis • Simulation • Strategic decision making • Supply chain science • Electives (1 of): • Any of the foundation courses • An agreed elective • Minor thesis

39. Acknowledgement – this presentation included material by some of my coworkers: • Jolanta Ciuk • Peter Dortmans • Wayne Hobbs • Denis Shine • Niem Tri • Much of this work is public domain • neville.curtis@dsto.defence.gov.au

40. Questions? Adelaide