Portfolio and Scenario Analysis in the Cost-Effectiveness Evaluation of Weapon Systems

1. Portfolio and Scenario Analysis in the Cost-Effectiveness Evaluation of Weapon Systems Jussi Kangaspunta, Ahti Salo and Juuso Liesiö Systems Analysis Laboratory Helsinki University of Technology P.O. Box 1100, 02015 TKK, Finland http://www.sal.tkk.fi firstname.lastname@tkk.fi

2. Contents • Finnish Defense Forces • Challenges in the evaluation of weapon systems • Multi-criteria portfolio model for weapon systems • Numerical example and future research • Conclusions

3. Finnish Defense Forces • Key statistics • Annual budget ~ 2.3€ (~$2.8) billion • About 1.3% of GNP (in USA ~4.5%) • Peacetime strength • 13,000 regulars • 27,000 conscripts • 30,000 reservists trained annually • Wartime strength 430,000 • Population of Finland ~5.2 million • Tasks • Territorial surveillance • Safeguarding territorial integrity • Defense of national sovereignty in all situations

4. Challenges in the evaluation of weapon systems • Several impact dimensions must be accounted for • E.g. enemy and own casualties, mission success probability • Impacts depend on the context • Mission (attack/defence), weather conditions, enemy strategies etc. • There are strong interactions among systems • How can joint impacts be best attributed to constituent systems? • Yet earlier work mainly focused on individual systems • Impacts are often very non-linear • 16 artillery guns may not be twice as effective as 8 guns

5. Modelling of weapon systems • Weapon system portfolio • = number of different weapon systems • = number of weapon systems of the jth type in portfolio x • = cost of portfolio x • Feasible portfolios satisfy all relevant constraints • E.g. budget constraints C(x) ≤ B, logical constraints (incompatibilities etc.) • Impact assessment criteria • Portfolios evaluated with regard to different impact criteria • Enemy casualties, own casualties etc. • Overall impacts approximated by an additive value function

6. Impact assessment model • Estimates from ground battle simulator of Defense Forces • Battle scenario with pre-specified enemy, terrain and mission • Numbers of own weapon systems varied according to an experimental design • Numerous simulations with different portfolios of selected weapon systems • Simulation results extended by interpolation Battle simulator Scenario Criterion 1 Own weapon system Enemy Overall impact of the portfolio Impact model Criterion 2 ... Criterion n

7. Incomplete information and dominance • Feasible weight set • E.g. rank-ordering for criterion importance • Portfolio x’ dominates x if it has greater or equal overall impact for all feasible weights two criteria; w1≥w2 V(x’,w) V1 V2 V(x,w) w1=1 w2=0 w1=0 w2=1 w1=.5 w2=.5

8. Cost-efficient portfolios • Feasible portfolios that are not dominated by any less or equally expensive portfolio V2 V1 COST Cost-efficient portfolios Inefficient portfolios

9. Numerical example based on realistic data • Three weapon systems • Additive costs • Three impact criteria for different types of enemy casualties • Incomplete information on the value (i.e relevance) of the impacts • Analysis at different budget levels with the aim of identifying cost-efficient portfolios

10. Simulated and interpolated impact functions x3=0 x3=1

11. Cost-efficient portfolios ~25% Inefficient portfolios ~75% Impacts of weapon system portfolios

12. Cost-efficient portfolios Composition of cost-efficient portfolios (1/2)

13. Cost-efficient portfolios Inefficient portfolios Composition of cost-efficient portfolios (2/2) x3=0 x3=1

14. Extensions and future research • Complementing simulation data with expert evaluations • Simulations can be augmented with judgmental expert evaluations of impacts • This helps overcome the ”curse of dimensionality” with more weapon systems • Experimental design of simulations and/or expert evaluations • Considering multiple battle scenarios • Cost-efficiency is highly context dependent  many scenarios are needed for comprehensiveness • These can be integrated with the MAVT model using probabilities • Risk and/or robustness measures for weapon portfolios can also be formed

15. Multiple battle scenarios 1 p1 Overall expected value of the portfolio Weapon system portfolio p2 2 ... ... pm m Optimization

16. Conclusions • Portfolio approach is necessitated by strong interactions • Evaluation of individual weapon systems makes little sense • These interactions are captured by the battle simulator • Multi-criteria model aggregates several impact dimensions • Contextual importance of impacts captured through incomplete information • Cost-efficiency depends on both impacts and costs • Focus on the computation of cost-efficient portfolios

17. References • Liesiö, J., Mild, P., Salo, A. (2007) Preference Programming for Robust Portfolio Modelling and Project Selection, European Journal of Operational Research, forthcoming • Liesiö, J., Mild, P., Salo, A. (2007) Robust Portfolio Modeling with Incomplete Cost and Budget Information, European Journal of Operational Research, forthcoming. • Stafira, S., Parnell, G., Moore, J., (1997). A Methodology for Evaluating Military Systems in a Counterproliferation Role, Management Science, Vol. 43, No. 10, pp. 1420-1430. • Parnell, G., et. al. (1998). Foundations 2025: A Value Model for Evaluating Future Air and Space Forces, Management Science, Vol. 44, No. 10, pp. 1336-1350.

18. Questions and comments?