George MAVROTAS Olena PECHAK

1. 8th Multi-criteria Meeting (HELORS) 8-10 December 2011, Eretria, Greece Dealing with uncertainty in project portfolio selection: Combining MCDA, Mathematical Programming and Monte Carlo simulation: a trichotomic approach George MAVROTAS Olena PECHAK

2. Contents • The project portfolio selection problem • Methodology • MCDA & Mathematical Programming • Incorporating uncertainty • Monte Carlo simulation • Trichotomic approach • Case study • Results and discussion • Conclusions & Future research

3. Methodology

4. MCDA problematiques = What kind of problems we can address • 4 problematiques (B. Roy) • Description – Understanding of the MCDA problem • Selection of the most preferred alternative • Ranking of the alternatives • Sorting of the alternatives in categories • Belton – Stewart (2002) • Selection of a subset under constraints (Portfolio)

5. Description of the simple case of project selection (without constraints) MCDA Ranking Selection of the first n-projects Top 20

6. If there are constraints... • Example • Segmentation constrains • Geographical • Sectoral • Logical constraints • Precedence • Mutually exclusive projects • Budget constraints • . . . • The alternatives are no longer independent • The top-n can only by chance fulfill the constraints • Combinatorial problem

7. Remedy • Examine all possible, feasible combinations of projects and try to find the “most preferred” Input Decision variables (binary)  projects Xj=1 (j-th project selected), Xj=0 (j-th project not selected) Constraints  feasible region Objective function  Sum of multicriteria scores Mathematical Programming Output the best combination of projects that satisfy the constraints= optimal portfolio

8. Combination of MCDA and Math Prog • MCDA • Scoring of the projects • Mathematical Programming • Integer Programming • Most preferred portfolio • Typical example: Promethee V

9. Dealing with uncertainty • Project uncertainty • Project attributes (costs, performance…) • Environmental uncertainty • Weights of criteria • Total budget • … • We assume stochastic nature of the uncertainty (probabilities, distributions) • Monte Carlo simulation

10. M Monte Carlo simulation & Optimization value1(i) param1 Project portfolio 1 Project portfolio 2 value2(i) Project portfolio 3 Solution of MP model i =1…n param2 . . . Project portfolio n value3(i) paramN

11. Project allocation in sets In each iteration we obtain an optimal portfolio Each project can be present (Xj=1) or not (Xj=0) in the optimal portfolio Trichotomic allocation of projects green set Projects selected in all optimal portfolios red set Projects not selected in none optimal portfolio grey set Projects selected in some optimal portfolios

12. Project allocation in sets Projects Iterations

13. Remarks from phase 1 • Usually there is no dominating portfolio • 1000 iterations  about 1000 different portfolios • Trichotomic approach provides useful information • Green set  they are in under any circumstances • Red set  they are out under any circumstances • Grey set  we are not sure, we need more info • Exploit information from phase 1 and go to phase 2 • Only the grey set

14. The model of phase 2 • Fix the values of green and red projects • Use as objective function coefficients the frequencies (fj) of the projects from the 1st phase

15. Results from phase 2 Two cases: • No stochastic parameters in the constraints • One single run • Final selection: the unique optimal portfolio • Stochastic parameters in the constraints • Monte Carlo simulation – Optimization (1000 runs) • Final selection: the dominating optimal portfolio • The portfolio with the highest frequency in 2nd phase • If there is no clear winner  comparison among the first two  project-wise comparison  total budget adjustments

16. Illustration of the method 2nd phase 1st phase green set Set of projects Multiple criteria Multiple constraints Uncertainty selected grey set Not selected red set MCDA MC simulation MathProg 1st phase info MC simulation MathProg

17. Case study Input data Shakhsi-Niaei, M., Torabi, S.A., Iranmanesh, S.H. (2011) A comprehensive framework for project selection problem under uncertainty and real-world constraintsComputers and Industrial Engineering 61, 226-237. 40 projects for telecommunication company, classified in 3 groups: • Basic - 2,7,9,12,13,14,17,22,23,26,28,37,38,39 • Applied - 1,3,4,6,10,11,16,18,20,21,24,25,27,30,31,32,33,35,36 • Developing - 5,8,15,19,29,34,40 Constraints • Available total budget (we allow a 15% excess) • Limits by project type (at most 20% Basic, 70% Applied, 40% Developing)

18. Case study The projects are evaluated by 5 criteria: • Cost: Total project cost including all expenses required for project completion. • Proposed methodology: Degree of being step-by-step, well planned, scientifically-proven, disciplined, and proper for organization current status in the proposed methodology. • The abilities of personnel: Work experience of project team related to concerned project. • Scientific and actual capability: Scientific degree and educational certificates of project’s team. • Technical capability: Ability of providing technical facilities and infrastructures. Uncertainties: Criteria weights Budget Costs Methodology Personnel qualification Scientific Technological Environmental Internal

19. Incorporating uncertainty with distributions • Weights  triangular distribution • Total budget  normal distribution • max(6000; normal(6000, 300)) • Scores  uniform distribution

20. Project’s data Projects’ data

21. Results of the model – phase 1

22. Results • Phase 1 • No clear dominating portfolio • We may introduce threshold for green and red projects • 1%  projects are green if freq > 99% and red if freq < 1% • Results • Green set - 7 projects (5, 8, 15, 19, 29, 34, 40) • Red set - 3 projects (2, 9, 17) • Grey set - 30 projects • Phase 2 • Still no clearly dominating portfolio, but 2 combinations are most preferred • The difference between 2 most frequent portfolios --> 1 project.

23. Results Results for 1000 iterations of seed 1513. (we test 15 different seeds  15 MC experiments  no significant difference) Frequency 1 and 2 are the frequencies of top 2 portfolios. • The most frequent portfolios differ only by 1 project 16 or 24 (only one of two may be in): • Both are in the group of “applied” projects • Have similar characteristics (in some criteria 24 performs weaker) • Final decision still to be made by a person according to the main goals.

24. The naïve approach: Consider the expected values • Expected values  no uncertainty • Although the result may be almost the same… … we have more fruitful information than considering just the expected values • We know which are the sure projects (green and red) • We can identify the “borderline projects” • We know the probability of the preferred combination from phase 2

25. Conclusions • The trichotomic approach is a structured method that can deal with uncertainties in project selection • Combination of MCDA, MP and MC • Reduces information burden • Identification of the sure projects • The DM can focus only on the grey projects • Flexible, not black box • Can be adapted to a specific decision situation and DM • More fruitful information than the naïve “expected value” approach

26. Future work • To address separately the environmental (criteria weights, budget etc.) and internal project uncertainties • Iterative approach: to decrease the uncertainties on each new iteration • Ask for more information only for the grey projects • To apply the trichotomic approach in group decision making • Phase 1  unanimity principle • Phase 2  majority principle

27. Thank you!