Multiobjective Calibration with PADDS: Testing Alternative Selection Metrics

1. Multiobjective Calibration with PADDS: Testing Alternative Selection Metrics Masoud Asadzadeh Bryan Tolson

2. Outline • Objectives • PA-DDS algorithm • Alternative selection metrics • Experiment to choose proper selection metric • MO Performance Evaluation with CNHV • Validation of Selected Metric, MO Model Calibration • Conclusions and Future Work

3. Objectives • Evaluating PA-DDS performance: • Solving MOPs with more than 2 objectives • Using alternative selection metrics • Random (RND) • Crowding Distance (CD) • Hypervolume (HV) • Choosing proper selection metric • Validating selected metric, comparing modified PA-DDS against high quality MO algorithms: • AMALGAM vs. ɛ-NSGAIIvs. PA-DDS

4. Pareto Archive DDS Initialize starting solutions Perturb current ND solution Update ND solutions Create ND-solution set Pick a ND solution Pick the New solution New solution is ND? Y N Y STOP N Continue?

5. Alternative Selection Metrics • Random Selection (RND) • Crowding Distance (CD) • Deb et al. (2002) • Contribution to HyperVolume(HV) • Zitzler and Thiele 1999 • Used as selection metric in Emmerich et al. (2005) f2 f1

6. Experiment to Choose Selection Metric PA-DDS • Number of Trials: 50 • Budget: 1,000 and 10,000 • Performance Evaluation: CNHV RND CD HV 1 2 3 Mathematical Test Suites

7. Mathematical Test Problem, ZDT4Zitzler et al. (2000) • 10 decision variables • 2 objectives • 219 local fronts • Convex Pareto front

8. Mathematical Test Problem, WFG4Huband et al. (2006) • Scalable • 24 decision variables • 2 and 3 objectives • Highly Multi-modal • Concave front

9. Mathematical Test Problem, WFG4Huband et al. (2006)

10. MO Model Comparison • Comparative Normalized Hyper-Volume 1 Worst attained front Best attained front 1

11. CNHV vs. HV • Same as HV or NHV • CNHV always prefers dominating solution • CNHVA > CNHVB : B doesn’t weakly dominate A • CNHVmax = 1 & CNHVmin = 0 • Compares multiple trials of multiple algorithms • Measures performance across compared algorithms

12. Results: ZDT4 1 1 1 1

13. Results: ZDT4

14. Results: WFG4 Two Objectives

15. Results: WFG4 Two Objectives

16. Results: WFG4 Three Objectives

17. Results: WFG4 Three Objectives

18. Validating the Selected Metric PA-DDS • Number of Trials: 10 • Budget: 10,000 • Performance Evaluation: CNHV RND CD HV 1 2 3 Mathematical Test Suites ε-NSGAII PA-DDS AMALGAM Model Calibration

19. Model Calibration, Town Brook • Sub-watershed in Cannonsville • 37 km2 • SWAT2000 • 26 Parameters • Nash Sutcliffe • Flow, Phosphorus delivery Tolson and Shoemaker 2007

20. Model Calibration Results

21. Model Calibration Results

22. Model Calibration Results

23. Model Calibration Results

24. Model Calibration Results

25. Model Calibration Results

26. Conclusions & Future Work • PA-DDS inherits simplicity and parsimonious characteristics of DDS • Generates good Pareto approximate front in the modeller's time frame • Reduces the need to fine tune the algorithm parameters • Solves both continuous and discrete problems • PA-DDS can solve MOPs with more than 2 objectives • HV based selection clearly improves PA-DDS performance • PA-DDS with HV selection is promising compared to two high quality benchmark algorithms, AMALGAM and ε-NSGAII • Evaluate PA-DDS performance in solving Multi Objective model calibrations with more than 2 objective functions • Implement a more efficient archiving strategy and dominance check (e.g. Fieldsend et al. 2003)

28. Budget vs. Dimension

29. Results: ZDT4

30. Results: WFG4 Two Objectives

31. Results: WFG4 Three Objectives

32. Model Calibration Results

33. Model Calibration Results