Quality Indicators (Binary ε -Indicator)

1. Quality Indicators(Binary ε-Indicator) Santosh Tiwari

2. Background • Multi-objective optimization ̶ Outcome is an approximation set. • In a real scenario ̶ Actual pareto-optimal set often unknown. • Our motive is to compare approximation sets, not algorithms. • In case of algorithms ̶ multiple runs ̶ distribution of indicator values need to be considered. • Basic idea ̶ x1 is preferable to x2 if x1 dominates x2.

3. Performance Evaluation of an Outcome • Quality of an outcome ̶ Quantitative description of the result (approximation set) e.g. Convergence, Diversity etc. • Computational resources required ̶ Measured in terms of number of function evaluations required, running time of algorithm etc.

4. Quality Indicators Three basic types • Unary performance indicators ̶ require only one approximation set. • Binary performance indicators ̶ require more than one approximation set. • Attainment function approach (conceptually different) ̶ Estimating the probability of attaining arbitrary goals in objective space from multiple approximation sets.

5. Unary Quality Indicators(Few Examples) • Convergence metric ̶ average distance of the approximation set from the efficient frontier – Actual efficient frontier required. • Hyper-volume measure ̶ volume of the objective space dominated by an approximation set. • Diversity metric ̶ chi-square-like deviation measure.

6. Limitations of Unary Performance Indicators • Cannot indicate whether an approximation set A is better than an approximation set B. • Above statement holds even if a finite combination of unary indicators are used. • Most unary indicators only infer that an approximation set A is not worse than B. • Unary measures that can detect A is better than B are in general restricted in their use. • Binary quality measures overcome all such limitations.

7. Binary Quality IndicatorsFew Examples • Coverage indicator – fraction of solutions in B dominated by one or more solutions in A. • Binary ε-indicator (detailed description ahead). • Binary hyper-volume indicator – hyper-volume of the subspace that is weakly dominated by A but not by B. • Other indicators e.g. Utility function indicator, Lines of intersection (uses attainment surface) etc.

8. Domination Relation for Objective Vectors • Weak Domination • Domination • Strict Domination • Non-dominated (Incomparable) Approximation set is a set of incomparable solutions

9. Domination Relation in Approximation Sets Every objective vector in B is weakly dominated by at least one member in A. A weakly dominates B but B does not weakly dominate A. Every objective vector in B is dominated by at least one member in A. Every objective vector in B is strictly dominated by at least one member in A.

10. Binary ε-Indicator (Definition) • ε-domination (multiplicative) • Binary ε-indicator Iε(A,B) Minimum value of ε (>0) for which every member of an approximation set B is weakly ε-dominated by at least one member of approximation set A.

11. Computation of Iε(A,B) Time Complexity O(n.|A|.|B|)

12. Algorithm to compute Iε(A,B) • Step 1: Find the ideal point of the combined sets (A & B). • Step 2: Translate both the approximation sets such that ideal point is situated at (1, 1, …, 1) in n-dimensional hyper-space. • Compute • Finally,