Partially Ordered Sets for Multi-indicator System Evaluation

1. Satellite session on Multi-indicator systems and partially ordered setsViews from Official Statistics New Techniques and Technologies for Statistics Brussels, March 2017 Emanuele Baldacci *, European Commission, Eurostat emanuele.baldacci@ec.europa.eu @emibaldacci Dario Buono*, European Commission, Eurostat dario.buono@ec.europa.eu @darbuo *The views expressed are the author’s alone and do not necessarily correspond to those of the corresponding organisations of affiliation

2. Triggers for Socio-economic evaluation • Data-users call for condensedinformation • Allow multidimensional phenomena to be synthesized. • Need to ensure time and regional comparisons

3. Dashboards proliferation Pro: Separation of Statistics from Economic Reading Cons: not always straightforward

4. Dimensionality Reduction Indicator systems might leads to the need of computing composite indicators and introduce some degree of arbitrariness concerning: • The analytical approach • The choice of the weights • The aggregation technique

5. POSET@work • This is a non-parametric approach for ranking multi-indicator data sets • Poset theory can help to overcome the conceptual and computational drawbacks of the standard aggregative procedures by exploiting the relational structure of the data, so as to compute evaluation scores in purely ordinalterms.

6. Features for Official Statistics • Based on some Eurostat pilot projects • Posetpermits to deal with complex multi-dimensional systems of indicators • Posetallows to derive rankings and synthetic indicators, without variable aggregations • Poset have already been applied in a variety of areas (Refugees' relocation in the EU, European opinions on services, Formal Concept Analysis, Fiscal policies)

7. Drawbacks for Official Statistics • Based on some Eurostat pilot projects • Poset does not have a temporal dimension (all variables refer to a fixed point in time) • Posetis computationally complex and tools are currently under development

8. POSET@Eurostat (1) • Paper on "Complementing scoreboards with composite indicators: the new • business cycle clock", by Mazzi (Eurona 2015). • 4 possible applications of POSET to the PEEIs (with mixed results) • detect the presence of cross-sectional effects in financial markets () • explain the economic phases () • explain the country ranking with another variable (V) • measure the diffusion of a crisis (?)

9. POSET@Eurostat (2) • Applications of POSETs in macro-economics, • by Ruggeri,Mazzi,Fattore(NTTS 2017) • Case of Regional Competitiveness Index in IT • set of triples; regions are ranked; use of "benchmarking profiles" • The interesting point: no synthetic indicators, incomparability remains but the “distance” between two elements could be quantified • Use of the R package, PARSEC (Partial Orders in Socio-Economics), for implementing the procedure for multidimensional evaluation.

10. Lessons learned • The effectiveness of the partial order approach • is particularly evident in the way the weighting and the aggregation problem is addressed and solvedwith an objective approach.

11. Challenges ahead • Computational issues • New skills with deep learning curve • Communication and dissemination of methods and approach

12. Next Steps • Call for scaled-up partnership between official statisticians, researchers and analysts • Further research, in order to improve the efficiency of algorithms to avoid too heavy numerical computations • Education of usersabout the possibilities and limitations of using indicators including communication of detailed information on the underlying assumption and selection method Lisbon Memorandum, (DGINS Sept 2015)

13. Thanks!