Policy effects of social research Labour market policies

1. Jacques BROSIUS CEPS/INSTEAD, Luxembourg Policy effects of social researchLabour market policies Joint seminar March 18 – 20, 2008 on EU policy making – EU reality www.ceps.lu

2. Short presentation of CEPS/INSTEAD • Public research centre in Luxembourg • Socio-economic research • Independent research programme +Contracts with different government departments • Main topics: poverty, labour market analysis, behaviour of firms, spatial aspects of the economy, living conditions • Our role: national research & international representation of Luxembourg

3. Today’s presentation • The pitfalls of presenting research results to policy makers • Examples from labour market research

4. What you learn at University (1/2) • At University: focus on academic research • Writing and presenting rules are imposed by standard layout of journals • Typical example: • Introduction • Literature review • Data • Theoretical chapter • Empirical chapter • Conclusions

5. What you learn at University (2/2) • Writing style should be short and concise • Scientific wording and mathematical formulations are standard • Common terminology of your research field are known and do not need to be explained • Emphasis is put on the value added of your theory or your empirical results • There is a preselection of papers by the journals

6. When you work for a policy maker • New public -> New rules • Previously: academic research communityNow : national (international) decision maker • The tools and methods stay the same (up-to-date methodology with best models) • The presentation changes!

7. Real world examples • Three examples from real-life experiences with the decision makers in Luxembourg: • Are your results statistically significant? • Are your statistical techniques confusing? • Is your writing style too technical?

8. Example 1: Are your results statistically significant? • Topic: participation rate of women in Luxembourg • Data used: labour force survey • Result: 2006 (65%), 2007 (63%) • Interpretation by the government: the participation rate has decreased • In reality: difference is not statistically significant • In statistics you usually have confidence intervals due to survey data • Difficulty: how to explain this technical aspect to a non-specialist audience

9. Example 2: Are your statistical techniques confusing? • Topic: evaluation of active labour market policies • Difficulty: estimation of the counterfactual • We concluded: « An additional 10% of unemployed found a job thanks to the policy » • They concluded: « 10% of those benefitting from the labour market policy found a job » • We had to clarify results in meeting with Minister

10. Example 3: Is your writing style too technical? • Topic: Why do cross-border workers come to work in Luxembourg? • Presentation of simple descriptive statistics with results for sub-populations: 45% of cross-border workers are interested by higher wages; 60% of these have a low qualification • One member of parliament said in a speech: 60% of all cross-border workers have a low qualification • Too many descriptive statistics seems to confuse some of the decision makers

11. Avoid the pitfalls • The following slides show some typical situations you will be confronted with when working with government officials. • Suppose you are the only expert working in your country on a specific problem. This is realistic even in a large country as the UK if your topic is sufficiently specialised. • Let’s call the decision maker M. (as in Minister)

12. M. does not have the technical abilities to check your results and has confidence in your research • Do not misuse this confidence! • Do not pretend having answers to everything just because you are afraid M. could believe that you are incompetent • Double check all your results (advantage of working in groups)

13. M will use your results for changing policies that could affect many people in your country. • Be aware of the impact that your research can have for the population • Make sure you apply the correct methods and that your results are robust (not very large variations of results with different methods) • Only present results that are statistically significant • Search for alternative explanations for your results

14. M. is interested by political conclusions • Do not just interpret the coefficients of your regression, go a step further and present policy implications • But do not go as far as to impose decisions; your role is to inform, the role of M. is to make informed decisions • Make sure to give all elements that you think are needed for making a good policy decision • Verify that the research topic is clearly defined between you and M. Do not announce something you cannot do (because of data limits for example)

15. M. might want to influence your research by suggesting expected results • You need to avoid this pitfall by staying scientific and not trying to satisfy M. (even if M. pays the research contract) • Always remember that independent researcher could be asked to check your results! • Stay objective, avoid being subjective (also during your presentations) • Be aware that, in the political arena, not everyone will like your results

16. M. has certain time constraints (governmental meetings where results should be presented, elections,…) • Make sure you respect these deadlines • Clearly explain to M. that these deadlines could limit the extent of your research • Also make sure M. understands your constraints: data accessibility, time constraints, budget constraints

17. M. usually does not have a degree in statistics • Keep your explanations simple • Define the concepts that you use • Do not use abbreviations without defining them • Add a methodological appendix, detailing the more sophisticated research results that could not be put in the main text • You are the expert and have to decide which results are the most reliable; make sure to explain the impact of data quality, weighting, number of observations,…

18. M. has hired you because you are the expert • Keep updated about the recent policy changes • Inform yourself about policies applied in other countries • Continue to read about new statistical methods • Try and get access to the best possible datasets • Use the appropriate graphical presentations

19. M. needs to be convinced of the quality of your presentation • Appear confident • Be prepared for additional questions • Keep the message clear • Use a graphical approach • Avoid technical jargon

20. Conclusions • When working with a policy maker, you need to use the tools you have learned in your statistics courses but you need to adapt the presentation of these results to the non specialist audience • Always keep in mind that from now onwards your research does no longer concern only yourself but it can have an important impact on thousands of people affected by a possible policy change due to your analysis