Microdata for policy research: lessons from the UK experience

1. Microdata for policy research: lessons from the UK experience Felix Ritchie

2. Structure of presentation • The value of microdata • The development of microdata use in policy • Challenges to microdata use

3. Part 1 The value of microdata

4. Why microdata? Ecological fallacy Town A has average income of NZ$42,000 and votes conservative Town A has average income of NZ$38,000 and votes liberal Therefore, wealthier people vote conservative… • The evidence-based policy need • how factors do interact at the individual level? • Technological developments • Secure access not previously considered practical • We managed without it before… • Limits of aggregate relationships • Inferences on a small number of variables • The ‘ecological fallacy’

5. 520 515 510 derived 505 500 495 490 490 495 500 505 510 515 520 stated Where microdata works best • Innovative perspectives • marginal effects • linked datasets

6. Impact: examples from the UK • National Health Service cost factors • Treasury productivity reports • Programme evaluation • National accounts • Low pay • Unsuccessfully!

7. Part 2 The development of microdata use

8. The historical perspective • Enormously successful Data Archive since 1960s • By 21st Century, distributed data increasingly limited • Detail increasingly locked up in confidential data • Need for secure facilities • Technology suggested flexible alternatives • Early days of the ‘Virtual Microdata Laboratory’ (VML) • Adopted standard ONS practices, but • Designed and run by researchers… • …Things changed

9. What changed? Internal drivers • In the VML • New models of disclosure control • New ‘incentive compatible’ operating models • ‘Open innovation’ • Active approach to security management • Active policies to identify and address gaps in research • Across ONS • New models of risk and risk perceptions • Re-evaluations of the range of data resources

10. What changed? External drivers • Evidence-based policy • Early results of direct policy interest • Funding for basic research (HMT, industry dept) • Requirement for programme evaluation • Increased awareness of possibilities • Practical problems solved by existence of VML • Data linking and consent improved • Law

11. ONS’ perspective on data access today • All data is accessible • Procedures are in place to cover most needs • if not, we have ways to identify them • Security and risk are multi-dimensional • Risk reflects real-life scenarios • Risk reflects all possible outcomes for data • including non-use • Distribute access, not data, where possible

12. Distributed access • Why is this good? • Data always under ONS control • Live monitoring • Simpler, but safer, disclosure control • How does this work in practice? • VML accessible from all ONS computers • Access points in govt. offices in Glasgow and Belfast • More coming • VML-duplicate set up on academic network • VML set to become exception rather than default data store

13. Impact of new world-view • When taking decisions about access to data… • Infrastructure is not important • Procedures are not important • Risk is not important • These are solved problems • The important problem: how much data do you want to give to whom?

14. Part 3 Challenges to microdata use

15. Three major problems • Supply • Demand • Capacity

16. The supply of data • Outside ONS, legal barriers to access • Concerns over researcher trustworthiness • Concerns over cost • Data not first priority • Research not first priority • Practical issues with administrative or survey data • Over-enthusiasm

17. Demand for research: some superficial limitations • Awareness of possibilities • Awareness of data • Research skills (analysis and commissioning)

18. Demand for research: some deeper limitations • Timeliness of research • Relevance of research • Relevance of data

19. Capacity issues • IT development lags badly behind demand • Too much data • Too much research • Communication is a major headache • Insufficient commitment to an ‘infrastructure’

20. Part 4 Concluding comments

21. What have we learnt, (1)Providing access • Principles are the key to efficient operation • Some key principles • Clear need to assert that re-use of data is good • Infrastructure is an investment, not an expense • Risk-assessment is holistic (include risk of non-use) • Research centres need to be a collective effort • Technology is small stuff • There is a lot of experience out there

22. What have we learnt, (2)Making research valuable • Policy-relevant results essential for credibility of data policies • Need to actively champion data use • Need for intermediary between researchers and government • This is a long-term strategy • Also investment in human capital to be made

23. Conclusion • Many benefits from microdata use, at relatively little cost • User engagement essential • Supplier engagement essential • Design infrastructure on principles, not data • Achieving the benefits needs work

24. Questions? • Felix Ritchie • Microdata Analysis and User Support • felix.ritchie@ons.gsi.gov.uk • Virtual Microdata Laboratory (VML) • Microdata Analysis and User Support • maus@ons.gsi.gov.uk