The Possibilities of Enhancing or Generating Longitudinal Studies through Record Linkage

1. The Possibilities of Enhancing or Generating Longitudinal Studies through Record Linkage Michael Noble and George Smith Social Disadvantage Research Centre

2. The Study • Overview of developments in admin records availability • Strengths and weaknesses of admin records • Methods of Data Matching • Using admin records in conjunction with surveys • As sampling frame • Individual level linkage • Ecological linkage • Linking admin records to admin records • Linking repeated extracts of the same records to generate a longitudinal data set • Linking such longitudinal data set to records from other sources • Data Protection Issues • Lessons from Northern Europe • The way forward Social Disadvantage Research Centre

3. Outline of Presentation • Surveys, Census and Registers • An ideal and the reality in the UK • Current UK Use of Administrative Data • Linking Administrative Data in the UK – an example • Ways Forward? Social Disadvantage Research Centre

4. Routes to the Development of Longitudinal Data • Census and Surveys (e.g. UK) • ‘Registers’ (e.g. Denmark and Sweden) or ‘Administrative Data’ (UK) • Convergence? Social Disadvantage Research Centre

5. The Ideal? • Large sets of administrative data linked over time and across different fields. • Data of high quality and consistency • Accessible to researchers on a standard basis, either as a ‘stand alone’ research resource or as a possible link to existing surveys Social Disadvantage Research Centre

6. The Reality in Denmark • Development of ‘data registers’ since the 1970s to replace census and surveys • Linkable at individual level through the individual ID number introduced nationally in 1968. • Data held by ‘Statistics Denmark’; but accessible to researchers under carefully controlled procedures. Social Disadvantage Research Centre

7. Example of Register Based Longitudinal Research Study • Study of Attempted Suicide Among Young People Born in 1966. (Cristopherson et al, 2003), to look at risk factors • Takes complete age cohort (c 85,000) and draws in data from a large number (?up to 16) ‘registers’ for the cohort group and also in some cases their parents. • Data is prospective but drawn out retrospectively. • Focussed on ages 13-24 but able to use data from an earlier period • Can only identify factors/events recorded in the registers. Lacks 'soft data’ such as psychological factors and also some ‘protective’ factors (e.g. supportive friendship network) Social Disadvantage Research Centre

8. The Reality in the UK? • Followed the census and surveys route to social statistics and research. • Same growth of administrative data in a potentially linkable and increasingly individual format (as in Denmark and Sweden). • Growth of Administrative data system within Government. Linking initially 1% , later 5% and now potentially 100% of records in a longitudinal format. • Lack a single ID, but growing bodies of data than can in principle be linked both longitudinally and across registers. • Researchers outside government have played relatively little part in these development, though health data record linkage in the UK stimulated by the long term Oxford Medical Record Linkage Study (ORLS) • Potential convergence with the systems in Denmark and Sweden? Social Disadvantage Research Centre

9. Access for UK Researchers? • Increasing amounts of usage by researchers of such data sets within Government. • Largely restricted either to groups working directly for Government or on major national studies where there is strong Government interest or involvement. • No routine access for researchers in general. Each case has to be negotiated. • Clear impression is that individual ‘deals’ vary in ways that may reflect the different legal positions, but are often ad hoc. • No consistent pattern detectable. Social Disadvantage Research Centre

10. Types of Access to Administrative Data for Research • Sampling frame/enhancing the longitudinal element by linking to an admin data set for updates. Normally indirect (e.g. through Govt or Agency). • Surveys using data from admin system as supplement to what can be directly collected (e.g. major longitudinal surveys, with ALSPAC probably collecting more data of this type that any other similar project – but even here each link has to be negotiated separately). Usually indirect • Evaluations of major government programmes (particularly New Deals) where both surveys and admin data is being used longitudinally. Sometimes direct access to the data in these cases. • Studies relying solely on admin data linked over time and across different records. Also may get direct access. Social Disadvantage Research Centre

11. Linking Administrative Data in the UK – a (composite) example • Draws on three Studies • Changing Patterns of Income and Wealth/Growing together Growing Apart • Claiming Matters • NDC Evaluation • Measures ‘Income Deprivation’ by reference to ‘out of work’ means tested benefits • Measures worklessness by reference to JSA/Incapacity Benefit receipt Social Disadvantage Research Centre

12. The Data • Records of Claimants of Income Support and Income based Job Seekers Allowance • Data at 1995, 1998 and 2000 • Allows distinction between the following groups • Lone parents • Sick and Disabled • Older People • Unemployed people Social Disadvantage Research Centre

13. Advantages and Disadvantages of Data • Includes all cases – 100% not a sample • Geo-coding allows for localised information to be presented • Held at individual level allowing linkage to create longitudinal data • BUT • Limited by variation in take-up • Limited by administrative definitions • Paucity of explanatory variables Social Disadvantage Research Centre

14. Longitudinal Analysis • Creating individual level longitudinal data base by linking by e-NINO or using GMS-ONE • Descriptive Statistics using Transition Matrices • Creation of ‘designer’ control areas • Modelling exits to examine impact of NDC activity • Multilevel • Proportional hazard models Social Disadvantage Research Centre

15. Social Disadvantage Research Centre

16. Social Disadvantage Research Centre

17. Social Disadvantage Research Centre

18. Social Disadvantage Research Centre

19. Social Disadvantage Research Centre

20. Social Disadvantage Research Centre

21. Social Disadvantage Research Centre

22. Percentage of 1995 claimants who had left the data set by 2000 Social Disadvantage Research Centre

23. % of 1995 lone parents and unemployed who became ‘disabled’ by 2000 Social Disadvantage Research Centre

24. Issues • Variability of Access for Research (no blanket ban, but no general right of access). • Admin data quality and variability • Concerns over quality of linkage if done remotely. Social Disadvantage Research Centre

25. Proposals • More consistent and clear route to data linkage for surveys or other research? • Controlled access to linked administrative data by more researchers, using some form of standardised setup, along the lines now possible for Danish researchers? Social Disadvantage Research Centre