Key issues for longitudinal research design: Lessons from the Growing Up in Scotland study

1. Key issues for longitudinal research design:Lessons from the Growing Up in Scotland study Paul Bradshaw

2. Definitions of longitudinal research • In the broadest sense longitudinal research involves the follow up of any set of entities in which changes can be observed over time. • Individuals • Households • Institutions such as hospitals or schools • Nations • Most commonly focussed on individuals • No recognised definition of what period or what number of follow-ups constitutes ‘longitudinal’ • Aim is usually to determine causes and processes which lead to change and/or to particular outcomes

3. The UK experience • The UK has a long, and highly regarded, history of (particularly quantitative) longitudinal social enquiry • A number of ongoing internationally renowned longitudinal studies providing information on the life histories of people as they move from birth to old age • Current QNLR being undertaken in Britain includes: • Birth cohort studies • Age cohort studies • Family/Household panel studies • Area studies • Census-based studies

4. The challenges of longitudinal research • “The success of a longitudinal study depends on stable leadership from a committed principal investigator and a team of highly skilled researchers” • (Bynner et al, 2006) Data collection Data analysis Sample Cost Ethics Duration (& relevance)

5. The Sample • Two main issues: • Sample design • Non-response and attrition

6. Sample design considerations • What is the population of interest? • All vulnerable children? • Sub-groups of interest? E.g. with particular characteristics – age, family circumstances, area where they live, interventions received • What do you want to be able to say about them? • Use sample to generalise to population? • Compare outcomes and experiences between children in different groups? • How will you find/recruit them? • Is there a sampling frame with the info you need? • Will you need the help of an agency/organisation to recruit?

7. Sampling precision (1) • When using a sample, the data produces ‘estimates’ of what the real value may be in the population. The precision of these estimates is affected by: • Sample size • Sample clustering • Measuring sample precision: • Use ‘confidence intervals’ and ‘standard errors’ • Confidence interval (CI) • Typically 95% meaning 95 times out of 100 this interval will capture the true population value that we are trying to estimate • Expressed as e.g. 64% (+/- 6%) suggesting the true value is somewhere between 58% and 70% • Differences will usually have to be of the magnitude of the CI for it to be statistically significant – e.g. in example above, an increase or decrease of around 6% would be necessary. • Smaller sample sizes produce larger CIs requiring greater change for statistically significant differences to be detected

8. Sampling precision (2) • Sample numbers required in each group to demonstrate significantdifferences (with a power of 0.8)

9. GUS response and attrition rates

10. GUS: Sweep 5 response by age of mother

11. Non-contact and refusal rates

12. Sample maintenance strategies • Keeping in touch • Tracing • Keeping respondents informed • Valuing and appreciating respondents (including, but not restricted to, use of incentives) • Boosts of key sub-groups of interest and on-going sample refreshment • Making reasonable demands

13. Sample maintenance: keeping in touch • Potentially willing respondents can be lost by virtue of moving home. • Important to establish effective procedures for obtaining updated contact information. • Techniques include: • Collecting as much contact information as possible at first contact including telephone/mobile numbers and e-mail • Information about ‘stable contacts’ – someone who knows the respondent and would know where they had moved to • Regular mailings with return address – undelivered items act as address checks ahead of fieldwork • Updates from administrative and service databases

14. Sample maintenance: keeping respondents informed • In any research project, it is considered important to provide good information to respondents about the purpose and nature of research • Respondents must understand the longitudinal nature of the study so that they also recognise why they are repeatedly visited and why they are irreplaceable • Many studies adopt a ‘branding’ approach – a name and logo that clearly identifies the study and which is easily recognisable by respondents • Websites can be extremely useful in providing more detailed information

19. Sample maintenance: valuing and appreciating respondents • If respondents are going to continue to take part in a longitudinal study they need to feel that their time and effort is valued and is worthwhile. • This can be demonstrated through: • The interviewer • Thank you letters sent after each interview • Providing evidence that survey findings have been used • Illustrating the impact they have had on government policy • Use of incentives • Non-financial rewards – gifts such as pens, fridge magnets, calendars

20. Being part of NCDS • There’s one thing I’m guaranteed, all of my entire life, is one birthday card. • I’ve always liked being part of it, I’ve always enjoyed being part of it because it’s different and I’m quite proud of it really. • I think I could almost put this in as a landmark in my life. • I haven’t kidded on about anything in my life, warts and all, I’ve been honest and I’ve said it all. • ….. I think, you know, it does make you feel special in certain ways, like really….. • I haven’t ever seen anything negative in it actually, not anything negative at all…..I’ve always felt comfortable and I always know that if I don’t want to answer a question I don’t have to………… I think it’s just been fascinating

21. Recognising the importance of NCDS • I can remember becoming part of the NCDS, I didn’t understand it.... I think I’ve just grown to understand it more as I’ve got older and the importance of it and how it’s helping everybody really, that’s what I think. • Well, I think to a certain extent you’ve got to say it’s mainly for helping others, you know. Like you say it’s no benefit to me to do it, but then again it’s no skin off my nose not to do it, so. It’s one of those things like, you know….I don’t see a reason [not to]…… • I think it’s interesting that they’re actually following all these people, right through their life and ……they find out comparisons. Aye, I think it’s really…. that’s why I take the bother to, aye, come. I want to be part of it because I’ve been in it all my life.

22. Data collection: what to ask and when • Decisions on content of data collection require good planning – important to ask the right questions at the right time • Take a ‘longitudinal’ view: • What ‘stage’ are your subjects at? • What stage will they reach? • What situations, characteristics or contexts might you want to compare between those two stages? • What might be significant at stage 1 when you are comparing outcomes at later stages? • Decisions on content will usually be theory or hypothesis-based • What if you miss something?

23. Data collection: intervals between fieldwork • How often do you follow-up your sample? This is dependent upon: • Respondent burden • Developmental stages, processes or transitions that you are interested in • Budget • There is no recognised nor legitimate pattern – it’s largely dependent upon the objectives and focus of the research

24. Follow-up intervals of selected UK child cohort studies

25. GUS: Sources of data

26. Data analysis • Benefits of longitudinal data • Measuring change over time: • Flows into and out of ‘states’ (e.g. poverty, unemployment, being looked after) • The effects of change, or of state durations, on outcomes • Impact of interventions • ‘Individual’ development • Temporal ordering of events • Improved control for omitted explanatory variables • Improved control for the effects of previous states • Exploring the effects of ageing and cohort membership

27. Data analysis • Drawbacks of longitudinal data • Data management • Complex structure/relations • Complex variable/samples • Resultant file and variable management requires training and skills of good practice • Software issues • Complexity of methods • Some methods only available via specific software packages

28. Longitudinal models • Two main modelling approaches in social science research: • Event history analysis, time to an event (Also known as: duration analysis; survival analysis; failure time; duration economics; hazard modelling) • Panel data analysis • Regression models suitable for repeated observations • Time generally conceptualised as being discrete • Extension of standard regression models • Closely related to multilevel modelling • (Simple methods can also be used)

29. Ethical issues (1) • Confidentiality, data security and data access • Safeguarding confidentiality through • Restricting the data that are released • Controlling the arrangements under which potentially disclosive data is released • Data access/release • Providing data for researchers to analyse on their desktop with the level of sensitive detail restricted • Providing more sensitive data under a special licence • Remote access where raw data is never released and analyses are run in house on behalf of researchers who receive edited outputs • Safe settings where researchers are required to visit a protected site where access to the data is carefully managed.

30. Ethical issues (2) • Informed consent • What is ‘informed consent’? • Respondents must understand that the project is longitudinal in nature and what that means for them • They are free to withdraw at any time from the project as a whole or any aspect of it • One-off or repeated consent? • No universal practice, but • Some form of repeated consent would normally be involved

31. Summary • Longitudinal research in any methodological discipline presents a set of core challenges • The key issue is to consider, in detail, how the fundamental temporal nature of the project affects the basic aspects of its design including: • The sample • Data collection • Data analysis • Ethics • Requires a more complex design, but a good design will produce tremendously valuable data