1 / 22

Unit 3 Siobhan Carey Department for International Development

Making cross-national comparisons using micro data. Unit 3 Siobhan Carey Department for International Development. Content. Why it’s important to get it right Preparation Assessing the data Organising your analysis Presentation of results. Does it matter if it’s wrong?.

nell
Download Presentation

Unit 3 Siobhan Carey Department for International Development

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Making cross-national comparisons using micro data Unit 3 Siobhan Carey Department for International Development

  2. Content • Why it’s important to get it right • Preparation • Assessing the data • Organising your analysis • Presentation of results

  3. Does it matter if it’s wrong? • Don’t be famous for the wrong reasons • Importance of reproducibility • Be able to defend your analysis if challenged • Two examples of why it’s important

  4. International Adult Literacy Survey • A survey run by very reputable organisations on literacy skills of adults • How do you explain the differences? • Various theories

  5. ….cont’d • Methodological review found • differences in sampling methods • differences in fieldwork practices • deviation from the survey design • A second review • translation • scoring of tests • …………

  6. Programme for International Student Achievement (PISA) • How do you explain the differences? • didn’t lead to any investigation • results accepted but lead to huge debate • Was IALS worse that other surveys? • Why did PISA not lead to the same questions?

  7. Preparation • Know the data • Read the documentation before you do the analysis • Have clearly defined research questions • Which countries are you going to include • Small number and expand? • Large number and reduce? • Combine countries?

  8. What do you need to consider • Was the survey intended to support cross-national research from the outset? • Is the survey design the same across all countries? • In what areas do the countries differ in design or execution and are these important? e.g. – all school types included?

  9. Understanding the data • Comparability of overall design • What questions were asked – who was asked • Sample design and coverage • Mode • Survey response • Data collection methods • Translation • Data processing and imputation • Missing values • Topic measurement – any special features (anthropometry, biochemical…)

  10. Understanding the variables • Which variables are derived • Which are imputed • How have they been imputed • What other modifications have been made e.g. hours spent watching TV – a categorical variable modified into a continuous variable

  11. Preparation • Creation or transformation of variables • Recording, derived variables, dummy variables, reference group • Reducing dataset to improve speed • Document your analysis - always • Develop good habits – in-flight v programme based - Naming conventions

  12. Weights • Design weights • Population weights • Compound weights • Replicate weights Which you use will depend on what comparisons you are making

  13. Analysis • Rarely unique • Use existing analyses to learn • Check and double check – • Reproduce, check bases, check population, check text to tables, check tables to source…. • Check significance

  14. Analysis - output • Does it pass the common sense test? • Are the bases right? • Are the right weights on? • Is it plausible? • What can you triangulate against? • Is your analysis simply a reflection of distribution? • What could be behind it? – e.g. poverty reduction and population growth

  15. Analysis • Horse race? • Gets media attention but … • Correlates are more interesting • e.g. relationship between sexual behaviour and HIV prevalence • Analysis needs to be set in context • Take into account system factors – e.g. school characteristics, hierarchical features, standardisation

  16. Presenting your analysis • Borrow from others • OECD, World Bank, UN, Lancet … • Try to keep it simple – • what’s the story • and why is it interesting

  17. Rewards? • Help understand social condition • Help unpick complex relationships • Contribute to evidence base so as to improve policy and outcomes • Interesting – only if you’re curious!

  18. Activity • Using ESS – on a topic of interest • formulate a research question - which countries are of interest? • investigate some variables that are relevant • look at difference when weights applied • are there differences in distribution between countries? Why might that be? • what is the unit of analysis and which weights should you use? • Develop an analysis plan

  19. Suggestions • Crime and fear of crime • Religiosity • Attitudes to organs of the state • Values – attitudes to cheating on tax • attitudes to sexual behaviour

More Related