140 likes | 221 Views
Dissemination to support Research & Analysis. John Cornish. Global trends. Users around the world are seeking access to more detailed data for research & analysis More analytical/quantitative academic research Evidence-based policy
E N D
Dissemination to support Research & Analysis John Cornish
Global trends • Users around the world are seeking access to more detailed data for research & analysis • More analytical/quantitative academic research • Evidence-based policy • More international comparisons and exchange of information amongst researchers, incl for monitoring MDGs • More NSOs are finding ways of providing access to detailed data, including microdata • Better understanding of user needs • Improved usability & functionality of software interfaces • Improved database technology • Growth of Data Archives • Trend towards ‘self help’ for most users • 2003 Conference of European Statisticians agreed that more needed to be done to support research and analysis • But must not go beyond what is legal or acceptable to the public
Ways of supporting users doing research & analysis • Have relevant data which is fit for use • Detailed aggregated data & Microdata • Good documentation, metadata and archiving - users need to find what is available and all about it • Access mechanisms • Use standards, maintain consistency • Partnerships • Staff who understand researcher needs • Involvement with professional organisations (eg population associations, economists)
Macrodata • Data more detailed than in publications • Detailed tables • Datacubes • With or without tabulation software • Detailed Indicators - ability to ‘drill down’ • Thematic databases • Time series • Geographic/area based/international • industry
Indicators • Web ideal for allowing users to drill down, draw graphs/maps • More for assessing trends rather than reasons for trends
Microdata • There are limits to analysis which can be done with macrodata • Users want more flexibility with different cross-classifiers, different aggregations, etc • Lots of models of analysis require working at the micro level • Linking of datasets
International Guidelines • Guidelines on “Managing Statistical Confidentiality and Microdata Access” by Trewin et al • Contains many case studies on good practices • Available on the UNSD website
Guideline Principles • It is appropriate to use microdata for statistical research & analysis as long as confidentiality is protected • Microdata should only be made available for statistical purposes • Provision must be legal • The procedures for researcher access to microdata, as well as the uses and users, should be transparent and publicly available
Confidentiality • Usefulness & confidentiality risk are inversely related • Usefulness increases with detail • Confidentiality risk increases with detail • Usefulness reduces with confidentiality adjustments (eg collapsing of detail) Confidentiality risk = f (sampling fraction + file detail + adjustments + management of access) also quality Greatest risk is with business records & families/households
Options for microdata access • Release of confidentialised microdata files -direct to user or via data archive • Remote access to more detailed confidentialised microdata files • Microdata services - submit jobs, NSO runs & vets • Data Laboratories • Research collaborations
Managing the Risks • Microdata Review Panel • Provide a variety of access methods • Legally enforceable undertaking is important • Withdrawal of service • Training of researchers before allowing access • Audits of actual practices
Data Archives & Microdata Services • International Census Microdata Service (IPUMS) provided by University of Minnesota • SADA, Data First • Key benefits to NSOs • Archiving and data recovery services • Harmonisation of variables to facilitate comparisons across countries and time • Allows access to international data sets • Documentation of microdata in English AASDA
Infrastructure issues • Data warehouse/data management - consistency of outputs • Metadata • Standards - data and presentation • Confidentiality knowledge & measures • Interface tools • Access facilities & vetting • Archiving
Some success factors • Good user engagement • Sound data management • Management strategies for risk- risk management, not risk avoidance • Overall vision for website & a transition path • Research & analysis done by NSO