160 likes | 303 Views
Evaluating the quality of vital statistics. UN Expert Group Meeting, 27-30 June 2011 Dr AJ Lanyon Australian Bureau of Statistics 30 June 2011. Outline. Chapter V: some comments Utility of an assessment framework and guidance tool Example: WHO/UQ HIS-Hub tool
E N D
Evaluating the quality of vital statistics UN Expert Group Meeting, 27-30 June 2011 Dr AJ Lanyon Australian Bureau of Statistics 30 June 2011
Outline • Chapter V: some comments • Utility of an assessment framework and guidance tool • Example: WHO/UQ HIS-Hub tool • Country Developmental Pathways • Conclusion?
Chapter V: comments • Evaluating quality can be challenging and complex • Chap V currently provides comprehensive overview of different types of approaches and methods (direct and indirect) • Should, in general, be carried forward
Chapter V: comments • However: • elements of the Chapter are very technical and dense • could benefit from reference to an assessment framework that provides guidance about why, when and how to assess different elements of the quality of vital statistics • could consider a recommendation on how to manage revisions of data and subsequently vital statistics
Example framework • Work by WHO / UQ HIS-Hub across past 5 years has demonstrated value in improvement of CR and VS systems of: • overarching system assessment framework • assessment guidance tool. • Help to: • Identify strengths / areas for improvement • Develop feasible action plans for CR and VS system improvement.
Context • Tool developed in light of slow progress in a number of countries in strengthening their CRS • Framed to empower countries to develop action-based plan with the support of key stakeholders in their CR and VS systems
Framework focus? • Reviewing the system underpinning the production, quality and use of birth, death and cause-of-death information • Not just focussed on the product or the output i.e. vital statistics • Did not include foetal deaths as many countries not able to collect relevant data – could be expanded to
Evidence that it works . . • Countries who have successfully used the framework now have: • Strong understanding of strengths/weaknesses of their systems • Complete understanding of key stakeholders necessary to support effective system functioning • Action plan in place to achieve functional and sustainable system and quality VS • Intra-agency co-ordinating committee moving forward
What’s in the framework? • Inputs – Legal framework for CR and VS systems; Registration infrastructure and resources. • Processes – Registration practices, coverage and completeness; Death certification and cause-of-death; ICD mortality coding practices. • Outputs – Data access, use and quality checks.
What’s in the guidance tool? • Comprises series of questions for each sub-component • Helps investigate how CR and VS systems have necessary inputs and processes in place to produce quality outputs i.e. vital statistics • Assists in better description of: • any issues or problems, and • possible strategies to address.
Review approach • Promotes establishment of an inter-agency review team • Required to gather relevant (specified) evidence • Basic assumption: • Buy-in of key stakeholders to process and ownership of final action plan • Collaboration = KEY TO SUCCESS
Component E1 • Data quality and plausibility checks • Levels of fertility and mortality • Causes of death • Asks questions like: • Are fertility indicators routinely calculated? • If so, which ones? • What data sources are used as the denominators to calculate rates? • Describe plausibility and consistency checks carried out (gives examples)
Developmental pathways • Rapid assessment version of the tool has been applied in 26 Asia Pacific countries • Results help to differentiate between strength of countries CR and VS systems • Identified 3 broad groups of countries: • dysfunctional to weak systems: • Functional but inadequate; • Satisfactory (minor adjustments possible).
Developmental pathways • Analysis of 26 assessments • Cross-cutting issues: • Coder qualification and training alongside quality of coding • Routine procedures to check quality of fertility and mortality data produced • Certification practices • Lack of collaboration across government departments.
Developmental pathways • Analysis also re-affirmed different countries: • different levels of system maturity • different strengths, weaknesses, opportunities and threats, and • different developmental priorities and pathways. • Cross-cutting issues could be addressed through regional programmes.
Conclusion • Can’t replace technical sophistication required to evaluate quality of VS • Can provide a framework to support countries to work through (what can be) a complex and challenging process to understand quality strengths and develop strategies to address issues.