Verbal Autopsy: overview of applications worldwide

1. Verbal Autopsy: overview of applications worldwide Professor Alan Lopez Melbourne School of Population and Global Health The University of Melbourne

2. The importance of cause of death information • Cause of death information at a population level helps to identify health priorities for planning • If we know what most people are dying from we can develop public health policy and direct resources towards these diseases • This information also helps us to evaluate health interventions and whether they have succeeded in reducing deaths due to specific causes • Provides trend data on overall mortality patterns • Supports ongoing population health surveillance, including emergence of new epidemics, etc. • Helps identify health research priorities

3. How do we obtain useful cause of death information for populations? • Optimal method is physician certification of sequence of morbid events leading to death, using WHO International Form of Medical Certificate of Cause of Death • This certificate is coded by trained coders to select the underlying cause of death using ICD rules (currently ICD-10) • However, limited availability of physicians in many LMIC, plus poor training, means this is only done for about one-third (15-20 million) of 55 million annual deaths worldwide. • For remainder, we need a simple, cost-effective method to diagnose CoD reliably. Some form of “Verbal autopsy” is required, based on symptom reporting by families.

4. What is Verbal Autopsy? • Process for deriving a cause of death from responses provided by the family to a structured set of questions on signs/symptoms experienced by the deceased prior to death • 3 components a) Questionnaire to seek information from family about symptoms, durations, etc. b) Methodto diagnose cause of death from questionnaire (typically, physician review) c)Target cause of death list that a) & b) above are compatible with (typically, 50-60 causes)

5. When to use VA? • VA is useful for deaths that occur outside of a hospital or health facility (e.g. home death, DoA, death on public roads, etc.) • VA is intended to provide essential, hopefully reliable information on causes of death in populations with limited access to health care (i.e. Cause-Specific Mortality Fractions or CSMF) • Other goal of VA is to correctly diagnose cause of individual deaths • Principle is to mirror the clinical process in terms of symptom assessment/medical history i.e. verbal autopsy ≡ pathology/autopsy

6. How has VA been applied? (1) • Most commonly, VA has been applied in research settings where it is important to know the true cause of death of individuals (i.e. HDSS, longitudinal follow-up studies of individuals with known baseline exposures, etc.) • Widely applied in HDSS in the INDEPTH network ( 30,000-100,000 population) primarily in Africa, but without any validation data to assess diagnostic accuracy. • In principle, this validation step provides information on performance characteristics of your VA questionnaire (e.g. for which causes does the questionnaire not work?). Need reliable ‘gold standards’ to validate VA performance.

7. How has VA been applied? (2) • Best current practice is to have physicians review the VA responses, including the Open Narrative, and to complete a death certificate according to rules and procedures of the ICD. This death certificate should then be coded to determine Underlying Cause of Death (UCOD) • Common practice is to have 2 physicians review each VA. If they disagree, ask a 3rd physician. If still disagreement (rare), case is usually dropped. How much confidence can we have in this procedure?

8. Challenges with Physician Coded VA (PCVA) • Diagnostic variability (skill?) • Replicability (intra and inter-physician diagnostic consistency) • Time • Cost • Accuracy??

9. SOLUTIONS • If VA is to be routinely applied in VR systems, we need to solve these problems. Need to replace physicians, but by what? • Can we use computer logic that recognizes patterns of responses in the data and associates these patterns with the true cause of death • Collectively, these methods are known as Automated Diagnostic Methods (ADM)

10. Typology of cause of death diagnosis methods T

12. Are computers more accurate than physicians? At the individual level, Tariff is 20% more accurate than Physicians in diagnosing VA’s; but medically certified death certificates in hospitals are still the most accurate source of cause of death information +20% Physician certified VA Hospital Death Certificates InterVA Tariff

13. Can computers accurately diagnose the cause of death? Accuracy of computer methods versus doctors in diagnosing causes of death, i.e. ensuring the cause of death distribution is correct at the population level +9% +13% Hospital Death Certificates InterVA Physician certified VA KL Tariff

14. What is the Tariff method to diagnose VA’s? Tariff is based on the “signal to noise” concept. Each questionnaire item gets a score, according to the number of times it was endorsed by the respondent (i.e., respondents answered “yes” to a question about the presence of a symptom during a VA interview; e.g.“didthe decedent have yellowish eyes?”). Any symptom reported in the VA that is known to be highly predictive of a particular cause of death (“signal”) will have a significantly higher score (“tariff”) assigned to it for that particular cause of death, compared with other, less strongly associated symptoms (“noise”). What is Tariff? How does Tariff work? The cause with the highest score (“tariff”) for a particular death is assigned as the cause of death for that individual. Tariff scores can be + or - depending on the strength of the signal.

15. How to calculate Tariff scores for various causes of death The table below gives the tariff values for a sample of cause-symptom pairs that have been calculated from previous research. Suppose the family member said that their deceased had fever, and was diagnosed with TB (i.e. these symptoms were endorsed and get a score of 1). We multiply the endorsement score for each symptom by the tariff value and sum them to get a set of tariff scores for various causes. In this example, AIDS has the highest Tariff score (6), and is therefore the CoD assigned for this case. Symptoms

16. Which VA questionnaire should I use? Choosing the right VA questionnaire is just as important as selecting the best method of analysing the questionnaire responses. The choice should be based on: • Time to administer the interview (the less time required, the better) • Possibility for automated diagnostic methods to be run on the questionnaire (such as the Tariff method) • Ability to diagnose an adequate number of leading causes of death (Tariff allows identification of 34 causes for adults, 21 for children and 10 for neonates) Choosing the right VA questionnaire The following four VA questionnaire options, all based on the same questions, are the most commonly used: • WHO – Full-version • WHO – Reduced version (based on expert opinion) • Population Health Metrics Research Consortium (PHMRC) – Full version • PHMRC – Item reduced version What are my options?

17. Short form VA questionnaire +Tariff method = SmartVA The Bloomberg Data for Health (D4H) Initiative, in collaboration with the University of Melbourne and IHME in Seattle, have developed a software application called Smart VA which incorporates the short questionnaire and the Tariff method. Using the short form VA questionnaire SmartVA can be run on any android smartphone or tablet, and can include both picture and language capabilities

18. Using Smart VA to generate policy-relevant data: application in Vietnam

19. Myanmar: application of SmartVA to nationally representative sample of 6,000 deaths,2017: Comparison with broad cause categories of GBD

20. Advantages of using tablets for collecting and processing VAs Strategic • Speed – diagnoses available the same day • Accuracy – minimises errors in data entry; diagnoses CoD more accurately ( SmartVA) than physicians reading VAs • Cost – largely costless except for cost of tablet • Standardisation/comparability – much higher inter-rater reliability than doctors Operational • Automated data quality checks (i.e., pregnancy can’t last for more than 10 months) • Automated data entry from VA questionnaire via Smart VA software • Interviewers obliged to follow survey skip patterns • Questionnaire is easily translatable into other languages • Picture and audio capability

21. Global applications of VA in national CRVS/SRS systems • SmartVA (used as input into physiciancertification): China (Shanghai, plus field trial in 14 DSP sites), Philippines, Colombia (?), Brazil, Mexico (child deaths) • SmartVA (primarily without physician input):PNG, Solomon Islands, Myanmar, Bangladesh, Rwanda, Peru(?), Sri Lanka (?) • SmartVA/InterVA/InSilicoVA from WHO2016: Kenya, Tanzania, Ghana, Morocco, Colombia (?) • SRS: physician review of VA in India, Bangladesh, Indonesia (SmartVA ?)

22. Conclusions • Verbal autopsy only real alternative to improve information on CoD patterns outside of hospitals in most countries • Recent advances in IT and research on methods, VA questionnaire design, etc. now mean automated VA methods ready for widespread application in VR systems (pending further trials in D4H countries) • Automated VA being used in many forms: optimal is as input to physician certification (e.g. Philippines), but also for DoA, home deaths, deaths with ill-defined causes etc). • Automated VA: most important advance in public health measurement in last 50 years