PRECOG: Developing a practical, evidence-based approach to assessing cataract surgical outcomes

1. PRECOG: Developing a practical, evidence-based approach to assessing cataract surgical outcomes Zhongshan Ophthalmic Center, Preventive Ophthalmology Unit, Guangzhou, China ORBIS International Nathan Congdon, MD, MPH

2. Financial interest No financial interest

3. The problem of un-operated cataract • The key to solving this problem, still the world’s leading cause of blindness, is training additional surgeons • The critical issue is outcome quality, for which the WHO has set standards: • Presenting acuity >= 6/18 in 80% of post-operative patients

4. Barriers to assessment of cataract outcomes • The proportion of patients returning after surgery is often very small in many parts of the developing world. • It is un-known whether vision outcomes among patients who do present for follow-up spontaneously are representative of all persons undergoing operations.

5. A new approach to outcomes assessment? • Wide adoption of small-incision, sutureless surgery mean more rapid recovery of vision post-operatively • Many surgical facilities, especially in rural areas, admit patients for 1-3 days after surgery • Can the principal assessment of post-operative vision be carried out at time of hospital discharge?

6. Advantages of early outcomes assessment • Collect data on all patients readily • Avoid bias in data collection • Reduce costs to patients and hospitals for follow-up

7. PRECOG: Prospective Review of Early Cataract Outcomes and Grading • Objectives: • Early assessment: • Assess validity of visual acuity measured at hospital discharge after cataract surgery as a predictors of medium-term (>= 50 days) vision (“Study hypothesis”) • Better use of existing data • Assess extent to which vision of persons spontaneously returning for follow-up care >= 50 days after cataract surgery are predictive of VA for entire operated cohort (“Traditional approach”)

8. PRECOG: Setting • Urban and rural facilities providing cataract surgery (n = 41): • East Asia: • China (18) • Vietnam (4) • Indonesia (2) • India: • All Aravind centers (5) • Latin America: • Peru (2), Ecuador (1), Paraguay (1), Guatemala (1), Mexico (2) • Africa: • Eritrea (2) • Ethiopia (3)

9. PRECOG: Participants and Sample Size • 50-100 consecutive persons aged > 30 years and under-going surgery for age-related cataract at each participating facility • Exclusion criteria: • Traumatic cataract • Ocular co-morbidities including glaucoma, retinal disease, corneal abnormalities or uveitis.

10. PRECOG: Follow-up • Target of > 90% follow-up at >= 50 days post op, either through: • Spontaneous return to clinic • Return to clinic potentiated by special intervention (phone call, offer of free transport, etc.) • Home visit • Type of follow-up recorded, so that patients returning spontaneously, under usual conditions (WITHOUT phone call, home visit etc.) can be studied

11. PRECOG Results: Participants • Hospitals (n = 41) • Annual surgical output: Range from < 500 (several) to 91,759 (Aravind Madurai) • Public: 31/41 (75.6%) • Rural: 24/44 (58.5%) • Cases • A total of 3547, of which: • 2246 (63%) SICS • 776 (22%) phaco • Remainder ECCE (15%)

12. PRECOG Results: Surgery • Pre-op VA <= 6/60 in operated eye: 84.6% • Final (>= 50 days) uncorrected VA • >= 6/18: 2089 (63.7%) • <= 6/60: 338 (10.3%) • Complications: • Intra-op: 7.79% • Post-op: 1.99%

13. PRECOG Results: Follow-up • The proportion of subjects with follow-up vision measured at >= 50 days after surgery was 3178/3547 (92.5%) • By region, follow-up was: • China 89.8% • India 93.6% • Vietnam/Indonesia 90.1% • Latin America 98.3% • Africa 95.6% • Spontaneous follow-up at clinic: 43% (Range from China 26% to Latin America 80%)

14. Correlation of early vision with final vision • What we want to know: How do hospitals rank according to final VA outcome? (proportion with VA >= 6/18) • We can compare two strategies to estimate this: • Using discharge vision to rank hospitals (the goal of PRECOG) • Using the final vision among those patients who do return spontaneously (what we have traditionally done)

15. The method we are testing in PRECOG The method we have traditionally used Discharge VA for all patients Final VA for 40% of patients who DO return spontaneously to clinic Final VA for ALL patients What we are trying to estimate

16. Correlation of early vision with final vision • Discharge vision and final vision are highly correlated for all patients: Spearman r = 0.59 • Hospital rankings using uncorrected discharge vision appear better-correlated with rankings using final vision than are rankings using the 43% of patients who return spontaneously: • Spearman r = 0.50 for discharge vision • Spearman r = 0.28 for patients who return spontaneously

17. Can we do even better? • Using best-corrected vision does not improve the performance of discharge VA in predicting hospital rankings based on final VA (r = 0.45) • Dropping patients (15%) with ECCE has little impact on performance of discharge VA (r = 0.56)

18. Can we do even better? • When we measure discharge vision as an index of outcome, there are inevitably some patients with temporary poor VA due to corneal edema or other problems • What if we could improve performance of poor vision by dropping these patients? • When we drop the 20% of patients at each hospital with the worst vision, discharge vision is better-correlated with final VA: r = 0.67

19. Concrete example using PRECOG data* • As a program planner in MOH or NGO, you want to separate hospitals into three categories: • Good (Top 25%: Can provide training to others) • Medium (Middle 50%: No intervention needed) • Problem (Bottom 25%: Further training needed) • How well does early vision assessment work for this? *Omitting data from 3 hospitals in Ethiopia for whom data not yet cleaned

20. Concrete example: Uncorrected VA, drop worst 20% by vision • 26/38 hospitals (68%) have the same ranking using discharge VA that they would have had using final VA • No hospitals went from Good to Poor or Poor to Good

21. Concrete example • 68% (26/38) of hospitals had the identical ranking based on discharge and final vision • If the vision of patients returning spontaneously was used to rank hospitals, only 18/38 (47%) had the same ranking • Based on chance alone, two such ranking systems would be expected to agree on 13/38 (34%) of hospitals

22. PRECOG Results: Standards for Early Vision Assessment • If discharge vision will be used as an index for surgical quality, the current WHO standard of 80% of patients with uncorrected VA >= 6/18 will likely need to change • In PRECOG, hospitals achieved the following standards for the % of patients with uncorrected discharge VA >= 6/18: • 90th percentile: 71.8% • 75th percentile: 60.6% • 50th percentile: 45.3% • 25th percentile: 31.1%

23. CAVEATS • Though hospitals in PRECOG included rural and urban, government and private facilities from many regions: • They were not chosen at random • We don’t know if they are truly representative of all facilities • Patients were chosen at random (consecutive surgeries), and follow-up was very good, but not 100% • Room for bias

24. PRECOG: Summary • If hospitals can measure discharge vision on 50-100 consecutive patients, they can provide a robust index of cataract surgical outcome usable by themselves and program planners • No need to be able to refract (using BCVA does not improve accuracy of data) • Works for hospitals performing ECCE as well as small incision cases • Even small, rural hospitals throughout the world have now proven their ability to collect these data

25. PRECOG: Next steps • Analyze other data we collected to further guide optimal follow-up: • Prevalence of refractive error and other conditions requiring treatment: how useful is follow-up? • Cost to patients and hospitals: how cost-effective is follow-up? • Look at simple adjustments to improve accuracy of discharge vision even further • Work with WHO, IAPB, NGOs and governments to disseminate and begin using these results to evaluate surgical quality in practice