Case Completeness and Data Accuracy in the National Program of Cancer Registries

1. Case Completeness and Data Accuracy in the National Program of Cancer Registries KK Thoburn, CDC/NPCR Contractor RR German, M Lewis, P Nichols, F Ahmed, CDC J Jackson-Thompson, University of Missouri-Columbia 2007 Annual NAACCR Conference Detroit, Michigan Tuesday, June 5, 2007

2. NPCR Cancer Surveillance System (NPCR-CSS) • Established in 2000 • In collaboration with SEER, collects cancer incidence data for the entire US population • NPCR-CSS data submissions are evaluated for case completeness (CC), percentage of death-certificate-only (DCO) cases, percentage of unknown/missing values for selected required data elements, and percentage of cases passing critical edits for data quality • Measures incidence rates by various characteristics • Data estimates CCR progress in meeting NPCR Program Standards

3. NPCR Technical Assistance and Audit Program (NPCR-TAA) • ORC Macro conducts NPCR-TAA • Performs case completeness (CC) and data accuracy (DA) audits at the level of the hospital reporting to the CCR • Uses sample of cases from selected facilities for reabstraction • Lung and bronchus, colon and rectum, prostate, and female breast • These primary sites represent over 50% of cases reported to the NPCR-CSS • Provides technical assistance to CCRs

4. NPCR Annual Program Evaluation Instrument (NPCR-APEI) • Web-based instrument completed annually by all NPCR-funded CCRs • Assessment of NPCR program objectives, registry operations, and data use

5. Present Study • Usefulness of NPCR-CSS data depends on completeness of case reporting and accuracy of submitted information • NPCR-CSS data are frequently used to measure incidence and survival • Race, sex, age, subsite, and stage at diagnosis • Present national-level study: • Summary analysis of NPCR-TAA information by primary site • Covariate analysis of NPCR-TAA results with information from the NPCR-APEI, and CCR achievement of USCS publication and NAACCR certification standards

6. Methods • NPCR-TAA audit results for 34 of 45 NPCR-supported CCRs audited at the time of analysis were included • 1998 through 2001 audit diagnosis years were included • Overall average case completeness and data accuracy rates, and site-specific data accuracy rates were calculated • DA rates calculated by site for each of the 13 data audited data • Average CC and site-specific DA rates were examined across the covariates • Small-sample Student’s t tests (α = 0.05)

7. Methods CC rates (%) 100 - (# of missed cases / total # of cases identified) x 100 DA rates, overall and site-specific (%) = (# of data elements with no discrepancies / total # of data elements reabstracted) x 100 DA rates, audited data elements (%) = (# of reabstracted cases with no discrepancies on data element / total # of cases reabstracted) x 100

8. NPCR-TAA: CC and DA rates Primary site Audited data elements Source of missed cases USCS publication: Met publication standards? NAACCR Web site: Certified by NAACCR? Certification level NPCR-APEI: Type of current funding? Ratio of FTE positions to central registry caseload? Ratio of CTRs to central registry caseload? Supplementary reporting sources reporting cases? Case-finding audits at reporting facilities? Reabstracting audits at reporting facilities? Annual report issued? Methods: Sources of Variables

9. Results: Case Completeness • 34 CCRs, diagnosis years 1998 to 2001 • 41,521 lung and bronchus, colon and rectum, prostate, and female breast were identified • 1,503 cases were missed • Overall case completeness rate of 96.4%

10. Distribution of Missed Cases (N = 1, 503) Lung and bronchus 19.9% (n = 299 ) Female breast 32.7% (n = 491) Colon and rectum 18.8% (n = 283) Prostate 28.6% (n = 430)

11. Distribution of Missed Cases by Type of Case-finding Source Lung and bronchus Colon and rectum Prostate Female breast n = 324* n = 315* n = 458* n = 544* 0.3% 0 . 7 % 0.6% 3 . 7 % 2.2% 0.6% 0.3% 1.3% 0.2% 0.2% 4 .6 % 3.1% 29.0% 40.6% 45.8% 43.2% 53.9% 52.0% 55.6% 62.0% Medical records disease index Radiation therapy log Autopsy record Pathology report Cytology report Other *Counts include missed cases from more than one case-finding source.

12. Site-Specific Data Accuracy 99.7 99.8 Gender 100.0 99.9 Lung and bronchus (n = 2,448) 99.8 99.9 State of residence 99.7 Colon and rectum (n = 2,063) 99.5 99.5 Prostate (n = 2,289) 99.2 Date of birth 99.4 99.3 Female breast (n = 3,210) 99.8 98.7 Behavior 99.9 98.9 99.4 97.4 Primary site 100.0 99.9 95.4 99.4 Laterality 99.4 99.1 98.4 Race 97.8 97.5 98.0 96.9 Sequence number 97.1 98.2 97.5 95.5 Diagnosis date 98.5 95.3 96.2 89.1 88.3 Histology 97.6 89.7 89.1 Subsite 93.6 100.0 81.2 89.7 Grade 93.9 89.9 85.9 83.5 84.3 SEER Summary Stage 92.0 93.5 0 10 20 30 40 50 60 70 80 90 100 Percentage of records containing no discrepancies

13. Covariate Analysis Ratio of FTEs and CTRs to CCR Caseload • Higher CC and DA rates were found for the more well-staffed registries (not significant; P > .05). • Higher DA rates were found for registries staffed with a greater number of CTRs (significant for colon and rectum, prostate) • Higher CC rate was found for registries staffed with a greater number of CTRs (not significant; P >.05).

14. Covariate Analysis Supplementary Reporting Sources • Higher CC rates were found for registries with pathology laboratories and/or radiation therapy centers facilities reporting (P < .01). • Higher site-specific DA rates were found for registries with pathology laboratories and/or radiation therapy centers facilities reporting (not significant; P >.05).

15. Covariate Analysis Met USCS Publication Standards/Achieved NAACCR Certification • Higher CC and average site-specific DA rates were found for CCRs that achieved these 2 milestones (only colon and rectum significant; P = .02, USCS; P = .04, NAACCR certification) • NAACCR-certified CCRs generally had higher data element-specific DA rates than non-certified CCRs

16. ConclusionsSummary Analysis of NPCR-TAA Data • Underscores importance and effectiveness of conducting CC and DA audits at reporting hospitals • Enables identification of general and site-specific case-finding and abstracting issues • Demonstrates overall high accuracy and completeness of NPCR-CSS incidence data on cancers of the lung and bronchus, colon and rectum, prostate, and female breast • Overall CC (96%) higher than NPCR-CSS 24-month standard (95%) • Overall DA rate (95%), and high DA rates found by demographic data elements and primary site lend confidence in incidence rates • Provides guidance to users of the data

17. ConclusionsCovariate Analysis • Shows that combining data from the NPCR-TAA and the NPCR-APEI provides additional valuable information that neither program can provide individually • Offers perspective on how CCR operations effect case completeness and data quality • Underscores importance of CCRs having well-trained staff; having supplemental reporting sources; and attaining compliance with national data standards

18. Conclusions • NPCR funding and technical assistance help CCRs in developing and enhancing effective registry operations—especially in areas such as staffing, training, and monitoring and in improving the completeness and quality of registry • Present study demonstrates: • Positive outcome of enhanced registry operations • Benefit of NPCR support for a high-quality, statewide, population-based CCR

19. Arizona Arkansas Colorado District of Columbia Florida Georgia Idaho Illinois Kansas Maine Maryland Massachusetts Michigan Minnesota Missouri Montana Nebraska Nevada New Hampshire New York North Carolina North Dakota Ohio Oklahoma Oregon Pennsylvania Rhode Island Tennessee Texas Vermont Virginia Washington West Virginia Wisconsin Special Appreciation