Measuring Accuracy of Street Centerline Datasets

1. Measuring Accuracy of Street Centerline Datasets Donald Cooke Founder, GDT CLEM2001 August 6-7, 2001

2. Accuracy of Centerline Files • History: NMAS • History: GDT involvement • NSSDA, July 1998 • GDT procedure • Results • 3 meter accuracy

3. History, NMAS • NMAS: National Map Accuracy Standard • Promulgated in 1940’s • Production Standard: “Build to this Spec.” • Binary (pass-fail) standard • Tied to scale/paper/”Gutenberg Disease” • Did they ever test the quads? How?

4. History, GDT Involvement • Geocoding accuracy

6. Tampa, Florida, April 1997 Red: DGPS positions of addresses Black: Interpolated geocoded addresses Most of error budget comes from storing potential address ranges…….

7. History, GDT Involvement • Geocoding accuracy • GDT inherited spatial accuracy of TIGER • Need to realign streets and improve accuracy • Need to qualify sources for realignment • Need to Q/C results of realignment operation

8. NSSDA • NSSDA: “National Standard for Spatial Data Accuracy” • July 1998; draft available well in advance

9. NSSDA Procedure • “accuracy testing by an independent source of higher accuracy is the preferred test” • Compile a test suite of coordinate measurements for well-defined points in the study area • Extract corresponding points from the dataset being evaluated • Statistically compare the two samples by computing RMSE and "accuracy" measures • ("accuracy" = RMSE * 1.7308)

10. GDT Procedure • Well-defined points: “T” or “Cross” intersections • Park in center of intersection; average for 1 minute • Post-processed code-phase differential • ~30, and later ~45 points collected per sample • Sample skewed to exclude major road intersections • Sample sometimes adjusted to fall within DOQQ or 7.5 minute quad tile.

11. Sample point spacing; Deerfield Illinois

12. Results • 56 areas surveyed; more in process • TIGER: 16-100+ meter RMSE • Dynamap aligned to DOQQ: 4-6 meter RMSE • See some results...

13. Test Area Number TIGER Dynamap of Points RMSE RMSE (meters) (meters) Sarasota, FL 39 26.5 4.3 Ann Arbor, MI 29 46.7 4.5 Deerfield, IL 29 27.0 4.2 Manchester, NH 36 25.5 4.6 Morristown, NJ 48 48.2 6.1 Greenfield, MA 70 21.3 6.0 Colma/Pacifica, CA 51 57.1 6.4 W Palm Beach, FL 38 44.2 3.0* Utility GIS Denver, CO 37 16.7 3.4 Dataset Warwick, RI 35 37.6 5.2 San Diego, CA 31 20.9 4.1

14. Absolute error for 29 check points, Ann Arbor, MI Error, Meters Check point #

15. Absolute error for 29 check points, Ann Arbor, MI Error, Meters Check point # RMSE: 46 meters RMSE of “good” pts: 20 meters; bad: 102 meters

18. Results • TIGER spatial errors are not normally distributed • Large population from 1:100K DLGs • Many from 1980 GBF/DIME files • Some streets “cartooned” freehand • Picking sample is crucial • Cannot make a blanket statistical statement that describes TIGER spatial error

19. Results, continued • 1:24,000 DLGs are spotty; South Carolina exp. • Some (25%?) 7.5 minute quads don’t pass NMAS • Impossible to test accuracy until recently • 1 meter DOQs appear to be boringly reliable

20. Conclusions on NMAS vs NSSDA • NSSDA is a workmanlike improvement • No longer a binary test • No longer tied to scale of analogue map • Procedure extensible to testing GPS accuracy • But still considers “map” to be a monolithic, one-time compilation to a single standard • Need to carry accuracy metadata on each object

21. Census TIGER/2010 Accuracy • 3-meter or better accuracy (=1.7 meter RMSE) • Cannot achieve this from off-the-shelf DOQs • Must use new imagery or drive DGPS • Code-phase differential GPS is marginal for test suite • We use carrier-phase for test points