ASPRS 2006 Annual Conference Reno, Nevada

1. Overview of the USGS Plan for Quality Assurance of Digital Aerial Imagery ASPRS 2006 Annual Conference Reno, Nevada

2. Outline • Background • ASPRS Study recommendations • Who, what, why, how • USGS Plan for Quality Assurance of Digital Aerial Imagery • When • Summary

3. Background Why calibration at USGS? • Photogrammetric methods for map generation • Map production primarily done in-house • Quality assurance measure for aerial photography from aerial contractors • Leadership role in standards development • Unbiased, independent agency with technical expertise • Quality assurance for The National Map

4. USGS Camera Calibration History • USGS responsible for calibration services for film camera in United States since 1973 • USGS operates Optical Sciences Lab (OSL) in Reston, VA with a custom-built calibration instrument • Current policy requires current (within 3 years) camera calibration report on file before award of contract • The “Catch-22” problem for digital sensors

5. ASPRS Study Recommendations • Continue to provide analog calibration services • Develop and implement digital calibration capabilities • Develop standards for camera and sensor calibrations • Develop and implement in-situ calibration methods

6. Stages of New Technology • Stage 1: Initiation • Stage 2: Contagion • Stage 3: Control • Stage 4: Integration

7. Example: DOQs • Initiation: Research and pilot projects • Contagion: Wild enthusiasm and demand • Control: Define standards and format • Integration: DOQs critical to enterprise operation

8. The Barriers to New Technology • The unknown: Does it perform as claimed? • Lack of standards: What are the specifications? • Lack of experience: What I can or can not do? • Business decision: Does it make business sense? • Operational changes: What are the best practices? • Integration: How to integrate into existing processes? • Overcoming inertia: Acceptance by users?

9. The Formula for Quality • An independent validation of manufacturer’s specifications • Selection of well-qualified Data Providers with experience and established “best practices” • Standardized procurement specifications and selection criteria • Quality control measures

10. What is the USGS Strategy for Quality? • Development of a comprehensive quality plan • Acceptance and use by Inter-Agency Digital Image Working Group composed of 14 Federal agencies • Extension of requirement to other agencies and contracting offices • Acceptance by general user community

11. What’s Your Perspective? The major players: Manufacturers Data Providers Procurement officials End-users

12. The USGS Perspective • Image products and services for other agencies and cooperators via contracts with Data Providers • Occasional procurement of sensor systems • Quality assurance of end products for The National Map • Development and establishing standards for geospatial community

13. USGS Plan for Quality Assurance • Four major parts covering two major processes: • Data Production • Manufacturers Certification • Data Providers Certification • Data Purchasing & Acceptance • Contracting Guidelines • Data Acceptance Standards

14. The USGS Plan Data Purchasing and Acceptance Domain Data Procurement: Contracting Guidelines & Boilerplate Tool Data Users and Inspectors: Acceptance Standards Final Product User Needs Sensor Manufacturers: Manufacturers Certification Data Providers: Data Providers Certification Data Production Domain

15. Manufacturers Certification • Aerial Digital Imaging is in its “Wild West” phase • Anything & everything being tried • Some metric-quality systems • Many “other” systems • How does the customer know which can produce mapping-quality data? • USGS to offer “type certification” of mapping-quality digital aerial sensors • Must be stable, well-quantified, repeatable • Able to routinely generate mapping-quality data • When operated properly!

16. Benefits of Manufacturers Certification • Communicates specifications • Provides evidence of system performance • Independent certification helps to promote sensor systems • Supports verification of Data Provider’s system • Type certification eliminates burden of calibration for each sensor sold in the United States (1 time vs. n times) • Eliminates need for USGS to have custom-built calibration instrument for calibration purposes

17. Review Team Lead - Gregory L. Stensaas Remote Sensing Technologies Project Manager USGS Earth Resources Observation and Science Center, Sioux Falls, SD stensaas@usgs.gov Systems Engineering Team Member - Jon Christopherson Science Applications International Corporation (SAIC) Contractor to the USGS EROS, Sioux Falls, SD jonchris@usgs.gov Photogrammetric Engineering Team Member - Dr. George Y. G. Lee U.S. Geological Survey, Menlo Park, CA gylee@usgs.gov Geo-Spatial and Software Engineering Team Member - Donald Moe Science Applications International Corporation (SAIC) Contractor to the USGS EROS, Sioux Falls, SD dmoe@usgs.gov Radiometric and Physics Team Member - Dr. Robert Ryan Science Systems and Applications, Inc. Contractor to NASA Stennis Space Center, MS Robert.Ryan@ssc.nasa.gov USGS Manufacturers Review Team

18. Data Providers Certification • Focused on processes and process control • Ensures that Data Providers are operating sensors in accordance with manufacturer’s instructions and limitations • Ensures that Data Providers follow quality procedures • Focused on documentation and monitoring of procedures and best practices • ISO-like certification process • Desire to ensure reliability, repeatability, and quality

19. Benefits of Data Providers Certification • Provides evidence of performance of products • Independent certification helps to promote product specifications and Data Provider’s capabilities • Documents Data Provider’s quality assurance plan and “best practices” • One certification for Data Provider and not for each camera • Data Providers no longer have to send cameras to OSL for calibration, reducing down-time and shipping expenses

20. Contracting Guidelines • User community is not sure how to contract for digital imagery • New terms, capabilities, standards, lexicon • Inhibits digital contracting • Addresses boilerplate requiring “USGS Certificate” • Goal is to remove barriers to digital aerial contracts • Encourage digital imaging • Created Federal Digital Imagery General Contract Guideline

21. Benefits of Contracting Guidelines • Standardized terms and descriptions make the contracting process easier and more uniform among agencies • Guidelines help acceptance of digital sensors and educate end-users on benefits of digital technology • Standardized terms and guidelines help contracting officers describe their users needs • Standardized performance measures • USGS certifications provide a priori acceptance of systems and Data Provider’s “best practices” • Manufacturer and Data Providers Certification reduces necessary documentation in the RFP process

22. Digital Data Acceptance Standards • End-users unsure of how to judge digital aerial data quality • New terms & capabilities (resolution, spectral, etc.) • Each customer understands things differently • There is a need for common, uniform definitions and methods for evaluating quality of image data • USGS to work with Inter-Agency Digital Image Working Group to develop these standards • Goal is a Web-based tool illustrating quality problems, measurement techniques, and standards

23. Benefits of Acceptance Standards • Data consumers have common standards to evaluate data products • More consistent acceptance/rejection criteria among contracting agencies • Clearer standards and guidelines helps to eliminate false expectations • Ensures high quality products • Increases customer satisfaction

24. Funding Strategy • Manufacturers Certification (Cost-shared by USGS and manufacturers) • Data Providers Certification (100% by Data Providers) • Contracting Guidelines (100% USGS and IADIWG funded) • Acceptance standards (100% USGS and IADIWG funded)

25. Status • Manufacturers Certification Guidelines drafted • Up to 4 manufacturers to be certified this fiscal year • Applanix and Intergraph factory visits completed, reports pending • A few details but no major issues; certification to be issued soon • Leica and Vexcel-Microsoft visits being discussed now • Four Data Providers have expressed interest working on the initial round of Data Providers Certification • First draft of Digital Imagery Contracting Guideline completed and reviewed by limited group • A Web-based tool to help generate contracting language is being developed • IADIWG to begin drafting guidelines in Fall

26. Next Steps • USGS Plan for Quality Assurance of Digital Aerial Imagery briefed during ASPRS panel session • Invite comment from broader community • Finalize and obtain ASPRS approval • Complete initial four Manufacturers Certifications • Begin initial Data Providers Certifications • Provide briefings to geospatial community • Participate in international forums to communicate plans used in the United States

27. Closing Thought Four Phases of Technology Assimilation: • Phase 1: Identification and investment • Phase 2: Learning and adaptation • Phase 3: Management control and standards • Phase 4: Maturity and widespread acceptance

28. Summary • To implement a comprehensive, meaningful process that ensures the quality of data products and services • To cooperatively develop the plan with all elements of the geospatial community • Good for one is good for all • Education and training for the community • To establish a model to support other new technologies in the future

29. For more information • IADIWG Web site at: http://calval.cr.usgs.gov/

30. For issues or comments Contact: • Gregory L. Stensaas Remote Sensing Systems Characterization Manager USGS EROS Data Center 47914 252nd Street Sioux Falls, SD 57198 605-594-2569 stensaas@usgs.gov

31. The USGS Quality Plan Questions or comments?