WAAS Test Program Overview & In-service Monitoring

1. WAAS Test Program Overview & In-service Monitoring Tom McHugh, ATO-P (ACB-430)DOT/FAA W.J.H. Technical CenterWAAS Technical Directortom.mchugh@faa.gov(609) 485-5183

2. Test Program Structure • WAAS Test Program • Contractor Testing • Design Qualification • Production Acceptance • Analysis & Simulation • FAA Testing • Operational Test & Evaluation • Shakedown Testing • Testing of Upgrades • In-service Monitoring • Performance Evaluation • WIPP Off-line Monitoring • Anomaly Detection & Resolution

3. Test Challenges • The WAAS test program was structured on historic FAA processes used for developing and fielding new systems • Some of those processes are not applicable outside of the FAA • Those processes required tailoring because they were not designed for SBASs • SBASs have unique features that require innovation in the test program • Integrity performance numbers that cannot be proven by test or demonstration • Performance is a function of the constellation and the ionosphere which are outside the control of the SBAS • SBASs are single systems with geographically distributed components • The navigation service in the immediate vicinity of the component is not directly tied to that component • Difficult (nearly impossible) to make corrections to the DO-229 • Access to GEOs • SBASs are new • The FAA was still learning how to build and test WAAS while we were doing it • How SBASs work is generally not well understood by all stakeholders

4. System Test and Monitoring Facilities • WAAS reference stations (WRS) IP multicast their data onto the WAAS network • Real time one-way access to that data is provided to: • The WAAS prime contractor's facility in Fullerton CA • The FAA Technical Center in Atlantic City NJ • FAATC also forwards the measurement data to NOAA/NGS for incorporation into the CORS network • The Mike Monroney Aeronautical Center in OKC • Access to the real time measurement data allows WAAS master station equipment in the prime contractor's facility to function as if it were operational • WAAS Shadow System • Extensively used for system level testing • Currently WAAS has 5 shadow systems in the Fullerton facility • Independent measurement data from the FAA National Satellite Test Bed is also fed from FAATC to the Raytheon facility in Fullerton CA

5. Contractor Testing • Contractor testing • Hardware • Subsystem • Factory acceptance • Site acceptance • COTS • OEM vendor data and testing • Software • FAA required processes (FAA-STD-026) • RTCA DO-178B • System • Algorithm validation and tuning • Comparison runs • Build verification testing • System integration and stability testing • Design qualification tests • 60 Day Stability & Performance Test • Regression tests • Service volume modeling • Safety analyses • Reliability analyses

6. Test Capabilities / Tools • All measurement data is always recorded • Simulations can be generated from that recorded data • Modified or un-modified • Test Support Software (TSS) used for unit and configuration item level software testing • Sim C&V • Off-line version of WAAS corrections and verification subsystem master station software • Much quicker than real time • Created from the operational software code using compile time switches • Can repeatedly test WAAS performance on any recorded data set • Algorithm implementation validation • Anomaly resolution • Regression testing • Traceability maintained in a DORS database

7. Service Volume Modeling (SVM) Tools • MITRE SVM • Stanford University SVM • Raytheon SVM • Volpe Service Prediction NOTAMS Tool • FAATC Coverage and Airport Availability Tools • Raytheon Service Monitoring System (SMS)

8. Upgrade Testing • Scope of testing is dependent on the nature of the upgrade • Standard testing • DO-178B objective satisfaction • Requirements change verification • Shadow system stability testing • Build validation testing • Cutover testing • Unique testing • Algorithm validation • HMI analysis • Comparison run tests • Integration and checkout of new hardware

9. FAA Testing(in support of IOC) • Operational Test and Evaluation • Flight testing • 8 locations spread over the LPV service area • 40 approaches per location • Static testing • Simulate aircraft at each of the WAAS and NSTB reference stations by applying the WAAS corrections to the receiver measurements • 24/7 1 Hz for the entire 60 day test • FAATC has been monitoring WAAS performance 24/7 since November 1998 • Degraded operations • simulated major system failures • Shakedown Testing • Demonstrates that the operators and maintainers were sufficiently proficient to operate the system

10. FAA In-service Monitoring(continuous and upgrades) • Three types of in-service monitoring • WAAS PAN Report (Quarterly) • WAAS user service performance • Availability and coverage • Accuracy • Outages • Anomaly reporting • GPS PAN Report (Quarterly) • Civilian GPS performance • WIPP off-line monitoring • Broadcast ephemeris accuracy statistics • Antenna multipath environment • Signal Quality Monitoring (SQM) • Periodic antenna position survey updates • GPS failure rate statistics