1 / 25

Using Program Generation to Support V&V in the Next Generation Air Transportation System

Using Program Generation to Support V&V in the Next Generation Air Transportation System. Ewen Denney (joint work with Xavier Bouyssounouse Johann Schumann Karen Tung-Cate) August, 2007. NGATS. US air transportation system (airports and airspace) rapidly reaching capacity

isaiah
Download Presentation

Using Program Generation to Support V&V in the Next Generation Air Transportation System

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Using Program Generation to Support V&V in the Next Generation Air Transportation System Ewen Denney (joint work with Xavier Bouyssounouse Johann Schumann Karen Tung-Cate) August, 2007

  2. NGATS • US air transportation system (airports and airspace) rapidly reaching capacity • Major effort to develop integrated solutions for a safe, efficient, and high-capacity airspace system • Dynamic Airspace Configuration • Traffic Flow Management • Separation Assurance • Safety analysis

  3. Trajectory Prediction Trajectory Prediction predicts the future progress of individual aircraft on the basis of • the current aircraft state, • estimates of pilot and controller intent, • expected environmental conditions, and • models of aircraft performance and procedures • What is a trajectory? • Actual flight path (“ground truth”) seldom known • Tracking data: radar, altitude, weather • Intent: flight-plan (pilot/ATC), way points, jet-routes • Route constraints

  4. TP Client Applications • Flight Plan Distribution and Sector Loading: prediction of the Center Sequence (i.e., Air Traffic Control facilities to be transited) • Flight Data Processing Systems: estimation of hand-over conditions, collision avoidance • Basic Decision Support Tools: monitor flight plan and tracking data to advise controllers of suggested metering/sequencing and potential conflicts • Advanced Decision Support Tools: directly suggest solutions (e.g., instructions for conflict resolution, accurate prediction of cruise-descent and climb-cruise profiles in presence of tactical maneuvers)

  5. Sources of Uncertainty • Data errors: incomplete, inaccurate, corrupted or untimely data (e.g., flight plan and radar track data inputs, airspace/navigation database) • Modeling errors: simplifications, omissions and uncertainty in the mathematical models used in the system (e.g., aircraft performance modeling database) • Intent errors: unpredictability of pilot and controller intent

  6. Tracking Database • Trajectory info in NAS • Track reading (radar position and barometric altitude): noisy measurement of actual AC movement • Flight plan: “intended path” of the AC • Initial flight plan • Amendments to flight plan (often not in system) CM_SIM database assembles aircraft trajectory data from multiple noisy sources: raw, cleaned, and derived data, weather, pilot intent, etc.

  7. Tracking Data • AC_DATA: radar tracking data • Noisy radar readings (every 12 secs) • ADD_FLIGHT_PLAN: flight plan data • AMEND_FLIGHT_INFO: additional intent data • weather and waypoint data are separate AMEND_FLIGHT_INFO 15 FDX1169/AFW.0015 000000002000 -NS- F -NS- AFW/1043 GRABE/1049 OKMX/1049 GRABE012038/1052 TIKYS260015/1055 EAKER/1057 ATOKA313002/1059 MLC304033/1104 LOBOE085022/1104 LOBOE058028/1106 69194/1106 69195/1107 TUL200033/1107 69184/1109 TUL/1111 ADD_FLIGHT_PLAN 15 AAL8/PHNL.0016 ZFW AAL8 0016 -NS- -NS- 1114 H/B763/QHNL./.GATES247077..3254N/9702W..KDFW TXO221069 1100 390 455 -NS- ESTIMATED_FP ATC_ARRIVAL AC_DATA 52 BKA302/ATL.0002 806.6335 540.9157 333850.00N 0913712.00W 28100 0.00 400.65 0.00 259.26 0.00 41.29 1 -NS- 28 N N F F ZFW 0.0000 403.0000 259.2621 AC_DATA 52 AAL8/PHNL.0016 217.1337 539.1776 333634.00N 1032302.00W 39000 0.00 450.52 0.00 97.61 0.00 52.69 2 93 93 N N F F ZFW 0.0000 510.0000 97.6074 ASSIGN_CONFIG 15 AAL8/PHNL.0016 CM KDFW SOUTH_4_VFR

  8. Trajectory Synthesis • Mature discipline with complex algorithms to • integrate equations of motion, aircraft models • analyze weather • interpret pilot intent • Can represent trajectories using Generalized Altitude Profile • Domain-specific language for trajectories: GenAlt • Input request constrains possible trajectories • TS tool outputs parametric trajectory and point sequence of states

  9. V&V Challenge • TS tool highly safety-critical but difficult to test • Complex algorithms • Lots of domain knowledge • Many different “kinds” of trajectories • Many different “kinds” of requests • Huge database of raw data • US national airspace tracking data • Weather data • Aircraft manufacturer models • Field tests • Need flexible data selection and analysis mechanism • Want to write analysis queries at high-level • Goal: • develop DSL for trajectory analysis • use program synthesis to generate analysis code

  10. V&V Goals • Validate TS algorithm • Test with representatives and edge-cases of important trajectory classes • Populate trajectory database (based on tracking data, etc.) • Contractor building database infrastructure • Use for testing/assessment of trajectory prediction algorithms • Determine faulty assumptions in models • Compare trajectory predictors • Initial analysis techniques • Clustering wrt. • Point data • Trajectory data • Change-point detection • Decompose trajectories into segments • Can use as features for clustering

  11. Clustering • Given: data-points that originate from 3 different sources/processes • Data points are: • iid • Gaussian distributed Problem: you don’t see the colors

  12. Clustering • Clustering task: Estimate the parameters µ, such that P(x| µ,) is maximal Clustering also produces most likely class-assignment (“coloring”) of data points and class frequency . Usually, an iterative machine-learning algorithm (EM, k-means) is used

  13. AutoBayes • Customizable Bayesian data analysis code generator • Synthesizes custom algorithms • C/C++, Matlab, … • Different model domains: MLE, MAP, mixture models, change-point detections, etc. • Different PDFs • Different algorithms (EM, k-means, kd-trees) • Analysis or sampling (Monte Carlo simulation) • Can be extended with new algorithms • Input is statistical model of analysis problem

  14. Example Data Model • Specify: • num. classes • num. datapoints • distribution of data • goal function

  15. Trajectory Features • Each trajectory (i.e., sequence of AC locations over time) can be characterized by a set of scalar features: • % in level flight, climbing, descent • Endpoints • Min distance to other AC • Min/max/mean v/speed, g/speed, g/accel, Mach/CAS/TAS • Diff raw and cleaned data • Turning maneuvers: • Total, max, sum • Altitude and heading • Could smooth wrt. turbulence

  16. Clustering Analyses • Simple clustering • Cluster into disjoint sets of ACs: each represents set of AC trajectories • Cluster into disjoint time segments: each represents all AC info during that segment • Cluster into disjoint sets of ACs at specific time point • Cluster on • Tracking data • GenAlt requests • Synthesized trajectories

  17. Example Analysis • Approx. 300x300nm around DFW • Approx. 1 hour of data • 215 AC trajectories • Cluster wrt. several features • Blue and cyan are landing and approach to DFW

  18. Example: Endpoints • DFW forms one cluster (cyan) • Other points not obviously clustered • through flights • cut-off by time

  19. Example: Close AC-distance • AC in descent to/departure from DFW: small distance to airport (?) (d~1.2nm) • En route flights which exhibit small minimal distances to other AC (d~0.97nm)

  20. Change-point Detection • Data analysis technique for time series • Detect (rapid) changes in • Intensity • Noise • etc.

  21. Mach/CAS Transition • Mach number • Velocity relative to sound (M = v/a) • Depends on altitude • Calibrated airspeed (CAS) • Indicated airspeed, corrected for errors • Mach/CAS transition point • During ascent: constant CAS then constant Mach

  22. Climb/Descend Profiles mach • Accurate AC models for climb profiles are necessary for accurate trajectory prediction • Mach/CAS profiles can be extracted from radar trajectory data • Convert ground speed to Mach/CAS profile • Cluster according to Mach/CAS values for different aircraft types and maneuvers • Change-point detection in climb/descend maneuvers cas alt Generate code to find transition points: Mach/CAS, climb/cruise, …

  23. Finding Transition Points const nat N, t_0; const double sigma_sq double a,b,c. data double mach(0..N-1). mach(t) ~ N( if(T < t_0)? a+b*t : c, sigma_sq). max pr(mach|{a,b,c}) for {a,b,c}. • Pre-process to find “climbs” • Find where transition from slope to constant occurs • Noisy data • Should also find best Mach/CAS transition a = 0.69428 b = 0.0072091 c = 0.44532 t_0 = 35 Red: actual trajectory Blue: estimated profile

  24. Next Steps • Characterize important trajectory features • Generate test cases for TS from clusters • Combine extracted transitions and parameters with other features • Covariance analysis wrt. AC types • Extension of statistical model by domain knowledge (“priors”) • e.g., airport locations • Incorporation of other data • flight plans • weather data (e.g., for Mach/CAS) • Relating physical trajectories (tracking data), synthesized trajectories, and TS requests • Recursive clustering algorithms (large data sets)

  25. Longer Term • Domain-specific language to describe • analysis types • trajectory features • data filters • weather conditions • changes of flight plans • specific airports, flight types, flight segments • Integrate generator with • trajectory database • TS tool

More Related