SWIM-SUIT Final User Forum

1. SWIM-SUITFinal User Forum

2. Learning How to SWIM:Validation Results Gaetano VITO / Emanuele DI PASCALE SICTA WP4 SWIM-SUIT Final User Forum, Rome

3. WP4 – SWIM-SUIT Validation • The validation is based on the user requirements defined in WP1, which are expected to cover such topics as: • Information content, timeliness, and accuracy; • Information update rates; • Safety aspects related to criticality of data shared/managed; • Security required for data access and management. • The validation is based on the prototypes developed in WP2, which have been integrated in the overall SWIM-SUIT system within the WP3 context. • WP4 produces the Validation Report and Recommendations for Further Activities SWIM-SUIT Final User Forum, Rome

4. Validation Process • Technical Validation • No legacy systems involved • Core Functionality Validation • Performance Measurement • Pseudo Operative Validation • Legacy systems involved • Real or plausible simulated data (i.e. using Data Filler) • Measurement of operative impact on typical use cases Input WP2 Test Plan Technical Validation Local Integration WP3 Overall Integration Pseudo Operative Validation SWIM-SUIT Final User Forum, Rome

5. WP4.4 Process of Collaboration • The following process was adopted to produce the Validation Report, based on the inputs coming from previous sub-work packages (WP4.2 & WP4.3): Validation Feedback Collection WP4.3 (Raw Data) Qualitative Analysis WP4.2 (KPA-HLO) WP4.3 (Raw Data) Measurements/Criteria Comparison and Result Presentation Data Extraction for Results Analysis Measurements Calculation Quantitative Analysis keyword specification Producing the Validation Report Analysis Criteria Definition WP1 and WP4 (WP4.2) WP2 and WP3 SWIM-SUIT Final User Forum, Rome

6. Validation Goals & KPAs In regards with the Goals and KPAs defined in the Validation Plan, their status of verification is reported in the following table: SWIM-SUIT Final User Forum, Rome

7. Technical Validation Objectives • PROTOTYPE FEASIBILITY: • to demonstrate the possibility to implement the SWIM concept in its core aspects, focusing on the several operations that can be executed through the SWIM infrastructure • to have a technological assessment after the development of the prototype and its integration in the current ATM scenario, using simulators, actual systems and adapters for the communication • PERFORMANCE BENCHMARKING: • due to the lack of a proper reference framework, performance exercises have been designed to provide indicative measures that could represent a baseline for future implementations of the SWIM concept and allow the stakeholders to assess SWIM capability to respond to the interoperability expectations. • load scenarios, stress and endurance tests have been executed, varying the workload, acting on the number and type of operations, rate of operation invocations, etc. in order to provide some benchmark for the performance and the information exchange, without directly involving the network (which, at this prototypal stage, does not have specific requirements). SWIM-SUIT Final User Forum, Rome

8. Technical Validation Objectives • SCALABILITY: • To demonstrate the flexibility of the SWIM infrastructure, fitting to new needs (for example, in case of new actors interested in the subscription of the same data or information). Scalability in terms of operation number and rate is covered by performance tests; specific scalability tests with an increasing number of participants have also been designed and executed. • SECURITY: • The prototype includes mechanisms to enforce security policies at the service and message level including Authentication, Authorization and Access Control for changes of Domain Objects and also the way Managers may hand over responsibilities to other Managers. • The current implementation of the prototype has static security mechanisms; each swim-box has certificates identifying and authenticating every other possible stakeholders. Security developments are still on-going to reach a federated security mechanism in which certificates are exchanged at run-time. These new developments haven’t been deployed yet and concern only SWIM, not its prototype. 24-25 June 2010 SWIM-SUIT Final User Forum, Rome

9. Technical Validation Results • From an high level point of view, it is possible to state: • Feasibilityis proved for all steps and in all situation individuated in VP, and for both technologies (DDS and JMS) which makes the system flexible and technology independent. System integrity at a good level is confirmed. Availability: too early to be evaluated for a prototype. • Security has been demonstrated as feasible; its application introduces a significant latency, which however is not prohibitive for SWIM BOXes operations. Data security in all transactions is ensured and secure access to the network is provided. • In order to examine the system in more detail, network layer measurements should also be taken in future developments to be able to prove system stability and interface independence SWIM-SUIT Final User Forum, Rome

10. Performance Benchmarking Example:CreateFO Load Test • Both technologies reacts in similar way, especially in case of no Security Mediator enabled (JMS and DDS). • These two technologies can be both used with equal performance. • However, in the case of JMS with SM enabled, measurements are not uniform. • Security Mechanisms (JMS_SM and DDS_SM) introduce a significant overhead, although performance is still acceptable 24-25 June 2010 SWIM-SUIT Final User Forum, Rome

11. Performance Benchmarking Example:CreateFO Stress Test • The overhead of the security mechanisms lowers the overall efficiency of the prototype of approximately 5 times. • Throughput without the Security Mediator improves with the number of operations involved, regardless of the imposed wait time. • Throughput with the Security Mediator enabled doesn’t improve because it’s limited by authentication overhead • JMS and DDS technologies behave mostly in the same way under the same load and waiting time, with and without security mediator. 24-25 June 2010 SWIM-SUIT Final User Forum, Rome

12. Performance Benchmarking Example:CreateFO Scalability • When simultaneous connections increases above 5, the throughput decreases till the maximum of 9 connections, where it seems to be stabilized. • Both technologies behaves in a uniform way, proving complete technology independence. • Scalability tests on a higher number of nodes (e.g. on a cluster of machines) and including Security Mediator are required to provide a better estimation of the prototype scalability. 24-25 June 2010 SWIM-SUIT Final User Forum, Rome

13. Current Systems Limitations • The assumption to include legacy systems “as they are” poses severe limits to the extent of what could be achieved and validated • E.g.: No HMI development to present the newly available data to a human controller • No chance to implement radical changes to system interactions, as in the case of ATM-4 Level Services • Necessity to simulate specific events to circumvent limitations imposed by missing capabilities (i.e. trajectory recalculation via a new constraint) • Due to these limitations, it was not possible to validate the safety aspects of the prototype (no human operator in the loop) • Future systems will presumably require a re-design to be fully compliant with SWIM requirements SWIM-SUIT Final User Forum, Rome

14. Pseudo Operative Validation: Interoperability • Interoperability in terms of Flight Data is achieved by adopting a common data meta-model (ICOG v2.0) • The model potentially enables interchange of data regarding every aspect of a flight and complex interactions among partners • Structured in modular clusters (e.g. Arrival, Departure, Trajectory…) • Open standards (based on xml and xsd schemas) • However, most legacy systems are only able to produce/consume a subset of the data clusters defined in the model • In order to enable proper interactions, it is currently necessary to include components like adapters / data filler to mediate the gap between current capabilities and desired functionalities • Moreover, it has been necessary to extend the model to cover specific data fields produced by legacy systems • E.g.: stand allocation (Aerodromes), operator flight priority (AOCs), Start up and Taxi data (Ground Control), Estimated landing time (Aircraft)… SWIM-SUIT Final User Forum, Rome

15. Pseudo Operative Validation: Interoperability • Surveillance data shared through an open format (Asterix cat. 62) and disseminated regardless of national boundaries • AOCs: Real-time awareness of the current fleet status across the globe • ATSUs: Extended time horizon to detect potential conflicts • Further considerations can be extrapolated from IOP studies between SWIM-SUIT and FAA’s SWIM testbed in the USA • ERAM / ICOG comparison (e.g.: lack of a manager/contributor role in ERAM) • Potential benefits of a worldwide standard to avoid data translation and enhance interoperability • These results will be part of D4.5.1 – Lessons Learnt and Recommendations SWIM-SUIT Final User Forum, Rome

16. Pseudo Operative Validation: Flexibility • New systems can be added to the SWIM Pool with minimal effort • No need to modify the system code or its interface • All of the required mediation services are implemented in the adapter, which acts as a bridge • Legacy systems currently interconnected through the prototype cover all roles of the ATM world: • ATC, APP, En-Route, Tower, GND, Aircraft FMS, AOCs, Aerodromes… SWIM-SUIT Final User Forum, Rome

17. Pseudo Operative Validation: Feasibility • It is possible to interconnect several operational systems, shadow mode applications and simulation platforms through a single common network, based on today’s technologies and without any modification to the code of the involved “legacy systems” • Core services and functionalities have been tested and their impact has been assessed in terms of interoperability, operative implications (to a limited extent), security etc. • In other words… YES, we can! SWIM-SUIT Final User Forum, Rome

18. Questions and Feedback http://www.swim-suit.aero/swimsuit/ SWIM-SUIT Final User Forum, Rome

19. SWIM-SUITFinal User Forum