OSPF Validation

1. OSPF Validation Dirk Jacob dirk@d-jacob.net http://ssfnet.d-jacob.net

2. Overview SSFNet Conformance Testing OSPF OSPF Test Suites Implementation for SSFNet Summary

3. Overview SSFNet Conformance Testing OSPF OSPF Test Suites Implementation for SSFNet Summary

4. SSFNet - Overview • Simulation of very large networks • Focus on scalability • modeling scalability • computing scalability • Layered Architecture

5. Scalable Simulation Framework • Simulator Kernel – not dedicated to simulation of networks • „Models“ are built on top of the SSF API • consists of only five core classes: Entity, Event, inChannel, outChannel and process • API hides all simulator internals • C++ and Java bindings • models are standard programs which extend the standard SSF classes • Different SSF implementations available

6. SSF Network Models • Set of SSF Java models, which simulate the networking world • distributed under the GPL • consists of several sub-packages • SSF.OS Framework for protocol modeling • SSF.Net models for hardware components • SSF.Util.Random generation of multiple independent random number streams • SSF.Util.Streams efficient multi-point monitoring infrastructure

7. SSF.OS • Framework for modeling of protocols • three classes: • ProtocolSession, • ProtocolMessage, • ProtocolGraph • Packets are modeled by ProtocolMessages

8. SSFNet Protocols • SSFNet comes with a variety of protocols • IP • UDP • TCP • Sockets • HTTP • BGP • „Static“ OSPF • OSPFv2

9. SSF.Net • Models for the simulation of networking components • Networks • Hosts • Routers • Links • Network Interfaces (NIC) • additional Classes like packet queues, routing tables, etc.

10. Modeling Topologies • Topologies are modeled using DML • consists of nested key/value pairs • supports attribute substitution • supports inheritance • component configurations can be kept in a dictionary for reuse • Example Net [ router[ id 1 _extends .dict.myrouter ] router[ id 2 _extends .dict.myrouter ] link[ attach 1(0) attach 2(0) ] ] dict [ myrouter [ interface [ id 0 _extends .dict.if100Mbit] graph [ ProtocolSession[name ip use SSF.OS.IP] ProtocolSession[name ospf use SSF.OS.OSPF.sOSPF] ] ] ]

11. Role of Testing • Protocols must be implemented correctly • Conformance tests to prove conformance with standards • Regression tests to verify that things still work after changes • Testing is extremely important!

12. Overview SSFNet Conformance Testing OSPF OSPF Test Suites Implementation for SSFNet Summary

13. Categories of Testing • Conformance testing • Regression testing • Performance testing • Stress testing • Roustness tests • Here: focus on conformance testing

14. Conformance Testing • Verify that a protocol implementation performs as required in the standard • Black box testing: observe outputs, when IUT is feeded with various inputs • Exhaustive testing: apply every possible input sequence • not possible with complex protocols • Test only important input sequences • Main problem: identify important input sequences

15. OSI Testing Methodology • Framework for conformance testing of OSI protocols • Standardized in ISO 9646 • based on the OSI Reference Model • Standardized test procedure

16. Test Architectures Local Method Distributed Method

17. Test Architectures (2) Coordinated Method Remote Method

18. TTCN • standardized notation for the description of test cases • event trees, which describe the external behavior of a protocol • two forms: TTCN/gr and TTCN/mp • four parts • overview part • declarations part • constraints part • dynamic part LT!SYN_Packet START wait_timer LT?ACK_Packet LT!FIN_Packet TIMEOUT wait_timer LT!FIN_acket Example for dynamic part

19. Test Case Selection • not specified in OSI framework • but: central role for coverage and correctness of test suite • efforts to use formal techniques • provable correctess • provable coverage

20. Test Selection based on FSMs • several methods for test case selection based on FSMs • transition tours • distinguishing sequences • characterizing sequences (W-Method) • unique I/O sequences • apply an input sequence which must produce a specific output sequence • protocol must be specified as an FSM • restrictions • strongly connected • fully specified • not applicable to most complex protocols

21. Formal Description Techniques • data portion of protocols often are not specified as FSM • FDTs are formal languages for specification • SDL • MSC • LOTOS • Estelle • specifications can be represented as labelled transition systems • similar to FSMs • different methods/algorithms for derivation of test cases from LTS

22. Informal Testing • many protocols are specified using natural language • translating them into a FDT may cause errors • test cases may not reflect the standard • many formal methods cannot be applied to complex protocols • often informal methods are used • reading the standard and deriving tests directly • no provable correctness and coverage • but: „high degree of certainty“

23. Overview SSFNet Conformance Testing OSPF OSPF Test Suites Implementation for SSFNet Summary

24. OSPF Overview • Link-state intra-domain routing protocol • OSPF Router knows the whole topology of the network • Specified in RFC 2328 • Routing Information is exchanged with directly attached neighors • flooding procedure assures distribution over the whole network • Hierarchical Concept • support for large networks • reduce CPU and memory requirements • Support for multiple least cost routes to the same destination • Uses Dijkstra‘s SPF algorithm to calculate routing tables

25. OSPF Overview (2) • Routers identify their neighbors • Some neighbors form adjacencies in order to exchange routing information • Synchronization through the exchange of link state databases • Detect link failures through periodic sending/receiving of Hello Packets • Changes in the topology are flooded as Link State Updates • All information is re-flooded periodically to keep databases consistent • OSPF distinguishes different network types:broadcast, nonboadcast-multiaccess (NBMA), point-to-point and point-to-multipoint

26. Hierarchical Routing • each AS can be divided into Areas • each area has ist own Link State Database • detailed information of area topology only known inside the area • information about area is summarized into other areas • special backbone area (area 0) • all areas must be connected to the backbone • virtual links to connect areas to the backbone which have no physical connection • Stub areas to reduce database sizes

27. Link State Database • Contains information about area topology • Describes a graph • Routers and networks are vertices • Links between vertices are edges • Links are asociated with a cost/metric • Network vertices are represented by a designated router

28. Functional Areas • 4 mostly independent functional areas • detection and maintenance of neighbors • building adjacencies • the flooding procedure • routing table calculation

29. Neighbor Discovery and Maintenance • Send Hello Packets periodically • Include information about neighbors • Assure bi-directional communication • Assure that neighbors agree on certain parameters • Elect Designated Router on broadcast and NBMA networks • Represent the network in the Link State Database • reduce the number of adjacencies

30. Building Adjacencies • neighbors with bi-directional communication become adjacent • dependent of network type • Synchronization of Link State Databases • Negotiate, who is the master in this process • Master controls the exchange process • Send summary of own Database in Database Description Packets • Request unknown or more recent LSAs from the neighbor • Send requested LSAs to the neighbor

31. Reliable Flooding • Each router must keep synchronized with every other router in the same area • Udates are flooded to all adjacent neighbors • Every Update must be acknowledged • Explicitly through an Ack packet • Implicitly with an Update of the same LSA flooded back • Retransmit Update if there is no such acknowledgement • Possibility of delayed Acks to minimize number of packets

32. Link State Advertisements • different types of LSAs • Router LSAs – represent a router with all its associated links • Network LSAs – represent broadcast and NBMA networks • Network Summary LSAs – represent routes to destinations in other areas • ASBR Summary LSAs – represent routes to AS boundary routers • AS-external LSAs – represent routes to AS external destinations • LSAs have an age associated with them • aging in databases and during the flooding procedure • premature aging to remove LSAs from databases • don‘t use LSAs that are older than 1 hour for routing calculation • refresh own LSAs that are older than ½ hour • use of sequence numbers to determine, which instance of an LSA is the most recent one

33. Routing Table Calculation • Dijkstra‘s Shortest Path First algorithm • Calculate SPF tree for the area using the SPF algorithm • Consider only routers and network nodes from the LSDB • Add stub networks to the tree • Add inter-area routes to the tree • Examine summary LSAs to find better routes for destinations in areas connected through virtual links • Add AS external routes to the tree

34. Testing OSPF • take advantage from the functional structure • four smaller testing problems instead of a big one • each functional area can be tested independently • take care of dependencies • functional hierarchy rather than completely independent areas • test „lower level“ functionality first

35. Overview SSFNet Conformance Testing OSPF OSPF Test Suites Implementation for SSFNet Summary

36. OSPF Test Suites • most of the formal methods require protocol specification with FSMs • OSPF only partially specified by FSMs • many parts like LSA origination not covered • most parts are specified using natural language • OSPF consists of multiple concurrent processes • Formal methods require formal specifications • not provided for OSPF, must be derived manually from standard • Formal methods difficult to apply to OSPF

37. C-TTCN Based Test Suite • approach to testing OSPF with formal methods • modeling test cases with C-TTCN • extension to TTCN which allows modeling of concurrent behaviors • OSPF specification was translated into a FDT called CEBE • based on LTS, which are similar to FSMs • resulting test suite had over 4000 test cases • after elimination of unimportant test cases: 543 test cases • Problem: how to prove that formal specification covers the complete OSPF specification?

38. IOL Test Suite • IOL is a test lab, whose services are used by many vendors • Test suites are derived by „carefully reading the standards“ • good coverage and correctness • used by many vendors • refined over the time • covers the functional areas of OSPF presented before • Hello Protocol tests • Fooding and Adjacency tests • Link state advertisement tests • Route calculation tests • Additionally: Configuration and Formatting tests • 78 test cases with several test steps each

39. Commecial Test Products • ANVL • software solution for conformance and performance testing • used by many well-known vendors • provides automated testing • OSPF test suite consists of over 300 test cases from 12 groups • no formal methods are used • QARobot / RouterTester • hardware solution together with protocol and test suite software • conformance and stress testing • OSPF test sute has 70 test cases from 12 groups • no formal methods are used

40. Testing OSPF in Practice • in practice: mixture of several different test products and/or lab testing • Example: ZebOS Advanced Routing Suite • basic testing with ANVL • additionally performance testing using other test tools • after that: lab testing at the IOL and other labs • Testing against multiple test suites increases probability of finding any offences against the specification

41. Overview SSFNet Conformance Testing OSPF OSPF Test Suites Implementation for SSFNet Summary

42. OSPF Test Suite for SSFNet • OSI methods can be easily transferred into SSFNet • Test Procedure needs not to be modified • Extended test architecture to support distributed test behavior • Test cases can be modeled using appropriate DML scenarios • some test cases need implementation of specialized testers

43. SSF.OS.OSPFv2 • Goal: implement a complete OSPF for SSFNet • Must correctly implement the standard • until now: • Hello Protocol, Database Exchange, Flooding and Routing Table Calculation • only Router LSAs • restricted to point-to-point networks • no vitual links or authentication • work still in progress

44. Test Selection • depends on several criteria • correctness • coverage • relevance and acceptance in practice • ease of implementation • applicability early during development • availability • formal methods are difficult to apply and not very common • dedicated test equipment not applicable easily • software solution like ANVL also not applicable • IOL test suite seems the best choice

45. Design of the Test Suite • many tests can be implemented using standard OSPF behavior • some require specialized testers with non-standard behavior • topology changes such as link failures • simple errors like malformed packets • complex tests which are aware of the state • test behaviors are implemented in SSF.OS.OSPFv2.test • five subsidiary test suites (see IOL test suite) • additionally: tests for packet forwarding and equal-cost multipath

46. Design of the Test Suite (2) • each test is performed in several steps • simulation of the scenario • evaluation of the logs • assignment of a verdict • evaluation is done automatically by PERL scripts • verdicts: PASS, FAIL or INCONCLUSIVE • test suite passes only when all tests pass • common components for simulation are kept in a test suite dictionary

47. SSF.OS.OSPFv2.test • contains classes to implement test behavior • Configurator • Reset • UnreliableIP • IPwithErrorInjection • PacketGenerator • OSPFMonitor, OSPFDumpPro • main task: manipulate packets coming from or sent to OSPF

48. SSF.OS.OSPFv2.test (2) • class hierarchy to be able to use different classes of testers together

49. Implementation of Test Cases • model test setup in DML • replace broadcast networks by a set of point-to-point links • features specific to broadcast networks, virtual links, etc. not yet tested • use tester classes to model tester behavior whenever necessary

50. Example: old_lsa_rcpt • a router must discard an LSA that is older than the copy in ist own database • routers synchronize their databases • then the tester sends a new LSA with sequence no. 0x70000001 • after that, it sends another LSA with sequence no. 0x 8FFFFFFE • IUT must send back the newer instance and discard the second LSA IUT Tester