1 / 47

Solving InterOperability Problems Using Object-Oriented CSP Modeling

Solving InterOperability Problems Using Object-Oriented CSP Modeling. Mohammed H. Sqalli Eugene C. Freuder University of New Hampshire msqalli,ecf@cs.unh.edu. -The first author is currently at the of Canada Ltd.

jeri
Download Presentation

Solving InterOperability Problems Using Object-Oriented CSP Modeling

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Solving InterOperability Problems UsingObject-Oriented CSP Modeling Mohammed H. Sqalli Eugene C. Freuder University of New Hampshire msqalli,ecf@cs.unh.edu -The first author is currently at the of Canada Ltd. -This material is based in part on work supported by the National Science Foundation.

  2. Outline • Example • Domain of Application • Motivations and Contributions • CSP Modeling • Constraint-Based Diagnosis • Explanation & Evaluation • Summary IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  3. Outline • Example • Domain of Application • Motivations and Contributions • CSP Modeling • Constraint-Based Diagnosis • Explanation & Evaluation • Summary IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  4. Specifications (What should Happen) Device B Device A Observations (What really happens) InterOperability Testing Communication line InterOperability Testing <=> Compare (Specifications, Observations) IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  5. Example from a Test Suite • Test Case ID: V4202H__004 • Test Purpose: Verify that the first Hello sent from both sides contains Remote node ID and Remote port ID set to zero (1Way). • Pre-requisite: Both SUTs (A, B) are in different lowest level peer groups(Out). • Verdict Criteria: The first Hello packet observed from each SUT will have the Remote node ID field and Remote port ID field set to zero(1WayOut). IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  6. 1WayOut 1WayOut Example - Test Description Switch A Switch B IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  7. 1WayOut(A) {Source, Time, Status, Node_ID, ...} 1WayOut(B) {Source, Time, Status, Node_ID, ...} ... Variables 1WayOut(A).Status = Mandatory 1WayOut(A).Remote_Node_ID != 0 Unary Constraints 1WayOut(A).Time < 1WayOut(B).Time 1WayOut(A).Peer_Group_ID != 1WayOut(B).Peer_Group_ID ... Binary Example - CSP Model 1WayOut(A) 1WayOut(B) < Status Type Time Status Type Time 1 1 Hello Hello != Port ID Node ID Source Port ID Node ID Source Peer Group ID Peer Group ID != Remote Port ID Remote Node ID Remote Port ID Remote Node ID 0 0 0 0 IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  8. Panel showing the CSP model of a test case Panel for declaring variables and domains Panel for declaring constraints ADIOP Modeling - Test Suite Builder GUI IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  9. ADIOP Diagnosis - Test Suite Menu GUI IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  10. Test Suites CSP #2 Test #1 Test #2 CSP #1 Protocol Specification Monitored Observations … Decoded Observations CSP Models … ADIOP – The big picture (Four modules) Decoding Modeling Update Diagnosis Result Debugging OK ? Store Next Step CBR + Expert Result Report Yes No IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  11. Test Suites Test #2 Test #1 CSP #2 CSP #1 Protocol Specification Monitored Observations … Decoded Observations CSP Models … ADIOP – Modeling & Diagnosis Decoding Modeling Diagnosis Result IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  12. Outline • Example • Domain of Application • Motivations and Contributions • CSP Modeling • Constraint-Based Diagnosis • Explanation & Evaluation • Summary IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  13. InterOperability Lab • Verify the interoperability and/or conformance of the computer communications products • Consortiums: ATM, FDDI, Wireless, … • Protocols tested: UNI, LANE, PNNI, … • Test Suites for IOP testing: LANE, PNNI, … • Testing Equipment: Snifters, Generators,... • Testing Problems: Data Analysis, Similar Problems, ... IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  14. ATM and PNNI • Asynchronous Transfer Mode • supports all classes of traffic: voice, video, and data • provides quick and easy switching and multiplexing • connection oriented • Private Network Network Interface • provides dynamic routing: includes discovery of the topology of the network • supports QoS and hierarchical routing • scales to very large networks IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  15. Protocol Specifications Observations Compare to Report InterOperability Testing Network Device B Device A Monitor IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  16. Outline • Example • Domain of Application • Motivations and Contributions • CSP Modeling • Constraint-Based Diagnosis • Explanation & Evaluation • Summary IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  17. Motivations – The ADIOP System • Model IOP test cases efficiently. • Diagnose IOP problems in a timely manner • Analyze a large amount of data • Provide an explanation of the diagnosis • Keep trace of previous problems and their solutions to be reused in future similar cases • Debug IOP tests IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  18. Solution - The ADIOP System • Automate the process of InterOperability Testing • CSP modeling of IOP tests • Diagnosing IOP problems quickly and efficiently using CSP models (including Explanation) • Repeatability of testing • Reusable diagnosis and results: experience is kept • Explanation and Reports generation • Debug the InterOperability tests IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  19. Outline • Example • Domain of Application • Motivations and Contributions • CSP Modeling • Constraint-Based Diagnosis • Explanation & Evaluation • Summary IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  20. X X: red Y: blue Z: green Red green blue • Search • Inference Red green blue Red green blue Y Z X Y Z Variables CSP Algorithm Solution CSP Representation Constraint Satisfaction Problems X Y Z Coloring Problem Problem Statement Values Constraints IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  21. Test Suites Test #1 Test #2 CSP #1 CSP #2 Protocol Specification Monitored Observations … Decoded Observations CSP Models … Result CSP Modeling and Diagnosis of InterOperability Testing NB: The decoders have built in functionality that provides the information (protocols, packets, packet’s fields) to build the CSP models. IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  22. Modeling with objects • An IAAI-01 presentation on August 7, 2001 at 10AM entitled: “Constraint-Based Modeling of InterOperability Problems Using an Object-Oriented Approach” will provide more details on the OO-based CSP modeling of ADIOP. • One-Model vs. Many-models architectures • Modeling with Objects • Test cases as objects • Advantages of CSP modeling using OOP • Modeling language definition IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  23. Modeling Interface (GUI) Protocols List: loaded from the class hierarchy of the Packet class Packet Types List: loaded from the class hierarchy of the protocol’s class Packet’s Parameters (CSP variables) List: loaded from the class definition of a packet. IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  24. Test Suite Builder GUI Step 1: Choose the protocol Step 2: Start CSP Model Step 3: Define the packets Step 4: Define the constraints Step 5: This generates the test case object IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  25. Example of CSP Modeling- Modeling Language - $CSP $PROTOCOL PnniRout $PACKET OneWayOutA Hello $PACKET OneWayOutB Hello $UNARY_CONSTRAINT OneWayOutA.status D_Mandatory $UNARY_CONSTRAINT OneWayOutA.remote_port_id == 0 $UNARY_CONSTRAINT OneWayOutB.remote_port_id == 0 $UNARY_CONSTRAINT OneWayOutA.remote_node_id == 0 $UNARY_CONSTRAINT OneWayOutB.remote_node_id == 0 $BINARY_CONSTRAINT OneWayOutA.time < OneWayOutB.time $BINARY_CONSTRAINT OneWayOutA.source != OneWayOutB.source $BINARY_CONSTRAINT OneWayOutA.peer_group_id != OneWayOutB.peer_group_id $ENDCSP Protocol Metavariables Optional statement Unary Constraints Binary Constraints IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  26. Test Suite Menu (GUI) IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  27. Test Suite Directory Hierarchy adiop adiopx testsuite pnnirout lane mpoa V4301H_002 V4301H_003 V4301H_001 • Test cases are stored in the appropriate directory. • Menu of test cases is generated dynamically from this directory structure. IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  28. Outline • Example • Domain of Application • Motivations and Contributions • CSP Modeling • Constraint-Based Diagnosis • Explanation & Evaluation • Summary IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  29. Constraint-Based Diagnosis of IOP Problems • Diagnosis of Interoperability problems is the detection of problems that occur when two devices running the same networking protocol are connected to each other through a network. The devices are assumed to have passed conformance testing which states that each device by itself is conformant in its behavior to the protocol specification. IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  30. Test Suites Test #2 CSP #2 CSP #1 Test #1 Protocol Specification Monitored Observations … Decoded Observations CSP Models … ADIOP – Modeling & Diagnosis Decoding Modeling Diagnosis Result Builds CSP models for test cases [Metavariables/ Constraints] Generates decoded observations [Metavalues] Compares decoded observations to CSP model IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  31. Algorithms for Diagnosis • CSP is used for modeling IOP test cases  Use CSP for diagnosis and take advantage of the already existing algorithms for solving CSPs. • Main goal of diagnosis: to get a “good” explanation of the problems. • Algorithms: search supplied by consistency inference methods in a CSP context. IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  32. Search • Simple backtracking adapted to OO-based CSPs. It uses metavariables and metavalues. • Search for the first solution if it exists. • Explanation provided to testers: • Test passes: which packets constitute the solution (useful to testers) • Test fails: solution obtained is not very useful to testers IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  33. Inference and Consistency Checking • Get a better explanation in case of failure of a test case • Specialized Node Consistency inference for and OO-based CSP [MVC]. • Domains of all metavariables are set to all observed packets. • Use some variables to reduce the domain of metavalues for metavariables: protocol and packetType variables. • If no packet observed belongs to the protocol in the CSP model, then there is no solution. • Delete all metavalues that do not appear in the CSP model as one of the metavariables types (packetType). • If size of a domain shared by n metavariables is reduced to r with r < n, then there is no possible solution (overconstrained n-clique) IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  34. Outline • Example • Domain of Application • Motivations and Contributions • CSP Modeling • Constraint-Based Diagnosis • Explanation & Evaluation • Summary IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  35. Explanation • ADIOP uses templates to generate human-like explanations. • Example: There are less observed packets of type packetType than what is stated in the model of this test. • Used when there is an overconstrained clique. IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  36. Reports generation • Individual test case reports • Section test cases reports IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  37. Evaluation • 4 real-world observations: 3 from the PNNI Routing protocol and 1 from the LANE protocol. • Run only test cases that belong to the protocol used when the observations were captured. • Compare time and explanation when using preprocessing (inference) + backtracking vs. backtracking only. • Preprocessing lead to a reduction of time effort needed for solving the problems and produced a better explanation. IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  38. Evaluation - Summary • Out of 69 test cases run: • 36 passed  useful explanation • 33 failed: • 14 using only preprocessing  useful explanation • Total of 50 test cases produces a useful explanation (73%) IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  39. Complexity • s: size of the domains of metavalues. • r: reduced size of s after preprocessing (r ≤ s) • n: number of metavariables defined in a model • Backtracking: O(sn) • Preprocessing: O(n2s) • Preprocessing + Backtracking: O(rn) IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  40. Outline • Example • Domain of Application • Motivations and Contributions • CSP Modeling • Constraint-Based Diagnosis • Explanation & Evaluation • Summary IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  41. Test Suites CSP #2 Test #1 Test #2 CSP #1 Protocol Specification Monitored Observations … Decoded Observations CSP Models … ADIOP – The big picture (Four modules) Decoding Modeling Update Diagnosis Result Debugging OK ? Store Next Step CBR + Expert Result Report Yes No IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  42. Test Suite Builder Decoder Diagnoser Test Generation and Implementation Test Debugger Decoder s Builder Decoder Usage Test Execution (Algorithms) Menus and Reports Test Description from IOP test suite CSP Model Protocol Specification Hierarchy of Classes Search (hard explanation) Automate Menus Creation CBR + User Interaction Natural Language Processing Frame Format (Defined language) Packet Inference (better explanation) Reports Generation Correct/Update CSP Model PnniR MPOA Graphical Representation Java file (Frame decoder) Notes: Existent Documents Hello DBS New CSP Model CSP Model Java Class (Frame decoder) Implemented Note: These classes are used by Decoder, Diagnoser, and Test Builder. Java file To be implemented Java Class (used by diagnoser) Not implemented ADIOP (Automated Diagnosis of InterOperability Problems) IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  43. Summary • CSP Model building: Simple, declarative, transparent, GUI-based, and complete modeling language for stating and generating IOP tests. • OO provides a more natural, concise, scalable and reusable framework for model statement of IOP tests (+ Inheritance, Encapsulation) • Combination of CSP and OO: metavariables and metavalues to represent packets and generate test cases as objects. • Scalability: More protocols can be made available for modeling by adding the corresponding decoders. • Modeling Interface: dynamic loading of protocols and packets being used. • Modeling Language: • Generic across protocols: Independent of the protocol being used. • Constraint expressiveness: unary and binary constraints. • Many-models architecture: test suites are used instead of protocol specifications. Each test case is represented by a CSP model. IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  44. Summary (Cont.) • The test cases built using the ADIOP’s modeling component are dynamically accessible through a menu. • Motivations of Diagnosis of IOP Problems: save time, reduce repetitive testing, store and reuse knowledge, automate reports generation. • Advantages of using CSP for modeling and diagnosis: take advantage of existing CSP algorithms. • Algorithms for Diagnosis: Search and Inference. • Generate an accurate and human-like explanation. • Explanation & Evaluation: Diagnosis and Explanation of IOP problems. • Evaluation and summary numbers: 50 (73% ) test cases out of 69 built produced a meaningful explanation. IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  45. Constraint-Based Modeling of InterOperability Problems Using an Object-Oriented Approach Mohammed H. Sqalli Eugene C. Freuder University of New Hampshire msqalli,ecf@cs.unh.edu Discussion

  46. Contributions • Model building: Simple, declarative, transparent, GUI-based, and complete modeling language for stating and generating IOP tests. • OO provides a more natural, concise, scalable and reusable framework for model statement of IOP tests. • Combination of CSP and OO: metavariables and metavalues to represent packets and generate test cases as objects. • Modeling is generic across protocols: independent of the protocol being used. • Scalability: More protocols can be made available for modeling by adding the corresponding decoders. • Dynamic loading of protocols and packets being used. • Many-models architecture: each test case is represented by a CSP model. • Use of test suite instead of protocol specification. • Constraint expressiveness: unary and binary constraints. • CSP algorithms for diagnosing IOP problems: search and specialized inference consistency methods. • Provide an explanation of the cause of IOP problems. IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

  47. Specifications (What should Happen) Device B Device A Observations (What really happens) InterOperability Testing Communication line InterOperability Testing <=> Compare (Specifications, Observations) IJCAI-01 Workshop - CONS-1: Sqalli & Freuder

More Related