1 / 46

Constraint-Based Modeling of InterOperability Problems Using an Object-Oriented Approach

Constraint-Based Modeling of InterOperability Problems Using an Object-Oriented Approach. Mohammed H. Sqalli Eugene C. Freuder University of New Hampshire msqalli,ecf@cs.unh.edu. -The first author is currently at the of Canada Ltd.

kobe
Download Presentation

Constraint-Based Modeling of InterOperability Problems Using an Object-Oriented Approach

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Constraint-Based Modeling of InterOperability Problems Using an Object-Oriented Approach Mohammed H. Sqalli Eugene C. Freuder University of New Hampshire msqalli,ecf@cs.unh.edu -The first author is currently at the of Canada Ltd. -This material is based in part on work supported by the National Science Foundation.

  2. Outline • Example • Domain of Application • Motivations and Contributions • CSP Modeling • Modeling with Objects • Modeling Interface and Language • Test Cases as Objects • Application & Evaluation • Summary IAAI-01: Sqalli & Freuder

  3. Outline • Example • Domain of Application • Motivations and Contributions • CSP Modeling • Modeling with Objects • Modeling Interface and Language • Test Cases as Objects • Application & Evaluation • Summary IAAI-01: Sqalli & Freuder

  4. Specifications (What should Happen) Device B Device A Observations (What really happens) InterOperability Testing Communication line InterOperability Testing <=> Compare (Specifications, Observations) IAAI-01: Sqalli & Freuder

  5. Example from a Test Suite • Test Case ID: V4202H__004 • Test Purpose: Verify that the first Hello sent from both sides contains Remote node ID and Remote port ID set to zero (1Way). • Pre-requisite: Both SUTs (A, B) are in different lowest level peer groups(Out). • Verdict Criteria: The first Hello packet observed from each SUT will have the Remote node ID field and Remote port ID field set to zero(1WayOut). IAAI-01: Sqalli & Freuder

  6. 1WayOut 1WayOut Example - Test Description Switch A Switch B IAAI-01: Sqalli & Freuder

  7. 1WayOut(A) {Source, Time, Status, Node_ID, ...} 1WayOut(B) {Source, Time, Status, Node_ID, ...} ... Variables 1WayOut(A).Status = Mandatory 1WayOut(A).Remote_Node_ID != 0 Unary Constraints 1WayOut(A).Time < 1WayOut(B).Time 1WayOut(A).Peer_Group_ID != 1WayOut(B).Peer_Group_ID ... Binary Example - CSP Model 1WayOut(A) 1WayOut(B) < Status Type Time Status Type Time 1 1 Hello Hello != Port ID Node ID Source Port ID Node ID Source Peer Group ID Peer Group ID != Remote Port ID Remote Node ID Remote Port ID Remote Node ID 0 0 0 0 IAAI-01: Sqalli & Freuder

  8. Panel showing the CSP model of a test case Panel for declaring variables and domains Panel for declaring constraints ADIOP- Test Suite Builder GUI IAAI-01: Sqalli & Freuder

  9. ADIOP - Test Suite Menu GUI IAAI-01: Sqalli & Freuder

  10. Test Suites CSP #1 Test #1 Test #2 CSP #2 Protocol Specification Monitored Observations … Decoded Observations CSP Models … ADIOP – The big picture (Four Modules) Decoding Modeling Update Diagnosis Result Debugging OK ? Store Next Step CBR + Expert Result Report Yes No IAAI-01: Sqalli & Freuder

  11. Test Suites Test #1 Test #2 CSP #1 CSP #2 Protocol Specification … CSP Models … Modeling ADIOP - Modeling IAAI-01: Sqalli & Freuder

  12. Outline • Example • Domain of Application • Motivations and Contributions • CSP Modeling • Modeling with Objects • Modeling Interface and Language • Test Cases as Objects • Application & Evaluation • Summary IAAI-01: Sqalli & Freuder

  13. InterOperability Lab • Verify the interoperability and/or conformance of the computer communications products • Consortiums: ATM, FDDI, Wireless, … • Protocols tested: UNI, LANE, PNNI, … • Test Suites for IOP testing: LANE, PNNI, … • Testing Equipment: Snifters, Generators,... • Testing Problems: Data Analysis, Similar Problems, ... IAAI-01: Sqalli & Freuder

  14. ATM and PNNI • Asynchronous Transfer Mode • supports all classes of traffic: voice, video, and data • provides quick and easy switching and multiplexing • connection oriented • Private Network Network Interface • provides dynamic routing: includes discovery of the topology of the network • supports QoS and hierarchical routing • scales to very large networks IAAI-01: Sqalli & Freuder

  15. Protocol Specifications Observations Compare to Report InterOperability Testing Network Device B Device A Monitor IAAI-01: Sqalli & Freuder

  16. Outline • Example • Domain of Application • Motivations and Contributions • CSP Modeling • Modeling with Objects • Modeling Interface and Language • Test Cases as Objects • Application & Evaluation • Summary IAAI-01: Sqalli & Freuder

  17. Motivations – The ADIOP System • Diagnose IOP problems in a timely manner • Analyze a large amount of data • Keep trace of previous problems and their solutions to be reused in future similar cases • Debug IOP tests IAAI-01: Sqalli & Freuder

  18. Solution - The ADIOP System • Automate the process of InterOperability Testing • CSP modeling of IOP tests • Diagnosing IOP problems quickly and efficiently using CSP models (including Explanation) • Debug the InterOperability tests IAAI-01: Sqalli & Freuder

  19. Contributions • Model building: Simple, declarative, transparent, GUI-based, and complete modeling language for stating and generating IOP tests. • OO provides a more natural, concise, scalable and reusable framework for model statement of IOP tests. • Combination of CSP and OO: metavariables and metavalues to represent packets and generate test cases as objects. • Modeling is generic across protocols: independent of the protocol being used. • Scalability: More protocols can be made available for modeling by adding the corresponding decoders. • Dynamic loading of protocols and packets being used. • Many-models architecture: each test case is represented by a CSP model. • Constraint expressiveness: unary and binary constraints. IAAI-01: Sqalli & Freuder

  20. Outline • Example • Domain of Application • Motivations and Contributions • CSP Modeling • Modeling with Objects • Modeling Interface and Language • Test Cases as Objects • Application & Evaluation • Summary IAAI-01: Sqalli & Freuder

  21. X X: red Y: blue Z: green Red green blue • Search • Inference Red green blue Red green blue Y Z X Y Z Variables CSP Algorithm Solution CSP Representation Constraint Satisfaction Problems X Y Z Coloring Problem Problem Statement Values Constraints IAAI-01: Sqalli & Freuder

  22. One Model (Protocol Specification) [Sqalli & Freuder 1996] Only one model to use for testing, but too complex Less inconsistencies No redundant testing Many Models (Test Suite) [Sqalli & Freuder 1998] Easy and small models for specific tests To state To fix/update Suitable for interoperability testing Generate reports Preferred by vendors/testers Explanation of the cause of failure One Model vs. Many Models IAAI-01: Sqalli & Freuder

  23. Test Suites Test #1 Test #2 CSP #1 CSP #2 Protocol Specification Monitored Observations … Decoded Observations CSP Models … Result CSP Modeling of InterOperability Testing NB: The decoders have built in functionality that provides the information (protocols, packets, packet’s fields) to build the CSP models. IAAI-01: Sqalli & Freuder

  24. Outline • Example • Domain of Application • Motivations and Contributions • CSP Modeling • Modeling with Objects • Modeling Interface and Language • Test Cases as Objects • Application & Evaluation • Summary IAAI-01: Sqalli & Freuder

  25. Packet Time Source Type Node ID ... Object (Metavariable/ Metavalue) getNodeID • Source • Time • Node ID getTime • Type getPacket Modeling with Objects Metavariable: X = {x1, x2, …, xn}, where xi is a variable CSP Metavalue: V = {v1, v2, …, vn}, where vi is a value Binary Metaconstraint: Cxy= {c1, c2, …, cm}, where ck is a constraint between xi and yj IAAI-01: Sqalli & Freuder

  26. PNNI Routing Packet Time Source PNNI Routing Header Info ... Hello Packet Time Source PNNI Routing Header Node ID Packet PnniRout Lane Mpoa Dbs Hello Class Hierarchy Packet Time Source Info IAAI-01: Sqalli & Freuder

  27. Advantages of Object-Orientation • Inheritance between objects allows for a hierarchical definition of packets which matches the way protocols are specified. • Encapsulation • Information hiding of the objects definition as the users do not need to know the details but only the functionality. • Modularity: Each object is a separate entity with its own functionality. • Scalability: Adding more protocols and packets does not affect the already existing system. • Reusability: The same objects are being used for decoding packets as well as for CSP model definition. • Natural: The CSP model obtained using objects is concise and expressive. IAAI-01: Sqalli & Freuder

  28. Outline • Example • Domain of Application • Motivations and Contributions • CSP Modeling • Modeling with Objects • Modeling Interface and Language • Test Cases as Objects • Application & Evaluation • Summary IAAI-01: Sqalli & Freuder

  29. Modeling Interface (GUI) Protocols List: loaded from the class hierarchy of the Packet class Packet Types List: loaded from the class hierarchy of the protocol’s class Packet’s Parameters (CSP variables) List: loaded from the class definition of a packet. IAAI-01: Sqalli & Freuder

  30. Advantages of CSP Modeling using OOP • No need for the tester to know what each packet contains (built in the hierarchy). • No need to know the syntax of the modeling language (generated by the GUI) • Possibility to add more protocols just by adding the decoder for such protocol (protocol independent modeling). • Dynamic loading of information about protocols, packets, etc. (done at runtime). IAAI-01: Sqalli & Freuder

  31. Example from a Test Suite • Test Case ID: V4202H__004 • Test Purpose: Verify that the first Hello sent from both sides contains Remote node ID and Remote port ID set to zero (1Way). • Pre-requisite: Both SUTs (A, B) are in different lowest level peer groups(Out). • Verdict Criteria: The first Hello packet observed from each SUT will have the Remote node ID field and Remote port ID field set to zero(1WayOut). IAAI-01: Sqalli & Freuder

  32. Metavariable Metavariable 1WayOut(A) {Source, Time, Status, Node_ID, ...} 1WayOut(B) {Source, Time, Status, Node_ID, ...} ... Variables 1WayOut(A).Status = Mandatory 1WayOut(A).Remote_Node_ID != 0 Unary Constraints 1WayOut(A).Time < 1WayOut(B).Time 1WayOut(A).Peer_Group_ID != 1WayOut(B).Peer_Group_ID ... Binary CSP Model 1WayOut(A) 1WayOut(B) < Status Type Time Status Type Time 1 1 Hello Hello != Port ID Node ID Source Port ID Node ID Source Peer Group ID Peer Group ID != Remote Port ID Remote Node ID Remote Port ID Remote Node ID 0 0 0 0 IAAI-01: Sqalli & Freuder

  33. Test Suite Builder GUI Step 1: Choose the protocol Step 2: Start CSP Model Step 3: Define the packets Step 4: Define the constraints Step 5: This generates the test case object IAAI-01: Sqalli & Freuder

  34. Example of CSP Modeling- Modeling Language - $CSP $PROTOCOL PnniRout $PACKET OneWayOutA Hello $PACKET OneWayOutB Hello $UNARY_CONSTRAINT OneWayOutA.status D_Mandatory $UNARY_CONSTRAINT OneWayOutA.remote_port_id == 0 $UNARY_CONSTRAINT OneWayOutB.remote_port_id == 0 $UNARY_CONSTRAINT OneWayOutA.remote_node_id == 0 $UNARY_CONSTRAINT OneWayOutB.remote_node_id == 0 $BINARY_CONSTRAINT OneWayOutA.time < OneWayOutB.time $BINARY_CONSTRAINT OneWayOutA.source != OneWayOutB.source $BINARY_CONSTRAINT OneWayOutA.peer_group_id != OneWayOutB.peer_group_id $ENDCSP Protocol Metavariables Optional statement Unary Constraints Binary Constraints IAAI-01: Sqalli & Freuder

  35. Modeling Language Definition • $CSP • $ENDCSP • $PROTOCOL protocolTested • $PACKET packet_name packet_type • $DOMAIN domain value1 value2 … • $UNRAY_CONSTRAINT variable operation [domain|value] • $BINARY_CONSTRAINT variable1 operation variable2 • $CONSTRAINT variable1 variable2 f(variable1, variable2) IAAI-01: Sqalli & Freuder

  36. Outline • Example • Domain of Application • Motivations and Contributions • CSP Modeling • Modeling with Objects • Modeling Interface and Language • Test Cases as Objects • Application & Evaluation • Summary IAAI-01: Sqalli & Freuder

  37. Hello Example of CSP Modeling- Test Cases as Objects - • This will generate an object in which fields are domains or metavariables, and methods are constraints. (This is the input to the Diagnosis module) OneWayOutA_Status • Mandatory OneWayOutB_Status OneWayOutA_Type OneWayOutA • Hello OneWayOutB OneWayOutB_Type OneWayOutA_Source_OneWayOutB_Source OneWayOutA_Time_OneWayOutB_Time IAAI-01: Sqalli & Freuder

  38. Test Suite Menu (GUI) IAAI-01: Sqalli & Freuder

  39. Test Suite Directory Hierarchy adiop adiopx testsuite pnnirout lane mpoa V4301H_002 V4301H_003 V4301H_001 • Test cases are stored in the appropriate directory. • Menu of test cases is generated dynamically from this directory structure. IAAI-01: Sqalli & Freuder

  40. Outline • Example • Domain of Application • Motivations and Contributions • CSP Modeling • Modeling with Objects • Modeling Interface and Language • Test Cases as Objects • Application & Evaluation • Summary IAAI-01: Sqalli & Freuder

  41. Test Suites CSP #1 Test #1 Test #2 CSP #2 Protocol Specification Monitored Observations … Decoded Observations CSP Models … ADIOP – The big picture (Four Modules) Decoding Modeling Update Diagnosis Result Debugging OK ? Store Next Step CBR + Expert Result Report Yes No IAAI-01: Sqalli & Freuder

  42. Test Suite Builder Decoder Diagnoser Test Generation and Implementation Test Debugger Decoder s Builder Decoder Usage Test Execution (Algorithms) Menus and Reports Test Description from IOP test suite CSP Model Protocol Specification Hierarchy of Classes Search (hard explanation) Automate Menus Creation CBR + User Interaction Natural Language Processing Frame Format (Defined language) Packet Inference (better explanation) Reports Generation Correct/Update CSP Model PnniR MPOA Graphical Representation Java file (Frame decoder) Notes: Existent Documents Hello DBS New CSP Model CSP Model Java Class (Frame decoder) Implemented Note: These classes are used by Decoder, Diagnoser, and Test Builder. Java file To be implemented Java Class (used by diagnoser) Not implemented ADIOP (Automated Diagnosis of InterOperability Problems) IAAI-01: Sqalli & Freuder

  43. Application of CSP Modeling - ADIOP Diagnosis • The test cases built using the ADIOP’s modeling component are dynamically accessible through a menu. • Motivations of Diagnosis of IOP Problems: save time, reduce repetitive testing, store and reuse knowledge, automate reports generation. • Advantages of using CSP for modeling and diagnosis: take advantage of existing CSP algorithms. • Algorithms for Diagnosis: Search and Inference. • Generate an accurate and human-like explanation. • Evaluation and summary numbers: 50 (73% ) test cases out of 69 built produced a meaningful explanation. IAAI-01: Sqalli & Freuder

  44. Outline • Example • Domain of Application • Motivations and Contributions • CSP Modeling • Modeling with Objects • Modeling Interface and Language • Test Cases as Objects • Application & Evaluation • Summary IAAI-01: Sqalli & Freuder

  45. Summary • CSP Model building: Simple, declarative, transparent, GUI-based, and complete modeling language for stating and generating IOP tests. • OO provides a more natural, concise, scalable and reusable framework for model statement of IOP tests (+ Inheritance, Encapsulation) • Combination of CSP and OO: metavariables and metavalues to represent packets and generate test cases as objects. • Scalability: More protocols can be made available for modeling by adding the corresponding decoders. • Modeling Interface: dynamic loading of protocols and packets being used. • Modeling Language: • Generic across protocols: Independent of the protocol being used. • Constraint expressiveness: unary and binary constraints. • Many-models architecture: test suites are used instead of protocol specifications. Each test case is represented by a CSP model. • Application & Evaluation: Diagnosis and Explanation of IOP problems. IAAI-01: Sqalli & Freuder

  46. Constraint-Based Modeling of InterOperability Problems Using an Object-Oriented Approach Mohammed H. Sqalli Eugene C. Freuder University of New Hampshire msqalli,ecf@cs.unh.edu Discussion

More Related