1 / 21

Interoperability Testing

Interoperability Testing. Work done so far. WSDL subgroup Generated Web Service Description with aim for maximum interoperability between various SOAP stacks WSDL tested against tooling supplied by JAX-RPC RI AXIS .NET Basic and informal interop tests .NET <-> AXIS.

roy
Download Presentation

Interoperability Testing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Interoperability Testing

  2. Work done so far • WSDL subgroup • Generated Web Service Description with aim for maximum interoperability between various SOAP stacks • WSDL tested against tooling supplied by • JAX-RPC RI • AXIS • .NET • Basic and informal interop tests • .NET <-> AXIS

  3. WSDL subgroup (cont’d) • Released a subgroup report (Andre) • Philosophy followed • e.g. choosing document/literal style • Extension mechanism • Found problems/concerns • Raised bug reports against the stacks • AXIS begun work on all of them • .NET reported fixes were in progress • JAX-RPC RI has not responded yet • Fixes/workarounds reflected in the WSDL and interfaces

  4. Interoperabilty • WSDL subgroup work • Allows various stacks to generate code from WSDL definitions • Ensures interoperability on SOAP level • Need for interoperability on WSRP protocol level • cross-vendor testing • Compliance testing ?

  5. Cross-vendor testing • Release open source project which provides sample implementation • Allows vendors to test their Producers and Consumers against this implementation • Does not guerantee compliance • Vendors willing to make their Producers accessible to others for interop tests? • Allows others to test their Consumers • In turn vendors return feedback on their Producer implementations • Requires similar (spec) level of implementation • Should be easy once version 1 is released • Still no compliance testing in a formal sense

  6. Cross-vendor Testing - Matrix

  7. Compliance Testing • Testing a particular implementation for compliance with the specification. • Compliance testing is strictly “black box” testing

  8. Compliance Testing - Tasks • Develop a set of testable assertions which represent the specification • Derive testable assertions from the specification • Determine test cases implied by each assertion • Develop a Compliance Test Suite available for public download so that vendors and customers can verify that their implementations are compliant with the WSRP 1.0 Version of the spec • Design, develop, and test the tests for each assertion • Create a test framework to run, evaluate, and report on the tests

  9. Deriving Testable Assertions • Significant Effort to recast the spec as assertions • Requires committee involvement

  10. Test Suite Framework • A technical framework for running compliance tests. • The framework shall be used to run, evaluate, and report test cases. • Should be highly scriptable and configurable so test cases may be added easily • This may be a significant effort by itself, depending on our ability to leverage existing frameworks

  11. Testing Producers • Develop a Consumer which executes test cases against a Producer and reports the results. • Test cases are configured as XML input • Reports are generated as XML output

  12. Testing Consumers • Develop a RI Producer • conforms to the spec (I.e. passes all the compliance tests) • produces reports on the behavior of the Consumer, if in error. • For example, validating the state that was supposed to be kept and returned by the Consumer

  13. Testing - Roadmap • Form interop test subgroup • Start interop tests February, 15th? • Establish interoperability April, 15th • Form compliance test subgroup • Draft assertion suite March, 15th • Finalize assertion suite May, 1st

  14. Interop • Mike • Alan • Richard • Andre • Nigel • Subbu • Gil(?)

  15. Compliance • Ross • Dan • Rich

  16. Backup

  17. Existing Examples • Java Community Process • Calls for developing a Technology Compatibility Kit (TCK) side by side with Reference Implementation • Well developed process and tools • W3C • The W3C standardization process only produces specification documents (Recommendations, etc), and does not formally address compliance testing. • Apache • Each project defines its own testing practices and standards

  18. WS-I Testing Working Group2 • Deliverables • The Test Tools Development Working Group will produce the following deliverables: • 1.WS-I Test Methodology White paper • 2.WS-I Test Tools Specification • 3.Two or more implementations of Test Tools and supporting documentation • 4.Assertions and Test Conditions used as input to the Test Tools • 5.WS-I Tool Configuration Template • 6.WS-I Experience Report: Tool Development as a component of WS-I Profile Development 2WS-I Testing Working Group Charter, http://www.ws-i.org/docs/charters/Test_Charter.pdf

  19. Scope of the Task1 • Derive testable assertions from the specification • Determine test cases implied by each assertion • Design, develop, and test the tests for each assertion • Create a test framework to run, evaluate, and report on the tests 1TCK Project Planning and Development Guide, Sun Microsystems, Inc

  20. Existing Frameworks • JUnit -- open source project • BaRT – Batch generator for Regression Tests -- IBM open source tool used for testing JVMs. • Sun’s Compatibility Test Toolkit – licensed, limited to JCP projects

  21. Testing Optional Functionality • Separate, or at least configurable, test suites for discrete levels of function • The discrete levels of function currently mentioned are Simple and Sophisticated • Finer distinctions may have to be made • Perhaps the test suite can determine at runtime which level of function the Producer supports and be able to apply the correct tests.

More Related