The METOER-S Framework for Semantic Web Process Composition

1. The METOER-S Framework for Semantic Web Process Composition Kaarthik Sivashanmugam Large Scale Distributed Information Systems (LSDIS) Lab, Department of Computer Science, The University of Georgia

2. Acknowledgements • Advisory Committee • Dr. John A. Miller (Major Advisor) • Dr. Amit P. Sheth (Co-advisor) • Dr. Hamid R. Arabnia • LSDIS Student Members • Kunal Verma, Deepti Chafekar, Ivan Vasquez, Preeda Rajasekaran and others

3. Outline • Introduction to Web services, Web services conceptual stack and Web processes • Challenges in Web services adoption • Semantics at different layers of Web services conceptual stack • Semantics for Web service life-cycle • METEOR-S project at LSDIS lab • Discovery Infrastructure (MWSDI) • Composition Framework (MWSCF)

4. Outline (contd) • MWSDI • P2P network of Web service registries • Semantics based publication and discovery of Web services • MWSCF • Need and advantages of Semantic Web process composition • Template based process generation • Tool for Template construction, Web service discovery and Process generation • Conclusion and Future Work • References

5. Web Services • Software applications (conforming set of standards) that can communicate with other software applications independent of their operating systems, programming languages etc. • to enable interoperability and to deliver complex value added services • A Web service is a software application identified by a URI, whose interfaces and binding** are capable of being defined, described and discovered by XML artifacts and supports direct interactions with other software applications using XML based messages via Internet-based protocols. (W3C definition) • **- An association between an Interface, a concrete protocol and a data format

6. Web Service Conceptual Stack1 Description Messaging Network • Description:Web Service Description Language (WSDL) • To describe Web Service interfaces and implementations • Details in WSDL files (data types, operations, binding details, access location) are used for service invocation • Messaging:(SOAP) • XML based messaging protocol • Network:(HTTP) • Network protocol 1 [Kreger]

7. Web Service Conceptual Stack1 Discovery Publication Description Messaging Network • Publication:(UDDI) • To make service descriptions available for search • Discovery:(UDDI) • To locate service descriptions Flow • Flow:(BPEL4WS, WSCI etc.) • To compose web services to form a composite web service / process 1 [Kreger]

8. Web Processes • Web Processes are next generation workflow technology to facilitate the interaction of organizations with markets, competitors, suppliers, customers etc. supporting enterprise-level and core business activities • encompass the ideas of both intra and inter organizational workflow. • created from the composition of Web Services • When all the tasks involved in a Web process are semantically described, we may call such process as Semantic Web Processes

9. Globalization of Processes B2B E-Services DistributedWorkflows Workflows Web Processes Enterprise Inter-Enterprise Global

10. BIG Challenges • Heterogeneity and Autonomy • Solution: Machine understandable descriptions • Dynamic nature of business interactions • Demands: Efficient Discovery, Composition etc. • Scalability (Enterprises  Web) • Needs: Automated service discovery/selection and composition Proposition: Semantics is the most important enabler to address these challenges

11. METEOR-S Project @ LSDIS lab • METEOR-S exploits Workflow, Semantic Web, Web Services, and Simulation technologies to meet these challenges in a practical and standards based approach. • Applying Semantics in Annotation, Quality of Service, Discovery, Composition, Execution of Web Services • Adding semantics to different layers of Web services conceptual stack • Use of ontologies to provide underpinning for information sharing and semantic interoperability

12. Semantics at Different Layers Discovery Publication • Description Layer: • Why: • Unambiguously understand the functionality of the services and the semantics of the operational data • How: • Using Ontologies to semantically annotate WSDL constructs (conforming to extensibility allowed in WSDL specification version 1.2) to sufficiently explicate the semantics of the • data types used in the service description and • functionality of the service • Present scenario: • WSDL descriptions are mainly syntactic (provides operational information and not functional information) • Semantic matchmaking is not possible Flow Description Messaging Network

13. Semantics at Different Layers (contd..) Discovery Publication • Publication and Discovery Layers: • Why: • Enable scalable, efficient and dynamic publication and discovery (machine processable / automation) • How: • Use of ontology to categorize registries based on domains and characterize them by maintaining the • properties of each registry • relationships between the registries • Capturing the WSDL annotations in UDDI • Present scenario: • Suitable for simple searches ( like services offered by a provider, services that implement an interface, services that have a common technical fingerprint etc.) • Categories are too broad • Automated service discovery (based on functionality) and selecting the best suited service is not possible Flow Description Messaging Network

14. Semantics at Different Layers (contd..) Discovery Publication • Flow Layer: • Why: • Design (composition), analysis (verification), validation (simulation) and execution (exception handling) of the process models • To employ mediator architectures for automated composition, control flow and data flow based on requirements • To employ user interface to capture template requirements and generate template based on that • How: • Using • Functionality/preconditions/effects of the participating services • Knowledge of conversation patterns supported by the service • Formal mathematical models like process algebra, concurrency formalisms like State Machines, Petri nets etc. • Simulation techniques • Present Scenario: • Composition of Web services is static. • Dynamic service discovery, run-time binding, analysis and simulation are not supported directly Flow Description Messaging Network

15. Semantics in WS stack and METEOR-S Discovery Publication MWSCF: Semantic Web Process Composition Framework Flow MWSDI: Scalable Infrastructure of Registries for Semantic publication and discovery of Web Services Description MWSDI: Semantic Annotation of WSDL (WSDL-S) Messaging Network

16. Semantics for Web Services • Data/Information Semantics • What: Formal definition of data in input and output messages of a web service • Why: for discovery and interoperability • How: by annotating input/output data of web services using ontologies • Functional/Operational Semantics • Formally representing capabilities of web service • for discovery and composition of Web Services • by annotating operations of Web Services as well as provide preconditions and effects; Annotating TPA/SLA • Execution Semantics • Formally representing the execution or flow of a services in a process or operations in a service • for analysis (verification), validation (simulation) and execution (exception handling) of the process models • using State Machines, Petri nets, activity diagrams etc. • QoS Semantics • Formally describing operational metrics of a web service/process • To select the most suitable service to carry out an activity in a process • using QoS model [Cardoso and Sheth, 2002] for web services

17. Semantics for Web Service Life-Cycle Execution Development / Description / Annotation WSDL, WSEL DAML-S Meteor-S (WSDL Annotation) BPWS4J, Commercial BPEL Execution Engines, Intalio n3, HP eFlow BPEL, BPML, WSCI, WSCL, DAML-S, METEOR-S (SCET,SPTB) UDDI WSIL, DAML-S METEOR-S (P2P model of registries) Composition Publication / Discovery Data / Information Semantics

18. Semantics for Web Service Life-Cycle Execution Development / Description / Annotation WSDL, WSEL DAML-S Meteor-S (WSDL Annotation) BPWS4J, Commercial BPEL Execution Engines, Intalio n3, HP eFlow Data / Information Semantics BPEL, BPML, WSCI, WSCL, DAML-S, METEOR-S (SCET,SPTB) UDDI WSIL, DAML-S METEOR-S (P2P model of registries) Composition Publication / Discovery

19. Semantics for Web Service Life-Cycle Execution Development / Description / Annotation WSDL, WSEL DAML-S Meteor-S (WSDL Annotation) BPWS4J, Commercial BPEL Execution Engines, Intalio n3, HP eFlow Functional / Operational Semantics BPEL, BPML, WSCI, WSCL, DAML-S, METEOR-S (SCET,SPTB) UDDI WSIL, DAML-S METEOR-S (P2P model of registries) Composition Publication / Discovery

20. Semantics for Web Service Life-Cycle Execution Development / Description / Annotation WSDL, WSEL DAML-S Meteor-S (WSDL Annotation) BPWS4J, Commercial BPEL Execution Engines, Intalio n3, HP eFlow QoS Semantics BPEL, BPML, WSCI, WSCL, DAML-S, METEOR-S (SCET,SPTB) UDDI WSIL, DAML-S METEOR-S (P2P model of registries) Composition Publication / Discovery

21. Semantics for Web Service Life-Cycle Execution Development / Description / Annotation WSDL, WSEL DAML-S Meteor-S (WSDL Annotation) BPWS4J, Commercial BPEL Execution Engines, Intalio n3, HP eFlow Execution Semantics BPEL, BPML, WSCI, WSCL, DAML-S, METEOR-S (SCET,SPTB) UDDI WSIL, DAML-S METEOR-S (P2P model of registries) Composition Publication / Discovery

22. Semantics for Web Service Life-Cycle Execution Semantics Data / Information Semantics QoS Semantics Functional / Operational Semantics Execution Development / Description / Annotation WSDL, WSEL DAML-S Meteor-S (WSDL Annotation) BPWS4J, Commercial BPEL Execution Engines, Intalio n3, HP eFlow Semantics Required for Web Processes BPEL, BPML, WSCI, WSCL, DAML-S, METEOR-S (SCET, SPTB) UDDI WSIL, DAML-S METEOR-S (P2P model of registries) Composition Publication / Discovery

23. METEOR-S components for Semantic Web Services • Discovery Infrastructure (MWSDI) • Semantic Annotation and Discovery of Web Services1 • Semantic Peer-to-Peer network of Web Services Registries 2 • Composer • SCET: Service Composition and Execution Tool 3 • Semantics Process Template Builder and Process Generator4 • QoS Management • Specify, compute, monitor and control QoS (SWR algorithm) 5 • Orchestrator(Under development) • Analysis and Simulation6 • Execution • Monitoring6 • 1 [Sivashanmugam et al.-1], 2 [Verma et al.], 3 [Chandrasekaran et al.], 4 [Sivashanmugam et al.-2], • 5 [Cardoso et al.], 6 [Silver et al.]

24. Service Discovery METEOR-S Web Service Discovery Infrastructure (MWSDI) - uses Functional, Data and QoS semantics

25. METEOR-S Web Service Discovery Infrastructure (MWSDI) - uses Functional, Data and QoS semantics Service Selection

26. The problem in discovery.. Registry is universal and provides non-semantic search Search retrieves lot of services (irrelevant results included) UBR Keyword match, taxonomy • Which service to select ? • How to select?

27. Scalable Solution.. Select relevant registries (semantic filtering) Registries are categorized Ontology Domain Registry Registry is domain specific and supports semantic search Select service(s) of interest

28. Search for services to book an air ticket (using categories)* • unspsc-org: unspsc:3-1 • Travel, Food, Lodging and Entertainment Services • Travel facilitation • Travel agents • Travel agencies • Services: 3 records found. • AirFares Returns air fares from netviagens.com travel agent • Hotel reservations Reservations for hotels in Asia, Australia and New Zealand • Your Vacation Specialists Web enabled vacation information • Providers: 2 records found. * Search carried out in one of the Universal Business Registries

29. Search for services to book an air ticket (using keywords)* • air ticket • 1 record with name air tickets booking • airticket, ticketbooking, airtravel, air travel, travel agent, airticketbooking, air ticket booking, travel agency, travelagency • 0 records were returned • travelagent • 1 record with name travelagent test • 4 services: BookFlight, cancelFlightBooking etc. • Descriptions say that both these services are “XML based Web services” • No URL for WSDL • Travel • 15 records. Purpose/functionality understood from descriptions • 2 services : TravelBooks • 4 services : TravelInformation • 2 services : Reservation and cancallation of travel tickets • 1 service : Emergency Services for travellers • 1 service : Travel documentation and itinerary • 5 services : Description is ambiguous/not present * Search carried out in one of the Universal Business Registries

30. Semantic Discovery: Overview • Annotation and Publication • WSDL file is annotated using ontologies and the annotations are captured in UDDI • Discovery • Requirements are captured as templates that are constructed using ontologies and semantic matching is done against UDDI entries • Functionality of the template, its inputs, outputs, preconditions and effects are represented using ontologies • Use of ontologies • brings service provider and service requestor to a common conceptual space • helps in semantic matching of requirements and specifications

31. Semantic Publication and Discovery Use of ontologies enables shared understanding between the service provider and service requestor For simplicity of depicting, the ontology is shown with classes for both operation and data

32. WSDL-S (WSDL with Semantic Annotation) • Mapping Input and Output Message Parts to Ontology • XML Schema elements used in Input/Output messages do not reflect the semantics of the data involved in Web Service operation • Use of ontologies or standard vocabulary* provides well defined semantics for operational data • Mapping Operations to Ontology • Service selection involves discovering appropriate WSDL description and locating an operation to invoke • Operations with same signature could have different functionalities • Ontology or vocabulary* depicting functionality is used for annotation • Additional tags to represent pre-conditions and effects of each operation • Preconditions and effects are added for each operation • Can be optionally used for service discovery and selection *RosettaNet Business/Technical dictionary or ebXML Core Component catalog/dictionary * Current implementation uses vocabularies The focus of our work is not in developing ontologies for representing functionality/preconditions/effects but to use such ontologies for semantic annotation

33. Annotation Syntax* • Each Operation in WSDL is annotated using an fully qualified attribute name-value pair in the operation element under portType element. The attribute name is operation-concept • Each Message part is annotated using a fully qualified attribute name-value pair in the part element under message element. The attribute name is onto-concept • Preconditions and effects are respectively represented using fully qualified additional tags with the names precondition and effect. These elements have two attributes name (optional) and precondition-concept (or effect-concept). Each operation can have multiple precondition and effectelements. * conforms to extensibility support in WSDL version 1.2

34. WSDL Annotation Example <?xml version="1.0" encoding="UTF-8"?> <wsdl:definitions xmlns:TravelOnto=lsdis.cs.uga.edu//METEORS/TravelServiceOntology.daml ….. > <wsdl:message name="OperationRequest"> <wsdl:part name="in0" type="tns:TravelDetails" LSDISExt:ontoconcept= “TravelOnto:TicketInformation"/> </wsdl:message> <wsdl:portType name="TravelArragement"> <wsdl:operation name="buyTicket" parameterOrder="in0" LSDISExt:operation-concept=“TravelOnto:TicketBooking"> <wsdl:input message="intf:OperationRequest" name="buyTicketRequest"/> <wsdl:output message="intf:OperationResponse" name="buyTicketResponse"/> <LSDISExt:precondition name="ValidCreditCard" LSDISExt:precondition-concept=“TravelOnto:ValidCreditCard"/> </wsdl:operation> </wsdl:definitions>

35. Semantics in UDDI • tModels are used to categorize and characterize service entries in UDDI (limited form of semantics) • Our approach categorizes* (using metadata constructs tModels and CategoryBags) the services in UDDI based on the semantic annotations * similar to [Paolucci et al.]

36. Semantic Categorization of Services in UDDI* For the example discussed earlier: Travel Arrangement Service with two operations buyTicket and cancelTicket Service CategoryBag TmodelKey:OperationalTModelKey, Value:TicketBooking, Name:buyTicket KeyedReferenceGroup (SemanticGroupTModelKey) TmodelKey:InputTModelKey, Value:TicketInformation TmodelKey:OutputTModel, Value:ConfirmationMessage KeyedReferenceGroup (SemanticGroupTModelKey) TmodelKey:OperationalTModelKey, Value:TicketCancellation, Name:cancelTicket TmodelKey:InputTModelKey, Value:TicketInformation TmodelKey:OutputTModel, Value:ConfirmationMessage * conforming to UDDI Version 3 spec [UDDI-v3]

37. Semantic Categorization of Services in UDDI* Functional/Operational Semantics: Operation-ontology mapping in WSDL for buyTicket operation <operation name=“buyTicket” operation-concept=“TravelOnto:TicketBooking”> Service CategoryBag TmodelKey:OperationalTModelKey, Value:TravelOnto:TicketBooking, Name:buyTicket KeyedReferenceGroup (SemanticGroupTModelKey) TmodelKey:InputTModelKey, Value:TravelOnto:TicketInformation TmodelKey:OutputTModelKey, Value:GeneralTradeOnto:ConfirmationMessage KeyedReferenceGroup (SemanticGroupTModelKey) TmodelKey:OperationalTModelKey, Value: TravelOnto:TicketCancellation, Name:cancelTicket TmodelKey:InputTModelKey, Value: TravelOnto:TicketInformation TmodelKey:OutputTModelKey, Value: GeneralTradeOnto: ConfirmationMessage * conforming to UDDI Version 3 spec [UDDI-v3]

38. Semantic Categorization of Services in UDDI* Data/Information Semantics: Input Message part-ontology mapping in WSDL for buyTicket operation <part name=“input1”type=“tns:TravelDetails” onto-concept=“TravelOnto:TicketInformation”> Service CategoryBag TmodelKey:OperationalTModelKey, Value:TicketBooking, Name:buyTicket KeyedReferenceGroup (SemanticGroupTModelKey) TmodelKey:InputTModelKey, Value:TravelOnto:TicketInformation TmodelKey:OutputTModelKey, Value: GeneralTradeOnto:ConfirmationMessage KeyedReferenceGroup (SemanticGroupTModelKey) TmodelKey:OperationalTModelKey, Value:TravelOnto:TicketCancellation, Name:cancelTicket TmodelKey:InputTModelKey, Value: GeneralTradeOnto:TicketInformation TmodelKey:OutputTModel, Value: GeneralTradeOnto:ConfirmationMessage * conforming to UDDI Version 3 spec [UDDI-v3]

39. Semantic Categorization of Services in UDDI* Data/Information Semantics: Output Message part-ontology mapping in WSDL for buyTicket operation <part name=“output”type=“xsd:String” onto-concept=“GeneralTradeOnto:ConfirmationMessage”> Service CategoryBag TmodelKey:OperationalTModelKey, Value:TicketBooking, Name:buyTicket KeyedReferenceGroup (SemanticGroupTModelKey) TmodelKey:InputTModelKey, Value:TravelOnto:TicketInformation TmodelKey:OutputTModelKey, Value: GeneralTradeOnto:ConfirmationMessage KeyedReferenceGroup (SemanticGroupTModelKey) TmodelKey:OperationalTModelKey, Value:TravelOnto:TicketCancellation, Name:cancelTicket TmodelKey:InputTModelKey, Value: GeneralTradeOnto:TicketInformation TmodelKey:OutputTModel, Value: GeneralTradeOnto:ConfirmationMessage * conforming to UDDI Version 3 spec [UDDI-v3]

40. Discovery using UDDI V1 API • Our implementation used UDDI Version 1 API • KeyedReferenceGroups are not supported • Each operation is grouped with its operation-concept, input and output onto-concepts each as a keyedReference in the keyedReferenceVector as tModelKey = “OpTModel” KeyValue = “operation-concept” KeyName = “OpName” tModelKey = “InTModel” KeyValue = “onto-concept” KeyName = “OpName” tModelKey = “OutTModel” KeyValue = “onto-concept” KeyName = “OpName” OpTModel: Key for the tModel representing functional semantics of the operation named “OpName” in a WSDL file linked to the UDDI entry InTModel: Key for the tModel representing semantics of the inputs of the operation named “OpName” in the WSDL OutTModel: Key for the tModel representing semantics of the outputs of the operation named “OpName” in the WSDL operation-concept: Fully qualified Id of a class in a functional ontology represented by OpTModel onto-concept: Fully qualfied Id of a class in a ontology that is used to annotate inputs (or outputs) represented by InTModel (or OutTModel)

41. Summary of Steps in Discovery • Services selection based on the functional requirements • Using operation-ontology mapping • Ranking based on semantic similarity based on input/output semantics of candidate services and requirement template • Using message part-ontology mapping • Optional step includes semantic similarity based on semantics of preconditions/effects of the candidate services and requirement template • Using precondition and effect tags

42. METEOR-S Web Service Composition Framework (MWSCF) - needed for the world where business processes never stop changing.

43. MWSCF Architecture UDDI UDDI UDDI UDDI UDDI UDDI Process Execution 1. Validation and deployment 2. Executing the process using a client Execution Engine Discovery Infrastructure (MWSDI) Process Designer 1. Template Construction - interfaces - services - semantic activity templates - other details 2. Process Generation - Service discovery and selection - Data flow Template Builder Process Generator Process Designer Activity Interfaces Process Templates Ontologies Repositories are used to store 1. Web Service Interfaces 2. Ontologies 3. Process Templates XML Repositories

44. Web Process Life-Cycle Design Create Process WSDL Create Process Template and Add Activities Discovery Find Ontologies & Annotate Activity Requirements Find Matches Add Control Flow Rank Services Select a Service Execution Composition Add to Process Generate Process Data Transformation Validate Syntax Data Flow Execute

45. Template Construction Discovery not needed • Process Template can be constructed with 3 types of activities • Concrete Web Service Implementation • activity is bound to Web service by linking it to a WSDL file and an operation in it • Web Service Interface • activity is bound to a Web service interface by linking it to a interface identifier (which is linked to a WSDL file ) and an operation in it • Semantic Activity Template • activity is semantically enriched by linking it to semantic specifications of its inputs/outputs/functionality/precondition and effects

46. Template Construction QoS requirements are specified too • Process Template can be constructed with 3 types of activities • Concrete Web Service Implementation • activity is bound to Web service by linking it to a WSDL file and an operation in it • Web Service Interface • activity is bound to a Web service interface by linking it to a interface identifier (which is linked to a WSDL file ) and an operation in it • Semantic Activity Template • activity is semantically enriched by linking it to semantic specifications of its inputs/outputs/functionality/precondition and effects

47. Template Construction (contd) • Activities are linked by a control flow constructs • Template can have protocol variables that do not have any assignment • These protocol variables are assigned a value (output of a WS) during process generation • Process generator will handle replacing the protocol variable with a relevant container and message details • Example: <case condition= “ (inventory-availability, '=', 'no')" > will be converted into <case condition = “ bpws:getContainerData('inventoryResponse', 'avail') = 'no' ”> During process generation inventory-availability is manually assigned to output of activity whose output container name is inventoryResponse and the output message part name is avail

48. Activity as Concrete Web Service Implementation Input, output messages, portType (of reserveRoom operation) and targetNamespace details are extracted from the given WSDL <invoke-activity name=“RoomReservation” type=“ServiceImpl” wsdl-URL=“http://lsdis.cs.uga.edu/proj/meteors/wsdl/Hotel.wsdl” operation-name=“reserveRoom” />

49. Activity as Web Service Interface <invoke-activity name=“RoomReservation” type=“WSInterface” tModel-id=“uuid:f4a6574b-49f4-a657-f908-45fa62354d84” operation-name=“reserveRoom” qos-spec=“qos-2” ranking-weight=“ranking-2” discovery-URL=“http://westpoint.cs.uga.edu:8080/registry- server/RegistryServerServlet” />

50. Activity as Web Service Interface <invoke-activity name=“RoomReservation” type=“WSInterface” tModel-id=“uuid:f4a6574b-49f4-a657-f908-45fa62354d84” operation-name=“reserveRoom” qos-spec=“qos-2” ranking-weight=“ranking-2” discovery-URL=“http://westpoint.cs.uga.edu:8080/registry- server/RegistryServerServlet” /> All the services that implement the interface (a WSDL without service/binding/ port details identified by a tModel-id) are bound to the given tModel (using binding template construct in UDDI). Discovery is based on this binding.