Grid Technology A Web Services Globus OGSA & Grid Architecture CERN Geneva April 1-3 2003

1. Grid Technology AWeb ServicesGlobus OGSA& Grid ArchitectureCERN GenevaApril 1-3 2003 Geoffrey Fox Community Grids Lab Indiana University gcf@indiana.edu

2. With Thanks to • Tony Hey my co-speaker and • I adapted presentations from • Marlon Pierce • Dennis Gannon • Globus • Malcolm Atkinson • Davis de Roure

3. Fermilab Experiments 1975-1980 0X 0X 0X0 0X0 Regge Theory 1978 Hadron Jets in 1977 Compared to Feynman Field (Fox) Model E350 -t E260 200 GeV hp

4. Caltech Hypercube JPL Mark II 1985 Chuck Seitz 1983 Hypercube as a cube

5. History New York Times 1984 • One of today's fastest computers is the Cray 1, which can do 20 million to 80 million operations a second. But at $5 million, they are expensive and few scientists have the resources to tie one up for days or weeks to solve a problem. • ``Poor old Cray and Cyber (another super computer) don't have much of a chance of getting any significant increase in speed,'' Fox said. ``Our ultimate machines are expected to be at least 1,000 times faster than the current fastest computers.''(80 gigaflops predicted. Earth Simulator is 40,000 gflops) • But not everyone in the field is as impressed with Caltech's Cosmic Cube as its inventors are. The machine is nothing more nor less than 64 standard, off-the-shelf microprocessors wired together, not much different than the innards of 64 IBM personal computers working as a unit. • The Caltech Hypercube was “just a cluster of PC’s”!

6. History New York Times 1984 • ``We are using the same technology used in PCs (personal computers) and Pacmans,'' Seitz said. The technology is an 8086 microprocessor capable of doing 1/20th of a million operations a second with 1/8th of a megabyte of primary storage. Sixty-four of them together will do 3 million operations a second with 8 megabytes of storage. • Computer scientists have known how to make such a computer for years but have thought it too pedestrian to bother with. • ``It could have been done many years ago,'' said Jack B. Dennis, a computer scientist at the Massachusetts Institute of Technology who is working on a more radical and ambitious approach to parallel processing than Seitz and Fox. • ``There's nothing particularly difficult about putting together 64 of these processors,'' he said. ``But many people don't see that sort of machine as on the path to a profitable result.'‘ • So clusters are a trivial architecture (1984) …… • So architecture is unchanged ; unfortunately after 20 years research, programming model is also the same (message passing)

7. What is a Grid I? • Collaborative Environment (Ch2.2,18) • Combining powerful resources, federated computing and a security structure (Ch38.2) • Coordinated resource sharing and problem solving in dynamic multi-institutional virtual organizations (Ch6) • Data Grids as Managed Distributed Systems for Global Virtual Organizations (Ch39) • Distributed Computing or distributed systems (Ch2.2,10) • Enabling Scalable Virtual Organizations (Ch6) • Enabling use of enterprise-wide systems, and someday nationwide systems, that consist of workstations, vector supercomputers, and parallel supercomputers connected by local and wide area networks. Users will be presented the illusion of a single, very powerful computer, rather than a collection of disparate machines. The system will schedule application components on processors, manage data transfer, and provide communication and synchronization in such a manner as to dramatically improve application performance. Further, boundaries between computers will be invisible, as will the location of data and the failure of processors. (Ch10)

8. What is a Grid II? • Supporting e-Science representing increasing global collaborations of people and of shared resources that will be needed to solve the new problems of Science and Engineering (Ch36) • As infrastructure that will provide us with the ability to dynamically link together resources as an ensemble to support the execution of large-scale, resource-intensive, and distributed applications. (Ch1) • Makes high-performance computers superfluous (Ch6) • Metasystems or metacomputing systems (Ch10,37) • Middleware as the services needed to support a common set of applications in a distributed network environment (Ch6) • Next Generation Internet (Ch6) • Peer-to-peer Network (Ch10, 18) • Realizing thirty year dream of science fiction writers that have spun yarns featuring worldwide networks of interconnected computers that behave as a single entity. (Ch10)

9. What is Grid Technology? • Grids support distributed collaboratories or virtual organizations integrating concepts from • The Web • Distributed Objects (CORBA Java/Jini COM) • Globus Legion Condor NetSolve Ninf and other High Performance Computing activities • Peer-to-peer Networks • With perhaps the Web being the most important for “Information Grids” and Globus for “Compute Grids” • Use Information Grids and not usual Data Grids as “distributed file systems” (holding lots of data!) are handled in Compute Grids

10. PPPH: Paradigms Protocols Platforms and Hosting I • We will start from the Web view and assert that basic paradigm is • Meta-data rich Web Services communicating via messages • These have some basic support from some runtime such as .NET, Jini (pure Java), Apache Tomcat+Axis (Web Service toolkit), Enterprise JavaBeans, WebSphere (IBM) or GT3 (Globus Toolkit 3) • These are the distributed equivalent of operating system functions as in UNIX Shell • Called Hosting Environment or platform

11. Some Basic Observations • Grids manage and share asynchronous resources in a rather centralized fashion • Peer-to-peer networks are “just like” Gridswith different implementations of services like registration and look-up • Web Services interact with messages • Everything (including applications like PowerPoint will be a WS?) – see later short discussion • Computers are fast and getting faster. One can afford many strategies that used to be unrealistic • All messages can be publish/subscribe • Software message routing • XML will be used for most interesting data and meta-data • One will store/consider data and meta-data separately but often use same technology to manage both of them. • Need Synchronous and AsynchronousResource Sharing • IntegrateGrid and Collaboration technology

12. Classic Grid Architecture Database Database Resources Content Access Composition Middle TierBrokers Service Providers Netsolve Security Collaboration Computing Middle Tier becomes Web Services Clients Users and Devices

13. What is a Web Service I • A web service is a computer program running on either the local or remote machine with a set of well defined interfaces (ports) specified in XML (WSDL) • In principle, computer program can be in any language (Fortran .. Java .. Perl .. Python) and the interfaces can be implemented in any way what so ever • Interfaces can be method calls, Java RMI Messages, CGI Web invocations, totally compiled away (inlining) but • The simplest implementations involve XML messages (SOAP) and programs written in net friendly languages like Java and Python • Web Services separate the meaning of a port (message) interface from its implementation • Enhances/Enables Re-usable component model of ANY electronic resource

14. Web Service (WS) WS WS WS WS WS WS RawResources Raw Data Raw Data (Virtual) XML Data Interface WS WS etc. XML WS to WS Interfaces (Virtual) XML Knowledge (User) Interface Render to XML Display Format (Virtual) XML Rendering Interface Clients

15. What is a Web Service II PaymentCredit Card WSDL interfaces Security Catalog Warehouse shipping WSDL interfaces • Web Services have important implication that ALL interfaces are XML messages based.In contrast • Most Windows programs have interfaces defined as interrupts due to user inputs • Most software have interfaces defined as methods which might be implemented as a message but this is often NOT explicit

16. What is a Web Service III • “Everything electronic” is a resource • Computers; Programs; People • Data (from sensors to this presentation to email to databases) • “Everything electronic” is a distributed object • All resources have interfaceswhich are defined in XML for both properties (data-structure) and methods (service, function, subroutine) (Resources are Services) • We can assume that a data-structure property has getproperty() and setproperty(value) methods to act as interface • All resources are linked by messages with structure, which must be specifiable in XML • All resources have a URIsuch asunique://a/b/c …….

17. WSDL Abstractions • WSDL abstracts a program as an entity that does something given one or more inputs with its results defined by streams on one or more outputs. • Functions are defined by method name and parametersmethodname(parm1,parm2, … parmN) • Where parameters are “Input” “Output” or both • In WSDL, we will have a Web Service which like a (Java or CORBA Program) can be thought of as a (distributed) object with many methods • Instead of a function call, the “calling routine” sends an XML message to the Web Service specifying methodname and values of the parameters • Note name of function is just another parameter

18. Details of WSDL Protocol Stack UDDI or WSIL WSFL WSDL SOAP or RMI HTTP or SMTP or IIOP or RMTP TCP/IP Physical Network • UDDI finds where programs are • remote( (distributed) programs are just Web Services • (not a great success) • WSFL links programs together(under revision as BPEL4WS) • WSDL defines interface (methods, parameters, data formats) • SOAP defines structure of message including serialization of information • HTTP is negotiation/transport protocol • TCP/IP is layers 3-4 of OSI • Physical Network is layer 1 of OSI

19. Education as a Web Service • Can link to Science as a Web Service and substitute educational modules • “Learning Object” XML standards already exist from IMS/ADL http://www.adlnet.org – need to update architecture • Web Services for virtual university include: • Registration • Performance (grading) • Authoringof Curriculum • Online laboratories for real and virtual instruments • Homework submission • Quizzes of various types (multiple choice, random parameters) • Assessmentdata access and analysis • Synchronous Delivery of Curricula • Scheduling of courses and mentoring sessions • Asynchronous access, data-mining and knowledge discovery • Learning Plan agentsto guide students and teachers

20. What are System and Application Services? • There are generic Grid system services: security, collaboration, persistent storage, universal access • OGSA (Open Grid Service Architecture) is implementing these as extended Web Services • An Application Web Service is a capability used either by another service or by a user • It has input and output ports – data is from sensors or other services • Consider Satellite-based Sensor Operations as a Web Service • Satellite management (with a web front end) • Each tracking station is a service • Image Processing is a pipeline of filters – which can be grouped into different services • Data storage is an important system service • Big services built hierarchically from “basic” services • Portals are the user (web browser) interfaces to Web services

21. Application Web Services Prog1WS Prog2WS Filter1WS Filter2WS Filter3WS Build as multiple interdisciplinaryPrograms Build as multiple Filter Web Services Sensor Data as a Webservice (WS) Simulation WS Simulation WS Data Analysis WS Data Analysis WS Sensor ManagementWS Visualization WS Visualization WS • Note Service model integrates sensors, sensor analysis, simulations and people • An Application Web Service is a capability used either by another service or by a user • It has input and output ports – data is from users, sensors or other services • Big services built hierarchically from “basic” services

22. The Application Service Model • As bandwidth of communication (between) services increases one can support smaller services • A service “is a component” and is a replacement for a library in case where performance allows • Services (components)are a sustainable model of software development – each service has documented capability with standards compliant interfaces • XML defines interfaces at several levels • WSDLat Service interface level and XSIL or equivalent for scientific data format • A service can be written as Perl, Python, Java Servlet, Enterprise JavaBean, CORBA (C++ or Fortran) Object … • Communication protocol can be RMI (Java), IIOP (CORBA) or SOAP (HTTP, XML) ……

23. Application with W3C DOM Structure as a Web Service Selected W3C DOM Semantic Events W3C DOM Raw(UI) Events W3C DOM User Interface Data Resource Facing Ports Application as a Web service Application Model Remaining W3C DOM Semantic Events MVCM: Model Control User FacingPorts View C:Control Events as Messages Rendering as Messages Application Viewand SelectedControl V: View

24. 7 Primitives in WSDL • types: which provides data type definitions used to describe the messages exchanged. • message: which represents an abstract definition of the data being transmitted. A message consists of logical parts, each of which is associated with a definition within some type system. • operation– an abstract description of an action supported by the service. • portType: which is a set of abstract operations. Each operation refers to an input message and output messages. • binding: which specifies concrete protocol and data format specifications for the operations and messages defined by a particular portType. • port: which specifies an address for a binding, thus defining a single communication endpoint. • service: which is used to aggregate a set of related ports

26. <?xml version="1.0" encoding="UTF-8"?> <wsdl:definitions> <wsdl:message name="execLocalCommandResponse"> <wsdl:message name="execLocalCommandRequest"> <wsdl:portType name="SJwsImp"> <wsdl:operation name="execLocalCommand" parameterOrder="in0"> <wsdl:input message="impl:execLocalCommandRequest" name="execLocalCommandRequest"/> <wsdl:output message="impl:execLocalCommandResponse" name="execLocalCommandResponse"/> </wsdl:operation> </wsdl:portType> <wsdl:binding name="SubmitjobSoapBinding" type="impl:SJwsImp"> <wsdlsoap:binding style="rpc" transport="http://schemas.xmlsoap.org/soap/http"/> <wsdl:operation name="execLocalCommand"> <wsdlsoap:operation soapAction=""/> <wsdl:input name="execLocalCommandRequest"> <wsdl:output name="execLocalCommandResponse"> </wsdl:operation> </wsdl:binding> <wsdl:service name="SJwsImpService"> <wsdl:port binding="impl:SubmitjobSoapBinding" name="Submitjob"> </wsdl:service> </wsdl:definitions>

27. Discussion of 7 WSDL Primitives • types specify data-structures which are equivalent to arguments of methods • message specifies collections of types and is equivalent to set of arguments in a method call. Note that it is an “abstract method” in Java terminology • operationis a a collection of input output and fault messages; there are 4 types of operation one-way(service just receives a message), request-response(RPC), solicit-response, notification (services pushes out a message) • portTyperepresents a single channel that can support multiple operations. It is “abstract” as specified as a set of operations. It is equivalent to a “interface or abstract class” in Java • binding tells you transport and message format for a porttype (which can have multiple bindings to reflect say performance-portability trades) • port combines a binding and an endpoint network address (URL) and is like a “class instance” • service consists of multiple ports and is equivalent to a “program” in Java

28. OGSA OGSI & Hosting Environments • Start with Web Services in a hosting environment • Add OGSI to get a Grid service and a component model • Add OGSA to get Interoperable Grid “correcting” differences in base platform and adding key functionalities

29. Functional Level above OGSA • Systems Management and Automation • Workload / Performance Management • Security • Availability / Service Management • Logical Resource Management • Clustering Services • Connectivity Management • Physical Resource Management • Perhaps Data Access belongs here

30. Two-level Programming I Nugget Data • The paradigm implicitly assumes a two-level Programming Model • We make a Service (same as a “distributed object” or “computer program” running on a remote computer) using conventional technologies • C++ Java or Fortran Monte Carlo module • Data streaming from a sensor or Satellite • Specialized (JDBC) database access • Such nuggets accept and produce data from users files and databases • The Grid is built by coordinating such nuggets assuming we have solved problem of programming the nugget

31. Two-level Programming II Nugget1 Nugget3 Nugget2 Nugget4 • The Grid is discussing the linkage and distribution of the nuggets with the onlyaddition runtime interfaces to Grid as opposed to UNIX data streams • Familiar from use of UNIX Shell, PERL or Python scripts to produce real applications from core programs • Such interpretative environments are the single processor analog of Grid Programming • Some projects like GrADS from Rice University are looking at integration between nugget levels but dominant effort looks at each level separately

32. Why we can dream of using HTTP and that slow stuff • We have at least three tiers in computing environment • Client (user portal discussed Thursday) • “Middle Tier” (Web Servers/brokers) • Back end (databases, files, computers etc.) • In Grid programming, we use HTTP (and used to use CORBA and Java RMI) in middle tier ONLY to manipulate a proxy for real job • Proxy holds metadata • Control communication in middle tier only uses metadata • “Real” (data transfer) high performance communication in back end

33. Portal Services SystemServices SystemServices Application Service Application Metadata Middleware SystemServices SystemServices SystemServices Raw (HPC) Resources Actual Application Database UserServices GridComputingEnvironments “Core”Grid

34. OGSA OGSI & Hosting Environments • Start with Web Services in a hosting environment • Add OGSI to get a Grid service and a component model • Add OGSA to get Interoperable Grid “correcting” differences in base platform and adding key functionalities

35. PPPH: Paradigms Protocols Platforms and Hosting II • Self-describing programs/interfaces are key to scaling • Minimize amount of work system has to do • Hide as much as possible in services and applications • Protocols describe (in “principle” at least) those rules that system obeys and uses to deliver information between services (processes) • Interfaces tell the service what to do to interpret the results of communication • HTTP is the dominant transport protocol of the Web • HTML is the “interface” telling browser how to render • But you can extend interface to allow PDF, multimedia, PowerPoint using “helper applications” which are (with more or less convenience) which are “automatically” downloaded if not already available • “Mime types” essentially self-describe” each interface

36. Analogy with Web II • HTTP and HTML are the analogies on the client side • A “Web Service” generalizes a CGI Script on server side • CGI is essentially a Distributed Object technology allowing server to access an arbitrary program labeled by a URL plus an ugly syntax to specify name and parameters of program to run • Roughly WSDL (Web Service Description Language) is a better to specify program name and its parameters • Web uses other protocols – HTTPS for secure links and RTP etc. for multimedia (UDP) streams • These again are required to integrate system – codecs like MPEG are interfaces interpreted by client • There are further protocols like H323 and SIP which will be placed (IMHO) by HTTP plus RTP etc. We should minimize number of protocols to get maintainable systems

37. PPPH: Paradigms Protocols Platforms and Hosting III • There are set of system capabilities which cannot be captured as standalone services and permeate Grid • Meta-data rich Message-linked Web Services is permeating paradigm • Component Model such as “Enterprise JavaBean” or OGSI describes the formal structure of services • Invocation Framework describes how you interact with system • Security in fine grain fashion to provide selective authorization (Globus and EDG WP6) • Policy context describes rules for this particular Grid • Transport mechanisms abstract concepts like ports and Quality of Service • Messaging abstracts destination and customization of content • Network (monitoring, performance) EDG WP7 • Fabric (resources) EDG WP4

38. Virtualization • The Grid could and sometimes does virtualize various concepts • Location: URI (Universal Resource Identifier) virtualizes URL • Replica management (caching) virtualizes file location generalized by GriPhyn virtual data concept • Protocol: message transport and WSDL bindings virtualize transport protocol as a QoS request • P2P or Publish-subscribe messaging virtualizes matching of source and destination services • Semantic Grid virtualizes Knowledge as a meta-data query • Brokering virtualizes resource allocation • Virtualization implies references can be indirect

39. IFS: Interfaces and Functionality and Semantics I • The Grid platform tries to minimize detail in protocols and maximize detail in interfaces to enhance scaling • However rich meta-data and semantics are critical for correct and interesting operation • Put as much semantic interpretation as you can into specific services • Lack of Semantic interoperation is in fact main weakness of today’s Grids and Web services • Everything becomes a service (See example of education) whether system or application level • There are some very important “Global Services” • Discovery (look up) and Registration of service metadata • Workflow • MetaSchedulers

40. IFS: Interfaces and Functionality and Semantics II • There are many other generally important services • OGSA-DAI The Database Service • Portal Service linked to by WSRP (Web services for Remote Portals) • Notification of events • Job submission • Provenance – interpret meta-data about history of data • File Interfaces • Sensor service – satellites … • Visualization • Basic brokering/scheduling

41. Globus in a Nutshell from IPG • GT2 (or Globus Toolkit 2) is original (non web service based) version which is basis of EDG (European Data Grid) work • C programs and libraries • See Chapter 5 of book with background in chapters 2-4 and 37 • http://www.ipg.nasa.gov/ipgusers/globus/ • http://www.globusworld.org/globusworld_web/jw2_program_tut.htm

42. Globus GT2 from IPG • The goal of the Globus GT2 is to provide dependable, consistent, pervasive access to high-end resources. • This is original Grid “start” general recently to virtual organizations and data grids • The Globus Project offers the most widely used computing grid middleware. The Globus Project is a joint effort of Argonne National Laboratory, the Informational Sciences Institute of the University of Southern California, in collaboration with numerous other organizations including  NCSA, NPACI, UCSD, and NASA. See http://www.globus.org/ for history, goals, release and usage notes, software distributions, and research papers.

43. Globus GT2 II • Grid Fabric: Layer OneThe fabric of the Grid comprises the underlying systems, computers, operating systems, networks, storage systems, and routers—the building blocks. • Grid Services: Layer TwoGrid services integrate the components of the Grid fabric. Examples of the services that are provided by Globus Toolkit 2: • GRAMThe Globus Resource Allocation Manager, GRAM, is a basic library service that provides capabilities to do remote-submission job start up. GRAM unites Grid machines, providing a common user interface so that you can submit a job to multiple machines on the Grid fabric. GRAM is a general, ubiquitous service, with specific application toolkit commands built on top of it • MDSThe Monitoring and Discovery Service, also known as GIS, the Grid Information Service, provides information service. You query MDS to discover the properties of the machines, computers and networks that you want to use: how many processors are available at this moment? What bandwidth is provided? Is the storage on tape or disk? Is the visualization device an immersive desk or CAVE? Using an LDAP (Lightweight Directory Access Protocol) server, MDS provides middleware information in a common interface to put a unifying picture on top of disparate equipment. • Contd …

44. Globus GT2 III • GSI gss-api library for adding authentication to a program. GSI provides programs, such as grid-proxy-init, to facilitate login to a variety of sites, while each site has its own flavor of security measures. That is, on the fabric layer, the various machines you want to use might be governed by disparate security policies; GSI provides a means of simplifying multiple remote logins. The standard installation is based on a PKI security system; the Kerberos installation of Globus is less standard. (Some installations with DoE and DoD insist on Kerberos) • GridFTPA new (in Globus 2.0) protocol for file transfer over a grid. This is a Global Grid Forum standard • GASS Globus Access to Secondary Storage, provides command-line tools and C APIs for remotely accessing data. GASS integrates GridFTP, HTTP, and local file I/O to enable secure transfers using any combination of these protocols..

45. Globus GT2 IV • Application Toolkits: Layer ThreeApplication toolkits use Grid Services to provide higher-level capabilities, often targeted to specific classes of application. • For example, the Globus development team has created a set of Grid service tools and a toolkit of programs for running remotely distributed jobs. These include remote job submission commands ( globusrun, globus-job-submit, globus-job-run), built on top of the GRAM service, and MPICH-G2, a Grid-enabled implementation of the Message Passing Interface (MPI). • A more modern interface is through CoG Kits (Commodity Grid) to different languages – Perl Python Java – see chapter 26 of Book • The Java CoG kit provides a natural way to link GT2 to a Web service framework • Globus Toolkit 3 (GT3) effectively integrated CoG Kit interface with core Globus by wrapping all Globus Services as Web services

46. Job Submission in Globus • Very similar to UNIX Shell – build Portal Web Interfaces to specific or general Shell commands. Some example commands • globusrun Runs a single executable on a remote site with an RSL specification. • globus-job-cancel Cancels a job previously started using globus-job-submit. • globus-job-run Allows you to run a job at one or several remote resources. It translates the program arguments to an RSL request and uses globusrun to submit the job. • globus-job-clean Kills the job if it is still running and cleans the information concerning the job. • globus-job-status Display the status of the job. See also globus-get-output to check the standard output or standard error of your job. • These are all controlled by metadata specified by the Globus Resource Specification Language (RSL) which provides a common language to describe jobs and the resources required to run them. • http://www.globus.org/gram/gram_rsl_parameters.html • The simplest RSL expression looks something like the following. (executable=/bin/ls)

47. Virtual Data Toolkit VDT from GriPhyn • http://www.lsc-group.phys.uwm.edu/vdt/ • Trillium (PPDG from DoE GriPhyn and iVDgL from NSF) is major US effort building Grid application software with a strong particle physics emphasis • VDT is their major software release and its heart is Condor and GT2. • There is some “virtual data” software as well but not clear if this is of interest in production use (interesting research area) • Condor (Chapter 11 of Book) is powerful job scheduler for clusters and “cycle scavenging” • It has a well developed interface (ClassAds) for defining requirements of jobs and matching to compute capabilities

48. OGSA/OGSI Top Level View Domain - specific services Other models More specialized services: data replication, workflow, etc., etc. Broadly applicable services: registry, authorization, monitoring, data & other entities Models for resources access, etc., etc. OGSI Host. Env . & Protocol Bindings Hosting Environment Transport Hosting Environment Protocol Chapters 7 to 9 of Book http://www.gridforum.org/Meetings/ggf7/docs/default.htm http://www.globusworld.org/globusworld_web/jw2_program_tut.htm • OGSA is the set of “core” Grid services • Stuff you can’t live without • If you built a Grid you would need to invent these things

49. OGSI Open Grid Service Interface • http://www.gridforum.org/ogsi-wg • It is a “component model” for web services. • It defines a set of behavior patterns that each OGSI service must exhibit. • Every “Grid Service” portType extends a common base type. • Defines an introspection model for the service • You can query it (in a standard way) to discover • What methods/messages a port understands • What other port types does the service provide? • If the service is “stateful” what is the current state? • A set of standard portTypes for • Message subscription and notification • Service collections • Each service is identified by a URI called the “Grid Service Handle” • GSHs are bound dynamically to Grid Services References (typically wsdl docs) • A GSR may be transient. GSHs are fixed. • Handle map services translate GSHs into GSRs.

50. OGSI and Stateful Services • Sometimes you can send a message to a service, get a result and that’s the end • This is a statefree service • However most non-trivial services need state to allow persistent asynchronous interactions • OGSI is designed to support Stateful services through two mechanisms • Information Port: where you can query for SDE (Service Definition Elements) • “Factories” that allow one to view a Service as a “class” (in an object-oriented language sense) and create separate instances for each Service invocation • There are several interesting issues here • Difference between Stateful interactions and Stateful services • System or Service managed instances