infop.lancs.ac.uk/msc/ads/index.htm

MSc Course in Advanced Distributed SystemsSession 3.1a: QoS-aware Middleware: the Challenge of Multimedia http://info.comp.lancs.ac.uk/msc/ads/index.htm

The Challenge of Multimedia • Overview of lecture • Requirements of multimedia in middleware; approaches to the support of multimedia in middleware; case study: Sumo and RM-ODP • Requirements of real-time; real-time middleware; case study: RT CORBA • Additional reading • Blair and Stefani, esp. chapters 1, 3 and 4 (http://info.comp.lancs.ac.uk/msc/ads/BlairandStefani-Part1.pdf) • http://www.comp.lancs.ac.uk/computing/research/sumo/ • www.enterprise-architecture.info/Images/Documents/RM-ODP.pdf • http://www.omg.org/cgi-bin/apps/doc?formal/05-01-04.pdf • http://www.cs.wustl.edu/~schmidt/PDF/orc.pdf • Acknowledgement • Doug Schmidt, UC Irvine, for basis of RT CORBA overview

What is meant by multimedia? • “A multimedia system is one which allows end users to store, transmit, interchange, present, represent and perceive a variety of forms of information (media types) in an integrated manner” Blair and Stefani, 1998

Requirements for multimedia middleware • Support continuous media, both in terms of the programming model, and also in the underlying system infrastructure • Provide both static and dynamic quality of service (QoS) management functions as an integral part of the middleware platform • Support both inter- and intra-media synchronisation in a distributed environment • Support multimedia specific interaction models such as streams and multiparty multimedia communications

Impact on middleware • In general, existing middleware platforms and standards …. • do NOT support continuous media • do NOT support QoS management • do NOT support real-time synchronisation • do NOT offer a full range of interaction types

An overview of some relevant research areas • Extended middleware platforms • DIMMA, ReTINA (Jonathan), Sumo • Adaptation and QoS management • QuO, Agilos, Quasar, DJINN, Lancaster’s Adapt • Multimedia service architectures • IMA MSS (and PREMO), CORBA A/V Streams • Multimedia component frameworks • Simon Gibbs’ framework, VuSystem, CMT, Mash, DirectShow, Java Media Framework, Lancaster’s TOAST

CORBA’s A/V Streams • Architecture for ‘open’ multimedia streaming applications • standardised flow spec and stream setup • supports multipoint topologies • supports multiple data-transfer protocols and endpoints • streams are out of band!

Provocative statements • “All media types are equal” • continuous media should not be a service! • “Communication is a first class entity” • cf. Birman’s ‘communication as an abstract data type’ [Middleware 2000]

Case study: the Sumo project • Aims • Develop an approach to supporting multimedia in the context of RM-ODP • Develop an object-oriented programming model (cf. the RM-ODP Computational Language) to meet the requirements of distributed multimedia programming • Examine system support techniques, focusing on the (soft) real-time requirements of multimedia

Sumo’s object model(cf. RM-ODP) • Objects as unit of encapsulation, offering one or more interfaces (described in IDL) • Interfaces have associated environmental contracts that specify constraints on the use of the object • Objects interact through explicit bindings

Operational interfaces Stream interfaces Signal interfaces Extensions for multimedia #1: additional styles of interface N.B. this approach required extensions to IDL

Extensions for multimedia #2: QoS annotations on interfaces • A contractual approach: • provided clause • what an interface can offer in terms of QoS • required clause • what an object expects from the environment in terms of QoS • Specifying offers/ expectations (in the environmental contract) • based on the emission or reception of signals • development of a simple real-time logic (QL)

Extensions for multimedia #3: explicit bindings • What is explicit binding? • the act of creating a communication channel between compatible interfaces, resulting in the creation of a explicitbinding object • Explicit binding and QoS management • static QoS management functions (e.g. resource reservation) are encapsulated by the creation process • dynamic QoS management functions (e.g. adaptation) are then supported by an interface on the binding object

Extensions for multimedia #4: reactive objects • What are reactive objects? • real-time controllers that maintain a permanent interaction with their environment (accept signals, react, emit signals, etc) • adherence to synchrony hypothesis (instantaneous execution) • Their role in Sumo • real-time synchronisation • (dynamic) QoS management

Â An example: lip synchronisation lip-sync (reactive object) Â QoS monitor (reactive object) video camera video window video stream binding microphone audio stream binding speaker

Example (continued): the extended IDL • speaker interfaces • video window (control interface only) typedef timestamp integer; interface <stream> speakerIn { flowIn audioIn(audio); } interface <signal> speakerControl { signalOut audioReady(timestamp); signalOut audioPresented(); } interface <signal> videoWindowControl { signalOut videoReady(timestamp); signalIn videoDoPresent(); signalOut videoPresented(); }

Example (continued): use of QL • video binding object Provided clause n, (videoBindingOut.videoOut.SE, n+24) (videoBindingOut.videoOut.SE, n) + 1000 AND n, (videoBindingOut.videoOut.SE, n) (videoBindingIn.videoIn.SR, n) + 60 Required clause n, (videoBindingIn.videoIn.SR, n+24) (videoBindingIn.videoIn.SR, n) + 1000

Example (continued): use of Esterel • QoS monitor (cf. equivalent QL statement) module QoSMonitor: input videoSent (integer); videoDelivered (integer), msec; output QoSViolation; [ every videoDelivered do await 24 videoDelivered watching 1000 msec timeout emit QoSViolation end || every videoSent do await videoDelivered watching 60 msec timeout emit QoSViolation end ].

Other aspects of Sumo • Development of a Sumo ORB (later versions: ReTINA, Jonathan) • extended CORBA platform featuring all the extensions given above • novel architecture based on a micro-ORB structure • Development of Sumo CORE • focus on operating system support for continuous media (based on Chorus) • investigation of split level resource management

Sumo and RM-ODP • Support for multimedia • the RM-ODP computational language now includes extensions #1, #2 and #3 above

Expected Learning Outcomes • You should have a clear understanding of the requirements of distributed multimedia and real-time applications and the problems of supporting them in middleware • You should be able to discuss alternative approaches to supporting multimedia in ORBs (service vs. first class) • You should be able to place the above in the general context of the expanding requirements imposed on middleware

infop.lancs.ac.uk/msc/ads/index.htm