Composite subscriptions in content based pub sub systems
Download
1 / 31

Composite Subscriptions in Content-based Pub/Sub Systems - PowerPoint PPT Presentation


  • 82 Views
  • Uploaded on

Composite Subscriptions in Content-based Pub/Sub Systems. Guoli Li. Agenda. Motivation PADRES System Composite Subscriptions Case Study: Workflow Management Evaluations Conclusions and Future Work. Motivation. A large number of atomic events Lack of event correlation

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Composite Subscriptions in Content-based Pub/Sub Systems ' - phelan-olson


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Agenda
Agenda

  • Motivation

  • PADRES System

  • Composite Subscriptions

  • Case Study: Workflow Management

  • Evaluations

  • Conclusions and Future Work

Middleware 2005 Grenoble France


Motivation
Motivation

  • A large number of atomic events

  • Lack of event correlation

  • Potential Applications:

    • Workflow processing and business process execution

    • Network management

Middleware 2005 Grenoble France


The padres system
The PADRES System

  • PADRES: Publish/Subscribe Applied to Distributed REsource Scheduling

  • Features:

    • Predicate-based subscription language

      • (attribute, op, value) pair

      • Extended to supporting composite subscriptions

    • Rule-based publication/subscription matching

      • Publications are mapped to facts

      • Subscriptions are mapped to rules

    • Advertisement-based routing

Middleware 2005 Grenoble France


Content based routing
Content-based Routing

  • Advertising

Advertisement

Distributed Overlay

Broker Network

PADRES

Middleware 2005 Grenoble France


Content based routing1
Content-based Routing

  • Subscribing

Subscription

Distributed Overlay

Broker Network

PADRES

Middleware 2005 Grenoble France


Content based routing2
Content-based Routing

  • Publishing

Publication

Distributed Overlay

Broker Network

PADRES

Middleware 2005 Grenoble France


Several concepts
Several Concepts

  • Message

    • Advertisement

    • Subscription

    • Publication

  • Event

    • Atomic event: publication in pub/sub literature

    • Composite event: a pattern of events

  • Subscriptions

    • Atomic subscription: atomic event

    • Composite subscription: composite event

Middleware 2005 Grenoble France


Composite subscriptions

&

S5

&

||

||

S1

S2

S3

S4

Composite Subscriptions

  • Composite subscription consists of primitive subscriptions

    • Provide a higher level view for subscribers

    • Save network bandwidth

  • Subscription language features

    • Operators: AND (&), OR (||)

    • Variables: ($x)

    • Advanced event patterns

CS={{S1 OR S2} AND{S3 OR S4} AND S5}

Middleware 2005 Grenoble France


Composite subscriptions1
Composite Subscriptions

  • Parallelization pattern: s1 & s2

  • Alternation pattern: s1 || s2

  • Sequence pattern: (s1;[timespan:ts]s2)[within:wi]

  • Repetition pattern: Repetition(S, n, attr, v)

e.g. {rule ((job (appl = $X) (name = A) (state = succ)) &

(job (appl = $X) (name = B) (state = succ)))

=> (forward a notification to proper destination)}

Middleware 2005 Grenoble France


Composite subscriptions2
Composite Subscriptions

  • Parallelization pattern: s1 & s2

  • Alternation pattern: s1 || s2

  • Sequence pattern: (s1;[timespan:ts]s2)[within:wi]

  • Repetition pattern: Repetition(S, n, attr, v)

e.g. {rule ((job …. (name = A) (time = $Y)) &

(job …. (name = B) (time > $Y+ts) (time < $Y+wi)))

=> (forward a notification to proper destination)}

e.g. {rule ((job …. (name = A) (attr = $Z)) &

(job …. (name = A) (attr = $Z+v)) &

… &

(job …. (name = A) (attr = $Z+(n-1)v)) )

=> (forward a notification to proper destination)}

Middleware 2005 Grenoble France


Composite subscription routing

&

||

S3

S1

S2

Composite Subscription Routing

CS={{S1 AND S2} ANDS3}

P

Publishers

Subscribers

CS’={S1 AND S2}

S

P1

Distributed Overlay Broker Network

CS

B1

B5

S

S1

B3

CS’

B4

S2

S3

B2

B6

P2

P3

Middleware 2005 Grenoble France


Composite event detection

AND

AND

S3

S1

S2

Composite Event Detection

CS={{S1 AND S2} ANDS3}

P

Publishers

Subscribers

S

CS’={S1 AND S2}

P1

P1

Distributed Overlay Broker Network

S1

CS

B1

B5

S

CS’

CS

P12

B3

B4

P123

B2

B6

P2

S2

P2

S3

P3

P3

Middleware 2005 Grenoble France


Advantages
Advantages

  • Rule-based approach naturally enables the composite subscription

  • Event detection results can be shared

  • Network traffic is reduced

    • Composite subscription is forwarded into the network as far as possible before it is split

    • Composite events are detected close to the data source and not widely disseminated

Middleware 2005 Grenoble France


Case study workflow management
Case Study: Workflow Management

  • Decentralized Workflow Management based on Pub/Sub

  • Functions

    • Workflow transform

    • Workflow deployment

    • Workflow execution

    • Workflow control and monitoring

Middleware 2005 Grenoble France


Architecture
Architecture

Middleware 2005 Grenoble France


Workflow descriptions

E

F

Workflow Descriptions

  • Workflows are described using XML-based languages, such as BPEL

  • Activities defined in a workflow have dependencies among each other

  • A workflow instance is generated by a trigger

  • Failures can be handled in the workflow description

trigger

A

B

C

D

Middleware 2005 Grenoble France


Workflow transform
Workflow Transform

  • The first job(s) subscribe to trigger messages

    e.g. Job A subscribes to trigger publication: {[class,eq,trigger],[workflow,eq,payroll], [instanceID,=,$x]}

  • Job dependencies are implied by subscriptions

    e.g. Job E subscribes to job A failed:

    {[class,eq,job_info],[workflow,eq,payroll], [nstanceID,=,$x], [job,eq,A], [status,eq,fail]}

    e.g. Job D subscribes to B and C finished, which is a composite subscription

  • Advertisements are generated from a workflow specification file

Middleware 2005 Grenoble France


Workflow deployment

BPEL

A

B

C

D

Workflow Deployment

sub/advs

Job D

sub/advs

Job C

sub/advs

Job B

sub/advs

Job A

Deployer

sub/advs

Job D

Distributed Overlay Broker Network

Job D

Agent

B1

B5

B3

B4

sub/advs

Job A

B2

Job A

Agent

B6

sub/advs

Job C

sub/advs

Job B

Job C

Agent

Job B

Agent

Middleware 2005 Grenoble France


Workflow execution success

BPEL

A

B

C

D

Workflow Execution (Success)

trigger

Workflow

Manager

END

Distributed Overlay Broker Network

Job D

Agent

B1

B5

BC

B3

A

B4

A

sub/advs

Job D

B2

Job A

Agent

B6

A

sub/advs

Job A

Job C

Agent

sub/advs

Job C

Job B

Agent

B

C

sub/advs

Job B

Middleware 2005 Grenoble France


Workflow execution failure

BPEL

A

E

B

C

F

D

Workflow Execution (Failure)

trigger

Workflow

Manager

sub/advs

Job D

Distributed Overlay Broker Network

Job D

Agent

B1

B5

B3

B4

END

B2

Job A

Agent

B6

Job F

Agent

A

Job C

Agent

sub/advs

Job F

sub/advs

Job A

Job B

Agent

E

Job E

Agent

sub/advs

Job E

sub/advs

Job C

sub/advs

Job B

Middleware 2005 Grenoble France


Workflow monitoring
Workflow Monitoring

  • Monitor is a Pub/Sub client

  • Monitor/control workflow execution status

    • Workflow level

    • Activity level

  • Support dynamic workflow modification

  • Visualize topology of broker overlay network

  • Visualize message routing

Middleware 2005 Grenoble France


Evaluations
Evaluations

  • Environment

    • Intel Xeon 3GHz, 1GB RAM

    • JDK 1.4.2

  • Workload description

    • Predicates based on 20 attributes with random operators

    • Values are uniformly distributed in the value range

    • Two workflows with different number of jobs

  • Metrics

    • Matching time

    • Routing delay per composite subscription

    • Network traffic

Middleware 2005 Grenoble France


Publication matching time
Publication Matching Time

Middleware 2005 Grenoble France


Composite event detection1
Composite Event Detection

Middleware 2005 Grenoble France


Routing delay
Routing Delay

Middleware 2005 Grenoble France


Number of notifications
Number of Notifications

Middleware 2005 Grenoble France


Workflow traffic
Workflow Traffic

Middleware 2005 Grenoble France


Related work
Related Work

  • P.R. Pietzuch, B. Shand, and J. Bacon. Composite event detection as a generic middleware extension. IEEE Network Magazine, Special Issue on Middleware Technologies for Future Communication Networks, Jan 2004

  • SIENA

  • REBECA

Middleware 2005 Grenoble France


Conclusions
Conclusions

  • An expressive subscription language

  • A rule-based approach of message matching and composite event detection

  • Case study: a decentralized workflow management system based on PADRES

  • Future Works

    • Support BPEL4WS

    • QoS based workflow execution

Middleware 2005 Grenoble France


Questions
Questions?

Middleware 2005 Grenoble France


ad