Bpm in cloud architectures business process management with slas and events
This presentation is the property of its rightful owner.
Sponsored Links
1 / 92

BPM in Cloud Architectures: Business Process Management with SLAs and Events PowerPoint PPT Presentation


  • 82 Views
  • Uploaded on
  • Presentation posted in: General

BPM in Cloud Architectures: Business Process Management with SLAs and Events. BPM 2010, Hoboken, NJ. Hans-Arno Jacobsen Bell University Laboratory Chair University of Toronto. http://www.padres.msrg.utoronto.ca. An e QoS ystem for declarative distributed applications with SLAs.

Download Presentation

BPM in Cloud Architectures: Business Process Management with SLAs and Events

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Bpm in cloud architectures business process management with slas and events

BPM in Cloud Architectures:Business Process Management withSLAs and Events

BPM 2010, Hoboken, NJ

Hans-Arno Jacobsen

Bell University Laboratory Chair

University of Toronto

http://www.padres.msrg.utoronto.ca

An eQoSystem for declarative

distributed applications with SLAs

  • http://eqosystem.msrg.org/


Business process example

Business Process Example

Pid=c1

Loan Application Processing

service RTT < 100ms

uptime > 99.99 %

Store inDB

service time < 2s

Reject

< 0.3

gid=c001

Creditcheck 2

< 0.5

Pid=c001

Pid=c3301

Pid=c401

gid=c001

c001 end

Checkscore

Checkscore 2

Creditcheck

Approve

service cost < $0.02

> 0.7

Send toofficer

else

else

BPM 2010, Hoboken, NJ


Large scale business processes

Large-scale Business Processes

Vendor

Goods

selection

Goods delivery

Dispatch B

Packaging

Pick-up goods

  • Case Study (Chinese Electronics Manufacturer):

  • Department-level processes with 26 to 47 activities

  • Global processes that compose departmental ones

  • Thousands of concurrent instances

  • Hundreds of collaborating partners

  • Geographically distributed

  • Administrative boundaries

Out-stock B

FedEx

Delivery

Pick up

Sale

prediction

Sign

Contract

Sale

Fill order

Determinate

plan

Process

Check order

CCC

administrate

Fill out-stock

bill

Check stock

Manufactory

Confirm

features

Design

Fill dispatch

bill

Determinate

plan

Control

Prototype

Out

Take

Raw

materials

Execute

plan

Warehouse

Material

Out-stock B

Pay

Credit card

Check

Assign

Audit

Process

control

Make plan

Target price

Signature

Raw

Check dealer

Check credit

Finance

Confirm

Approval

Approval

Monitoring

Feature

selection

Print receipt

Validate

Statistic

Monitor

Marketing

Requirement

collection

Feedback

Affirm order

Chart

Strategy

Design

Marketing

Manufactory

Order

Payment

BPM 2010, Hoboken, NJ


What support is required

What Support is Required ?

  • De-coupling and loose coupling

  • Fine-grained event filtering

  • In-network event processing

  • Composite event detection

  • Event correlation

BPM 2010, Hoboken, NJ


Conflicting goals

Conflicting Goals

End user:Performance

Administrator:Cost, utilization

Developer:Flexibility, simplicity

BPM 2010, Hoboken, NJ


Agenda

Agenda

  • Enabler

  • The PADRES approach

  • Event-driven BPM with PADRES

  • SLA-driven BPM with PADRES

BPM 2010, Hoboken, NJ


What abstractions enable bpm

What Abstractions Enable BPM?

BPM 2010, Hoboken, NJ


What abstractions do not work

What Abstractions Do Not Work?

Cum grani salis

  • Databases

    • Great for managing historic data

    • But what about future data (e.g., events)

  • Data streams

    • Great for managing structured streams of tuples

    • But what about un-structured, multi-typed, sporadic, un-ordered events from many sources

  • Rule-based expert systems

    • Great for inference and reasoning

    • But what about managing large numbers of fined-grainedfilters in distributed environments

BPM 2010, Hoboken, NJ


What abstractions enable bpm1

What Abstractions Enable BPM?

  • It is our opinion that the afore-mentioned requirements can best be addressed by

    • The content-based publish/subscribe paradigm

    • Realized by content-based message routing

  • Events represent state transitions in the environment.

    • Conveyed as publications to the pub/sub system

  • Event filtering and correlation is based on

    • Subscriptions managed by the pub/sub system

BPM 2010, Hoboken, NJ


Content based publish subscribe

B

B

Matching Engine

&

B

Routing Table

Output

queue

d2

Input queue

Notification

Notification

subscription

dest

Class = Loan d2

B

Service RTT < 2s d3

B

Output

queue

d3

Content-based Routing

Content-based Publish/Subscribe

3. Publish

Publisher

Publisher

Publications

Service = credit check

1. Advertise

class = Loan,

status = approved,

Event-Based

Content Routing

Flexible

Service RTT < 2s

amount > $500K

10

Decoupled

Declarative

Responsive

Uptime > 99.99%

Subscriptions

Service RTT < 150ms

2. Subscribe

Subscriber

Subscriber

BPM 2010, Hoboken, NJ


Application modeling

Application Modeling

  • Advertisements (schema or types)

    • [class=Loan], [action=*], [customerID=*], [amount<$100K], [region=East]

  • Publications & events (data)

    • [class, Loan], [action, request],

      [customerID, 876594], [amount, $50K]

  • Subscriptions (query)

    • [class=Loan], [amount>$500K], [region=*]

      Application semantics is expressed via

      advertisements (data sources), publications

      (data sources), and subscriptions (data sinks)

S

A

P

BPM 2010, Hoboken, NJ


Benefits of publish subscribe

Benefits of Publish/Subscribe

  • Simplifies IT development and maintenance by decoupling enterprise components

  • Supports sophisticated interactions among components using expressive subscription languages

  • Supports fine-grained subscriptions for event processing

  • Achieves scalability with in-network filtering and processing

BPM 2010, Hoboken, NJ


Many applications are event based benefit from publish subscribe

A

E

B

C

F

D

Many Applications are Event-based &Benefit from Publish/Subscribe

Workflows, business processes and job scheduling

Supply chain and logistics

Job A done

In flight

Triggered

Delivered

Fault!

Ordered

Event-Based

Callback

RFID: Razor SKU

Invoke Loan

Light

Temperature

Transform

Service oriented architectures

RFID and sensor networks

BPM 2010, Hoboken, NJ


Our approach

Our Approach

SLA model

An SLA model can communicate goals to cloud infrastructure providers.

Process monitoring

Distributed execution

Service selection

An event-based system is an enabling technology for a number of the listed features.

Content-based routing & messaging middleware

BPM 2010, Hoboken, NJ

14


Our padres esb for event driven bpm

Our PADRES ESB for Event-driven BPM

Enterprise Services Bus

(Events & Services Bus)

  • First generation of students, when I looked away 

    • Peng Alex David aRno Eli Serge

  • PADRES is Publish/subscribe Applied to Distributed

    Resource Scheduling

  • PAdres is Distributed REsource Scheduling

  • http://www.padres.msrg.utoronto.ca

Web start & open source

padres.msrg.org

Implemented in Java

Acknowledgements (2004-present):

15

BPM 2010, Hoboken, NJ


Our padres esb stack vision

Our PADRES ESB Stack & Vision

BPM 2010, Hoboken, NJ


Padres esb is an overlay of p s brokers

PADRES ESB is an Overlay of P/S Brokers

dest2

Matching Engine

B

+

Publications

Routing Table

B

B

dest1

output queue dest2

output queue dest1

input queue

subscription

dest

B

dest3

output queue dest3

S

P

service time < 3s dest2

service time < 2s dest3

service time = 3s

service time = 2.5s

service time = 1s

S

P

= publisher

= subscriber

Try out and download at:

http://www.padres.msrg.org

S

BPM 2010, Hoboken, NJ


Innovative padres features

A

E

B

C

F

D

Innovative PADRES Features

HistoricAccess

CSRG TR 2009

ACM DEBS’2007

Management

CompositeEvents

ACM Middleware’2007

ACM Middleware’2004

IEEE ICDCS’2005

IEEE ICDCS’2009

ACM Middleware’2008

ACM DEBS’2007

Security

Robustness

IEEE ICDCS’2010

ACM Middleware’2006

LoadBalancing

BPM 2010, Hoboken, NJ


Event driven bpm with padres

Event-driven BPM with PADRES

But not

as complex

BPM 2010, Hoboken, NJ


Modeling business processes

A

B

C

D

Exception

A

E

E

B

C

F

F

D

B

C

D

Modeling Business Processes

  • Dependency in processes and more complex process patterns require event correlation

  • Event correlation is enabled by the detection of composite events

  • Composite events are expressed via composite subscriptions

  • Composite subscription consists of atomic subscriptions

  • Subscription language features for BPM modeling

    • E.g., AND, OR, and variables ($x)

  • Example: D executes, if BandC

    have completed (D depends on

    B and C).

BPM 2010, Hoboken, NJ


Composite subscription examples

Composite Subscription Examples

  • Expresses a structural property of a process

    [class = Activity Status], [cmd. = Archived],[Process ID = $X] AND

    [class = Activity Status], [cmd. = Signed Off],[Process ID = $X]

  • Expresses a performance property of a process

    [cmd. = Credit check request], [Process ID = $X] AND

    [status = Approved], [Process ID = $X]

Loan Process

Archive

Process

Approve

> $50K

Checkscore

Creditcheck

Singed Off

Reject

BPM 2010, Hoboken, NJ


Business process management

E

F

Business Process Management

  • Transformation of process into pub/sub language

  • Deployment of transformed process

  • Execution of process by triggering instances

  • Monitor process & instance execution

  • Manage, i.e., control, version,…

trigger multiple

instances concurrently

trigger

Exception & compensation

A

B

C

D

BPM 2010, Hoboken, NJ


Business process execution

BPEL

Receive

Assign

Flow

Invoke

Wait

Reply

Business Process Execution

END

WS Gateway Agent

WS client

PADRES ESB

6

Invoke

1

3

4

Web Service

2

5

Receive

Assign

Web Service

Reply

Wait

pub/sub

http/soap

BPM 2010, Hoboken, NJ


Bpel transformation example

BPEL Transformation Example

Sub1 (flow agent):

[class = ACTIVITY_STATUS],

[process = ’Process5’],

[activityName = ’activity1’],

[IID isPresent any],

[status = ’SUCCESS’]

Sub4 (actvity6 agent):

( [class = ACTIVITY_STATUS],

[process = ’Process5’],

[activityName = ’activity5’],

[IID = $X],

[status = ’SUCCESS’] )

AND

( [class = LINK_STATUS],

[process = ’Process5’],

[activityName = ’activity2’],

[IID = $X],

[status isPresent any] )

Sub1:

[class,eq,ACTIVITY_STATUS],

[process,eq,’Process5’],

[activityName,eq,’activity1’],

[IID,isPresent,any],

[status,eq,’SUCCESS’]

Pub3:

[class,LINK_STATUS],

[process,’Process5’],

[activityName,’actitiy2’],

[IID,’g001’],

[status,’POSITIVE’]

Sub5:

[class,eq,ACTIVITY_STATUS],

[process,eq,’Process5’],

[activityName,eq,’activity4’],

[IID,isPresent,any],

[status,eq,’SUCCESS’]

&&

[class,eq,ACTIVITY_STATUS],

[process,eq,’Process5’],

[activityName,eq,’activity7’],

[IID,isPresent,any],

[status,eq,”SUCCESS”]

Process 5

activity1

Pub1:

[class, ACTIVITY_STATUS],

[process,’Process5’],

[activityName,’flow1’],

[IID,’g001’],

[status,’STARTED’]

Sub4:

[class,eq,ACTIVITY_STATUS],

[process,eq,’Process5’],

[activityName,eq,’activity2’],

[IID,eq,$X],

[status,eq,’SUCCESS’]

&&

[class,eq,LINK_STATUS],

[process,eq,’Process5’],

[activityName,eq,’activity2’],

[IID,eq,$X],

[status,isPresent,any]

flow1

activity2

activity5

activity3

activity6

Sub2:

[class,eq,ACTIVITY_STATUS],

[process,eq,’Process5’],

[activityName,eq,’flow1’],

[IID,isPresent,any],

[status,eq,’STARTED’]

Pub5:

[class, ACTIVITY_STATUS],

[process,’Process5’],

[activityName,’actitiy7’],

[IID,’g001’],

[status,’SUCCESS’]

activity4

activity7

activity8

Pub2:

[class, ACTIVITY_STATUS],

[process,’Process5’],

[activityName,’actitiy2’],

[IID,’g001’],

[status,’SUCCESS’]

Pub4:

[class, ACTIVITY_STATUS],

[process,’Process5’],

[activityName,’actitiy6’],

[IID,’g001’],

[status,’SUCCESS’]

Cf. our ACM Trans Web’2010

for full BPEL mapping

BPM 2010, Hoboken, NJ


Evaluation changing request rate

Evaluation: Changing Request Rate

P/S

Clustered

20 servers

P/S

Centralized

P/S

P/S

P/S

Distributed

PADRES ESB

BPM 2010, Hoboken, NJ


Sla driven bpm

SLA-driven BPM

More

organized!

An eQoSystem for declarative

distributed applications with SLAs

  • http://eqosystem.msrg.org/

BPM 2010, Hoboken, NJ


Bpm in cloud architectures business process management with slas and events

Currently, business goals must be manually considered at every stage of the business process development cycle

Only trusted partners

service time < 3s

Find flight

Y

Far?

Validaterequest

Getdestination

Find train

N

cost < $0.02

BPM 2010, Hoboken, NJ


Service level agreements slas

Service Level Agreements (SLAs)

SLAs are contracts between service consumers and providers that specify the expected behavior of each party and the penalties of violating the contract.

SLAs specify business goals declaratively.

BPM 2010, Hoboken, NJ


Runtime uses of slas

p

q

A

B

C

D

Runtime Uses of SLAs

Process

Dynamic service discovery Discover services with capabilities that satisfy goals.

MonitoringOnly monitor the business events related to goals.Feed back measurements to support runtime adaptations.

Distributed executionFine-grained allocation of process to available resources.Move portions of process to strategic locations.

ESB adaptationReconfigure the ESB topology to satisfy goals.

ESB broker topology

C

A,B

service time < 2s

1a

M

service time < 1s

1b

D

2

Web service

Execution engine

ESB node (PADRES broker)

M

Monitor

BPM 2010, Hoboken, NJ


Distributed execution example

Distributed Execution Example

  • Execution of one process

  • instance incurs 5 events

    • 3 from P to T

    • 2 from T to S

  • In practice many concurrent

  • instances (a rate)

Process

Activity

P

T

S

Execution engine

hosting activity P

ESB broker topology

T

P

2

5

8

1

4

7

S

3

6

9

Broker

Execution engine

BPM 2010, Hoboken, NJ


Sla cost function example

SLA & Cost Function Example

  • SLA is minimize traffic

  • SLA metric is message hops

  • Minimize message hops

    • f()= msg_hops_rate= msg_rate * engine_distance

  • Measure the msg_hops_rate between activities

  • Measure the distance between execution engines

  • Strive to find better and better deployments subject to execution constraints

T

P

2

5

1

4

BPM 2010, Hoboken, NJ


Sla management stack

RedeploymentManager (CASCON)

SLAs, cost models

Estimators

Ranking algorithms

SLA Management Stack

Execution Engine

Execution

Engine

Candidate Engine Discovery

(ACM DEBS’2009)

Activity Profiler

P Q R

Activities

Engine Profiler

P/S

broker

Activity Manager

Latency

Bandwidth

Engine resource

Activities

(ACM TWEB’2010)

Instance states

Atomic Redeployer

(IEEE ICDCS’2009)

Input, output queues

...

PADRES messaging layer

BPM 2010, Hoboken, NJ


Cost model components

Cost Model Components

The cost of a process is based on metrics

1. Distribution cost Cdist = f(Cd3, Cd1, Cd2)

Cd1 Message rate

Cd2 Message size

Cd3Latency

2. Engine cost Ceng = f(Ce1, Ce2,Ce3)

Ce1 Load (number of instances)

Ce2 Resources (CPU, memory, etc.)

Ce3 Activity complexity

3. Service cost Cserv = f(Cs1, Cs2, Cs3)

Cs1 Latency of external service

Cs2 Execution time of external service

Cs3 Marshalling/unmarshalling

Cost(activity) = f(wiCi)

Cost(process) = ∑cost(activity)

These metrics can be weighted to achieve different objectives

Optimize time

wd1 = wd3 = we3 = wserv = 1, other wi = 0

Optimize network overhead

wd2 = wd3 = 1, other wi = 0

Various optimization criteria can be specified

Threshold criteria: ∑wiCi > x

E.g., Report SLA violations within 3 s.

Minimized criteria:min( ∑wiCi )

E.g., Minimize distribution overhead

BPM 2010, Hoboken, NJ


Redeployment manager

Redeployment Manager

  • Estimator: Computes an estimate of the metric cost ck(ai,ej) of hosting an activity ai at engine ej

  • Cost model: Computes an estimate of the cost c(ai,ej) of hosting activity ai on engine ej

  • Check deployment: Determines what to do with an activity ai

    • Determine best engine e

    • Compute benefit: c(ai) – c(ai,e)

    • Compute resident time at current engine

    • If resident long enough

      • If benefit is large enough move ai to e

      • Otherwise, apply pressure to other activities

BPM 2010, Hoboken, NJ


Evaluation and design issues

Evaluation and design issues

BPM 2010, Hoboken, NJ


Process hotspot illustration

Process Hotspot – Illustration

Process

p = 90%

q = 10 %

B

G

A

D

E

F

I

C

H

Red activities

are pinned to

brokers

ESB broker topology

SLAMinimize traffic

AB

2

5

8

GI

D

1

4

7

F

MetricMessage hops

E

C

3

6

9

H

Broker

Execution engine

BPM 2010, Hoboken, NJ


Process hotspot results

Process Hotspot – Results

10% of

static

40

  • Traffic with redeployment is 47% of the static case

    • Post-redeployment traffic is 10%

    • Redeployment triggered in about 30 sec

BPM 2010, Hoboken, NJ


Varying hotspot illustration

Varying Hotspot – Illustration

Process

p

q

B

G

A

D

E

F

I

C

H

ESB broker topology

AB

2

5

8

GI

Red activities

are pinned to

brokers

D

1

4

7

F

E

C

3

6

9

H

Broker

Execution engine

BPM 2010, Hoboken, NJ


Varying hotspot results

Varying Hotspot – Results

Dynamic redeployment suffers temporarily after process hotspot moves

Traffic with redeployment is 42% of the static case

BPM 2010, Hoboken, NJ


Larger process results

Larger Process - Results

  • Larger process with several parallel branches

  • Traffic with redeployment is 48% of the static case

    • 14% in steady state

BPM 2010, Hoboken, NJ


Summary on sla driven bpm

Summary on SLA-driven BPM

  • Distributed execution engine has qualitative and quantitative advantages

  • Redeployment algorithm can optimize SLA in many cases

  • Challenges and design issues

    • Deployment stuck in local optima

    • Coordination among re-deployment decisions

    • SLA granularity

    • Interference of processes

BPM 2010, Hoboken, NJ


Benefits of content based publish subscribe for bpm

Benefits of Content-based Publish/Subscribe for BPM

  • Naturally enables centralized and distributed business process coordination

  • Coordination can span administrative domains and physically distributed resources

  • Supports process orchestration and choreography

  • Monitoring & control is integral part of paradigm

  • Agile on the fly process adaptation and versioning

  • Correlation of application events with low-level infrastructure events

BPM 2010, Hoboken, NJ


Summary the padres stack

Summary: The PADRES Stack

Download PADRES @ http://padres.msrg.org

BPM 2010, Hoboken, NJ


Outlook

Outlook

Enabled by the PADRES Events & Services Bus

  • Ad hoc and dynamic business processes

  • Business performance management

  • Business impact assessment

  • Business entity management

BPM 2010, Hoboken, NJ


Conclusions

Conclusions

  • Many applications are inherently event-driven.

  • Effective BPM requires capable event processing abstractions.

  • Content-based publish/subscribe is a powerful event processing abstraction and paradigm.

  • PADRES is based on the pub/sub paradigm.

  • PADRES is an ESB targeted at event-based BPM.

  • PADRES enables real-time business analytics and business activity monitoring.

BPM 2010, Hoboken, NJ


Acknowledgements current padres team

Acknowledgements: Current PADRES Team

  • Chen Chen

  • Alex Cheung

  • Alton Chiu

  • Amer Farroukh

  • Patrick Lee

  • Guoli Li

  • Bala Maniymaran

  • Vinod Muthusamy

  • Reza Sherafat

  • Naweed Tajuddin

  • Chunyang Ye

  • Young Yoon

Countless alumni (see our web site)

http://padres.msrg.org

BPM 2010, Hoboken, NJ


Questions discussion

P

A D R E S

Questions & Discussion?

BPM 2010, Hoboken, NJ


Selected literature about padres

Selected Literature about PADRES

  • All our papers are available from

    • http://msrg.org/tags/padres & hyperlinked below

  • A Distributed Service Oriented Architecture for Business Process Execution. ACM Trans. on the Web, 2010.

  • Publisher Placement Algorithms in Content-based Publish/Subscribe. IEEE ICDCS, 2010.

  • Transactional Mobility in Distributed Content-Based Publish/Subscribe Systems. IEEE ICDCS, 2009.

  • Distributed Automatic Service Composition in Large-Scale Systems. ACM DEBS, 2008.

  • Adaptive Content-based Routing in General Overlay Topologies.ACM Middleware, 2008.

  • Historic Data Access in Publish/Subscribe. ACM DEBS, 2007.

  • Papers on managing SLAs with PADRES

    • http://msrg.org/tags/sla

BPM 2010, Hoboken, NJ


References

References

DEBS Conference

http://www.debs.org

July 2011 @ IBM in New York, US

  • The PADRES ESB project home

    • http://padres.msrg.utoronto.ca

  • An eQoSystem for declarative distributed applications with SLAs

    • http://research.msrg.utoronto.ca/Eqosystem/

  • The Micro-ToPSS event processing middleware for sensor networks (RFID)

    • http://microToPSS.msrg.utoronto.ca/

  • Mobile-ToPSS – publish/subscribe for mobile and location-based applications

    • http://research.msrg.utoronto.ca/Mobile/

  • ToPSS - the Toronto Publish/Subscribe System Family

    • http://www.ToPSS.biz (coming soon  )

  • Quantifying events in software to increase modularity & customization in C-based systems and software-based product lines

    • http://www.AspeCtC.net (ACC - the AspeCt-oriented C compiler)

  • The Middleware Systems Research Group

    • http://www.msrg.utoronto.ca

  • My web site

    • http://www.eecg.toronto.edu/~jacobsen

@ the University

of Toronto

BPM 2010, Hoboken, NJ


Bpm in cloud architectures business process management with slas and events

BPM 2010, Hoboken, NJ


Additional slides

Additional slides

BPM 2010, Hoboken, NJ


Local optima illustration

Local Optima – Illustration

Process

p = 90%

q

B

G

A

D

E

F

I

C

H

ESB broker topology

DE

ABC

2

5

8

F

1

4

7

GHI

3

6

9

Red activities

are pinned to

brokers

Broker

Execution engine

BPM 2010, Hoboken, NJ


Local optima problem

Local Optima – Problem

  • Handcrafted static deployment shows effect of being stuck in a locally optimal deployment

  • Finding non-local optima increases complexity

    • Becomes centralized at the limit

p = 90%

q

D

E

F

ooo

ooo

DE

ESB broker topology

ABC

2

5

8

F

1

4

7

3

6

9

GHI

BPM 2010, Hoboken, NJ


Local optima initial conditions

Local Optima – Initial Conditions

ABC

2

5

8

D

F

1

4

7

E

3

6

9

GHI

58

  • If D moves to 7 first

    • Average 34.4 messages per instance

  • If E moves to 8 first

    • Average 36.3 messages per instance

BPM 2010, Hoboken, NJ


Local optima solution illustration

Local Optima – Solution Illustration

Process

p

q

B

G

A

D

E

F

I

C

H

ESB broker topology

DE

ABC

2

5

8

F

1

4

7

GHI

3

6

9

Red activities

are pinned to

brokers

Broker

Execution engine

BPM 2010, Hoboken, NJ


Local optima set optimization solution

Local Optima – Set Optimization Solution

ESB Fragment

8

1

4

7

7

4

1

60

  • Static

    • Activities D and E remain at engine 8

    • Average 18.6 messages per instance

  • Dynamic

    • Activities D and E move together to engines 7, 4, 1

    • Average 9.1 messages per instance (over entire run)

BPM 2010, Hoboken, NJ


Local optima algorithm extensions

Local Optima – Algorithm Extensions

  • Compute the cost of deploying contiguous local activities ai at candidate engines ej

  • Compute the cost of deploying contiguous local activities ai at candidate engines ej

  • Compute the cost of deploying contiguous local activities ai at candidate engines ej

Measurements

C(ai): Cost at local engine

E(P(ai)): Location of predecessors

E(S(ai)): Location of successors

Cost Model

Given

F(ai): Cost function

P(ai): Predecessors

S(ai): Successors

Complexity

Memory: O( |ai|d |ej| )

Computation: O( |ai|d |ej| |Navg(ai)| )

Compute

C(ai1,ej) + … + C(aid,ej) for ai1, …, aid, ej:

Estimated cost of deploying d contiguous activities at candidate engines

BPM 2010, Hoboken, NJ


Convergence illustration

Convergence – Illustration

Process

p

q

B

G

A

D

E

F

I

C

H

ESB broker topology

*

2

5

8

F

Red activities

are pinned to

brokers

1

4

7

3

6

9

Broker

Execution engine

BPM 2010, Hoboken, NJ


Convergence results

Convergence – Results

63

Increasing the candidate space to 2-hop neighbours stabilizes in half the time

Tradeoff convergence speed with complexity

Larger candidate space can also help escape local minima

BPM 2010, Hoboken, NJ


Sensitivity problem

Sensitivity – Problem

Initially p = 0.9, q = 0.1

Dynamic algorithm adapts properly

Between instances 75-80 there is a short blip where q = 0.999

While short, this is significant and causes activity E to move to engine 7

BPM 2010, Hoboken, NJ


Sensitivity solution

Sensitivity – Solution

By examining the distribution of messages we can discard outliers

In this case, the algorithm does not react to the temporary spike

There are fewer movements

The steady state cost is lower

BPM 2010, Hoboken, NJ


Common denominator

BPM applications, such as business processes, are driven by asynchronous state transitions.

Something happens, … an appropriate reaction is expected and required.

Asynchronous state transitions represent events.

A process is triggered, a request submitted, …

BPM applications require event management and processingcapabilities to run effectively.

Common Denominator

BPM 2010, Hoboken, NJ


Example

Example

  • Atomic subscription for Credit Check agent

    [class = Activity Status], [cmd. = Trigger], [Process ID = $X]

  • Composite subscriptions

    [class = Activity Status], [cmd. = Archived],[Process ID = $X] AND

    [class = Activity Status], [cmd. = Signed Off],[Process ID = $X]

    [cmd. = Credit check request], [Process ID = $X] AND

    [status = Approved], [Process ID = $X]

Structural property

of process

Performance property

of process

Loan Process

Archive

Process

Approve

> $50K

Checkscore

Creditcheck

BPM 2010, Hoboken, NJ

Send toofficer

Reject


Application modeling1

Application Modeling

  • Advertisements (schema or types)

    • A1: [class,=,reading], [shipID,=,*], [level,<,10]

    • A2: [class,=,manifest], [shipID,=,*], [firm,=,*], [content,=,*]

    • A3: [class,=,audit], [firm,=,*], [trust,>=,0]

  • Publications & events (data)

    • [class, reading],[shipID,ACME123],[level, 4](induced from A1)

    • [class, manifest],[shipID,ACME123],[firm,ACME](induced from A2)

    • [class, reading],[shipID,ACME123],[level, 12] (not induced from A1)

  • Subscriptions (query)

    • [class,=,reading], [level,>,3]

    • [class,=,audit], [firm,=,*],[trust, >=, 3]

BPM 2010, Hoboken, NJ


Business process deployment

BPEL

Receive

Assign

Flow

Invoke

Wait

Reply

Business Process Deployment

sub/advs

Job D

sub/advs

Job C

sub/advs

Job B

sub/advs

Receive

Deployer

sub/advs

Invoke

PADRES Broker Overlay

6

Invoke

1

3

4

sub/advs

Receive

2

5

receive

sub/advs

Reply

sub/advs

Assign

Reply

Assign

Activity Agent

Process Deployer

BPM 2010, Hoboken, NJ


What abstractions do not work1

What Abstractions Do Not Work?

Cum grano salis

  • Databases

    • Great for managing historic data

    • But what about future data (e.g., events)

  • Data streams

    • Great for managing structured streams of tuples

    • But what about un-structured, multi-typed, sporadic events from many sources

  • Rule-based expert systems

    • Great for inference and reasoning

    • But what about managing large numbers of fined-grainedfilters in distributed environments

BPM 2010, Hoboken, NJ


The toronto publish subscribe system family topss

The Toronto Publish/Subscribe System Family (ToPSS)

  • Matching algorithms

    • Language expressiveness vs. efficient matching

  • Routing protocols

    • Network architectures & scalability

  • Higher level abstractions

    • Process execution

    • Monitoring

A-ToPSS

(approximate)

ToPSS

(matching)

CS-ToPSS

(composite subs)

S-ToPSS

(semantic)

L-ToPSS

(location-based)

Rb-ToPSS

(rule-based)

X-ToPSS

(XML matching)

persistent-ToPSS

(subject spaces)

M-ToPSS

(mobile)

P2P-ToPSS

(peer-to-peer)

LB-ToPSS

(load balancing)

Federated-ToPSS

(federation of ToPSS brokers)

Ad hoc-ToPSS

(ad hoc networking)

Historic-ToPSS

(historic data)

FT-ToPSS

(fault tolerance)

JS-ToPSS

(job scheduling)

BPEL-ToPSS

(BPEL execution)

BPM 2010, Hoboken, NJ


Historic query processing

Historic Query Processing

BPM 2010, Hoboken, NJ


Historic query with event correlation

P

= publisher / sensor

= subscriber / doctor

S

Historic Query with Event Correlation

Monitoring patients who still have fever after taking Advil

heart rate = 150 bpm

P

[medication = NoNameDrugX]

Medical records database

[medication = NoNameDrugX] & [blood pressure > 100] & [heart rate > 130]

[blood pressure > 100]

B

[heart rate > 130]

S

B

medication = NoNameDrugX

Monitoring patients who still have high blood pressure after taking NoNameDrugX

P

S

B

B

B

temperature = 38°C

[temperature > 42] & [medication = Advil]

P

S

Monitoring patients having a fever but not having a cold

B

temperature = 40°C

[heart rate > 140 bpm] & [temperature > 39] & [illness != cold]

B

P

P

illness = diabetes

blood pressure = 60 mmHg

BPM 2010, Hoboken, NJ


Experimental results

Experimental results

BPM 2010, Hoboken, NJ


Publish subscribe

Publish/Subscribe

BPM 2010, Hoboken, NJ


Publish subscribe 101

Publish/Subscribe 101

  • Not all publish/subscribe is equal

  • Publish/Subscribe models and evolution

    • Channel-based

      • Ex.: OMG CORBA Event Service, …

    • Topic-based

      • Ex.: WS Notifications, …

    • Type-based

      • Ex.: OMG Data Dissemination Service (partially), …

    • Content-based

      • Ex.: The PADRES ESB (see below), …

    • State-based

      • a.k.a., Subject Spaces (active research)

BPM 2010, Hoboken, NJ


The content based pub sub model

The Content-based Pub/Sub Model

  • Language and data model

    • Boolean functions over predicates

    • Subscriptions are conjunctions of predicates

    • Publications are sets of attribute-value pairs

  • Matching semantic

    • A subscription matches if all its predicates match

    • Approximate semantic (e.g., close to, cheap, sunny)

    • Semantic and similarity-based matching

BPM 2010, Hoboken, NJ


In network processing

In-network processing

BPM 2010, Hoboken, NJ


Event correlation and in network processing

P

= publisher / sensor

= subscriber / doctor

S

Event Correlation and In-network Processing

Monitoring patients who needs critical attention

heart rate = 150 bpm

P

[medication = DrugX] & [heart rate > 120 bpm] & [blood pressure < 70 mmHg]

P

B

S

B

age = 70

Monitoring elder patients who are losing blood pressure

P

S

B

B

B

[heart rate < 30] | [temperature < 33] | [blood pressure < 50]

temperature = 38°C

P

S

Monitoring patients having a fever but not having a cold

B

temperature = 40°C

[heart rate > 140 bpm] & [temperature > 39] & [illness != cold]

B

P

P

illness = diabetes

blood pressure = 60 mmHg

BPM 2010, Hoboken, NJ


Illustration of in network processing

Illustration of In-network Processing

BPM 2010, Hoboken, NJ


Monitoring

Monitoring

BPM 2010, Hoboken, NJ


Monitoring client

Monitoring Client

  • Monitor is a Pub/Sub client

  • Visualizes topology of broker overlay network

  • Visualizes message routing

  • Monitor/control process execution

    • Process level

    • Activity level

  • Supports dynamic process modifications

BPM 2010, Hoboken, NJ


Padres on planetlab

PADRES on PlanetLab

BPM 2010, Hoboken, NJ


Bpm in cloud architectures business process management with slas and events

BPM 2010, Hoboken, NJ


Cost model

output queue B1

input queue

output queue B2

Cost Model

  • Routing cost of CS

    RC(CS) =

    +

    +

  • Subscription cardinality

    • |P(S)| : The number of matching publications per unit of time.

    • |P(S)| =

    • |P(CS)| = |P(Sl)| + |P(Sr)| if op = OR

Matching Engine

+

Routing Table

subscription

dest

temperature > 37 B1

temperature > 40 B2

BPM 2010, Hoboken, NJ


Dynamic cs routing dcsr

AND

AND

Dynamic CS Routing (DCSR)

Adv 3

Broker 5 and 8

are the join

point brokers

Adv 2

1

2

7

CS

CS’’

S3

3

4

8

CS’

S3

S2

Adv 1

S1

S2

5

6

9

S1

CS={{S1 AND S2} AND S3}

CS

BPM 2010, Hoboken, NJ


Bpm in cloud architectures business process management with slas and events

BPM 2010, Hoboken, NJ


Estimator example distance

Estimator Example – distance

  • Efficiently compute and cache information about candidate engines

  • Discover paths

    • BP→BT , BT→BS

  • Probe paths

    • BT→BC for each candidate C

  • Compute paths

    • BP→BC , BC→BS

    • Cases

      • C in BP→BT

      • C in BT→BS

      • otherwise

Process

P

T

S

Topology

P

2

5

T

8

1

4

7

S

C

3

6

9

Broker

Execution engine

BPM 2010, Hoboken, NJ


Engine resource monitor

Engine Resource Monitor

  • Profile execution of local tasks

Process

P

T

S

Topology

P

2

5

T

8

1

4

7

S

C

3

6

9

Broker

Execution engine

BPM 2010, Hoboken, NJ


Redeployment manager1

Redeployment Manager

  • Compute the cost of deploying local agents Ai at candidate engines Ej

Measurements

C(Ai): Cost at local engine

E(P(Ai)): Location of predecessors

E(S(Ai)): Location of successors

Cost Model

Given

F(Ai): Cost function

P(Ai): Predecessors

S(Ai): Successors

Complexity

Memory: O( |Ai| |Ej| )

Computation: O( |Ai| |Ej| |Navg(Ai)| )

Compute

C(Ai,Ej) for every Ai,Ej:

Estimated cost of deploying agent Ai at candidate engine Ej

BPM 2010, Hoboken, NJ


Cost model1

Cost Model

The cost of a process is based on three costcomponents

1. Distribution cost Cdist = Cd3 * (Cd1 + Cd2)

Cd1 Message rate

Cd2 Message size

Cd3Latency

2. Engine cost Ceng = f(Ce1, Ce2,Ce3)

Ce1 Load (number of instances)

Ce2 Resources (CPU, memory, etc.)

Ce3 Task complexity

3. Service cost Cserv = Cs1 + Cs2 + Cs3

Cs1 Latency of external service

Cs2 Execution time of external service

Cs3 Marshalling/unmarshalling

Cost(agent) = ∑wiCi

Cost(process) = ∑cost(agent)

These costs can be weighted to achieve different objectives

Optimize time

wd1 = wd3 = we3 = Cserv = 1, other wi = 0

Optimize network overhead

wd2 = wd3 = 1, other wi = 0

Various optimization criteria can be specified

Threshold criteria: ∑wiCi > x

E.g., Report SLA violations within 3 s.

Minimized criteria:min( ∑wiCi )

E.g., Minimize distribution overhead

BPM 2010, Hoboken, NJ


Bpm in cloud architectures business process management with slas and events

BPM 2010, Hoboken, NJ


Transformation example

AND

AND

S3

S1

S2

Transformation Example

[class,=,reading], [level,>,3]

AND

[class,=,audit], [firm,=,*], [trust, >=, 3]

[class,=,reading], [shipID,=,$X], [level,>,3]

AND

[class,=,manifest], [shipID,=,$X], [firm,=,$Y], [content,!=,fertilizer]

AND

[class,=,audit], [firm,=,$Y], [trust,>,7]

composite subscription

atomic

subscription

BPM 2010, Hoboken, NJ


Bpm in cloud architectures business process management with slas and events

BPM 2010, Hoboken, NJ


The padres esb for event driven bpm

The PADRES ESB for Event-driven BPM

Enterprise Services Bus

(Events & Services Bus)

  • First generation of students, when I looked away 

    • Peng Alex David aRno Eli Serge

  • PADRES is Publish/subscribe Applied to Distributed

    Resource Scheduling

  • PAdres is Distributed REsource Scheduling

  • http://www.padres.msrg.utoronto.ca

Web start & open source

padres.msrg.org

Supported and sponsored by

BPM 2010, Hoboken, NJ


The padres esb in r d

The PADRES ESB in R&D

  • Develop flexible BPM concept

    • With CA Labs, Sun Microsystems, OCE & NSERC

  • Develop SLA-based autonomic BPM concept

    • With IBM and NSERC

  • Evaluate and experiment with the use of the PADRES ESB for application integration

    • With Bell Canada and BUL

Acknowledgements

BPM 2010, Hoboken, NJ


  • Login