Probabilistic verification of discrete event systems using acceptance sampling
This presentation is the property of its rightful owner.
Sponsored Links
1 / 43

Probabilistic Verification of Discrete Event Systems using Acceptance Sampling PowerPoint PPT Presentation


  • 98 Views
  • Uploaded on
  • Presentation posted in: General

Probabilistic Verification of Discrete Event Systems using Acceptance Sampling. Introduction. Verify properties of discrete event systems Model independent approach Probabilistic real-time properties expressed using CSL Acceptance sampling Guaranteed error bounds. DES Example.

Download Presentation

Probabilistic Verification of Discrete Event Systems using Acceptance Sampling

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Probabilistic verification of discrete event systems using acceptance sampling

Probabilistic Verification of Discrete Event Systems using Acceptance Sampling


Introduction

Introduction

  • Verify properties of discrete event systems

  • Model independent approach

  • Probabilistic real-time properties expressed using CSL

  • Acceptance sampling

  • Guaranteed error bounds


Des example

DES Example

  • Triple modular redundant system

    • Three processors

    • Single majority voter

Voter

CPU 1

CPU 2

CPU 3


Des example1

DES Example

3 CPUs upvoter up

Voter

CPU 1

CPU 2

CPU 3


Des example2

DES Example

CPU fails

3 CPUs upvoter up

2 CPUs upvoter up

40

Voter

CPU 1

CPU 2

CPU 3


Des example3

DES Example

CPU fails

repair CPU

3 CPUs upvoter up

2 CPUs upvoter up

2 CPUs up

repairing CPU

voter up

40

19.2

Voter

CPU 1

CPU 2

CPU 3


Des example4

DES Example

CPU fails

repair CPU

CPU fails

3 CPUs upvoter up

2 CPUs upvoter up

2 CPUs up

repairing CPU

voter up

1 CPU uprepairing CPU

voter up

40

19.2

11

Voter

CPU 1

CPU 2

CPU 3


Des example5

DES Example

CPU fails

repair CPU

CPU fails

CPU repaired

3 CPUs upvoter up

2 CPUs upvoter up

2 CPUs up

repairing CPU

voter up

1 CPU uprepairing CPU

voter up

2 CPUs upvoter up

40

19.2

11

4

Voter

CPU 1

CPU 2

CPU 3


Sample execution paths

CPU fails

repair CPU

CPU fails

CPU repaired

3 CPUs upvoter up

2 CPUs upvoter up

2 CPUs up

repairing CPU

voter up

1 CPU uprepairing CPU

voter up

2 CPUs upvoter up

40

19.2

11

4

CPU fails

repair CPU

CPU repaired

voter fails

3 CPUs upvoter up

2 CPUs upvoter up

2 CPUs up

repairing CPU

voter up

3 CPUs up

voter up

3 CPUs upvoter down

82

13

16

4

Sample Execution Paths


Properties of interest

Properties of Interest

  • Probabilistic real-time properties

    • “The probability is at least 0.1 that the voter fails within 120 time units while at least 2 CPUs have continuously remained up”


Properties of interest1

Properties of Interest

  • Probabilisticreal-time properties

    • “The probability is at least 0.1 that the voter fails within 120 time units while at least 2 CPUs have continuously remained up”

CSL formula:

Pr≥0.1(“2 CPUs up”  “3 CPUs up” U≤120 “voter down”)


Continuous stochastic logic csl

Continuous Stochastic Logic (CSL)

  • State formulas

    • Truth value is determined in a single state

  • Path formulas

    • Truth value is determined over an execution path


State formulas

State Formulas

  • Standard logic operators: ¬, 1  2 …

  • Probabilistic operator: Pr≥()

    • True iff probability is at least  that  holds


Path formulas

Path Formulas

  • Until: 1 U≤t2

    • Holds iff 2 becomes true in some state along the execution path before time t, and 1 is true in all prior states


Verifying real time properties

Verifying Real-time Properties

  • “2 CPUs up”  “3 CPUs up” U≤120 “voter down”

CPU fails

repair CPU

CPU fails

CPU repaired

3 CPUs upvoter up

2 CPUs upvoter up

2 CPUs up

repairing CPU

voter up

1 CPU uprepairing CPU

voter up

2 CPUs upvoter up

40

19.2

11

4

False!


Verifying real time properties1

+

+

+

≤ 120

Verifying Real-time Properties

  • “2 CPUs up”  “3 CPUs up” U≤120 “voter down”

CPU fails

repair CPU

CPU repaired

voter fails

3 CPUs upvoter up

2 CPUs upvoter up

2 CPUs up

repairing CPU

voter up

3 CPUs up

voter up

3 CPUs upvoter down

82

13

16

4

True!


Verifying probabilistic properties

Verifying Probabilistic Properties

  • “The probability is at least 0.1 that ”

    • Symbolic Methods

      • Pro: Exact solution

      • Con: Works only for restricted class of systems

    • Sampling

      • Pro: Works for any system that can be simulated

      • Con: Uncertainty in correctness of solution


Our approach

Our Approach

  • Use simulation to generate sample execution paths

  • Use sequential acceptance sampling to verify probabilistic properties


Error bounds

Error Bounds

  • Probability of false negative: ≤

    • We say that  is false when it is true

  • Probability of false positive: ≤

    • We say that  is true when it is false


Acceptance sampling

Acceptance Sampling

  • Hypothesis: Pr≥()


Performance of test

1 – 

Probability of acceptingPr≥() as true

Actual probability of  holding

Performance of Test


Ideal performance

1 – 

False negatives

Probability of acceptingPr≥() as true

False positives

Actual probability of  holding

Ideal Performance


Actual performance

1 – 

False negatives

Probability of acceptingPr≥() as true

Indifference region

 – 

 + 

False positives

Actual probability of  holding

Actual Performance


Sequential acceptance sampling

True, false,or anothersample?

SequentialAcceptance Sampling

  • Hypothesis: Pr≥()


Graphical representation of sequential test

Number ofpositive samples

Number of samples

Graphical Representation of Sequential Test


Graphical representation of sequential test1

Accept

Number ofpositive samples

Continue sampling

Reject

Number of samples

Graphical Representation of Sequential Test

  • We can find an acceptance line and a rejection line given , , , and 

A,,,(n)

R,,,(n)


Graphical representation of sequential test2

Accept

Number ofpositive samples

Continue sampling

Reject

Number of samples

Graphical Representation of Sequential Test

  • Reject hypothesis


Graphical representation of sequential test3

Accept

Number ofpositive samples

Continue sampling

Reject

Number of samples

Graphical Representation of Sequential Test

  • Accept hypothesis


Verifying probabilistic properties1

Verifying Probabilistic Properties

  • Verify Pr≥() with error bounds  and 

    • Generate sample execution paths using simulation

    • Verify  over each sample execution path

      • If  is true, then we have a positive sample

      • If  is false, then we have a negative sample

    • Use sequential acceptance sampling to test the hypothesis Pr≥()


Verification of nested probabilistic statements

Verification of Nested Probabilistic Statements

  • Suppose , in Pr≥(), contains probabilistic statements

    • Error bounds ’ and ’ when verifying 


Verification of nested probabilistic statements1

True, false,or anothersample?

Verification of Nested Probabilistic Statements

  • Suppose , in Pr≥(), contains probabilistic statements


Modified test

Accept

With ’ and ’ = 0

Number ofpositive samples

Continue sampling

Reject

Number of samples

Modified Test

  • Find an acceptance line and a rejection line given , , , , ’, and ’:

A,,,(n)

R,,,(n)


Modified test1

With ’ and ’ > 0

Number ofpositive samples

Number of samples

Modified Test

  • Find an acceptance line and a rejection line given , , , , ’, and ’:

Accept

Continue sampling

Reject


Modified test2

Number ofpositive samples

Number of samples

Modified Test

  • Find an acceptance line and a rejection line given , , , , ’, and ’:

A’,’,,(n)

Accept

’ – ’= 1 – (1 – ( – ))(1 – ’)

’ + ’= ( + )(1 – ’)

Continue sampling

R’,’,,(n)

Reject


Verification of negation

Verification of Negation

  • To verify ¬ with error bounds  and 

    • Verify  with error bounds  and 


Verification of conjunction

Verification of Conjunction

  • Verify 12  … n with error bounds  and 

    • Verify each i with error bounds  and /n


Verification of path formulas

Verification of Path Formulas

  • To verify 1 U≤t2 with error bounds  and 

    • Convert to disjunction

      • 1 U≤t2 holds if 2 holds in the first state, or if 2 holds in the second state and 1holds in all prior states, or …


More on verifying until

More on Verifying Until

  • Given 1 U≤t2, let n be the index of the first state more than t time units away from the current state

  • Disjunction of n conjunctions c1 through cn, each of size i

  • Simplifies if 1 or 2, or both, do not contain any probabilistic statements


Performance

Performance

 = 0.01

 = 0.9

Average number of samples

 =  = 10-10

 =  = 10-1

Actual probability of  holding


Performance1

Performance

 =  = 10-2

 = 0.9

=0.001

=0.002

Average number of samples

=0.005

=0.01

=0.02

=0.05

Actual probability of  holding


Summary

Summary

  • Model independent probabilistic verification of discrete event systems

  • Sample execution paths generated using simulation

  • Probabilistic properties verified using sequential acceptance sampling

  • Easy to trade accuracy for efficiency


Future work

Future Work

  • Develop heuristics for formula ordering and parameter selection

  • Use in combination with symbolic methods

  • Apply to hybrid dynamic systems

  • Use verification to aid controller synthesis for discrete event systems


Prover probabilistic verifier

ProVer: Probabilistic Verifier

http://www.cs.cmu.edu/~lorens/prover.html


  • Login