Probabilistic verification of discrete event systems using acceptance sampling
Download
1 / 43

Probabilistic Verification of Discrete Event Systems using Acceptance Sampling - PowerPoint PPT Presentation


  • 128 Views
  • Uploaded on
  • Presentation posted in: General

Probabilistic Verification of Discrete Event Systems using Acceptance Sampling. Introduction. Verify properties of discrete event systems Model independent approach Probabilistic real-time properties expressed using CSL Acceptance sampling Guaranteed error bounds. DES Example.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha

Download Presentation

Probabilistic Verification of Discrete Event Systems using Acceptance Sampling

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Probabilistic Verification of Discrete Event Systems using Acceptance Sampling


Introduction

  • Verify properties of discrete event systems

  • Model independent approach

  • Probabilistic real-time properties expressed using CSL

  • Acceptance sampling

  • Guaranteed error bounds


DES Example

  • Triple modular redundant system

    • Three processors

    • Single majority voter

Voter

CPU 1

CPU 2

CPU 3


DES Example

3 CPUs upvoter up

Voter

CPU 1

CPU 2

CPU 3


DES Example

CPU fails

3 CPUs upvoter up

2 CPUs upvoter up

40

Voter

CPU 1

CPU 2

CPU 3


DES Example

CPU fails

repair CPU

3 CPUs upvoter up

2 CPUs upvoter up

2 CPUs up

repairing CPU

voter up

40

19.2

Voter

CPU 1

CPU 2

CPU 3


DES Example

CPU fails

repair CPU

CPU fails

3 CPUs upvoter up

2 CPUs upvoter up

2 CPUs up

repairing CPU

voter up

1 CPU uprepairing CPU

voter up

40

19.2

11

Voter

CPU 1

CPU 2

CPU 3


DES Example

CPU fails

repair CPU

CPU fails

CPU repaired

3 CPUs upvoter up

2 CPUs upvoter up

2 CPUs up

repairing CPU

voter up

1 CPU uprepairing CPU

voter up

2 CPUs upvoter up

40

19.2

11

4

Voter

CPU 1

CPU 2

CPU 3


CPU fails

repair CPU

CPU fails

CPU repaired

3 CPUs upvoter up

2 CPUs upvoter up

2 CPUs up

repairing CPU

voter up

1 CPU uprepairing CPU

voter up

2 CPUs upvoter up

40

19.2

11

4

CPU fails

repair CPU

CPU repaired

voter fails

3 CPUs upvoter up

2 CPUs upvoter up

2 CPUs up

repairing CPU

voter up

3 CPUs up

voter up

3 CPUs upvoter down

82

13

16

4

Sample Execution Paths


Properties of Interest

  • Probabilistic real-time properties

    • “The probability is at least 0.1 that the voter fails within 120 time units while at least 2 CPUs have continuously remained up”


Properties of Interest

  • Probabilisticreal-time properties

    • “The probability is at least 0.1 that the voter fails within 120 time units while at least 2 CPUs have continuously remained up”

CSL formula:

Pr≥0.1(“2 CPUs up”  “3 CPUs up” U≤120 “voter down”)


Continuous Stochastic Logic (CSL)

  • State formulas

    • Truth value is determined in a single state

  • Path formulas

    • Truth value is determined over an execution path


State Formulas

  • Standard logic operators: ¬, 1  2 …

  • Probabilistic operator: Pr≥()

    • True iff probability is at least  that  holds


Path Formulas

  • Until: 1 U≤t2

    • Holds iff 2 becomes true in some state along the execution path before time t, and 1 is true in all prior states


Verifying Real-time Properties

  • “2 CPUs up”  “3 CPUs up” U≤120 “voter down”

CPU fails

repair CPU

CPU fails

CPU repaired

3 CPUs upvoter up

2 CPUs upvoter up

2 CPUs up

repairing CPU

voter up

1 CPU uprepairing CPU

voter up

2 CPUs upvoter up

40

19.2

11

4

False!


+

+

+

≤ 120

Verifying Real-time Properties

  • “2 CPUs up”  “3 CPUs up” U≤120 “voter down”

CPU fails

repair CPU

CPU repaired

voter fails

3 CPUs upvoter up

2 CPUs upvoter up

2 CPUs up

repairing CPU

voter up

3 CPUs up

voter up

3 CPUs upvoter down

82

13

16

4

True!


Verifying Probabilistic Properties

  • “The probability is at least 0.1 that ”

    • Symbolic Methods

      • Pro: Exact solution

      • Con: Works only for restricted class of systems

    • Sampling

      • Pro: Works for any system that can be simulated

      • Con: Uncertainty in correctness of solution


Our Approach

  • Use simulation to generate sample execution paths

  • Use sequential acceptance sampling to verify probabilistic properties


Error Bounds

  • Probability of false negative: ≤

    • We say that  is false when it is true

  • Probability of false positive: ≤

    • We say that  is true when it is false


Acceptance Sampling

  • Hypothesis: Pr≥()


1 – 

Probability of acceptingPr≥() as true

Actual probability of  holding

Performance of Test


1 – 

False negatives

Probability of acceptingPr≥() as true

False positives

Actual probability of  holding

Ideal Performance


1 – 

False negatives

Probability of acceptingPr≥() as true

Indifference region

 – 

 + 

False positives

Actual probability of  holding

Actual Performance


True, false,or anothersample?

SequentialAcceptance Sampling

  • Hypothesis: Pr≥()


Number ofpositive samples

Number of samples

Graphical Representation of Sequential Test


Accept

Number ofpositive samples

Continue sampling

Reject

Number of samples

Graphical Representation of Sequential Test

  • We can find an acceptance line and a rejection line given , , , and 

A,,,(n)

R,,,(n)


Accept

Number ofpositive samples

Continue sampling

Reject

Number of samples

Graphical Representation of Sequential Test

  • Reject hypothesis


Accept

Number ofpositive samples

Continue sampling

Reject

Number of samples

Graphical Representation of Sequential Test

  • Accept hypothesis


Verifying Probabilistic Properties

  • Verify Pr≥() with error bounds  and 

    • Generate sample execution paths using simulation

    • Verify  over each sample execution path

      • If  is true, then we have a positive sample

      • If  is false, then we have a negative sample

    • Use sequential acceptance sampling to test the hypothesis Pr≥()


Verification of Nested Probabilistic Statements

  • Suppose , in Pr≥(), contains probabilistic statements

    • Error bounds ’ and ’ when verifying 


True, false,or anothersample?

Verification of Nested Probabilistic Statements

  • Suppose , in Pr≥(), contains probabilistic statements


Accept

With ’ and ’ = 0

Number ofpositive samples

Continue sampling

Reject

Number of samples

Modified Test

  • Find an acceptance line and a rejection line given , , , , ’, and ’:

A,,,(n)

R,,,(n)


With ’ and ’ > 0

Number ofpositive samples

Number of samples

Modified Test

  • Find an acceptance line and a rejection line given , , , , ’, and ’:

Accept

Continue sampling

Reject


Number ofpositive samples

Number of samples

Modified Test

  • Find an acceptance line and a rejection line given , , , , ’, and ’:

A’,’,,(n)

Accept

’ – ’= 1 – (1 – ( – ))(1 – ’)

’ + ’= ( + )(1 – ’)

Continue sampling

R’,’,,(n)

Reject


Verification of Negation

  • To verify ¬ with error bounds  and 

    • Verify  with error bounds  and 


Verification of Conjunction

  • Verify 12  … n with error bounds  and 

    • Verify each i with error bounds  and /n


Verification of Path Formulas

  • To verify 1 U≤t2 with error bounds  and 

    • Convert to disjunction

      • 1 U≤t2 holds if 2 holds in the first state, or if 2 holds in the second state and 1holds in all prior states, or …


More on Verifying Until

  • Given 1 U≤t2, let n be the index of the first state more than t time units away from the current state

  • Disjunction of n conjunctions c1 through cn, each of size i

  • Simplifies if 1 or 2, or both, do not contain any probabilistic statements


Performance

 = 0.01

 = 0.9

Average number of samples

 =  = 10-10

 =  = 10-1

Actual probability of  holding


Performance

 =  = 10-2

 = 0.9

=0.001

=0.002

Average number of samples

=0.005

=0.01

=0.02

=0.05

Actual probability of  holding


Summary

  • Model independent probabilistic verification of discrete event systems

  • Sample execution paths generated using simulation

  • Probabilistic properties verified using sequential acceptance sampling

  • Easy to trade accuracy for efficiency


Future Work

  • Develop heuristics for formula ordering and parameter selection

  • Use in combination with symbolic methods

  • Apply to hybrid dynamic systems

  • Use verification to aid controller synthesis for discrete event systems


ProVer: Probabilistic Verifier

http://www.cs.cmu.edu/~lorens/prover.html


ad
  • Login