Probabilistic verification of discrete event systems using acceptance sampling
Download
1 / 43

Probabilistic Verification of Discrete Event Systems using Acceptance Sampling - PowerPoint PPT Presentation


  • 150 Views
  • Uploaded on

Probabilistic Verification of Discrete Event Systems using Acceptance Sampling. Introduction. Verify properties of discrete event systems Model independent approach Probabilistic real-time properties expressed using CSL Acceptance sampling Guaranteed error bounds. DES Example.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Probabilistic Verification of Discrete Event Systems using Acceptance Sampling' - decker


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Introduction
Introduction Acceptance Sampling

  • Verify properties of discrete event systems

  • Model independent approach

  • Probabilistic real-time properties expressed using CSL

  • Acceptance sampling

  • Guaranteed error bounds


Des example
DES Example Acceptance Sampling

  • Triple modular redundant system

    • Three processors

    • Single majority voter

Voter

CPU 1

CPU 2

CPU 3


Des example1
DES Example Acceptance Sampling

3 CPUs upvoter up

Voter

CPU 1

CPU 2

CPU 3


Des example2
DES Example Acceptance Sampling

CPU fails

3 CPUs upvoter up

2 CPUs upvoter up

40

Voter

CPU 1

CPU 2

CPU 3


Des example3
DES Example Acceptance Sampling

CPU fails

repair CPU

3 CPUs upvoter up

2 CPUs upvoter up

2 CPUs up

repairing CPU

voter up

40

19.2

Voter

CPU 1

CPU 2

CPU 3


Des example4
DES Example Acceptance Sampling

CPU fails

repair CPU

CPU fails

3 CPUs upvoter up

2 CPUs upvoter up

2 CPUs up

repairing CPU

voter up

1 CPU uprepairing CPU

voter up

40

19.2

11

Voter

CPU 1

CPU 2

CPU 3


Des example5
DES Example Acceptance Sampling

CPU fails

repair CPU

CPU fails

CPU repaired

3 CPUs upvoter up

2 CPUs upvoter up

2 CPUs up

repairing CPU

voter up

1 CPU uprepairing CPU

voter up

2 CPUs upvoter up

40

19.2

11

4

Voter

CPU 1

CPU 2

CPU 3


Sample execution paths

CPU fails Acceptance Sampling

repair CPU

CPU fails

CPU repaired

3 CPUs upvoter up

2 CPUs upvoter up

2 CPUs up

repairing CPU

voter up

1 CPU uprepairing CPU

voter up

2 CPUs upvoter up

40

19.2

11

4

CPU fails

repair CPU

CPU repaired

voter fails

3 CPUs upvoter up

2 CPUs upvoter up

2 CPUs up

repairing CPU

voter up

3 CPUs up

voter up

3 CPUs upvoter down

82

13

16

4

Sample Execution Paths


Properties of interest
Properties of Interest Acceptance Sampling

  • Probabilistic real-time properties

    • “The probability is at least 0.1 that the voter fails within 120 time units while at least 2 CPUs have continuously remained up”


Properties of interest1
Properties of Interest Acceptance Sampling

  • Probabilisticreal-time properties

    • “The probability is at least 0.1 that the voter fails within 120 time units while at least 2 CPUs have continuously remained up”

CSL formula:

Pr≥0.1(“2 CPUs up”  “3 CPUs up” U≤120 “voter down”)


Continuous stochastic logic csl
Continuous Stochastic Logic (CSL) Acceptance Sampling

  • State formulas

    • Truth value is determined in a single state

  • Path formulas

    • Truth value is determined over an execution path


State formulas
State Formulas Acceptance Sampling

  • Standard logic operators: ¬, 1  2 …

  • Probabilistic operator: Pr≥()

    • True iff probability is at least  that  holds


Path formulas
Path Formulas Acceptance Sampling

  • Until: 1 U≤t2

    • Holds iff 2 becomes true in some state along the execution path before time t, and 1 is true in all prior states


Verifying real time properties
Verifying Real-time Properties Acceptance Sampling

  • “2 CPUs up”  “3 CPUs up” U≤120 “voter down”

CPU fails

repair CPU

CPU fails

CPU repaired

3 CPUs upvoter up

2 CPUs upvoter up

2 CPUs up

repairing CPU

voter up

1 CPU uprepairing CPU

voter up

2 CPUs upvoter up

40

19.2

11

4

False!


Verifying real time properties1

+ Acceptance Sampling

+

+

≤ 120

Verifying Real-time Properties

  • “2 CPUs up”  “3 CPUs up” U≤120 “voter down”

CPU fails

repair CPU

CPU repaired

voter fails

3 CPUs upvoter up

2 CPUs upvoter up

2 CPUs up

repairing CPU

voter up

3 CPUs up

voter up

3 CPUs upvoter down

82

13

16

4

True!


Verifying probabilistic properties
Verifying Probabilistic Properties Acceptance Sampling

  • “The probability is at least 0.1 that ”

    • Symbolic Methods

      • Pro: Exact solution

      • Con: Works only for restricted class of systems

    • Sampling

      • Pro: Works for any system that can be simulated

      • Con: Uncertainty in correctness of solution


Our approach
Our Approach Acceptance Sampling

  • Use simulation to generate sample execution paths

  • Use sequential acceptance sampling to verify probabilistic properties


Error bounds
Error Bounds Acceptance Sampling

  • Probability of false negative: ≤

    • We say that  is false when it is true

  • Probability of false positive: ≤

    • We say that  is true when it is false


Acceptance sampling
Acceptance Sampling Acceptance Sampling

  • Hypothesis: Pr≥()


Performance of test

1 – Acceptance Sampling

Probability of acceptingPr≥() as true

Actual probability of  holding

Performance of Test


Ideal performance

1 – Acceptance Sampling

False negatives

Probability of acceptingPr≥() as true

False positives

Actual probability of  holding

Ideal Performance


Actual performance

1 – Acceptance Sampling

False negatives

Probability of acceptingPr≥() as true

Indifference region

 – 

 + 

False positives

Actual probability of  holding

Actual Performance


Sequential acceptance sampling

True, false, Acceptance Samplingor anothersample?

SequentialAcceptance Sampling

  • Hypothesis: Pr≥()


Graphical representation of sequential test

Number of Acceptance Samplingpositive samples

Number of samples

Graphical Representation of Sequential Test


Graphical representation of sequential test1

Accept Acceptance Sampling

Number ofpositive samples

Continue sampling

Reject

Number of samples

Graphical Representation of Sequential Test

  • We can find an acceptance line and a rejection line given , , , and 

A,,,(n)

R,,,(n)


Graphical representation of sequential test2

Accept Acceptance Sampling

Number ofpositive samples

Continue sampling

Reject

Number of samples

Graphical Representation of Sequential Test

  • Reject hypothesis


Graphical representation of sequential test3

Accept Acceptance Sampling

Number ofpositive samples

Continue sampling

Reject

Number of samples

Graphical Representation of Sequential Test

  • Accept hypothesis


Verifying probabilistic properties1
Verifying Probabilistic Properties Acceptance Sampling

  • Verify Pr≥() with error bounds  and 

    • Generate sample execution paths using simulation

    • Verify  over each sample execution path

      • If  is true, then we have a positive sample

      • If  is false, then we have a negative sample

    • Use sequential acceptance sampling to test the hypothesis Pr≥()


Verification of nested probabilistic statements
Verification of Nested Probabilistic Statements Acceptance Sampling

  • Suppose , in Pr≥(), contains probabilistic statements

    • Error bounds ’ and ’ when verifying 


Verification of nested probabilistic statements1

True, false, Acceptance Samplingor anothersample?

Verification of Nested Probabilistic Statements

  • Suppose , in Pr≥(), contains probabilistic statements


Modified test

Accept Acceptance Sampling

With ’ and ’ = 0

Number ofpositive samples

Continue sampling

Reject

Number of samples

Modified Test

  • Find an acceptance line and a rejection line given , , , , ’, and ’:

A,,,(n)

R,,,(n)


Modified test1

With Acceptance Sampling’ and ’ > 0

Number ofpositive samples

Number of samples

Modified Test

  • Find an acceptance line and a rejection line given , , , , ’, and ’:

Accept

Continue sampling

Reject


Modified test2

Number of Acceptance Samplingpositive samples

Number of samples

Modified Test

  • Find an acceptance line and a rejection line given , , , , ’, and ’:

A’,’,,(n)

Accept

’ – ’= 1 – (1 – ( – ))(1 – ’)

’ + ’= ( + )(1 – ’)

Continue sampling

R’,’,,(n)

Reject


Verification of negation
Verification of Negation Acceptance Sampling

  • To verify ¬ with error bounds  and 

    • Verify  with error bounds  and 


Verification of conjunction
Verification of Conjunction Acceptance Sampling

  • Verify 12  … n with error bounds  and 

    • Verify each i with error bounds  and /n


Verification of path formulas
Verification of Path Formulas Acceptance Sampling

  • To verify 1 U≤t2 with error bounds  and 

    • Convert to disjunction

      • 1 U≤t2 holds if 2 holds in the first state, or if 2 holds in the second state and 1holds in all prior states, or …


More on verifying until
More on Verifying Until Acceptance Sampling

  • Given 1 U≤t2, let n be the index of the first state more than t time units away from the current state

  • Disjunction of n conjunctions c1 through cn, each of size i

  • Simplifies if 1 or 2, or both, do not contain any probabilistic statements


Performance
Performance Acceptance Sampling

 = 0.01

 = 0.9

Average number of samples

 =  = 10-10

 =  = 10-1

Actual probability of  holding


Performance1
Performance Acceptance Sampling

 =  = 10-2

 = 0.9

=0.001

=0.002

Average number of samples

=0.005

=0.01

=0.02

=0.05

Actual probability of  holding


Summary
Summary Acceptance Sampling

  • Model independent probabilistic verification of discrete event systems

  • Sample execution paths generated using simulation

  • Probabilistic properties verified using sequential acceptance sampling

  • Easy to trade accuracy for efficiency


Future work
Future Work Acceptance Sampling

  • Develop heuristics for formula ordering and parameter selection

  • Use in combination with symbolic methods

  • Apply to hybrid dynamic systems

  • Use verification to aid controller synthesis for discrete event systems


Prover probabilistic verifier
ProVer: Probabilistic Verifier Acceptance Sampling

http://www.cs.cmu.edu/~lorens/prover.html


ad