Loading in 5 sec....

Probabilistic Verification of Discrete Event Systems using Acceptance SamplingPowerPoint Presentation

Probabilistic Verification of Discrete Event Systems using Acceptance Sampling

- 150 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about ' Probabilistic Verification of Discrete Event Systems using Acceptance Sampling' - decker

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

### Probabilistic Verification of Discrete Event Systems using Acceptance Sampling

Introduction Acceptance Sampling

- Verify properties of discrete event systems
- Model independent approach
- Probabilistic real-time properties expressed using CSL
- Acceptance sampling
- Guaranteed error bounds

DES Example Acceptance Sampling

- Triple modular redundant system
- Three processors
- Single majority voter

Voter

CPU 1

CPU 2

CPU 3

DES Example Acceptance Sampling

CPU fails

3 CPUs upvoter up

2 CPUs upvoter up

40

Voter

CPU 1

CPU 2

CPU 3

DES Example Acceptance Sampling

CPU fails

repair CPU

3 CPUs upvoter up

2 CPUs upvoter up

2 CPUs up

repairing CPU

voter up

40

19.2

Voter

CPU 1

CPU 2

CPU 3

DES Example Acceptance Sampling

CPU fails

repair CPU

CPU fails

3 CPUs upvoter up

2 CPUs upvoter up

2 CPUs up

repairing CPU

voter up

1 CPU uprepairing CPU

voter up

40

19.2

11

Voter

CPU 1

CPU 2

CPU 3

DES Example Acceptance Sampling

CPU fails

repair CPU

CPU fails

CPU repaired

3 CPUs upvoter up

2 CPUs upvoter up

2 CPUs up

repairing CPU

voter up

1 CPU uprepairing CPU

voter up

2 CPUs upvoter up

40

19.2

11

4

…

Voter

CPU 1

CPU 2

CPU 3

CPU fails Acceptance Sampling

repair CPU

CPU fails

CPU repaired

3 CPUs upvoter up

2 CPUs upvoter up

2 CPUs up

repairing CPU

voter up

1 CPU uprepairing CPU

voter up

2 CPUs upvoter up

40

19.2

11

4

…

CPU fails

repair CPU

CPU repaired

voter fails

3 CPUs upvoter up

2 CPUs upvoter up

2 CPUs up

repairing CPU

voter up

3 CPUs up

voter up

3 CPUs upvoter down

82

13

16

4

…

Sample Execution PathsProperties of Interest Acceptance Sampling

- Probabilistic real-time properties
- “The probability is at least 0.1 that the voter fails within 120 time units while at least 2 CPUs have continuously remained up”

Properties of Interest Acceptance Sampling

- Probabilisticreal-time properties
- “The probability is at least 0.1 that the voter fails within 120 time units while at least 2 CPUs have continuously remained up”

CSL formula:

Pr≥0.1(“2 CPUs up” “3 CPUs up” U≤120 “voter down”)

Continuous Stochastic Logic (CSL) Acceptance Sampling

- State formulas
- Truth value is determined in a single state

- Path formulas
- Truth value is determined over an execution path

State Formulas Acceptance Sampling

- Standard logic operators: ¬, 1 2 …
- Probabilistic operator: Pr≥()
- True iff probability is at least that holds

Path Formulas Acceptance Sampling

- Until: 1 U≤t2
- Holds iff 2 becomes true in some state along the execution path before time t, and 1 is true in all prior states

Verifying Real-time Properties Acceptance Sampling

- “2 CPUs up” “3 CPUs up” U≤120 “voter down”

CPU fails

repair CPU

CPU fails

CPU repaired

3 CPUs upvoter up

2 CPUs upvoter up

2 CPUs up

repairing CPU

voter up

1 CPU uprepairing CPU

voter up

2 CPUs upvoter up

40

19.2

11

4

…

False!

+ Acceptance Sampling

+

+

≤ 120

Verifying Real-time Properties- “2 CPUs up” “3 CPUs up” U≤120 “voter down”

CPU fails

repair CPU

CPU repaired

voter fails

3 CPUs upvoter up

2 CPUs upvoter up

2 CPUs up

repairing CPU

voter up

3 CPUs up

voter up

3 CPUs upvoter down

82

13

16

4

…

True!

Verifying Probabilistic Properties Acceptance Sampling

- “The probability is at least 0.1 that ”
- Symbolic Methods
- Pro: Exact solution
- Con: Works only for restricted class of systems

- Sampling
- Pro: Works for any system that can be simulated
- Con: Uncertainty in correctness of solution

- Symbolic Methods

Our Approach Acceptance Sampling

- Use simulation to generate sample execution paths
- Use sequential acceptance sampling to verify probabilistic properties

Error Bounds Acceptance Sampling

- Probability of false negative: ≤
- We say that is false when it is true

- Probability of false positive: ≤
- We say that is true when it is false

Acceptance Sampling Acceptance Sampling

- Hypothesis: Pr≥()

1 – Acceptance Sampling

Probability of acceptingPr≥() as true

Actual probability of holding

Performance of Test1 – Acceptance Sampling

False negatives

Probability of acceptingPr≥() as true

False positives

Actual probability of holding

Ideal Performance1 – Acceptance Sampling

False negatives

Probability of acceptingPr≥() as true

Indifference region

–

+

False positives

Actual probability of holding

Actual PerformanceNumber of Acceptance Samplingpositive samples

Number of samples

Graphical Representation of Sequential TestAccept Acceptance Sampling

Number ofpositive samples

Continue sampling

Reject

Number of samples

Graphical Representation of Sequential Test- We can find an acceptance line and a rejection line given , , , and

A,,,(n)

R,,,(n)

Accept Acceptance Sampling

Number ofpositive samples

Continue sampling

Reject

Number of samples

Graphical Representation of Sequential Test- Reject hypothesis

Accept Acceptance Sampling

Number ofpositive samples

Continue sampling

Reject

Number of samples

Graphical Representation of Sequential Test- Accept hypothesis

Verifying Probabilistic Properties Acceptance Sampling

- Verify Pr≥() with error bounds and
- Generate sample execution paths using simulation
- Verify over each sample execution path
- If is true, then we have a positive sample
- If is false, then we have a negative sample

- Use sequential acceptance sampling to test the hypothesis Pr≥()

Verification of Nested Probabilistic Statements Acceptance Sampling

- Suppose , in Pr≥(), contains probabilistic statements
- Error bounds ’ and ’ when verifying

True, false, Acceptance Samplingor anothersample?

Verification of Nested Probabilistic Statements- Suppose , in Pr≥(), contains probabilistic statements

Accept Acceptance Sampling

With ’ and ’ = 0

Number ofpositive samples

Continue sampling

Reject

Number of samples

Modified Test- Find an acceptance line and a rejection line given , , , , ’, and ’:

A,,,(n)

R,,,(n)

With Acceptance Sampling’ and ’ > 0

Number ofpositive samples

Number of samples

Modified Test- Find an acceptance line and a rejection line given , , , , ’, and ’:

Accept

Continue sampling

Reject

Number of Acceptance Samplingpositive samples

Number of samples

Modified Test- Find an acceptance line and a rejection line given , , , , ’, and ’:

A’,’,,(n)

Accept

’ – ’= 1 – (1 – ( – ))(1 – ’)

’ + ’= ( + )(1 – ’)

Continue sampling

R’,’,,(n)

Reject

Verification of Negation Acceptance Sampling

- To verify ¬ with error bounds and
- Verify with error bounds and

Verification of Conjunction Acceptance Sampling

- Verify 12 … n with error bounds and
- Verify each i with error bounds and /n

Verification of Path Formulas Acceptance Sampling

- To verify 1 U≤t2 with error bounds and
- Convert to disjunction
- 1 U≤t2 holds if 2 holds in the first state, or if 2 holds in the second state and 1holds in all prior states, or …

- Convert to disjunction

More on Verifying Until Acceptance Sampling

- Given 1 U≤t2, let n be the index of the first state more than t time units away from the current state
- Disjunction of n conjunctions c1 through cn, each of size i
- Simplifies if 1 or 2, or both, do not contain any probabilistic statements

Performance Acceptance Sampling

= 0.01

= 0.9

Average number of samples

= = 10-10

= = 10-1

Actual probability of holding

Performance Acceptance Sampling

= = 10-2

= 0.9

=0.001

=0.002

Average number of samples

=0.005

=0.01

=0.02

=0.05

Actual probability of holding

Summary Acceptance Sampling

- Model independent probabilistic verification of discrete event systems
- Sample execution paths generated using simulation
- Probabilistic properties verified using sequential acceptance sampling
- Easy to trade accuracy for efficiency

Future Work Acceptance Sampling

- Develop heuristics for formula ordering and parameter selection
- Use in combination with symbolic methods
- Apply to hybrid dynamic systems
- Use verification to aid controller synthesis for discrete event systems

ProVer: Probabilistic Verifier Acceptance Sampling

http://www.cs.cmu.edu/~lorens/prover.html

Download Presentation

Connecting to Server..