Model Checking for Probabilistic Timed Systems

Model Checking for Probabilistic Timed Systems Jeremy Sproston Università di Torino VOSS Dagstuhl seminar 9th December 2002

The problem • Model checking probabilistic timed systems • In probabilistic systems: • Probabilistic choice between alternatives Example: electronic coin flipping in randomized algorithms • In timed systems: • Timing parameters are critical for the correct functioning of the system Example: the system must meet a certain deadline • In probabilistic timed systems: • Coexistence of probabilistic choice and timing

The focus • Probabilistic versions/extensions of timed automata (Alur and Dill 1994) • Timed automaton = finite-state graph + clocks + clock constraints • Clocks are an appropriate device for modelling time-dependent behaviour • A clock is a real-valued variable which increases at the same rate as real time Clocks can be reset when system transitions occur Therefore, clocks can measure the exact amount of time elapsed since a particular transition

{x:=0} on off {x:=0} x3 x2 Timed automata • Finite-state graph + clocks + clock constraints (examples: x3, x-y>5) • Example: light switch

Timed CTL • CTL: a request will always follow a response ⃞(request -> (⃟ response)) • TCTL: timed CTL • Alur, Courcoubetis and Dill (1993) • Henzinger et al. (1994) • A request will always follow a response within 5 milliseconds ⃞(request -> (⃟ 5 response)) • Use ⊨T for the satisfaction relation of TCTL

Off, x=3.5 Off, x=3.5 Off, x=3.7 … … Timed automata: semantics • Problem: underlying semantic model is • infinite-state: (node space) x R(number of clocks) • infinitely branching: for example • Model checking classically assumes a finite state space

Model checking for timed automata • Reduce to a finite state space: clock equivalence • Partitioning bounded by the maximal constant used in the timed automaton or the TCTL formula • Clock equivalent states satisfy the same clock constraints now and in the future y 2 1 x 1 2

Model checking for timed automata • Region equivalent states have the same • node • clock equivalence class • Construct finite-state region graph (transition system) • States: region equivalence classes • Transitions: Time transitions Discrete transitions E.g. crossing an edge with {x:=0}

Model checking for timed automata • Let: • TA be a timed automaton, • T be a TCTL formula, • RG(TA, T) be the region graph of TA, T • TA ⊨T T if and only if RG(TA, T) ⊨ • where ⊨and are “untimed” versions of ⊨T and T • Key result of Alur, Courcoubetis and Dill (1993)

Real-time probabilistic processes • Alur, Courcoubetis and Dill (1991:ICALP, 1991:Real-Time) • Similar to Generalized Semi-Markov Processes (Whitt (1980), Glynn (1989)) • A fully probabilistic model

{y} on off {y} y x,y {x} Real-time probabilistic processes • Finite-state graph + clocks + clock scheduling function + probabilistic branching over edges + probabilistic clock resetting • Example: light switch y:=Uniform(1,30) x:=3

Timed CTL revisited • Interpreting “branching-time” logic over fully probabilistic systems s ⊨ means “the probability that the computations starting in s satisfy  is > 0” s ⊨ means “the probability that the computations starting in s satisfy  is =1” • Alur, Courcoubetis and Dill (1991:ICALP) interpret TCTL (branching-time) over real-time probabilistic processes

Timed CTL revisited • For example: ⃞(request -> (⃟ 5 response)) With probability 1, a request is followed by a response within 5 milliseconds • Use R-TCTL to denote the logic, and ⊨R for its satisfaction relation

Real-time probabilistic processes: semantics • Real-time probabilistic processes use clocks, so are infinite-state Markov processes • Clocks are set to negative values drawn from continuous probability distributions • When at least one clock reaches 0, a transition is triggered

Model checking for real-time probabilistic processes • Again, reduce to a finite state space using (a version of) clock equivalence • The set of clocks to reach 0 first is the same for all clock equivalent states y -3 -2 -1 x -1 -2 -3

Model checking for real-time probabilistic processes • Construct finite-state region graph (transition system) • States: region equivalence classes • Transitions: Discrete transitions E.g. crossing an edge triggered by y; reset y within (1,2) Time transitions

Model checking for real-time probabilistic processes • Let: • RTPP be a real-time probabilistic process • R be a R-TCTL formula, • RG(RTPP, R) be the region graph of RTPP, R • RTPP ⊨R R if and only if RG(RTPP, R) ⊨ • where ⊨and are “untimed” versions of ⊨R and R • Key result of Alur, Courcoubetis and Dill (1991:ICALP)

Probabilistic timed automata • Introduced by Jensen (1995), Kwiatkowska et al. (2002) • Finite-state graph + clocks + clock constraints + probabilistic branching over edges • Example: light switch {x:=0} {x:=0} 0.01 0.99 0.99 on off x3 0.01 x2

Probabilistic timed CTL • PCTL (Probabilistic CTL): Hansson and Jonsson (1994), Bianco and de Alfaro (1995) • The system will fail with probability < 0.01 P<0.01[⃟failure] • PTCTL (timed PCTL): Kwiatkowska et al. (2002) • The system will fail within 5 hours with probability < 0.01 P<0.01[⃟ 5 failure] • Use ⊨P to denote the satisfaction relation of PTCTL

Model checking probabilistic timed automata • Probabilistic timed automaton semantics: • Infinite-state, infinite-branching Markov decision process • Again, reduce to a finite state space using clock equivalence y 2 1 x 1 2

Model checking probabilistic timed automata • Construct finite-state region graph (Markov decision process) • States: region equivalence classes • Transitions: • Time transitions are as standard • Discrete transitions: for example 0.99 0.99 {x:=0} 0.01 0.01 on on fail fail y<3 x<7

Model checking probabilistic timed automata • Construct finite-state region graph (Markov decision process) • States: region equivalence classes • Transitions: • Time transitions are as standard • Discrete transitions: for example on 0.99 {y:=0} 0.99 {x:=0} 0.01 0.01 on on fail fail y<3 x<7

Model checking probabilistic timed automata • Let: • PTA be a probabilistic timed automaton, • P be a PTCTL formula, • RG(PTA, P) be the region graph of PTA, P • PTA ⊨P P if and only if RG(PTA, P) ⊨ • where ⊨and are “untimed” versions of ⊨P and p • Key result of Kwiatkowska et al. (2002)

Continuous probabilistic timed automata • Introduced by Kwiatkowska et al. (2000) • Finite-state graph + clocks + clock constraints + probabilistic branching over edges + probabilistic clock resetting • Example: light switch y:=Uniform(0,29) x:=0 y=30 0.01 y=30 0.99 x2 off1 on off2 x,y y x3 ∧y30 y30 y30 0.99 0.01 y=30

Model checking continuous probabilistic timed automata • Continuous probabilistic timed automata semantics • Infinite-state, infinitely branching probabilistic-nondeterministic system with continuous probability distributions • Again, reduce to a finite state space using clock equivalence

Model checking continuous probabilistic timed automata • Problems with clock equivalence: an example by Alur • Clock x is reset within (0,1) in node A; clock y is arbitrary • Some time elapses in node A • Then we move to node B; clock y is reset within (0,1) • 3 cases: (1) x<y, (2) x=y, (3) x>y • Probability of (2) is 0, but we do not know the probabilities of (1) and (3) (clock equivalence abstracts from the duration of the time transition in node A) x=1 A B x y x<1 y=1

Model checking continuous probabilistic timed automata • A partial solution: change the granularity of the time scale • For example, from granularity of 1 to granularity of 0.5 • Say we know that x  (0,0.5) • Say that y is then set within (0.5,1) • We know that y>x 1 1 0.5 1 0.5 1

Model checking continuous probabilistic timed automata • Given a time granularity, construct a finite-state region graph (Markov decision process) • States: region equivalence classes • Transitions: • Time transitions are standard • Handling of probabilistic branching over edges is straightforward • But how do we deal with resetting clocks according to continuous probability distributions?

Model checking continuous probabilistic timed automata • Representing continuously distributed clock resets in the region graph: • Integrating over time-unit intervals gives the probability of a clock being set within an interval • E.g. with a time granularity of 1, we integrate over intervals such as (0,1), (1,2), … • E.g. with a time granularity of 0.5, we integrate over intervals such as (0,0.5), (0.5, 1), … • But the relationship between the ordering on the fractional parts of the newly set clocks and the clocks which keep their old values is not obtainable • The probabilistic choice regarding this relationship is replaced with a nondeterministic choice

Model checking continuous probabilistic timed automata • Let: • CPTA be a probabilistic timed automaton, • P be a PTCTL formula, • n1 be the chosen time granularity, • RG(CPTA, P, n) be the region graph of CPTA, P, n • CPTA ⊨P P if RG(CPTA, P, n) ⊨ • where ⊨and are “untimed” versions of ⊨P and p • Key result of Kwiatkowska et al. (2000)

Model checking continuous probabilistic timed automata • Replacing probabilistic choice with nondeterministic choice introduces the possibility of an error in the computed probabilities • But we know that the maximum probability that CPTA satisfies a path formula is bounded from above by the maximum probability that the RG(CPTA, P, n) satisfies the path formula (similar with minimum) • For example: CPTA ⊨PP<0.01[⃟failure] if RG(CPTA, P, n) ⊨ P<0.01[⃟failure]

Conclusions: model checking timed automata • Achieved success in the form of the development of tools such as UPPAAL (Uppsala/Aalborg) and KRONOS (Grenoble) • Use of zone-based algorithms • Manipulate sets of clock equivalence classes

Conclusions: model checking real-time probabilistic processes • Activity died off after Alur, Courcoubetis and Dill’s 1991 papers • Interest renewed by the development of process algebras with generally distributed delays (Bravetti et al., D’Argenio et al) • Model checking of Semi-Markov Chains: Infante-Lopez et al. (2001)

Conclusions: model checking probabilistic timed automata • Model checking using PRISM (Kwiatkowska, Norman and Parker (2002)) and: • Region graphs • Discrete-time semantics (given restrictions on clock constraints to xc and xc) • Based on discrete-time semantics for timed automata developed by Henzinger et al. (1992), Asarin et al. (1998), Bozga et al. (1999) • Case studies: FireWire (Kwiatkowska et al. (2002:FAC)), IEEE802.11 (Kwiatkowska et al. (2002:PAPM-PROBMIV))

Conclusions: model checking probabilistic timed automata • Zone-based algorithms for probabilistic timed automata: • Must carefully distinguish zones which have different probabilities • Kwiatkowska et al. (2001:CONCUR, 2002:TCS) • Case study: FireWire • Kwiatkowska et al. (2002:FAC), Daws et al. (2002)

Conclusions: model checking continuous probabilistic timed automata • Increasing the time granularity blows up the state space • Exists a need to concentrate on restricted subclasses

Model Checking for Probabilistic Timed Systems