Performance Model Checking Scenario-Aware Dataflow

1. Performance Model CheckingScenario-Aware Dataflow Bart Theelen, Marc Geilen, Jeroen Voeten

2. Overview • Dataflow Formalisms • Timed Probabilistic Systems • Performance Model Checking • Experimental Results • Conclusions & Outlook

3. Dataflow Formalisms Example digital signal processing areas Streaming Multi-Media Loop-Control in Mechatronics MP3 Decoder Dataflow formalisms describe task graphswhere potential parallelism is made explicit

4. Dataflow Formalisms: Expressivity vs Analyzability Kahn ProcessNetworks Scenario-AwareDataflow Synchronous Dataflow(Weighted Marked Graphs) Stuijk, et al. Scenario-Aware Dataflow: Modeling, Analysis and Implementation of Dynamic Applications. SAMOS’11

5. Scenario-Aware Dataflow (SADF) • Scenario = operation modes with similar resource usage • Detectors control processes by sending scenario-valued tokens • Detectors contain automata to capture scenario occurrences • Real-life: data-dependent control behaviour (normal state machine) • Modelling worst/best-case only: non-deterministic state machine • Modelling worst/best-case & average-case: Markov chain MPEG-4 Decoder kernel control channel data channel rate x = {30, 40, 50 ,60, 70, 80, 99} detector tokens

6. Scenario-Aware Dataflow (SADF) • Processes run in parallel according to ‘extended actor semantics’ • Determine scenario depending on • Kernels & Detectors: scenario-valued control tokens • Detectors: next state of Markov chain • Wait until sufficient tokens available • Perform the actual task (sample from discrete time distribution) • Produce and consume tokens MPEG-4 Decoder x = {30, 40, 50 ,60, 70, 80, 99}

7. Timed Probabilistic Systems (TPS) • Compositional semantic model with • guarded (interactive) action transitions • probabilistictransitions • deterministic timetransitions • Alternates action/time transitions with probabilistic fan-out • Pattern for generic discrete execution time distributions time advances exactlyti time units with probability pi for i=1,…,n a p t t1 p1 1 τ tn pn 1

8. Illustrative Example TPS for Detector D TPS for Kernel A

9. Semantic Properties • Model checking based on (relevant) state-space • Exploit semantic properties to limit state-space explosion • SADF satisfies various semantic properties • Time additivity, action persistency, action urgency, action determinacy • Only non-determinism between actions as a result of concurrency • Policy for resolving non-determinism does not effect net behaviour policy for resolving non-determinism may however effect performance result

10. Performance Model Checking Direct computation of quantitative resultsbased on model checking techniques • Broad variety of performance metrics • Mostly complex reward-based properties • Mostly time-related properties Policy for resolving non-determinism only affects Max Buffer Occupancy

11. Model Checking Strategy - Theory TPS per SADF Process Parallel Composition |S| TPS of Complete SADF Model S2 Resolve Non-Determinism p1 S1 a |S’| ≤|S| Deterministic TPS of Complete SADF Model S3 p2 Move Transition Labels into States S2,a p1 >|S’| |S| Discrete Markov Chain S1, - Remove Irrelevant States p2 S3, a Information on occurrence of actions and time available through reward functions on states only <<|S| |Sc| Reduced Discrete Markov Reward Model Compute Equilibrium Performance Number

12. Throughput MPEG-4 throughput = average number of frames per second = average number of RC firings per time unit Let {Xi | i ≥ 1} denote Markov chain with state space S Define reward c to identify firing completion action of RC If c(s) = 1 for state s, it is relevant, otherwise it is irrelevant Δ is temporalreward function denoting amount of time elapsed since previous RC firing

13. Metric-Dependent State-Space Reduction • If relevant positive recurrent state for ergodic Markov chain exists, then • Reduction yields ergodic Markov chain • Reduction preserves performance properties

14. Model Checking Strategy - Practice Transformed TPS per SADF Process TPS per SADF Process Parallel Composition |S| |S’’| Transformed TPS of Complete SADF Model TPS of Complete SADF Model Resolve Non-Determinism |S’| Deterministic TPS of Complete SADF Model Move Transition Labels into States |S| Discrete Markov Chain |S| Discrete Markov Chain Remove Irrelevant States |Sc| Reduced Discrete Markov Reward Model Parallel composition with on-the-fly reduction and resolving non-determinism Compute Equilibrium Performance Number

15. Experimental Results - requires more than 1.5GB of memory ¤ takes more than 6 hours Reduction after resolving non-determinism

16. Statistical Model Checking as Alternative • Statistical model checking supported by modelling SADF in POOSL • POOSL is much more expressive than SADF but also has TPS semantics • Compositional estimation of confidence intervals for long-run averages

17. Conclusions & Outlook • Performance model checking approach for SADF • Compositional TPS semantics with discrete time distributions • Exploit semantic properties • Removal of metric-dependent irrelevant states • On-the-fly construction of relevant state-space • Broad variety of pre-defined performance metrics • All expressible as temporal reward formula • Statistical model-checking for long-run averages as alternative • Increase flexibility to allow computing user-defined metrics • Lift Markov chain reduction to bisimulation reduction on TPS • Support temporal rewards as property specification language Could contemporary quantitative model checkers supporting Probabilistic Timed Automata be a suitable alternative?