Bridging the gap between specification and implementation

1. Bridging the gap between specification and implementation Insup Lee Department of Computer and Information Science University of Pennsylvania November 9, 2004

2. The gap between specification and implementation • Problem • gap between an abstract model and the implementation • scalability challenge (software size and complexity) • Approaches • Software model checking • Model-based code generation • Test generation from specification • Run-time verification/checking • Model checking • Formal, Complete • Does not scale well • Checks design, not implementation • Testing • Tests an implementation directly • Informal, Incomplete Requirements Design specification Implementation

3. Run-time Verification and its Application to Security • Run-time verification • The MaC framework • Java-MaC • Steering • Model-based security checking • Security automata • Model carrying code • Current work

4. Runtime Verification Program Verifier Execution Information Check Sat/Unsat Feedback User

5. Run-time verification • Run-time monitoring and checking w.r.t. formal specification • Ensures the runtime compliance of the current execution of a system with its formal requirement • Steps • Specifyformal requirements • Extractinformation from current executing program • Check the execution against formal requirements • Steer the computation to a safe state • Complementary methodology to formal verification and program testing • Validate implementation • Not complete: guarantee for current execution • Prevention, avoidance, and detection & recovery • Joint work with S. Kannan, M. Kim, U. Sammapun, O. Sokolsky, M. Viswanathan

6. The MaC Framework Human Automatic Instrumentation Automatic Translation Automatic Translation Program Filter Event Recognizer Run-time Checker Program Informal Requirement Spec Input Monitoring Scripts Low-level Specification High-level Specification Static Phase low-level behavior high-level behavior Run-time Phase [Kim et al, ECRTS 99]

7. Design of the MaC Languages • Must be able to reason about both time instants and information that holds for a duration of time in a program execution. • Events and conditions are a natural division, which is also found in other formalisms such as SCR. • Conditions, which are true or false for a finite duration of time (e.g., is variable x >5?) • Events, which are either present or absent at some instant of time (e.g., is the control right now at the end of method f?). • Need temporal operators combining events and conditions in order to reason about traces. raiseGate start(position==100) end(position==100) position == 100 1:00:10 1:00:15 1:00:30 Time

8. Logical Foundation e2 e1 [e1 ,e2) [e1 ,e2) • Conditions interpreted over 3 values: true, false and undefined. • [., .) pairs a couple of events to define an interval. • start and end define the events corresponding to the instant when conditions change their value. [Lee et al, ICPDP 99]

9. Meta Event Definition Language (MEDL) • Expresses requirements using the events and conditions, sent by event recognizer. • Expresses the subset of safety properties. • Describes the safety requirements of a system, in terms of conditions that must always be true, and alarms (events) that must never be raised. • propertysafeRRC = IC -> GD; • alarm violation = start (!safeRRC); • Auxilliary variables may be used to store history. • endIC-> { num_train_pass’ = num_train_pass + 1; } ReqSpec <spec_name> /* Import section */ import event <e>; import condition <c>; /*Auxiliary variable */ var int <aux_v>; /*Event and condition */ event <e> = ...; condition <c>= ...; /*Property and violation */ property <c> = ...; alarm <e> = ...; /*Auxiliary variable update*/ <e> -> { <aux_v'> := ... ; } End

10. The MaC languages MEDL PEDL SADL • PEDL: abstraction • MEDL: abstract transformation • SADL: feedback • Run-time state: • method call • object state • local variables • Abstract state: • events • conditions

11. PEDL (Primitive Event Definition Language) • Primitive Event Definition Language (PEDL) • Maps the low-level state information of the system to high-level events. • Depends on target program implementation • Design Issues • What can be observed? • Passive and active probes • Software probe vs. hardware probe • Programming language-based vs. systems API-based • Java-PEDL • Provides primitives to refer to values of variables and to certain points in the execution of the program. • PEDL is defined so that events can be recognized in time linear to the size of the PEDL specification [Kim et al, FMSD 04]

12. Java-MaC Static Phase Java Bytecode MaC Specifications PEDL MEDL SADL MaC Compilers Instrumented Bytecode Event Recognizer Checker Steerer MaC Verifiers

13. Java-MaC Dynamic Phase Instrumented Java Program MaC Verifier Event Recognizer (PEDL) Execution Information Checker (MEDL) Feedback Steerer Sat/Unsat User

14. MaC Language - PEDL Railroad Crossing Property: - If train is crossing, then gate must be down - Train is crossing when position is between 30 and 50 position = 0 position = 20 position = 40 cross = true Gate.down() startGD cross position = 55 cross = false Gate.up() endGD position = 60 Abstraction - When train position is between 30 and 50 - When gate starts/ends being down export event startGD, endGD; export condition cross; // specify what to monitor monobj Train.position; monmeth Gate.up(); monmeth Gate.down(); // specify abstraction condition cross = (30 < Train.position) && (Train.position < 50); event startGD = endM(Gate.down()); event endGD = startM(Gate.up()); Java Program PEDL

15. MaC Language - MEDL Railroad Crossing Property: - If train is crossing, then gate must be down - Train is crossing when position is between 30 and 50 cross = true Violation startGD cross cross cross = false gateDown endGD Abstraction - When gate is down Property - If train is crossing, then gate must be down import event startGD, endGD; import conditions cross; // specify abstraction condition gateDown = [startGD, endGD); // specify propertyproperty safeRRC = cross -> gateDown; PEDL MEDL

16. Instrumentation class Train { int position; main() { position = 0; send(x,0); position = 20; send(x,20); position = 40; send(x,40); position = 55; send(x,55); } } class Train { int position; main() { position = 0; position = 20; position = 40; position = 55; } } = + monobj Train.position; Sent to Event Recognizer: [ (position,0), (position,20), (position,40), (position,55) ]

17. MaC Language - Components Railroad Crossing Property: - If train is crossing, then gate must be down - Train is crossing when position is between 30 and 50 position = 0 position = 20 position = 40 cross = true Violation Gate.down() startGD cross cross position = 55 cross = false gateDown Gate.up() endGD position = 60 Abstraction - When train position is between 30 and 50 - When gate starts/ends being down • Abstraction • - When gate is down • Property • - If train is crossing, then gate must be down Java Program PEDL MEDL

18. Evaluation Overview • Static Phase: Each property is represented as a tree • The most basic events/conditions/aux vars are at the leaf level • For PEDL, variable updates, start/end method events are leaves • For MEDL, imported events from PEDL and auxiliary variables are leaves • Composition of events and conditions link to events/conditions that are composed of • PEDL and MEDL are forests • Dynamic Phase: In both Event Recognizer (ER) and Checker, evaluation starts at the leaves and traverses up to the root • Evaluation starts only at leaves representing occurred events or changed conditions • Otherwise, no evaluation is done • Efficient

19. PEDL Graph export event startGD, endGD; export condition cross; monobj Train.position; monmeth Gate.up(); monmeth Gate.down(); condition cross = (30 < Train.position) && (Train.position < 50); event startGD = endM(Gate.down()); event endGD = startM(Gate.up());

20. MEDL Graph import event startGD, endGD; import conditions cross; condition gateDown = [startGD, endGD); property safeRRC = cross -> gateDown;

21. Algorithm • Static Phase: • Create PEDL and MEDL graphs • Assign a height to each node in both graphs • Dynamic Phase: • Maintain an evaluation list sorted by height • Add all occurred primitive events and changed conditions to the evaluation list at height 0 • For each event/condition in the evaluation list, • Call evaluate() method • Add its parent in the evaluation list (if not already in) • Repeat until the list if empty • Finishing • ER sends occurred events/changed condition to checker for each exported events/condition • Checker notifies user for each event in the alarm list. • Complexity • The size of the PEDL or MEDL graph is linear in the size of the formula. • Evaluation of a MEDL or PEDL formula on a single observation is linear in the size of the graph.

22. Java-MaC Dynamic Phase Instrumented Java Program MaC Verifier Event Recognizer (PEDL) Execution Information Checker (MEDL) Feedback Steerer (SADL) Sat/Unsat User

23. Steering process action invocation received steering condition satisfied violation action executed system event received action invoked detection checker

24. Simplex Architecture • Experimental controllers provide improved performance but uncertain stability properties • Can be dynamically added or replaced • Safety controller has the largest stability region us Safety EX x0 Decision Module u x Physical System Experimental ue SC Equilibrium state [L. Sha]

25. Inverted Pendulum (IP) Example g l m q uf M Experimental Controller Experimental Controller x Experimental Controller Decision Module Switching logic Written in C Safety Controller angle, track Device Drivers [L. Sha] volts

26. Steering Action Definition Language • SADL script • identifies object used in steering // the target of steering is the object dm of type DecisionModule // located in the class IP DecisionModule IP:dm; • defines steering actions // setSC() method of dm is invoked steering action change2SC = { call (IP:dm).setSC(); } • specifies steering location • locations in the code where the actions can be executed before read DecisionModule:volts;

27. IP and MaC Experimental Controller Experimental Controller Experimental Controller Written in Java Safety Controller angle, track monitor J N I MaC: Switching logic Device Drivers Decision Module volts steer [Kim et al, RV 02]

28. Applications • Where can we get specifications? • During the design specification and analysis phase, many properties may be identified and verified. • Reuse properties model checked during the design phase • E.g., more than 1000 properties during designing of flight control systems • Extract from (informal) requirements and specification documents • Security Policy • Extract from the target program • To ensure that the program has not been tampered • Model Carrying Code • Other application areas • Network routing simulation • Hardware design • Adaptable sensor network systems • Cheat detection in distributed game • Annual run-time verification workshop (01, 02, 03, 04, …)

29. Monitoring and Checking for Security Properties

30. Similar techniques, different purposes • Check security policy • Security automata, edit automata • Model-Carrying Code (MCC) • Intrusion detection • Extract from the target program to ensure that the program has not been tampered • Signature-based approach

31. Security Policy in Security/Edit Automata a = (malloc(q), q’=q’+q, q’< n) a a = (malloc(q), q’=q’+q, q’< n) a = (malloc(q), q’=q’+q, q’ >= n) a halt Example (modified from [BLW02]): Limit the amount of memory that an application can allocate for itself Property: application must not allocate memory more than n

32. Must not allocate more than n PEDL export event mallocCall; monmeth int malloc(int); event mallocCall = startM(malloc(int)); MEDL import event mallocCall; import action halt; var int memory; alarm violateMemoryPolicy = end(memory < 1000); mallocCall -> { // value(mallocCall,0) returns arg of malloc() memory’ = memory + value(mallocCall,0); } violateMemoryPolicy -> { invoke(halt); } SADL steering action halt = // exit before next malloc() call { call System.exit(); } before call malloc(int);

33. Model-Carrying Code (MCC) • How can we run untrusted code on our machine? • Untrusted code comes with a model of its security-relevant behavior • Users have their own security policies • Employ two types of checking • Static checking: to ensure that untrusted program’s model respects user’s security policy • Use model checking to check that Beh(Model) are in Beh(Policy) • Run-time checking: to ensure that program behaves as specified by model • Use runtime checking with • Model is a specification (Automata) • Events are system calls

34. MCC Framework [SVB+03]

35. Current Work on MaC • Java-MaC available for download • www.cis.upenn.edu/~rtg/mac • Minimum trace of run-time verification • MEDL-RE: MEDL with regular expressions • MaC with probability • Hierarchical IDS • Using MaC to detect failed/malicious sensor nodes

36. Minimal Trace for run-time verification • Large amounts of monitored data hampers communication between the system and the checker • How to extract minimum information necessary to check a given property? • Redundant information in traces • alarm dropReq = fault when respondingcondition responding = [acceptReq,response)event acceptReq = request when ready==trueevent fault = start(abort == true) • Trace with redundant data: request ready(t) abort(t) request abort(f) request abort(t) • Trace without redundant data: ready(t) request abort(t) • Approach: define minimally adequate trace, and MEDL automata

37. MEDL-RE: MaC with regular expression • Regular expression over events • Statement: RE R {Ē} = < R >, • Grammar of R: R ::= e | R.R | R+R | R* • Alphabet of R contains events used in R and events in its relevant set {Ē} • Regular expressions are neither events nor conditions and cannot be used alone • Events associated with RE R:startRE(R), success(R), fail(R) • Example • Three components of a media must start in the following order: video, caption, audio • RE media {} = < startVideo . startCaption . startAudio > • alarm notOrdered = fail(media) • Challenges • (Possibly infinite) multiple instances of (possibly overlapping) regular expressions

38. MaC with probability • Definition of Probability • P(E) = |E| / |S| Ex. Sample Space S = {H,T} and Event E = {H}, P(E) = 0.5 • Long run frequency: P(E) = lim n(E) n  n • where n = the number of experiments performed, n(E) = the number of outcomes belonging to the event E • Ex. experiment is performing coin flipping, P(H) is the number of times the outcome is H in relative to the number of times the coin flipping is performed • Challenges • How to detect experiments from execution trace • Accuracy of probability calculated from execution trace • Possible approach • Detecting experiment using events or regular expressions • Calculate probability from execution trace with confidence interval • Possible syntax: E ~ [ p, Exp] and C ~ [ p, Exp] where  = < | > | <= | >=, p is probability, Exp is an experiment

39. Extend MaC Architecture for IDS

40. Thank You!