A Formal Monitoring-based Framework for Software Development and Analysis

1. A Formal Monitoring-based Framework for Software Development and Analysis Feng Chen Marcelo D'Amorim Grigore Rosu University of Illinois at Urbana-Champaign

2. Motivation :Increase Software Reliability Traditional formal verification methods High confidence, do not always scale up well Testing Scalable, but adhoc, no correctness guarantees What do we really want … ? • Prove a program correct? • Correct execution! A => B … but isn’t the price too big?

3. Overview • Monitoring Software Systems • Monitoring-Oriented Programming (MOP) • Java-MOP • An HTTP-Server Example • Conclusion and Future Work

4. Avoiding Complexity whileEnsuring Safety by Monitoring (Example by Lui Sha - Simplex) Joe is a student in an algorithms class Final exam is to develop a sorting algorithm If incorrect then Joe fails the class If provably correct but inefficient then he gets B If both correct and efficient then he gets A Hm, heap sort is efficient but hard to prove correct; insertion sort is trivial to prove correct but it is not efficient

5. Monitor if vector is sorted Insertion sort Heap sort no O(n log(n)) O(n2) provably correct yes O(n) Joe’s Solution Joe has an efficient and provably correct sorting algorithm! He avoided proving heap sort correct, which is hard!

6. Monitoring in Engineering Most engineering disciplines take monitoring as basic design principle Fire alarms in buildings Fuses in electronics Watchdogs in hardware Why not to do the same in software eng.? Monitoring-oriented programming

7. Monitoring-Oriented Programming (MOP) Proposed at RV 2003 Monitoring already used in software development Design By Contract, Runtime Verification (MaC, PaX, …) MOP underlying motivations and challenges • Can one generically and formally capture the various approaches to monitoring and runtime verification in a unified manner? • Monitoring should be a design and software development principle. Write programs having monitoring in mind!

8. 3. Recovery Specification S1 S2 . . . . . Sn 1. Observation 2. Checking What are the foundational principles of monitoring programs? Program

9. Goal of MOP The different approaches to RV rely on implicit, carefully chosen instance of the 3 principles • Observation • Checking • Recovery MOP aims at separation and individual advancement of the three principles Three ways to look at MOP • Merging specification and implementation • Extending languages with logic-based statements • A light weighted formal method

10. MoP Example: A traffic light monitor Following monitor can appear in a Java program ... /*@ logic=FTLTL { predicate red = tlc.state.getColor() == 1; predicate green = tlc.state.getColor() == 2; predicate yellow = tlc.state.getColor() == 3; formula : []( green -> (! red U yellow)); Violation handler : … any code } @*/ ... It is pre-compiled into actual Java code ... switch(bttFsmState) { case -1 : break; case 1 : bttFsmState = tlc.state.getColor()==3 ? 1 : tlc.state.getColor()==2 ? tlc.state.getColor()==1 ? 0 : 2 : 1; break; case 2 : bttFsmState = tlc.state.getColor()==3 ? 1 : tlc.state.getColor()==1 ? 0 : 2; break; } if(bttFsmState == 0) { … the “recovery” code …}} ...

11. MoP Features andDesign Principles

12. Extending Languages with Logics • Do not modify host languages • Able to reuse heritage systems • Evolve with languages and compilers • Specifications can be • Annotations in the source code • Especially for checkpoints • In separated specification files • Separation of concerns • Improved reusability

13. MOP Specification Syntax Example Definition /************** Heading starts ****************/ [attribute]* <Type> <Name> logic= <Logic Name> { /************* Body starts ********************/ ... Specification Body ... /************* Handler starts *****************/ [Violation Handler: ...handling the violation...] [Validation Hander: ...handling the validated...] } class-inv logic = ERE { var int flag = -1; event suspend: called(void suspend()) {flag = 1;}; event resume: called(void resume()) {flag = 2;}; formula: (suspend resume)* Violation Handler: if (flag == 2) { System.out.println("resume() called before suspend() in HttpClient! Adding client back to queue ..."); synchronized(suspendedClients){ suspendedClients.add(thisObject); } } else { System.out.println("suspend() called twice before resume() in HttpClient!"); } }

14. MOP Specification Syntax • Attributes • static, inline/outline, sync/async • Types of specifications: • class invariants, interface constraints, method pre-post conditions, checkpoints. • User-defined handlers • Violation and/or validation handlers • Not all combinations always possible

15. Sorting Example method (HeapSort(int[] a)) logic=JML { ensures isOrdered(a); Violation handler: InsertionSort(a); } … if (! isOrdered(a)){ InsertionSort(a); } …

16. Security Example class-inv logic=ERE { event auth = authentication(); event use = accessResource(); formula: (! auth)* use; Validation handler: authentication(); } switch ($state) { case 0 : $state = use ? 1 : ! auth ? 0 : -1; break; } if ($state == 1) authentication();

17. Extensible Logic Framework

18. Workflow of MOP

19. Logic Plug-ins • No silver-bullet logic to express any requirements • To add a new MOP requirements formalism, all one needs to do is to provide a logic plug-in • Logic plug-in consists of a language shell and a logic engine; several logic plug-ins can share a logic engine • The I/O of the logic plug-in is standardized to facilitate extensibility • Logic plug-in encodes a monitor synthesis algorithm: formula -> monitor • WWW repository of logic plug-ins fsl.cs.uiuc.edu/mop

20. Current Logic plug-ins • Linear Temporal Logic (LTL) • Past Time LTL • Future Time LTL • Extended Regular Expressions (ERE) • Design By Contract: Jass, JML (partly) • To be supported: • RTL, MTL, … • Users can easily add new logics via provided standardized interface

21. Future time LTL plug-in Input: [] (green -> (! red) U yellow) Output: Declaration. int state; Initialization. state = 1; Monitoring Body. switch (state) { case 1 : state = yellow ? 1 : green ? (red ? -1 : 2) : 1; case 2 : state = yellow ? 1 : red ? -1 : 2; } Failure Condition. state == -1 Algorithm jointly with Havelund (RV’01)

22. Past time LTL plug-in Input: start(P) -> [Q, end(R \/ S)) Output: Declaration. boolean now[3], pre[3] Initialization. now[3] = R || S; now[2] = Q; now[1] = P; Monitoring Body. now[3] = R || S; now[2] = (pre[2] || Q) && (now[3] || (! pre[3])); now[1] = P; Failure Condition. now[1] && (! pre[1]) && (! now[2]) Algorithm jointly with Havelund (TACAS’02)

23. Extended RE plug-in Input: ~((~ empty)(green red)(~ empty)) Output: Declaration. int state; Initialization. state = 0; Monitoring Body. switch (state){ case 0 : state = (yellow || red) ? 0 : green? 1 : -1; case 1 : state = green ? 1 : yellow ? 0 : -1; } Failure Condition. state == -1 Algorithm jointly with Sen (RV’03)

24. Java-MOP A prototype supporting part of the desired MOP features in Java

25. Java-MOP Overview • Provides GUI and command-line interfaces • Online interface: http://fsl.cs.uiuc.edu/mop • Supports FTLTL, PTLTL, ERE, Jass, JML • Uses AspectJ for monitor integration • Limitations • Can only monitor scalar fields of classes for class invariants • The event generation and analysis are not atomic

26. Java-MOP Architecture

27. Java-MOP at WorkA buggy HTTP server- taken from an IBM repository of buggy concurrent Java programs

29. Request access to the server Granted? No Yes work Put itself into the waiting queue 1 Get a client from the waiting queue, and resume it suspend itself 3 2 Exit

30. 2 suspend Clienti Clientj SuspendedClients Clienti 1 add Has been removed from the waiting queue! Stopped remove & resume 3 2 suspend Exit

31. A bad trace

32. The Bug • Caused by “unexpected” thread interleaving • Not easy to analyze and test • Root cause • Violation of the basic assumption for suspend() and resume(): (suspend() resume())* • Can we fix it on-the-fly in case of violation? • Yes: put the client back into the queue to allow another client to resume it later.

34. Trace with Monitor

35. Conclusion and Future Work • MOP: a uniform logic-independent monitoring framework. • Java-MOP: a prototype for Java • Incorporate more logics/paradigms, especially those that are widely accepted • Complete JML, RTL, MTL, … • WWW repository of logic plug-ins • Turn Java-MOP into a mature tool • Currently a prototype • Use it on real and large applications • Support more programming languages