1 / 64

Static and Dynamic Analysis at JPL

Static and Dynamic Analysis at JPL. Klaus Havelund. Mars Science Laboratory (MSL). planned launch 2011 biggest rover so far to be sent to Mars programmer team of 30 testing team of 10+ people programming language is C, 3 M LOC highly multi-threaded (over 160 threads).

Download Presentation

Static and Dynamic Analysis at JPL

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Static and Dynamic Analysisat JPL Klaus Havelund

  2. Mars Science Laboratory (MSL) • planned launch 2011 • biggest rover so far to be sent to Mars • programmer team of 30 • testing team of 10+ people • programming language is C, 3 M LOC • highly multi-threaded (over 160 threads)

  3. specification and programming Let’s see … a command should always succeed … program input output

  4. specification and programming specification formal program input output

  5. specification and programming specification formal relationship program input output

  6. specification and programming specification formal • code generation: • from state machines • graphical • textual • -from data formats (XML) refinement program input output

  7. specification and programming specification formal model checking [](request -> <>response) static analysis abstraction program input output

  8. specification and programming specification formal Good practice analysis using commercial static analyzers. static analysis program input output

  9. a checkable Java coding standard

  10. why a standard?two perspectives • more reliable code • code that is easier to read, leading to • code that is easier to develop • code that is easier to maintain • code that is easier to share style

  11. the choices good • industry standard • organizational standard • project standard • personal standard • no standard bad

  12. basic questions 1. do we need to agree on a standard? carefully designed? or lots of rules, which can be turned on/off? 2. should it be 100% checkable? 3. do we care about naming, style, doc? 4. how many rules?

  13. C coding standard

  14. free tools coding error checker coding standard checkers

  15. specification and programming specification formal dynamic analysis program input output

  16. model-based testing

  17. model-based testing http://spinroot.com/swarm/index.html reference file system do :: mkdir dir :: cd dir :: rm file :: .. … od SPIN real file system abstract state abstract

  18. runtime verification specification and programming specification formal program input output

  19. tool-based log file analysis

  20. problems with testing FSW • flight software engineers work under tight schedules: hard to access. • system = hardware + software: it is cumbersome to run. • difficult to determine what events to monitor. .h .c .h A B

  21. separation of concerns log log log log

  22. architecture log LogMaker [e1,e2,…,en] spec LogScope violations

  23. command execution command dispatch success dispatch failure failure

  24. example log ... COMMAND 7308 { Args := ['CLEAR_RELAY_PYRO_STATUS'] Time := 51708322925696 Stem := "POWER_HOUSEKEEPING" Number := "4" type := "FSW” } EVR 7309 { message := "Dispatched immediate command POWER_HOUSEKEEPING: number=4, seconds=789006392, subseconds=1073741824." Dispatch := "POWER_HOUSEKEEPING" Time := 51708322925696 name := "CMD_DISPATCH" level := "COMMAND" Number := "4” } ... EVR 7311 { name := "POWER_SEND_REQUEST" Time := 51708322925696 message := "power_queue_card_request- sending request to PAM 0." level := "DIAGNOSTIC” } EVR 7312 { message := "Successfully completed command POWER_HOUSEKEEPING: number=4." Success := "POWER_HOUSEKEEPING" Time := 51708322944128 name := "CMD_COMPLETED_SUCCESS" level := "COMMAND" Number := "4” } EVR 7313 { name := "PWR_REQUEST_CALLBACK" Time := 51708322944128 message := "power_card_request - FPGA request successfully sent to RPAM A." level := "DIAGNOSTIC” } CHANNEL 7314 { channelId := "PWR-3049" DNChange := 67 dnUnsignedValue := 1600 type := "UNSIGNED_INT" Time := 51708323217408 ChannelName := "PWR-BCB1-AMP” } ... COMMAND 9626 { Args := ['set_device(1)', 'TRUE'] Time := 51708372934400 Stem := "RUN_COMMAND" Number := "18" type := "FSW” } EVR 9627 { message := "Validation failed for command RUN_COMMAND: number=18." DispatchFailure := "RUN_COMMAND" Time := 51708372934499 name := "CMD_DISPATCH_VALIDATION_FAILURE" level := "COMMAND" Number := "18” } ... ?

  25. example log ... COMMAND 7308 { Args := ['CLEAR_RELAY_PYRO_STATUS'] Time := 51708322925696 Stem := "POWER_HOUSEKEEPING" Number := "4" type := "FSW” } EVR 7309 { message := "Dispatched immediate command POWER_HOUSEKEEPING: number=4, seconds=789006392, subseconds=1073741824." Dispatch := "POWER_HOUSEKEEPING" Time := 51708322925696 name := "CMD_DISPATCH" level := "COMMAND" Number := "4” } ... EVR 7311 { name := "POWER_SEND_REQUEST" Time := 51708322925696 message := "power_queue_card_request- sending request to PAM 0." level := "DIAGNOSTIC” } EVR 7312 { message := "Successfully completed command POWER_HOUSEKEEPING: number=4." Success := "POWER_HOUSEKEEPING" Time := 51708322944128 name := "CMD_COMPLETED_SUCCESS" level := "COMMAND" Number := "4” } EVR 7313 { name := "PWR_REQUEST_CALLBACK" Time := 51708322944128 message := "power_card_request - FPGA request successfully sent to RPAM A." level := "DIAGNOSTIC” } CHANNEL 7314 { channelId := "PWR-3049" DNChange := 67 dnUnsignedValue := 1600 type := "UNSIGNED_INT" Time := 51708323217408 ChannelName := "PWR-BCB1-AMP” } ... COMMAND 9626 { Args := ['set_device(1)', 'TRUE'] Time := 51708372934400 Stem := "RUN_COMMAND" Number := "18" type := "FSW” } EVR 9627 { message := "Validation failed for command RUN_COMMAND: number=18." DispatchFailure := "RUN_COMMAND" Time := 51708372934499 name := "CMD_DISPATCH_VALIDATION_FAILURE" level := "COMMAND" Number := "18” } ... ?

  26. example log ... COMMAND 7308 { Args := ['CLEAR_RELAY_PYRO_STATUS'] Time := 51708322925696 Stem := "POWER_HOUSEKEEPING" Number := "4" type := "FSW” } EVR 7309 { message := "Dispatched immediate command POWER_HOUSEKEEPING: number=4, seconds=789006392, subseconds=1073741824." Dispatch := "POWER_HOUSEKEEPING" Time := 51708322925696 name := "CMD_DISPATCH" level := "COMMAND" Number := "4” } ... EVR 7311 { name := "POWER_SEND_REQUEST" Time := 51708322925696 message := "power_queue_card_request- sending request to PAM 0." level := "DIAGNOSTIC” } EVR 7312 { message := "Successfully completed command POWER_HOUSEKEEPING: number=4." Success := "POWER_HOUSEKEEPING" Time := 51708322944128 name := "CMD_COMPLETED_SUCCESS" level := "COMMAND" Number := "4” } EVR 7313 { name := "PWR_REQUEST_CALLBACK" Time := 51708322944128 message := "power_card_request - FPGA request successfully sent to RPAM A." level := "DIAGNOSTIC” } CHANNEL 7314 { channelId := "PWR-3049" DNChange := 67 dnUnsignedValue := 1600 type := "UNSIGNED_INT" Time := 51708323217408 ChannelName := "PWR-BCB1-AMP” } ... COMMAND 9626 { Args := ['set_device(1)', 'TRUE'] Time := 51708372934400 Stem := "RUN_COMMAND" Number := "18" type := "FSW” } EVR 9627 { message := "Validation failed for command RUN_COMMAND: number=18." DispatchFailure := "RUN_COMMAND" Time := 51708372934499 name := "CMD_DISPATCH_VALIDATION_FAILURE" level := "COMMAND" Number := "18” } ... ?

  27. specification languagesfor trace analysis • programming languages (Python at JPL) • state machines • regular expressions • temporal logic • grammars most commonly used “formal” trace logics

  28. the first scripture trigger consequences look:DRILL_DMP\ evr(CMD_DISPATCH,positive)\ evr(CMD_COMPLETED_SUCCCESS,positive)\ evr(CMD_COMPLETED_FAILURE,negative)\ chan(id:CMD-0004,positive,contains opcode of last immediate command)\ chan(id:CMD-0007,positive)\ chan(id:CMD-0001,negative)\ chan(id:CMD-0009,negative)\ prod(name:DrillAll,1,*)

  29. property P1 P1: Whenever a flight software command is issued, then eventually an EVR should indicate success of that command

  30. recall log COMMAND 7308 { Args := ['CLEAR_RELAY_PYRO_STATUS'] Time := 51708322925696 Stem := "POWER_HOUSEKEEPING" Number := "4" type := "FSW” } . . . EVR 7312 { message := "Successfully completed command POWER_HOUSEKEEPING: number=4." Success := "POWER_HOUSEKEEPING" Time := 51708322944128 name := "CMD_COMPLETED_SUCCESS" level := "COMMAND" Number := "4” }

  31. recall log COMMAND 7308 { Args := ['CLEAR_RELAY_PYRO_STATUS'] Time := 51708322925696 Stem := "POWER_HOUSEKEEPING" Number := "4" type := "FSW” } P1: Whenever a COMMAND is issued with the Type field having the value "FSW”, the Stem field (command name) having some unknown value x, and the Number field having some unknown value y, then eventually an EVR should occur, with the field Success mapped to x and the Number field mapped to y. . . . EVR 7312 { message := "Successfully completed command POWER_HOUSEKEEPING: number=4." Success := "POWER_HOUSEKEEPING" Time := 51708322944128 name := "CMD_COMPLETED_SUCCESS" level := "COMMAND" Number := "4” }

  32. recall log COMMAND 7308 { Args := ['CLEAR_RELAY_PYRO_STATUS'] Time := 51708322925696 Stem := "POWER_HOUSEKEEPING" Number := "4" type := "FSW” } pattern P1: COMMAND{Type:"FSW", Stem:x, Number:y} => EVR{Success:x, Number:y} . . . EVR 7312 { message := "Successfully completed command POWER_HOUSEKEEPING: number=4." Success := "POWER_HOUSEKEEPING" Time := 51708322944128 name := "CMD_COMPLETED_SUCCESS" level := "COMMAND" Number := "4” }

  33. pattern P1: COMMAND{Type:"FSW", Stem:x, Number:y} => EVR{Success:x, Number:y}

  34. pattern syntax pattern ::= 'pattern' NAME ':' event '=>' consequence consequence ::= event | '!' event | '[' consequence1,...,consequencen ']’ | ‘{' consequence1,...,consequencen ‘}'

  35. property P2 P2: Whenever a COMMAND is issued with the Type field having the value "FSW”, the Stem field (command name) having some unknown value x, and the Number field having some unknown value y, Then an EVR should thereafter not occur, with the field Failure mapped to x and the Number field mapped to y. pattern P2: COMMAND{Type:"FSW", Stem:x, Number:y} => ! EVR{Failure:x, Number:y}

  36. pattern syntax pattern ::= 'pattern' NAME ':' event '=>' consequence consequence ::= event | '!' event | '[' consequence1,...,consequencen ']’ | ‘{' consequence1,...,consequencen ‘}'

  37. property P3 P3: Whenever a flight software command is issued, there should follow a dispatch and then exactly one success. No dispatch failure before the dispatch, and no failure between dispatch and success.

  38. formalization pattern P3: COMMAND{Type:"FSW", Stem:x, Number:y} => [ ! EVR{DispatchFailure:x, Number:y}, EVR{Dispatch:x, Number:y}, ! EVR{Failure:x, Number:y}, EVR{Success:x, Number:y}, ! EVR{Success:x, Number:y} ]

  39. expressed in first order LTL

  40. Python predicate definitions {: def within(t1,t2,max): return (t2-t1) <= max :} pattern P6: COMMAND{Type:"FSW",Stem:x,Number:y,Time:t1} where {: x.startswith("PWR_”) :} => EVR{Success:x, Number:y, Time:t2} where within(t1,t2,10000)

  41. scoped version of P4 pattern P9: COMMAND{Type:"FSW", Stem:x, Number:y} => { EVR{Dispatch:x, Number:y}, [ EVR{Success:x, Number:y}, ! EVR{Success:x, Number:y} ], ! EVR{DispatchFailure:x, Number:y}, ! EVR{Failure:x, Number:y} } upto COMMAND{Type: "FSW"}

  42. from patterns to automata • temporal patterns are translated into parameterized universal automata • automaton language more expressive • user can use both, in practice only temporal patterns have been used for testing MSL

  43. recall P3 pattern P3: COMMAND{Type:"FSW", Stem:x, Number:y} => [ ! EVR{DispatchFailure:x, Number:y}, EVR{Dispatch:x, Number:y}, ! EVR{Failure:x, Number:y}, EVR{Success:x, Number:y}, ! EVR{Success:x, Number:y} ]

  44. automaton A_P3 { always S1 { COMMAND{Type:"FSW",Stem:x,Number:y} => S2(x,y) } hot state S2(x,y) { EVR{DispatchFailure:x, Number:y} => error EVR{Dispatch:x, Number:y} => S3(x,y) } hot state S3(x,y) { EVR{Failure:x, Number:y} => error EVR{Success:x, Number:y} => S4(x,y) } state S4(x,y) { EVR{Success:x, Number:y} => error } }

More Related