Verifying Autonomous Planning Systems Even the best laid plans need to be verified

1. Prepared for the 2005 Software Assurance Symposium (SAS) Verifying AutonomousPlanning SystemsEven the best laid plans need to be verified MSL EO1 DS1 Gordon Cucullu Gerard Holzmann Rajeev Joshi Benjamin Smith Margaret Smith (PI) Affiliation: Jet Propulsion Laboratory

2. Importance • Autonomous Planning Systems (APSs) determine what the spacecraft / rover / installation should do. • Compared to conventional software, they are able to determine thisin a wide range of circumstances. • As a result, • no need for continual oversight (save on 24/7 operations staff) • more science is done (avoid delay of calling back to Earth) • improved safety (more proactive than just “safe mode”) • But because APSs must operate in a wide range of circumstances – far too many to test, even if you could predict them all, • how can you trust them to do the right thing??? This work is pursuing a solution! SAS_05_Verifying_Autonomous_Planners_Smith

3. How to get from A to B ? Consequences of a bad planWasted Resources missed science goal out of resources SAS_05_Verifying_Autonomous_Planners_Smith

4. How to get from A to B ? Consequences of a bad plan:Loss of Mission SAS_05_Verifying_Autonomous_Planners_Smith

5. Solution Challenge: Assure that all plans generated by the APS are safe for the spacecraft. The current empirical testing approach is insufficient because it lacks coverage. Solution: Replace current empirical testing with model checking. Model checking offers exhaustive or measurable test coverage leading to greater confidence in correctness. SPIN Model Checker • Logic Model Checker used to formally verify distributed software systems. • Development began in 1980 at Bell Labs • publicly distributed source code since 1991 • Most widely used logic model checker with over 10,000 users. • Recipient of 2002 System Software Award for 2001 from the Association for Computing Machinery (ACM) • Verifies software using a meta language called Promela • requires that system being verified be expressed in Promela • SPIN flags deadlocks, unspecified receptions, incompleteness, race conditions and unwarranted assumptions about relative speeds of processes SAS_05_Verifying_Autonomous_Planners_Smith

6. Approach requirements requirements Empirical Testing (current approach) Testing with the SPIN Model Checker (our work) input model input model properties of desirable plans Promela Model Testing Testing limited by time required to inspect sample plans limited only by memory and processor speed plans analyzes billions of plans ~100 plans Manually inspect plans to identify undesirable plans undesirable plan undesirable plan no errors all desirable (error trace) plans Adjust model to exclude undesirable plan Adjust model to exclude undesirable plan end testing end testing SAS_05_Verifying_Autonomous_Planners_Smith

7. testing Relevance to NASA software complexity • APS are needed by NASA projects to reduce operations costs and meet science return requirements. • Our work retires an important class of risks inherent to all missions using APS. • we replace an inadequate testing method with a method that has greatly improved and measurable test coverage. Testing methods must keep pace with the highly complex, autonomous systems we need and are developing. SAS_05_Verifying_Autonomous_Planners_Smith

8. Accomplishments • Selected Earth Observer 1 as a target mission for application of our work. – 100+ activities = more plans than atoms in the universe!!! • Current empirical method of where ~100 plans are tested is woefully inadequate. • Our approach: Use model checking to greatly improve testing coverage = billions of plans. • prune the search space through the use of constraints • For DS4 / Champollion APS model, used model checking to find a deadlock error – 10 activities =exploration of ~ 3 million plans deadlock: out of memory sample sample1 sample2 sample2 image image 2 image 1 compress data compress uplink uplink oven1 on off-warm off-warm off-cool on off-cool off-cool oven2 off-cool camera on off off drill location oven1 oven1 hole1 hole7 power use memory use • Currently working on a set of automated tools for automatically converting APS for model checking SAS_05_Verifying_Autonomous_Planners_Smith

9. Where we are Going • Our goal: to improve APS testing capabilities which have been an impediment to the acceptance of APS for other than experimental use. • How we will get there: • complete implementation of a set of tools to fully automate model checking of APS models • improve coverage from hundreds of test cases to billions of test cases. SAS_05_Verifying_ Autonomous_Planners_Smith