Gregory J. Barlow 1,2 and Choong K. Oh 2 1 The Robotics Institute, Carnegie Mellon University

1. Gregory J. Barlow1,2 and Choong K. Oh2 1 The Robotics Institute, Carnegie Mellon University 2 The U.S. Naval Research Laboratory Robustness Analysis of Genetic Programming Controllers for Unmanned Aerial Vehicles

2. Motivation • Evolutionary robotics (ER) controllers may evolve in simulation or on real robots, but the true test of performance must happen in real-world conditions • Testing unfit controllers may be dangerous or expensive for some robots

3. Transference • For controllers evolved in simulation, evaluation in a noisy environment does not ensure good transference if simulated noise is not consistent with true noise • If a controller performs well over a wide range of state and sensor noise conditions in simulation, prior work suggests that the controller should transfer well

4. Evolving controllers for unmanned aerial vehicles • Unmanned aerial vehicles (UAVs) require assurance of off-design performance • Even under noise not considered during evolution, controllers must still be able to efficiently accomplish the task • Poorly performing controllers could cause crashes, possibly destroying the UAV

5. Overview • Controller evolution (Barlow et al., 2004) • Goals • Robustness testing • Results • Conclusions

6. Controller evolution Evolve unmanned aerial vehicle (UAV) navigation controllers able to: • Fly to a target radar based only on sensor measurements • Circle closely around the radar • Maintain a stable and efficient flight path throughout flight

7. Controller Requirements • Autonomous flight controllers for UAV navigation • Reactive control with no internal world model • Able to handle multiple radar types including mobile radars and intermittently emitting radars • Robust enough to transfer to real UAVs

8. Simulation • To test the fitness of a controller, the UAV is simulated for 4 hours of flight time in a 100 by 100 square nmi area • The initial starting positions of the UAV and the radar are randomly set for each simulation trial

9. Sensors • UAVs can sense the angle of arrival (AoA) and amplitude of incoming radar signals

10. UAV Control Sensors Evolved Controller Roll angle UAV Flight Autopilot

11. Radars • Stationary, continuously emitting • Mobile, continuously emitting • Stationary, intermittently emitting with regular period • Stationary, intermittently emitting with irregular period • Mobile, intermittently emitting with regular period

12. Transference • To encourage good transference to real UAVs, during evolution: • Modeled only the sidelobes of radars • Added noise to the modeled radar emissions • Set accuracy of the angle of arrival values to be within ±10° • Evolved controllers were successfully tested on wheeled mobile robots (Barlow et al., 2005)

13. Multi-objective GP • We had four desired behaviors which often conflicted, so we used NSGA-II (Deb et al., 2002) with genetic programming to evolve controllers • Each evaluation ran 30 simulations • Each of 50 evolutionary runs had a population size of 500 • We used environmental incremental evolution to produce controllers evolved for a total of 1800 generations

14. Functions and Terminals Functions • Prog2, Prog3, IfThen, IfThenElse, And, Or, Not, <, <=, >, >=, < 0, > 0, =, +, -, *, /, X < 0, Y < 0, X > max, Y > max, Amplitude > 0, AmplitudeSlope > 0, AmplitudeSlope < 0, AoA > Arg, AoA < Arg Terminals • HardLeft, HardRight, ShallowLeft, ShallowRight, WingsLevel, NoChange, rand, 0, 1

15. Considerations • We have many acceptable controllers on the Pareto front, but we need to choose one “best” controller • Controllers may be optimized to the simulation parameters, may not be robust to other noise levels or sources • Fitness values are only measured over 30 trials

16. Goals • Choose a single “best” evolved controller for future flight tests • Evaluate the robustness of the best evolved controllers to sensor and state noise to assure off-design performance • Compare evolved controllers to human designed controllers

17. Test functions • Flying to the radar • Percent error in time to radar • Circling the radar • Average circling distance • Efficient flight • Percent error in flying with a roll angle of zero degrees • Stable flight • Cost of sharp, sudden turns

18. Test functions

19. Baseline Values Flying to the radar ≤ 0.2 • Error in flight time to radar less than 20% Circling the radar ≤ 2 • Average distance less than 2 nmi Efficient flight ≤ 0.5 • ~50% of time (not in-range) with roll angle = 0 Turn Cost ≤ 0.05 • Turn sharply less than 0.5% of the time

20. Performance metrics • Failures • Percent of trials that don’t meet the baseline values • Normalized maximum • Magnitude of failure normalized by the baseline value • Normalized mean • Means for each test function normalized by the baseline value and then averaged • Average rank • Combination of first three performance metrics

21. Performance metrics

22. Selecting controllers for testing • GP produced 25,000 controllers • Based on prior work, 1,602 had acceptable mean fitness values • We ran 100 simulations on each of the five radar types for each of these 1,602 and chose ~300. • We cut these down to 10 using the normalized maximum performance metric

23. Designed controllers • Hand-written • Based on functions and terminals available to GP and knowledge of good GP strategies • Proportional-derivative (PD) • Takes AoA as input (approximates derivative) • PID performed poorly with mobile radars, so integral term was not used

24. Robustness tests • Robustness tests fell into five categories: AoA error, amplitude error, varied airspeed, heading error, and wind effects (position error) • For every combination of radar type and controller, we performed 10,000 simulations, for a total of 50,000 simulations per controller per test

25. Robustness tests • Angle of arrival error ±{10, 15, 20, 30}° • Amplitude error {6, 12} dB • UAV airspeed {50, 80, 100} knots • Heading error {0, 0.5, 1, 1.5, 2}° • Wind (position error) {0, 5, 10, 20, 30} knots

26. Results • For each test, we ranked the 12 controllers based on the four performance metrics • We combined these results into an overall ranking to determine the best controller • The best evolved controller fails gracefully and compares well to the PD controller

27. Rankings

28. Comparison

29. Conclusions • Selected a single best controller for future flight tests • Established the off-design performance of evolved UAV controllers; evolved controllers failed gracefully • Performance of the best controller compares favorably with PD control • Established the limits of performance for these evolved controllers

30. Acknowledgements • Financial support was provided by Swampworks project office of the Office of Naval Research • The U.S. Naval Research Laboratory (Code 5730) provided computation time on their Beowulf cluster • Gregory J. Barlow is supported by a National Defense Science and Engineering Graduate Fellowship