Cyborg Astrobiologist -- Basic Idea:

1. Cooccurence 2D Histogram for Saturation Novelty Detection with the FamE (FamiliarityEnergy) Hopfield NeuralNet (after R. Bogacz, U. Bristol) Compute: averages: <H>,< S>, <I> for each segment x = (<H>,< S>, <I> )T If (Energy_Hopfield( x ) < -N/4 then Novel = FALSE; Else { Novel = TRUE; Store Pattern x in Hopfield Net; } Repeat for all segments; Repeat for all incoming images; Sat(pix[i][j+1]) Sat(pix[i][j]) Algorithm: Simple Uncommon Map Simple Image Uncommon Map Brighter = “more uncommon” (For the image on the left) • “The Cyborg Astrobiologist:Teaching Computers to Find Uncommon or Novel Areas of Geological Scenery in Real-time” • Network Conference of the Alexander von Humboldt Foundation, Muenster, Germany, November 2008. • Authors: Patrick C. McGuire1,5,6,*, Enrique Díaz Martínez1,3, Jens Ormö1, Javier Gómez Elvíra1, Virginia Souza-Egipsy1, Helge Ritter2, Markus Oesker2, Robert Haschke2, Jörg Ontrup2, Florian Schmidt2, Alexandra Bartolo4, Richard Bose5, Lorenz Wendt6 • Institutions: • 1Centro de Astrobiología (CAB, INTA/CSIC), Instituto Nacional Técnica Aeroespacial, Torrejón de Ardoz, Madrid, Spain: Robotics Laboratory, Planetary Geology Laboratory, Transdisciplinary Laboratory 2Universität Bielefeld, Germany, Neuroinformatics Group, Computer Science Department, Technische Fakultät 3(currently at) Dirreción de Geología y Geofísica, Instituto Geológico y Minero de España, Tres Cantos, Madrid, Spain 4Engineering Faculty, University of Malta, Malta 5McDonnell Center for the Space Sciences, Washington University, St. Louis, Missouri, USA 6Institute for Geosciences, Freie Universität Berlin, Department of Planetary Science and Remote Sensing, Berlin, Germany • *(corresponding author) Email: mcguire@epsci.wustl.edu The Cyborg Astrobiologist:Geological Field Missions to Rivas Vaciamadrid, Guadalajara and Malta McGuire, Ormöet al., Int’l Journal of Astrobiology (2004) McGuire, Diaz-Martinez et al., Int’l Journal of Astrobiology (2005) CONCEPT Question: “Can an autonomous Robotic Astrobiologist someday be taught to understand a geological scene like the one pictured below?” CyborgAstrobiologist -- Basic Idea: Geologists’ intuition & high-level planning Mission Riba1: Tripod Position “#2”,Inside the Cyborg Astrobiologist’s ‘Brain’ McGuire, Diaz-Martinez et al., Int’l Journal of Astrobiology (2005) Real-time Selection of Interest Points Wearable Computer Real-time Image-segmentation VISION ALGORITHMS Human Mobility Video Camera, Firewire communication (Cyborg Astrobiologist)++tested at Rivas Vaciamadrid (Sept 6, 2005) Uncommonness NoveltyDetection:CyborgAstrobiologistgets a memory (of Familiar patterns) Uncommon Map Orig (Cam or Micro) Full-color Image Segmentation Novel Segments Cyborg Astrobiologist, with microscope, studying lichens #1(Cam) FIELD TESTS Microscope in Field! #18(Micro) Lichen=White Sporing = Red Rock=Brown Tripod!! Not Yet #19(Micro) Lichen=White Sporing = Red Rock=Brown Head-mounted Display failed in field  Pictured here: tablet display Not Yet #23(Micro) Lichen= Algae + Fungus (in Symbiosis) Lichen = Black, Orange Gypsum Crystal = SmoothGray Summary of Concept: Developing computer-vision technologies to automate some of a Human Geologist’s low-level thought processes. We are testing these computer vision technologies in the field in real-time with low-cost platforms, with the intention of later putting these algorithms on robotic platforms for exploring the Moon and Mars. Currently working with collaborators in Madrid, Malta, St. Louis, Bielefeld and Berlin, in order to: a) Do more comprehensive field tests at sites of geological and astrobiological interest using both the uncommon-map and novelty-detection systems. b) Make the phone-cam system work with a field laptop enabled with Bluetooth, for automatic transfer of the images from the phone to the laptop and (marked- up) back to the phone. This should speed up the testing. Future:0) Improved image-segmentation (to include texture) 1) Improved Novelty Detection (better statistical measures) 2) Improved cameras (hyperspectral cameras) 3) Improve technological readiness for planetary robotics missions. The CyborgAstrobiologist:Ported from a wearable computer to the Astrobiology Phone-cam Before: HARDWARE IMPROVEMENTS Now: Bartolo, McGuire et al., International Journal of Astrobiology (2007)