1 / 38

Making a Robotic Dog See and Hear

Making a Robotic Dog See and Hear. Daniel D. Lee. World of Science 2000. Alternative images. Face recognition. Original image. Terminator. Arnold is looking for you. Robots. Hollywood versus reality. Gort. Data. HAL. Deep Blue. Computer beats world champion Gary Kasparov. Complexity.

dolph
Download Presentation

Making a Robotic Dog See and Hear

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Making a Robotic DogSee and Hear Daniel D. Lee World of Science 2000

  2. Alternative images Face recognition Original image

  3. Terminator • Arnold is looking for you...

  4. Robots • Hollywood versus reality Gort Data HAL

  5. Deep Blue • Computer beats world champion Gary Kasparov

  6. Complexity • Tic Tac Toe easy to program using brute force • Deep Blue evaluated 200 million chess positions per second Number of configurations Tic Tac Toe

  7. Images Pixel vector • Vector representation of pixel values(white=0.0, black=1.0).

  8. Combinatorial explosion • Impossible for a computer to search all possible images images 2 pixels 400 pixels images images Age of universe: seconds 3 pixels

  9. The brain • Vision occupies a large fraction of our brains

  10. Neurons • Approximately 1012 neurons in a human brain

  11. Neuronal properties • Neurons communicate with each other using action potentials

  12. Circuit diagram (Felleman & Van Essen, 1991) • Complex and hierarchical organization.

  13. Artificial neuron • Unit sums inputs x with synaptic weights w • Nonlinear transformation x1 Synaptic weights w1 x2 Input activities + Output x3 Squashing function x4 w5 x5

  14. x1 x2 t1 x3 t2 x4 xN Artificial neural network Weights • Transformation of input into output. • Change synaptic weights to maximize performance. Hidden layer Labelled data: W11 Input Output Output layer Input layer WNM

  15. Learning • How to set the connections between neurons to have the network do the right thing? Weights x1 Hidden layer W11 x2 t1 x3 Output layer t2 x4 Input layer WNM xN

  16. Optimization Gradient ascent Mount Everest • Like climbing a mountain blindfolded. • Small steps until top is reached.

  17. Robotic dog • Doesn’t have a name yet… any suggestions?

  18. Artificial sensorimotor system • Total cost of parts ~ $700 • You too can build your own!

  19. Video tracking

  20. Video processing • Conversion of video images into luminance, color, and motion channels.

  21. Face recognition neural network • Learns to associate saliency with face.

  22. Unsupervised learning • Database containing many different faces.

  23. Learning parts of faces

  24. Parts representation W: 49 hidden units V X  = Original: • Computer automatically decomposes the images into their constituent parts.

  25. Eye movements • Fast eye movements to scan visual environment Eye muscles (Yarbus, 1967)

  26. Goldfish eye movements

  27. Control of eye position

  28. Neural integrator (Pastor, et al., 1994)

  29. Vestibular system • Sense of balance and seasickness

  30. Vestibular-ocular reflex

  31. Auditory localization (Konishi, 1990) Barn Owl

  32. Auditory localization

  33. Walking

  34. Language Text Corpus Doc #1 Doc #2 Text Document lazy Doc #3 brown fox dogs Doc #4 Doc #5 jumped • Model text document as collections of words.

  35. Text and images analogy Word counts: Text Images Documents words pixels document picture Words word frequency grayscale intensity • Represent documents with word frequencies. • Analogy between learning algorithms.

  36. president (148) congress (124) power (120) united (104) constitution (81) amendment (71) government (57) law (49) Learned semantic topics court government council culture supreme constitutional rights justice president served governor secretary senate congress presidential elected flowers leaves plant perennial flower plants growing annual disease behavior glands contact symptoms skin pain infection ´ » Entry on “Constitution of the United States” metal processmethodpaper…glass copper lead steel person exampletimepeople…rules lead leads law • Grolier encyclopedia: 15276 words, 30991 articles. • Semantic features, word sense disambiguation.

  37. Multimodal integration • Vision, hearing and language combined (Knudson, 1997)

  38. Summary • Adaptation and learning in biological systems important for vision, hearing, motor control. • Mimic neural systems in computer algorithms. • Robotic systems can learn from experience. • But still cannot compete with your family dog or cat...

More Related