Review, Todorov (and a little Hutchins)

1. Review, Todorov (and a little Hutchins) Adrienne Moore 2-6-08

2. General Info • Next week, Quiz 2 in section Wednesday • Please check the website for new info periodically!! • Info for Adrienne’s Wed section – 9 views : ( • List of terms/concepts for Midterm 1 – 32 views : ( • What you need to know of Dr. Todorov’s lecture – 21 views : (

3. General Info • *Many* midterm scantrons were hand-graded – you may want to double-check your grading during office hours • must let TA know ahead of time so the scantron will be there • Plus, you should know what you got right and wrong (final’s cumulative)

4. 1. What are some differences between reinforcement learning and supervised learning? • Supervised: learning from example • Reinforcement: the learner must discover which actions are most rewarding • Maximize a numerical reward signal • Trial-and-error search • Delayed reward

5. 2. What is an example of supervised learning from past lectures? • Gary Cottrell’s neural networks for emotional face processing • Erik Viirre’s neural networks for ICA

6. 3. W hat do the terms exploration and exploitation mean with reference to reinforcement learning? • Exploration: searching for new actions that will produce more reward • Exploitation: repeating actions that are already known to produce reward • This is a challenging trade-off that arises in reinforcement learning

7. 4. What are the key components of a reinforcement learning system, beyond agent and environment? • Policy: defines learner’s way of behaving, like a stimulus-response association • Reward Function: defines the goal (maximize total reward over time); quantifies the immediate desirability of a given state • Value Function: defines long-term desirability of a given state; basis for choosing an action • (Model of Environment: allows you to think about likely consequences of an action)

8. What are two examples of low level processes that are studied by Cognitive Scientists, from this and previous lectures? What are two examples of high level processes studied by Cognitive Scientists? What does the distinction between low level and high level processes mean? Could you make an argument that a low level process can actually be harder than a higher level process (with an example from Emo’s lecture)? Low level – High level processes:

9. Maps in the Brain: • Explain the concept of a map in the brain. • The lecture mentioned four examples of maps in the brain, tell me where they are. • Define homunculus, and explain why the motor homunculus has proportions that are distorted from normal.

10. Homunculi

11. Circuitry of the Motor System

12. Circuitry of the Motor System • What happens if you stimulate neural tissue of cerebral cortex motor areas? • Describe an experiment mentioned in lecture that demonstrated that the circuit through the spinal cord alone is capable of generating movement?

13. Differences between Robotic and Human Movement: • What are some differences mentioned in lecture between robotic and human movement? • What is quasi-static walking? Is that how people walk, and if not, in what way do we differ? • How does the walking robot in Emo’s lecture video make use of the laws of physics to walk successfully? • How is the locomotion of the simulated “artificial life” creatures from Emo’s second lecture video more like real life than the previous machine examples (3 ways were mentioned in lecture)?

14. Differences between Robotic and Human Movement:

15. Hutchins lecture, terms: • Modus ponens • Modus tollens • Affirmation of the consequent • Denial of the antecedent • Decontextualization • Ethnography • Epistemology • And a little more Todorov: proprioception, muscle spindle