Artificial Intelligence in

1. Artificial Intelligencein Audio and Physiological Sensing S. Hamid Nawab

2. Agents • An agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through actuators • Human agent: eyes, ears, and other organs for sensors; hands, legs, mouth, and other body parts for actuators • Robotic agent: cameras, microphones, and infrared range finders for sensors; various motors for actuators

3. Environment types • Fully observable (vs. partially observable): An agent's sensors give it access to the complete state of the environment at each point in time. • Deterministic(vs. stochastic): The next state of the environment is completely determined by the current state and the action executed by the agent. (If the environment is deterministic except for the actions of other agents, then the environment is strategic) • Static (vs. dynamic): The environment is unchanged while an agent is deliberating. (The environment is semidynamic if the environment itself does not change with the passage of time but the agent's performance score does)

4. Percepts and Actions • Percepts at time t0 are readings from all sensors of the agent at given time instant t0. • Percept Sequence at time t0 (or Percept History) is the entire set of readings from all sensors of the agent up to time t0. • Actions are operations that the agent has the ability to perform in the environment.

5. Percept Sequences and Signals • In many domains, a percept sequence at time n0 may be viewed as a set of sensor-indexeddigital signals whose values have been determined till time n0.

6. Agent Function & Agent Program • The agentfunction maps from percept sequences to actions: [f: P* A] • The agentprogram runs on the physical architecture to produce f • agent = architecture + program

7. Rational agent • RationalAgent: For each possible percept sequence, a rational agent should select an action that is expected to maximize its performance measure, given the evidence provided by its percept sequence and whatever built-in knowledge the agent has. • Rationality maximizes expected performance, while perfection maximizes actual performance.

8. Performance Measure • When placed in an environment, an agent performs a sequence of actions that lead to the environment going through a sequence of states. • A performance measure assigns a numerical score to each possible sequence of environment states that the agent may thus produce. The sequence with the highest score is the rational choice.

9. Agent functions and programs • An agent is completely specified by the agent function mapping percept sequences to actions • One of the agent functions (or a small equivalence class) is rational, i.e., it maximizes a measure of expected performance given the information (percept sequence, model of environment etc.) the agent has about the environment. • Engineering Aim: find a way to implement the rational agent function concisely

10. Autonomous Rational agents • An agent is autonomous if its behavior is determined by its own experience (in turn obtained from its Percept Sequence since the beginning of time). • Autonomy has the potential to endow the agent with the abilities to adapt and learn. • Autonomous Rational Agent = Intelligent Agent.

11. Percept Sources • Percept Sources: Emissions/Reflections (e.g. sounds, light, infrared, etc.) from the environment that contribute to the percepts produced at the sensors of an agent. • In many domains, a percept sequence at time n0 may be viewed as having been produced by some type of combination of digital signals representing various percept sources of interest to the agent

12. Key Theme of this Course • How do you design an autonomous rational agent to go about determining what action to take at time n0 when many of its percepts are the result of mutually interacting percept sources (emissions/reflections) of interest.

13. AI-APS: Search, Build, Unlock • First, search for ideas and clues • Second, build and evaluate an app • Third, unlock new ideas and insights

14. AI-APS: Foundational Questions-I • How does Shazam work? HW 1 • Can you build a Shazam-like app? PG 1 • Is Shazam an artificial intelligence? • How does EMGdecomp work? HW 2 • Can you build an EMGdecomp-like app? PG 2 • Is EMGdecomp an artificial intelligence?

15. AI-APS: Foundational Questions-II • How does SUT work? HW 3 • Can you build a SUT-like app? PG 3 • Is SUT an artificial intelligence? • How does MDR work? HW 4 • Can you build an MDR-like app? PG 4 • Is MDR an artificial intelligence?

16. AI-APS: Foundational Questions-III • How does ArrayECG work? HW 5A • Is ArrayECG an artificial intelligence? • How does EEGinterpret work? HW 5B • Is EEGinterpret an artificial intelligence? • How does Siri work? HW 5C • Is Siri an artificial intelligence?

17. Course Grading • Three Quizzes (45%) – Tue Oct 1, Tue Nov 5, Thu Dec 5; all in-class, closed book. • Four Projects (20%) – Due by: Tue Oct 1, Tue Nov 5, Thu Dec 5, Wed Dec 11 (last day of classes). Teams of 2 and 3 allowed. Teams of 1 to 3 allowed; individual reports • 5 HWs (35%) – Due by TBA as the semester progresses. Teams of 1 to 3 allowed; individual reports. • Quizzes will be graded on accuracy; Projects and HWs will be graded on completeness. • Class will not meet Tue Oct 15 (Monday schedule at BU) and Tue Oct 22.

18. Required Background (Pre-Requisites) • Equivalent of EC401 (signals and systems). Please let me know if you have a more extensive signal processing background. Grading criteria will ensure that those without a background beyond EC401 are notdisadvantaged. • You should have some (e.g. taken a course) programming experience– Matlab or Pythonor C++ etc. Please let me know your programming background. Grading criteria will ensure that those without extensive programming backgrounds are not disadvantaged.

19. Agent types • Four basic types in order of increasing generality: • Simple reflex agents • Model-based reflex agents • Goal-based agents • Utility-based agents

20. Simple reflex agents

21. Model-based reflex agents

22. Goal-based agents

23. Utility-based agents

24. Learning agents