Dynamic Models of Augmented cognition

1. Young, Clegg, & Smith 2004 Dynamic Models of Augmented cognition

2. Closed & Open-Loop Systems • Microphone example • The microphone & human are both open-loop systems. • Human speaking into the microphone becomes a closed-loop system. • Feedback results from the sounds produced from the speakers feeding back into the microphone, & back out through the speakers. • Closed-loop system can be stable (Feedback decays) or unstable (Feedback increases). • Therefore, when two or more open-loop systems are brought together to form a closed-loop system, the result can be either stable or unstable. • Engineering control systems theory can be used to model the shift from open to closed-loop systems to better understand their characteristics.

3. Elements of a Control Systems Model of Augmented Cognition The paper utilizes a standard block model of human cognitive processing. Each block of the model has two important characteristics, lag and bandwidth. Lag is produced because each block requires a certain amount of time to process information moving through. Bandwidth limits the amount and frequency of the information that can be processed. Information that has a higher frequency than the system lag exceeds the bandwidth and will not be processed (flicker fusion rate). The lag and bandwidth of each block of the system can be modeled mathematically, the resulting equations provide time estimates that can be used to predict the behavior of the human open-loop system. By using the output of one block as the input for the next a cascading model that shows systemic effects of inputs is achieved.

4. Closed-Loop Dynamic Model • DARPA AugCog system with a human operator was used to model a closed-loop system. • A sensor was modeled in the system as a first-order lag with a time constant of 1 second. • The readings for the sensor were given to the Systems Interface Director which fed into adaptive automation. • The adaptive automation could pass off low priority tasks to system automation if it received input that the operator’s mental workload was increasing.

5. Results from the Model • Three distinctive patterns of error rates emerge from implementing the AugCog system in different ways and introducing increasing workload during the simulation • When workload is increased beyond the operator’s ability to cope, error rates climb dramatically then level off when augmented cognition is not present. • When simplistic augmented cognition is present, where the amount of system feedback is a constant, the result is closed-loop instability. When the amount of feedback provided by the augmented cognition always changes by the same amount, the result is rapid cluttering and clearing of the display. • When there are three variations on the level of feedback the system provides, which take into account the operator’s memory and anticipation, a stable closed-loop system is produced where error rates increase then decay over the trial.

6. Implications • The modeling techniques presented in this paper can be used to evaluate the effectiveness of a human-machine closed-loop system before the system is actually built. It can mathematically derive the parameters involved to produces estimates of stability, tracking, performance, and robustness. The model can also help answer questions about where improvements to augmented cognition can be made (How quickly should workload measures be taken? How frequently should feedback be updated?).

7. Beer 2000 Dynamical approaches to cognitive science

8. The Lexical and Grammatical Structure of Language • The experiments of Elman show that a computational model can be trained in such a way that it develops the ability to predict next words in a sentence in ways that mirror human language ability. The effects that words have on the state of the system change over time to reflect groupings based on semantic properties of the words themselves. The limitations of this system as well appear to coincide with the limitations a human would have while attempting to perform a similar task.

9. The A-not-B Error in Infant Reaching • The A-not-B reaching error in infants occurs when they are required to reach for a container with a desirable object inside. They have been previously trained that the object is always in container A, then they are shown the object being placed in container B, they are delayed for a time, and then required to choose a container. Under specific conditions, they will choose the wrong container. This was originally attributed to immature object permanence, but a dynamical theory approach produces different implications. The error rate is reduced by several conditions including reducing the time delay, visually distinct containers, desirability of the object, and the infants posture, none of which are explained by the object permanence hypothesis. Using dynamic probability functions to graph the inputs to different cognitive systems, it can be shown that it is likely that the infant’s immature goal-directed reaching system is more likely at fault for the error.

11. Active Categorical Perception in an Evolved Model Agent Dynamical systems approaches can be used in areas such as robotics as well. These approaches can aid in the development of evolutionary algorithms which are programs that receive environmental inputs and adapt over time to produce desirable outputs based on predetermined parameters. One such example is the evolved agent that was designed to move horizontally , catch falling circles, and avoid falling diamonds. To perceive the falling objects the agent had seven rays that informed it of the distance from the object to itself. By moving back and forth, the agent could use these rays to determine the object’s width. As seen on the color-coded graphs, the agent would make large amplitude movements horizontally when it first detected the falling object, then at a point roughly halfway down the screen the agent’s behavior altered. If it determined the object to be a circle, the movement amplitude continued to shrink until the agent was centered beneath the object. If the object was a diamond the agent would make an avoidance movement.

12. Dynamical, Symbolic, and Connectionist Approaches to Cognition • Symbolic Models • The inputs to the system are symbolic in nature, which active store symbolic representations which imply syntactic manipulations of the input to produce an output. • Connectionist Models • Layered networks of “neuron-like” elements. These elements are like neurons in that they code information numerically (a specific pattern and rate of firing). These numerical inputs produce numerical outputs that activate embedded networks of the elements. • Dynamical Models • Dynamical models are composed of a set of differential equations that explain changes in a system over time. The possible ways in which the system can change are expressed as different trajectories in the equation’s solution space. The system has a baseline trajectory that is determined by it’s internal state without input. Inputs are represented by perturbations in the dynamics that exist between variables, which alter the trajectory. The dynamics of the internal state can effect and are affected by the external state to produce new trajectories over time, even if exposed to the same perturbation.

13. Cognition and the Dynamics of Situated, Embodied Action • Dynamical systems do not function in isolation, rather they are situated within larger dynamical systems and coupled to them. While the nervous system, body, and environment can all be expressed as sets of differential equations, none of them alone produces cognitive behaviors. When the equations are coupled together, nested inside one another, then the system can start expressing behaviors and interactions complex enough to be deemed “cognitive”.