Topic: Cognitive Architecture The fixed parts of cognitive processes

1. Topic: Cognitive ArchitectureThe fixed parts of cognitive processes No matter what your favorite computational theory may be, it always assumes certain fixed properties of the system within which it functions

2. Among the architectural properties assumed by any computational theory: • The form of representations. Computational theories typically assume that representations consist of symbol structures. But how do they represent magnitudes? Numerals or analogs? • Available operations. The building blocks of processes that combine to form algorithms. • Information flow and constraints on subprocesses (e.g., encapsulated modules?) • Fixed capacities determined by the innate structures of the mind/brain, as opposed to those that depend on their niche environment.

3. The form of representation: The case of visual perception • Distinguishing form and content • What is the form of mental representations? • In memory? In perception? In thought? • Lectures by R.C. Gallistel & B. McLaughlan • In imagination (in mental imagery). • Does vision depend on cognition, or is it encapsulated so it cannot use knowledge? • Pylyshyn, Z. W. (1999). Is vision continuous with cognition? The case for cognitive impenetrability of visual perception. Behavioral and Brain Sciences, 22(3), 341-423.)

4. The Form of visual representation • Our subjective impressions (our intuitions) of what our representations are like are seriously unreliable and misleading. We do not experience the form of a representation, only of its content – what it is about or what it represents • A representation of a large square object is not largeand square • But the demands of scientific explanation are quite different; and they almost always lead us to unfamiliar and counterintuitive conclusions

5. This is what our conscious experience suggests goes on in vision…

6. This is what the demands of explanation suggests must be going on in vision…

7. Some reasons given for believing that there is a “pictorial” representation in vision • Visual input is highly incomplete (moving peephole view), but visual system operates over filled-in (completed) displays • Representations are in world, not retinal, coordinates • Our representation not only completes the missing retinal information, but it also represents panoramic (broad) and dynamic (moving) information • It’s obvious that we experience a rich and finely detailed 3D perceptual world – so it must be recreated in the brain as a 3D display

8. Standard view of saccadic integration by superposition

9. Partial patterns are seen as completed … Where’s Waldo?

10. Vision patterns can present many different percepts • Seeing as: It’s what you see the figure as that determines behavior – not its physical properties. • What you see one part as determines what you see another part as. • There must be a representation that contains the interpreted pictorial pattern

11. Is it possible to specify a set of ways of physically presenting a visual stimulus for it to be perceived in a certain way?

12. Can you think of other ways of presenting a stimulus so it is perceived as e.g., a Necker Cube?

13. Errors in recall suggest how visual information is encoded Children have very good visual memory, yet often make egregious errors of recall • Errors in relative orientation often take a canonical form • Errors in reproducing a 3D image preserve 3D information

14. Errors in recall suggest how visual information is encoded • Children more often confuse left-right than rotated forms • Errors in imitating actions is another source of evidence

15. Ability to manipulate and recall patterns depends on their conceptual, not geometric, complexity • Difficulty in superimposing shapes depends on how they are conceptualized • Look at first two shapes and superimpose them in your mind; then draw or select one that is their superposition ?

16. Many studies have shown that memory for shapes is dependent on the conceptual vocabulary available for encoding theme.g., recall of chess positions by beginners and masters

17. Other examples showing that it is how you represent something that is relevant to cognitive science • Example from color vision • “Red light and yellow light mix to produce orange light” • This remains true for any way of getting red light and yellow light: • e.g. yellow may be light of 580 nanometer wavelength, or it may be a mixture of light of 530 nm and 650 nm wavelengths… • So long as one light looks yellow and the other looks red the “law” will hold.

18. What is the evidence against an inner pictorial display? • We will see that constraints on explanation demand a symbolic form of representation. • Causal explanation requires strong equivalence • Some fields of study, such as History, can only provide retrospective analyses and “reasons” not causal models: Is Psychology like History?

19. Strong Equivalence and the role of cognitive architecture • For two models to be in strong equivalence they must have the same architecture (at least in theoretically relevant ways).

20. The concept of cognitive architecture • If differences among behaviors (including differences among individuals) is to be attributed to different beliefs or to different algorithms, then there must be some common set of basic operations and mechanisms. This is called the Cognitive Architecture • The concept of a particular algorithm, or of being “the same algorithm” is only meaningful if two systems have the same architecture. Algorithm is architecture-relative. • The architecture is the part of the system that does not change when beliefs change. So it defines the system’s Cognitive Capacity.

21. Recall our earlier example of a model of the Sternberg task • Store memory set as a list L. Call the list size = n • Read target item, call it  • Check if  is one of the letters in the list L • If found in list, assign Resp = “yes” otherwise Resp = “no” • If Resp =“yes”, set  = 500 + K * n  Random(20  x  50) • If Resp =“no”, set  = 800 + K * n  Random(20  x  50) • Print , Print  • Go to 2 Is this the way people do it? How do you know?

22. The program outputs Yes/No response and Time Notice that for this to be a model, various aspects of the architecture have to be specified • Store memory set as a list L. Call the list size = n • Read target item, call it  • Check if is one of the letters in the list L • If found in list, thenassign=“yes” else =“no” • If =“yes”, then set  = 500 + K set*n Rand(20  x  50) • If =“no”, then set  = 800 + K * n  Rand(20  x  50) • Print , Print  • Go to 2  HOW?

23. Tacit assumptions made in constructing a computational model But there are many other properties of algorithms that constitute assumptions about the cognitive architecture.One class of properties seems so natural that it goes unquestioned – it’s the control structure • Operations are carried out in sequence. No operation can begin until the previous one is completed. This seems so natural that it goes unnoticed as an assumption. • Another fundamental property that is assumed is that control is passed from one operation to another (e.g., “go to”), as opposed to being grabbedin a “recognize-act” cycle in architectures called Production System orBlackboard System. Rules are: Condition  Action (or IF  THEN)

24. Observed regularity versus capacity:The difference between explanations that appeal to mental architecture and those that appeal to tacit knowledge The parable of a found mysterious box: Suppose we observe some robust behavioral regularity of an unknown system. What does it tell us about the nature of the system or about its intrinsic properties?

25. Observed regularity versus capacity:The difference between explanations that appeal to mental architecture and those that appeal to tacit knowledge Walking through a field one day, a scientist comes upon a mysterious box. The box has a meter that records some aspect of its behavior. Suppose that after many days of recording, the scientist finds some robust behavioral regularity of the box’s recording. What does it tell him about the nature of the box or about its intrinsic causal properties? The parable of a found mystery box:

26. An illustrative example: Mystery Code Box What does this behavior pattern tell us about the nature of the box?

27. An illustrative example: Mystery Code Box Careful study reveals that pattern #2 only occurs in this special context when it is preceded by pattern A What does this behavior pattern tell us about the nature of the box?

28. It tells us (very nearly) nothing about the nature of the system under study • Why? Because the observed behavior, although it is an objective true record, is but a small part of what the box is capable of. • The sample we observed is attributable to the environment – including what the box is used for – rather than to the fixed structure (the architecture) or the natureof the box.

29. How can an objective record of behavior not tell you about the nature of a system?What more is there? • In this example, what the scientist found happens to be a box that transmits English messages in international Morse code. • Two short blips is a code for i, one blip is an e and a long-short-long-short sequence is a c. • Thus the regularity that the scientist found can be explained by the spelling rule in English: “i before e except after c”! • Nothing inside the box can explain that because the box has a capacity not revealed by its usual ecological behavior.

30. The Moral: Regularities in behavior may be due to either: The inherent nature of the system or its structure or architecture. The content of what the system represents (what it “knows”).

31. Why it matters: A great many regular patterns of behavior reveal nothing more about human nature than that people do what follows rationally from what they believe. An example from language understanding The example of human conditioning

32. An example from language understanding Examples from language. John gave the book to Fred because he finished it John gave the book to Fred because he wanted it • The city council refused to give the workers a permit for a demonstration because they feared violence • The city council refused to give the workers a permit for a demonstration because they were communists

33. Another example where it matters:The study of mental imagery Application of the architecture vs knowledge distinction to understanding what goes on when we reason using mental images

34. Examples of behavior regularities attributable to tacit knowledge • Color mixing, conservation of volume • The effect of image size ? • Scanning mental images ?

35. Color mixing example

36. Conservation of volume example

37. Our studies of mental scanning (Pylyshyn & Bannon. See Pylyshyn, 1981) There is even reason to doubt that one can imagine scanning continuously (Pylyshyn & Cohen, 1998)

38. Can you rotate a mental image? Which pair of 3D objects is the same except for orientation?

39. Do mental images have size?Imagine a very small mouse. Can you see its whiskers? Now imagine a huge mouse. Can you see its whiskers? Which is faster?

41. Why do so many people deny these obvious facts about mental imagery? • The power of subjective experience (phenomenology). The mind-body problem is everywhere: but subjective experience does not cause behavior! (e.g., conscious will) • The failure to make some essential distinctions • Content vs form (the property of images vs the property of what images are about) {compare the code box example} • An image of X with property P can mean • (An image of X) with property P or • An image of (X with property P) • Capacity vs typical behavior: Architecture vs knowledge

42. Are all the things we thought were due to internal pictures actually due to tacit knowledge? Other reasons for imagery phenomena: • Task demands: Imagine that X = What would it be like if you saw X?

43. Are there pictures in the brain? • There is no evidence for cortical displays of the right kind to explain visual or imaginal phenomena

44. So what is in the brain? • The best hypothesis so far (i.e., the only one that has not been shown to be clearly on the wrong track) is that the brain is a species of computer in which representations of the world are encoded in the form of symbol structures, and actions are determined by calculations (i.e., inferences) based on these symbolic encodings.

45. So why does it not feel like we are doing computations? • Because the content of our conscious experience is a very poor guide to what is actually going on that causes our experiences and our behavior. Science is concerned with causes, not just correlations. • Because we can’t assume that the way things seem has much to do with how it works (e.g., language understanding) • As in most sciences, the essential causes are far from obvious (e.g., why does the earth go around the sun? What is this table made of ? etc.). • In the case of cognition, what is going on is a delicate mixture of the obvious (what Granny or Shakespeare knew about why people do what they do) and the incredible

46. Often we get the wrong story because we have the wrong methods or instruments.Conscious experience may be one of those