Knowledge Representation --- Images and Propositions

1. Knowledge Representation--- Images and Propositions Lecturer: Siyun Liu

2. Knowledge Structures • Mental Representation of Knowledge » • declarative knowledge • facts that can be stated • knowing that • procedural knowledge • procedures that can be implemented • knowing how

3. Knowledge Representations • External Representations » • Internal Representations » Back

4. External Representations • Pictures » • Words » • The differences between picture representation and word representation » Home

5. Picture Representations • The picture is relatively analogous to the real-world object it represents; • The picture shows concrete attributes (shape & relative size) that are similar to the features and spatial properties of the real-world object the picture represents. • Pictorial representations convey all features simultaneously Home

6. Picture Representation Home

7. Word Representation • Words are symbolic representations • Words capture abstract and categorical information • Representations in words usually convey information sequentially Home

8. Words Are Symbolic • 拉丁符号：Ç Ì Ñ Ô Ø à ã Ë Đ • 希腊符号：ΎξλζέΪΠΣΘςΰ • 希伯来语：בהלףץסכזא • 汉语：中国 Home

9. Pictures vs. Words • Neither form of representation actually retains all of the characteristics of what is being represented • Pictures and words represent relationships in different ways Home

10. Internal Representations • Mental Images » • Propositions » Home

11. Mental Images • Definition » • Various Forms » Home

12. Definition of Mental Images • The mental representation of things that are not currently being sensed by the sense organs; • It may represent things that have never been observed by your senses at any time; • It may represent things that do not exist at all. Home

13. Various Forms • Seeing ------ visual images » • seeing in the absence of a visual stimulus • not presently visible to the eyes • Hearing • Smell • Taste Home

14. Mental Imagery • Memory Codes of Mental Imagery » • Empirical Investigations of Imagery» • The nature of mental imagery » • Questions about mental images • What are images? • What kinds of properties do images have? • How are these like or unlike the properties that real pictures have? • Neuropsychological Findings » Home

15. Memory Codes of Mental Imagery • The Dual-Coding Hypothesis » • The Relational-Organizational Hypothesis » Home

16. The Dual-Coding Hypothesis • Allan Paivio (1969, 1971) • We can use either verbal or imagined codes or both for representing information. • Verbal • Containing information about an item’s abstract, linguistic meaning • Imagined • Mental pictures of some sort that represent what the item looks like • Pictures and concrete words • Both verbal labels and visual images • abstract words • Only verbal label Evidence1 for Dual-Coding Home Evidence2 for Dual-Coding

17. Evidence for Dual-Coding • Paivio (1965) • four lists of noun pairs • CC: concrete word vs. concrete word • Book vs. Table • CA: concrete word vs. abstract word • Chair vs. Justice • AC: abstract word vs. concrete word • Freedom vs. Dress • AA: abstract word vs. abstract word • Beauty vs. Truth Continue…

18. Evidence for Dual-Coding • Participants learnt one of four lists, then had a recall test Continue…

19. Evidence for Dual-Coding • Interpretations for results (1969) • When items are coded by both images and verbal labels, the chances of the learner’s retrieving them are obviously better. • If the learner forgets the verbal label, he or she might still access the visual images, or vice versa. • Items coded only by verbal labels are disadvantages • If the verbal label is forgotten or “misplaced”, the learner has less to go on. Continue…

20. Evidence for Dual-Coding • Interpretations for results (1969) • The first noun in a pair (“stimulus” noun) serves as a conceptual peg on which the second (“response”) noun is hooked. • The stimulus noun serves as a “mental anchor”, a place to which the representation of the response noun can be attached. • The imaginability of the first noun is particularly important in improving memorability • This is why recall in the CA condition was significantly higher than in the AC condition Continue…

21. Evidence for Dual-Coding • Interpretations for results • Whenever possible, participants spontaneously formed visual images of the noun pairs • The formation was easiest with concrete nouns • Visual imagery, unlike verbal labelling, increases as a function of concreteness. • The more concrete the noun, the richer the image and the more elaborated the internal code. Return

22. Evidence for Dual-Coding • Lee Brooks (1968) • Provided evidence that images are distinct from verbal materials or at least use different processes from those used by verbal materials. • Two types of tasks • visual task: • briefly presented a picture, answer questions about it • verbal task: • briefly presented a sentence, make judgment about it • Two types of responses • Visual: point to an answer • Verbal: saying “yes” or “no” Continue…

23. Evidence for Dual-Coding • Lee Brooks (1968) • Material 1 used in the experiment • Task for material 1 • Findings for task 1 • Material 2 used in the experiment • Task for material 2 • Findings for task 2 Conclusion

24. Evidence for Dual-Coding A BIRD IN THE HAND IS NOT IN THE BUSH. Start at the corner marked with a dot, and indicate whether or not each corner is at the extreme top or bottom. For each word in the sentence above, indicate whether or not each word is a concrete noun. Return

25. Evidence for Dual-Coding • Participants responded in different ways • Verbal • Participants say “yes” or “no” • Spatial • Participants were given a response sheet on which the letters Y and N were printed in an irregular pattern, and were told to point to either a Y or an N in each row to indicate their response Return

26. Evidence for Dual-Coding • Task 1: • Participants took almost two and a half times longer when they responded by pointing than they did by responding verbally. Reason • Task 2: • Participants were faster to respond by pointing than they were to respond verbally. Reason Return

27. Evidence for Dual-Coding • It requires the formation of a visual image of an F. • The visual image probably has at least some picture-like qualities (spatial or visual), so a spatial or visually guided response (pointing) would be interfered with to a greater extend than would a verbal response. • The visual image is more disruptive of, and disrupted by, another spatial or visual type of task (pointing) than by a verbal kind of task (talking). Return

28. Evidence for Dual-Coding • Holding a sentence in memory (a verbal task) is easier to do with a concurrent visual/spatial task (such as pointing) than with another verbal task. Return

29. Evidence for Dual-Coding • Conclusion • Images and words use different kinds of internal codes (dual-coding hypothesis) • Visual images and words are separate codes Try it • Visual imagery interferes with other visual tasks • Verbal tasks interfere with other verbal tasks • Visual imagery does not interfere with verbal tasks Return

30. Which of the following tasks is (are) the hardest one(s)? • Imagine an elephant and at the same time try to draw a house. • Imagine the definition of “elephant” and at the same time write down a definition of short-term memory. • - Draw an elephant and at the same time provide a definition of short-term memory. Return

31. The Relational-Organizational Hypothesis • Bower (1970b) • The relational-organizational hypothesis • An alternative to the dual-coding hypothesis • Imagery improved memory by producing more associations between the items to be recalled, rather than being richer than verbal labels • Individuals create a number of links or hooks between the information to remember and other information in paired-associates learning. Home Continue…

32. The Relational-Organizational Hypothesis • Imagery facilitates the creation of a greater number of hooks that link the two to-be-remembered pieces of information • Bower (1970b) provided evidence to distinguish between the dual coding and the relational-organizational hypothesis Evidence

33. Evidence for relational-organizational hypothesis • Three groups of participants with different instructions for a paired-associates learning task • G1: to rehearse aloud • G2: to construct two images that did not interact and were “separated in imaginal space”see here • G3: to construct an interactive scene of the two words in a pair see here Continue…

34. Piano vs. Cigar Return

35. Dog vs. Bathtub Return

36. Evidence for relational-organizational hypothesis • Results • All participants recognized about 85% of the previously seen words, however, • G1: recalled 30% of the paired associates • G2: recalled 27% of the paired associates • G3: recalled 53% of the paired associates Interpretations

37. Evidence for relational-organizational hypothesis • If Dual-Coding hypothesis is correct • G2 = G3 • Imagery simply led to more elaborated coding of the paired associates • However • G2 < G3 • It is not imagery per se that helps memory • Rather, it is the way in which imagery is used Continue…

38. Evidence for relational-organizational hypothesis • Interacting images presumably create or suggest more links between the target information and other information, making the target information easier to retrieve • Evidence support for relational-organizational hypothesis Home

39. Empirical Investigations of Imagery • Mental Rotation of Images • Shepard & Metzler (1971) » • Cooper & Shepard (1973) » • Images Scanning • Functional-equivalence hypothesis » • Image Scaling » • Image Scanning » Home

40. Mental Rotation of Images • Shepard & Metzler (1971) • They showed participants perspectives line drawings of three dimensional objects. • Participants were presented two drawings • The drawings depicted the same object but with one rotated by some degree. See here • Rotation was in a picture plane • Rotation was in depth • The drawings depicted mirror-image reversals • The objects were similar but not identical. See here • The mirror images were also sometimes rotated. Findings

41. Mental Rotation of Images Return

42. Mental Rotation of Images Return

43. Mental Rotation of Images Return

44. Mental Rotation of Images • The amount of time it took participants to decide if the two drawings depicted the same object or a mirror-image reversal was directly proportional to the angle of rotation between the drawings. See Figure Here

45. Mental Rotation of Images 5 MeanRT (Sec) 0 0 20 40 60 80 100 120 140 160 180 Angle of rotation (degree) Return

46. Mental Rotation of Images • Changes in manipulations • Materials • Rotation of letters • Procedure • Participants given a cue showing the orientation to which the test stimulus would be rotated, before the test stimulus appeared Results

47. Mental Rotation of Images Result

48. Implications

49. Mental Rotation of Images • Participants’ performances were the same for all angles of rotation if the cues were presented early enough • 1000 msec before the test stimulus appeared • Participants were able mentally to rotate their images either clockwise or counterclockwise, depending on which direction led to a lesser angle. Home More questions

50. Mental Rotation of Images • Are participants in these experiments mentally rotating the whole stimulus, or are they looking only at certain parts? • Lynn Cooper (1975) • Used irregular polygons as materials ▶