William Kaye Estes

1. William Kaye Estes Behaviorist or Cognitivist?

2. W.K. Estes (1919- Statistical learning theorist Indiana University, Stanford and Harvard- three of oldest programs Worked with Skinner at U of Minn Major focus: attempt to quantify Guthrie�s model

3. Major Theoretical Concepts Assumption I: Learning situation involves large but finite # of stimulus elements (S): many things learner could experience at onset of learning trial Includes both internal and external stimuli Assumption II: All responses made in experimental situation fall into 1 of 2 categories: A1 response: response that is required/necessary for correct responding in experimental situations A2 responses: all other incorrect responses No gradation: just 1 or the other

4. Major Theoretical Concepts Assumption III: All elements in S attached to either A1 or A2 All or nothing: attached to one or other, not both Should see shift: at first attached to A2, soon move to A1 Assumption IV: Learner limited in ability to experience S Only samples some of S Constant proportion of S experienced at beginning of each learning trial is designated as theta (?) After learning trial, elements in ? returned to S Sampling with replacement

7. Major Theoretical Concepts Assumption V: Learning trial ends when response occurs Must be either A1 or A2 If A1: then ? elements conditioned to A1 As # of elements conditioned from S to A1 increases, likelihood that ? contains some of conditioned S elements increases Result: likelihood of A1 response to occur at beginning of learning trial increases over time This is learning State of the system at any given time = proportion of elements attached to A1 and A2 responses

8. Major Theoretical Concepts Assumption VI: Because elements in ? are returned to S at conclusion of trial, ? sampled at beginning of new trial essentially random proportion of elements conditioned to A1 in S reflected in elements in ? at beginning of new trial A1 may not occur for several trials (is probabalistic) Thus: determine likelihood or probability of an A1 by examining ? If all elements of S conditioned to A1, then prob = 1.0 If only 75% are conditioned: prob = 0.75 If only 25% are conditioned: prob = 0.25

9. Formalizing the assumptions: Stimulus Sampling theory Probability of an A1 response depends on the state of the system Probability of response A1 on any given trial n(Pn) = proportion of elements conditioned to A1 on that trial (pn) Pn=pn All elements are either A1 (with probability p) or A2 elements (probability of q) p+q=100 or p=1.0-q Elements not conditioned to A1 on any given trial n (reflected in q) must be: elements that were not preconditioned to A1 prior to 1st trial AND not conditioned to A1 on any previous trial On any trial n, probability that an element has no preconditioned S on trial 1 = (1-P1) On any trial n, probability that an element not conditioned previously to A1 = (1- ?)n-1 joint probability of 2 events occurring together = mathematical product of their individual probabilities: Therefore: q= (1-P1)(1- ?)n-1 Substituting from above: Pn = 1-(1-P1)(1- ?)n-1 ee

10. Let�s do an example Two learners: Learner 1: P1=0, ?=0.05 Learner 2; P1=0, ?=0.2 learner 1: Trial 1: P1=1-(1-0)(1�.05)0=0 Trial 2: P2=1-(1-0)(1�.05)1=0.05 Trial 3: P3=1-(1-0)(1�.05)2=0.1 Learner 1 will reach asymptote after 105 trials learner 2: Trial 1: P1=1-(1-0)(1�.2)0=0 Trial 2: P2=1-(1-0)(1�.2)1=0.2 Trial 3: P3=1-(1-0)(1�.2)2=0.36 learner 2 will reach asymptote at 23-25 trials Generates learning curve (similar to R-W and we will find out similar to Hull) Note again is negatively accelerated learning curve! As more and more S becomes conditioned, less and less change (fewer A2 to move over)

11. How can he apply this? Generalization: transfer takes place to extent that 2 situations have stimulus elements which overlap More overlap = more generalization Extinction: Extinction trial ends with subject doing something different Thus: stimulus elements conditioned to A1 revert back to A2 May not be complete, thus get spontaneous recovery

12. How can he apply this? Spontaneous Recovery: S includes all stimulus elements including transitory events and temporary body states Because many S events are transitory, may be part of S on one occasion but not on other When not part of S, cannot be sampled; when are available, can be sampled Possible for A1 responses to be conditioned to many transitory elements If not occur during extinction, not revert back to A2 Then, if reoccur- conditioning remains and get response!

13. Probability Matching Traditional method: Signal light first Then 2 other lights Must guess at signal light which of the two lights will turn on Experimenter varies probability of lights on Result: subject typically matches the probability of the light that turns on most often

14. Probability Matching According to Estes� theory: E1= left light on E2 = right light on A1 = predicting E1: When E1 occurs, evokes implicit A1 response A2 = predicting E2 When E2 occurs, evokes implicit A2 response p = probablity of E1 occuring: p(E1) 1- p = p(E2)

15. Probability Matching: On trial in which E1 occurs: All elements sampled from S on that trial become conditioned to A1 Opposite occurs for E2 trials Probability of an A1 response to any given trial (Pn) = proportion of elements in S that are conditioned to A1 (and vice versa) ? remains constant and = proportion of elements sampled on each trial Thus: the probability of an A1 response after n trials: Pn = p-(p-P1)(1- ?)n-1 Because (1- ?) is less than 1, with n getting larger, result is negatively accelerating curve: the learning curve Predicts that proportion of A1 responses made by subject will eventually match proportion of E1 occurrences as set by experiment

16. Estes Markov Model of Learning Remember: most learning theorists (Thorndike, Skinner, Hull) believe learning occurs gradually but Guthrie and Gestaltists believed was 1-trial Also remember: statistical learning theories = probabalistic Dependent variable = probability of a response Difference of opinion comes over what the changing response probabilities tell us: Gradual learning Complete (1 Trial) learning Estes Stimulus Sampling Theory (SST): Early on: accepted both incremental and all-or-none Sampled stimuli conditioned all or none Sampling occurred incrementally Those stimulus elements that were sampled on given trial conditioned in all-or-none manner, but only small number conditioned, thus took many trials Later on: took more all-or-none position When small # of elements to be conditioned, occurs all or none Found data could be explained by Markov process Markov process = abrupt, stepwise change in response probabilities rather than asymptotic curve

17. Evidence for Markov Process Paired associates learning: Learn pairs of items: when presented with 1, say the other Learn pairs, then given �multiple choice�: word and 4 choices Probability of correct = 0.25 If person guessed correctly on 1st trial: probability went to 1.0 and stayed there When pool the data across trials: average the probabilities and get more asymptotic curve Thus: individual learning appears to be all or none; averaged sessions or peoples = asymptotic curve Data basically support, although Hull�s model may be better interpretation (have to wait!). Bottom line: if you got it correct the first time, you are much more likely to get it right again! Important implications for education Don�t practice mistakes!

19. How can this be all or none 4 people are learning the task One gets the right answer Three do NOT get right answer Average of �correct� = 0.25 (1.0/4) But: would argue that must plot 4 individually 1 person at 1.0 3 people at 0.0 Each person �steps up� all or none, only get asymptotic curve when �average� the learners

21. Criticisms Estes: when something is learned, it is learned completely If it is not learned completely, then it is not learned at all Criticism though: Underwood and Keppel (1962): if all or none is correct, why were all the items that were correct on first test not also correct on second test? Evidence suggests that much learning is carried over, but not all or none Must rely on Hull�s model, instead: oscillation effects Oscillation of the stimulus elements Not every trial is equal Modern data typically support asymptotic learning, with nod that most learning occurs on first trial

22. Cognitive Psychology and Estes Remember, is a contiguity theorist but is also considered a cognitivist Why? Emphasizes importance of memory Stimuli and responses associated by contiguity These must be remembered Scanning model of decision making: Person will choose to make response that yields most valuable outcome Uses whatever info has stored in memory concerning response-outcome relationship Using this information, optimizes Memory critical for language, particularly grammatical rules and principles

23. Cognitive Array Model: Used to understand behaviors of classifying and categorizing People assumed to examine complex stimulus, attend to (sample) important/salient features Stimulus features AND their category/class membership learned all or none in 1 trial Array model: differs from SST Both stimulus characteristics AND category/class are stored in memory set or memory array New stimuli compared to these sets to determine where fits (comparitor model) Array model focuses on current classification of events, not just past Memory is �past� Comparisons are done in present

24. Differences between SST and Array models: SST assumes additive stimulus relationships: According to SST model, when compare stimuli, choose the label that covers most categories E.g.: Large RED circle; small blue triangle Shown a large red triangle: most like the LARGE RED circle, so more likely to choose that category Problem: data not support in complex situations Array model: assumes multiplicative stimulus relationships Compare stimulus attributes or elements Use similarity coefficient to describe degree of similarity Measure of similarity = product of these coefficients P(response transfer) from training to test situations = function of the product of the similarity coefficients. Model used to describe/predict how people judge stimuli to be members of specific categories, NOT how the stimuli are generalized

25. How use the model? Items within a category = similar to one another Assume s=0.7 Three matches: 1x1x1=1 Two matches, one mismatch: 1x1x.7=.7 One match, two mismatch: 1x.7x.7=.49 No matches: .7x.7x.7=.34

26. Now make it more complicated: Apply the array model to determine degree to which particular stimulus is representative of the category as a whole Construct similarity coefficient matrix: comparing elements within a category to other elements in that category Also comparing a single stimulus with itself Must add similarity to A = (1+s)

27. Let�s see how this works: Now can add all together: (1+s) -------------- (1+s)+(s2+s3) So: assume that s=0.7 (1+0.7) 1.7/2.53 = 0.67 ---------------- = (1+0.7)+(.49+.34) Interestingly, Estes assumes people actually compute these values innately (not just a descriptive equation) This model remains VERY controversial!

28. Estes and reinforcement Cognitive interpretation (remember, not a reinforcement theorist!) Rejects law of effect: no reward needed, simply association Reinforcement simply prevents unlearning Preserves association between certain responses and stimuli Reinforcement works because it provides information Not only learn S-R relationships, but also R-O (outcome) Learn which responses lead to which consequences Reinforcement and punishment are more performance variables than learning variables

29. Learning to Learn: Continuity-noncontinuity controversy: Incremental or all or none? Both all-or-none and incremental learning are correct Still VERY controversial Better interpretation may be learning to learn or learning set: Harry Harlow studies: Early on, lots of mistakes Later, few errors Forming a learning set or expectations about outcomes �animals can gradually learn insight� How? Error factors: Error factors = erroneous strategies that have to be extinguished before discrimination problem can be solved Response tendencies that lead to incorrect choice Must eliminate errors more than learn correct choices

30. Evaluation: Mathematical models of learning Not dead, by any means Taking area by storm (Staddon, Baum, Nevin, Davison, etc) Emerging and highly changing Estes Contributions Increased precision, added additional cognitive factors Mathematical Moved away from more simplistic early behaviorists (modern behaviorists look an awful lot like Estes) Criticisms: Model only has restricted use, not widely applicable or adaptable No allowance for mechanisms other than S�R contiguity Mathematical abstractions constrain experimental conditions- can only experiment on it if can model it