chapter 6 schedules or reinforcement and choice behavior
Download
Skip this Video
Download Presentation
Chapter 6 – Schedules or Reinforcement and Choice Behavior

Loading in 2 Seconds...

play fullscreen
1 / 52

Chapter 6 – Schedules or Reinforcement and Choice Behavior - PowerPoint PPT Presentation


  • 119 Views
  • Uploaded on

Chapter 6 – Schedules or Reinforcement and Choice Behavior. Outline Simple Schedules of Intermittent Reinforcement Ratio Schedules Interval Schedules Comparison of Ratio and Interval Schedules Choice Behavior: Concurrent Schedules Measures of Choice Behavior The Matching Law

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Chapter 6 – Schedules or Reinforcement and Choice Behavior' - margie


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
chapter 6 schedules or reinforcement and choice behavior
Chapter 6 – Schedules or Reinforcement and Choice Behavior
  • Outline
    • Simple Schedules of Intermittent Reinforcement
      • Ratio Schedules
      • Interval Schedules
      • Comparison of Ratio and Interval Schedules
    • Choice Behavior: Concurrent Schedules
      • Measures of Choice Behavior
      • The Matching Law
    • Complex Choice
      • Concurrent-Chain Schedules
      • Studies of “Self Control”
slide2

Simple Schedules of Intermittent Reinforcement

  • Ratio Schedules
    • RF depends only on the number of responses performed
  • Continuous reinforcement (CRF)
    • each response is reinforced
      • barpress = food
      • key peck = food
  • CRF is rare outside the lab.
    • Partial or intermittent RF
slide3

Partial or intermittent Schedules of Reinforcement

  • FR (Fixed Ratio)
    • fixed number of operants (responses)
      • CRF is FR1
    • FR 10 = every 10th response  RF
  • originally recorded using a cumulative record
    • Now computers
      • can be graphed similarly
slide5

The cumulative record represents responding as a function of time

    • the slope of the line represents rate of responding.
      • Steeper = faster
slide6

Responding on FR scheds.

    • Faster responding = sooner RF
      • So responding tends to be pretty rapid
    • Postreinforcement pause
  • Postreinforcement pause is directly related to FR.
    • Small FR = shorter pauses
      • FR 5
    • large FR = longer pauses
      • FR 100
        • wait a while before they start working.
    • Domjan points out this may have more to do with the upcoming work than the recent RF
      • Pre-ratio pause?
slide9

how would you respond if you received $1 on an FR 5 schedule?

  • FR 500?
    • Post RF pauses?
  • RF history explanation of post RF pause
    • Contiguity of 1st response and RF
      • FR 5
        • 1st response close to RF
        • only 4 more
      • FR 100
        • 1st response long way from RF
        • 99 more
slide10

VR (Variable ratio schedules)

    • Number of responses still critical
    • Varies from trial to trial
  • VR 10
    • reinforced on average for every 10th response.
    • sometimes only 1 or 2 responses are required
    • other times 15 or 19 responses are required.
slide11

Example (# = response requirement)

VR10FR10

      • 19  RF 10  RF
      • 2  RF 10  RF
      • 8  RF 10  RF
      • 18  RF 10  RF
      • 5  RF 10  RF
      • 15  RF 10  RF
      • 12  RF 10  RF
      • 1  RF 10  RF
  • VR 10
    • (19+2+8+18+5+15+12+1)/8 = 10
slide12

VR = very little postreinforcement pause

    • why would this be?
  • Slot machines
    • very lean schedule of RF
    • But - next lever pull could result in a payoff.
slide13

FI (Fixed Interval Schedule)

    • 1st response after a given time period has elapsed is reinforced.
  • FI 10s
    • 1st response after 10s  RF.
      • RF waits for animal to respond
      • responses prior to 10-s not RF.
  • scalloped responding patterns
    • FI scallop
slide16

Similarity of FI scallop and post RF pause?

    • FI 10s?
    • FI 120s?
  • The FI scallop has been used to assess animals’ ability to time.
slide17

VI (variable interval schedule)

    • Time is still the important variable
    • However, time elapse requirement varies around a set average
  • VI 120s
    • time to RF can vary from a few seconds to a few minutes
  • $1 on a VI 10 minute schedule for button presses?
    • Could be RF in seconds
    • Could be 20 minutes
  • post reinforcement pause?
slide18

Produces stable responding at a constant rate

    • peck..peck..peck..peck..peck
    • sampling whether enough time has passed
  • The rate on a VI schedule is not as fast as on an FR and VR schedule
    • why?
    • ratio schedules are based on response.
      • faster responding gets you to the response requirement quicker, regardless of what it is?
    • On a VI schedule # of responses don’t matter,
      • steady even pace makes sense.
slide19

Interval Schedules and Limited Hold

    • Limited hold restriction
      • Must respond within a certain amount of time of RF setup
    • Like lunch at school
      • Too late you miss it
slide20

Comparison of Ratio and Interval Schedules

    • What if you hold RF constant
  • Rat 1 = VR
  • Rat 2 = Yoked control rat on VI
    • RF is set up when Rat 1 gets to his RF
      • If Rat 1 responds faster, RF will set up sooner for Rat2
      • If Rat 1 is slower, RF will be delayed
slide22

Why is responding faster on ratio scheds?

    • Molecular view
      • Based on moment x moment RF
      • Inter-response times (IRTs)
        • R1……………R2 RF
          • Reinforces long IRT
        • R1..R2 RF
          • Reinforces short IRT
      • More likely to be RF for short IRTs on VR than VI
slide23

Molar view

    • Feedback functions
      • Average RF rate during the session is the result of average response rates
    • How can the animal increase reinforcement in the long run (across whole session)?
      • Ratio - Respond faster = more RF for that day
        • FR 30
        • Responding 1 per second RF at 30s
        • Respond 2 per second RF at 15s
slide24

Molar view continued

    • Interval - No real benefit to responding faster
      • FI 30
      • Responding 1 per second RF at 30 or 31 (30.5)
      • What if 2 per second 30 or 30.5 (30.25)
    • Pay
      • Salary?
      • Clients?
slide25

Choice Behavior: Concurrent schedules

    • The responding that we have discussed so far has involved schedules where there is only one thing to do.
    • In real life we tend to have choices among various activities
    • Concurrent schedules
      • examines how an animal allocates its responding among two schedules of reinforcement?
      • The animals are free to switch back and forth
slide27

Measures of choice behavior

    • Relative rate of responding
      • for left key

BL.

(BL + BR)

    • BL = Behavior on left
    • BR = Behavior on right

We are just dividing left key responding by total responding.

slide28

This computation is very similar to the computation for the suppression ratio.

    • If the animals are responding equally to each key what should our ratio be?

20 . = .50

20+20

    • If they respond more to the left key?

40 . = .67

40+20

    • If they respond more to the right key?

20 . = .33

20+40

slide29

Relative rate of responding for right key

    • Will be reciprocal of left key responding, but also can be calculated with the same formula

BR.

(BR + BL)

  • Concurrent schedules?
    • If VI 60 VI 60
    • The relative rate of responding for either key will be .5
      • Split responding equally among the two keys
slide30

What about the relative rate of reinforcement?

    • Left key?
      • Simply divide the rate of reinforcement on the left key by total reinforcement.

rL.

(rL + rR)

  • VI 60 VI 60?
    • If animals are dividing responding equally?
    • .50 again
slide31

The Matching Law

    • relative rate of responding matches relative rate of RF when the same VI schedule is used
      • .50 and .50
    • What if different schedules of RF are used on each key?
slide32

Left key = VI 6 min (10 per hour)

  • Right key = VI 2 min (30 per hour)

Left key relative rate of responding

BL. = rL .10 =.25 left

(BL + BR) (rL + rR) 40

Right key?

simply the reciprocal

.75

Can be calculated though

BR. = rR .30 =.75 right

(BR + BL) (rR + rL) 40

Thus - three times as much responding on right key .25x3 = .75

slide33

Matching Law continued: Simpler computation.

BL. = rL.

BRrR

10

30

again – three times as much responding on right key

slide34

Herrnstein (1961) compared various VI schedules

    • Matching Law.
      • Figure 6.5 in your book
slide36

Application of the matching law

    • The matching law indicates that we match our behaviors to the available RF in the environment.
    • Law,Bulow, and Meller (1998)
      • Predicted adolescent girls that live in RF barren environments would be more likely to engage in sexual behaviors
      • Girls that have a greater array of RF opportunities should allocate their behaviors toward those other activities
      • Surveyed girls about the activities they found rewarding and their sexual activity
      • The matching law did a pretty good job of predicting sexual activity
    • Many kids today have a lot of RF opportunities.
      • May make it more difficult to motivate behaviors you want them to do
        • Like homework
          • X-box
          • Texting friends
          • TV
slide37

Complex Choice

    • Many of the choices we make require us to live with those choices
      • We can’t always just switch back and forth
        • Go to college?
        • Get a full-time job?
      • Sometimes the short-term and long-term consequences (RF) of those choices are very different
        • Go to college
          • Poor now; make more later
        • Get a full-time job
          • Money now; less earning in the long run
slide38

Concurrent-Chain Schedules

    • Allows us to examine these complex choice behaviors in the lab
  • Example
    • Do animals prefer a VR or a FR?
      • Variety is the spice of life?
slide40

Choice of A

    • 10 minutes on VR 10
  • Choice of B
    • 10 minutes on FR 10
  • Subjects prefer the VR10 over the FR10
    • How do we know?
  • Subjects will even prefer VR schedules that require somewhat more responding than the FR
    • Why do you think that happens?
slide41

Studies of Self control

    • Often a matter of delaying immediate gratification (RF) in order to obtain a greater reward (RF) later.
      • Study or go to party?
      • Work in summer to pay for school or enjoy the time off?
slide42

Self control in pigeons?

    • Rachlin and Green (1972)
      • Choice A = immediate small reward
      • Coice B = 4s Delay  large reward
    • Direct choice procedure
      • Pigeons choose immediate, small reward
    • Concurrent-chain procedure
      • Could learn to choose the larger reward
        • Only if a long enough delay between initial choice and the next link.
slide44

This idea that imposing a delay between a choice and the eventual outcomes helps organisms make “better” (higher RF) outcomes works for people to.

  • Value-discounting function

V = M .

(1+KD)

      • V-value of RF
      • M- magnitude of RF
      • D – delay of reward
      • K – is a correction factor for how much the animal is influenced by the delay
    • All this equation is saying is that the value of a reward is inversely affected by how long you have to wait to receive it.
    • IF there is no delay D=0
      • Then it is simply magnitude over 1
slide45

If I offer you

    • $50 now or $100 now?

50 . = 50 100 . = 100

(1+1x0) (1+1x0)

    • $50 now or $100 next year?

50 . = 50 100 . = 7.7

(1+1x0) (1+1x12)

slide47

As noted above K is a factor that allows us to correct these delay functions for individual differences in delay-discounting

  • People with steep delay discounting functions will have a more difficult time delaying immediate gratification to meet long-term goals
    • Young children
    • Drug abusers
  • Madden, Petry,Badger, and Bickel (1997)
    • Two Groups
      • Heroin-dependent patients
      • Controls
    • Offered hypothetical choices
      • $ smaller – now
      • $ more – later
    • Amounts varied
      • $1,000, $990, $960, $920, $850, $800, $750, $700, $650, $600, $550, $500, $450, $400, $350, $300,$250, $200, $150, $100, $80, $60, $40, $20, $10, $5, and $1
    • Delays varied
      • 1 week, 2 weeks, 2 months, 6 months, 1 year, 5 years, and 25 years.
slide50

It has been described mathematically in the following way (Baum, 1974)

  • RA = b rAa
  • RB rB
  • RA and RB refer to rates of responding on keys A and B (i.e. left and right)
  • rA and rB refer to the rates of reinforcement on those keys
  • When the value of exponent a is equal to 1.0 a simple matching relationship occurs where the ratio of responses perfectly match the ratio of reinforcers obtained.
  • The variable b is used to adjust for response effort differences between A an B when they are unequal, or if the reinforcers for A and B were unequal.
ad