Punishment and Negative Reinforcement

1. Punishment and Negative Reinforcement

2. Avoidance

3. Discrete Trial Avoidance • SΔsignals that an aversive event will occur • If aversive stimulus already has begun, • “Escape” response terminates stimulus • If aversive stimulus has yet to begin, • “Avoidance response prevents/postpones stimulus • Shuttle Avoidance • long box with a small partition in the middle. • Light/tone signal s shock will occur • Must jump to other side to prevent/escape shock • Escape often is the precursor to avoidance

4. Free-Operant Avoidance • Also known as Sidman Avoidance • Named for Murray Sidman (1953) • No explicit stimulus serves as SD • Aversive stimulus is set to occur at set intervals • Response resets interval • No response results in presentation + move to “worse” schedule • When highly trained avoidance responses are consistent and efficient

5. An Example • Following a response, shock set to occur at 10 sec intervals • 10 s Response-Shock interval • Following a shock, shock set to occur at 5 sec intervals • 5 s Shock-Shock interval • Response resets interval back to 10 s response-shock interval

6. Punishment • A reduction in the probability of a specific response as a result of the immediate delivery of a stimulus of a stimulus for that response. The stimulus is designed as a punishing stimulus, the whole procedure is deigned as punishment • Azrin and Holz 1966

7. Types of Punishment • Positive • Response leads to presentation of Aversive stimulus, leads to reduction in behavior • Examples include: • Shock, spanking, yelling • Negative • Response leads to removal of Pleasant stimulus, leads to reduction in behavior • Examples include: • Response cost, time out

8. Terms • Punisher • Must lead to decrease in future responding • Aversive Stimulus • Something that one will work to avoid or escape • Differentiation not so clear… • But neither is simply something bad

9. Very Important • By itself… • Punishment only suppresses behavior • The punished behavior and other behavior • Could lead to “complete” suppression of most responses • Reinforcement of other behaviors aided by punishment of unwanted response

10. Critics of Punishment w/in Behavior Analysis

11. Arguments against Punishment • Thorndike • Small monetary losses as consequence in memory studies, did not significantly improve memory • Skinner • Argued on grounds of effectiveness not morality? • Argued that Sr+ and Sr- are not reciprocal processes • Punished behavior much more likely to reappear after punisher is removed • Sr- gives rise to aggressive counter attack

12. Skinner 1938 • Food reinforced lever pressing • Soon, lever slap was used as punisher • Responding suppressed but not weakened? • Behavior re-emerged at pre-punishment levels almost immediately after EXT

13. Avoidance - organisms responses prevent the occurrence of an aversive stimulus. - eg moving around a dark room - hold hands out front - feel with feet - the avoidance response is followed by nothing - no particular pleasure, you simply don’t get hurt - how can the absence of something provide reinforcement - 1st avoidance experiments by Bechterev 1913 - human subjects – finger on a metal plate – warnings – CS followed by shock (US) – subjects quickly lift finger off plate upon being shocked – learned to withdraw finger upon presentation of warning. - actually not formal Classical condit since delivery of US depends on subjects behaviour - starting in the 30’s avoidance was studied in detail - Guinea Pigs – rotating wheel - Tone CS – shock US - shock stimulated running on the wheel - CC group – shock always 2 sec after start of tone - avoidance group – if wheel moved during CS prior to shock then no shock delivery - so the performance of the avoidance group was superior to the classical group - what can explain this facilitation of performance? If viewed from a classical conditioning paradigm the avoidance trials are like extinction! And should have attenuated the development of the conditioned response. So if pairings are important the results should be opposite! Therefore the results require more than a classical conditioning explanation. Discriminated avoidance – signaled avoidance - discrete trials, each initiated by CS - the events that occur next depend on subject - if respond before shock, CS terminated & avoidance - if respond after shock, CS+ shock stay on until escape - common apparatus – shuttle box – 2 compartments, low barrier – animal shuttles back & forth between two sides of box – “shuttle avoidance” (animal can be shocked on either side of box) - a variation on this is one – way avoidance’ where 1 side of box is always the start side and one side the ‘safe side’ so here there is a ‘shock compartment’ and a ‘safe compartment’. This is usually learned more rapidly than shuttle avoidance. Two process Theory – Mowrer ’47 and Miller ‘51 - a long standing theory – no longer reviewed as a complete explanation of avoidance - 2 mechanisms 1) classical conditioning process activated by pairings of the warning stimulus (CS) and the aversive event (US) when subject fails to make avoidance response. Through association with US, CS comes to elicit fear. 2) Fear is an arousing state that motivates an organism – it is also aversive, so reduction in fear can provide negative reinforcement - since fear is elicited by CS, termination of CS results in reduction of fear! So, the 2nd component is instrumental reinforcement of the avoidance response through fear reduction. - the two processes, classical & instrumental conditioning are interdependent. Classical conditioning must occur 1st, after this the instrumental conditioning creates extinction trials for classical conditioning procedure. - This theory predicts a constant interplay between the two processes - it explains avoidance behaviour in terms of escape from conditioned fear rather than prevention of shock - Thus the instrumental response is reinforced by a tangible event (fear reduction) rather than merely the absence of something (aversive stimulation). Acquired Drive Experiments - attempt to demonstrate 2 diff processes in avoidance 1) condition fear to a CS with a “pure” classical conditioning procedure 2) animals periodically exposed to fear-eliciting CS and allowed to perform an instrumental response to terminate it (thereby reducing far) (No shocks in this phase) - If the 2 process theory is correct then subjects should be able to learn phase 2 – this is called an acquired drive study because the drive to perform the instrumental response (fear) is learned through classical conditioning and is not innate Brown & Jacobs ’49 – rats in shuttle box – confined to one side – light/tone CS + shock – control gp – no checks - phase 2 – subject place on one side of box, CS on until crossed to other side (no shocks, one-way procedure) - would rats learn to cross if only reinforcement was turning off CS? - two groups had similar latencies at the start of training. - As training progressed the shock conditioned animals learned to cross faster -  termination of a fear condition stimulus is sufficient to provide reinforcement for an instrumental report. These results provide strong support for the two process theory! Independent measurement of Fear during Acquisition of avoidance behaviour. - if 2 process theory correct then conditioning of avoidance and conditioning of fear should proceed together - however, conditional fear and avoidance responding are not always highly correlated - animals become less fearful as they learn the avoidance response. - look at conditioned suppression as a measure of amount of fear - animals learn to bar press for food, then a shock conditioned CS presented – measure suppression of lever press behaviour – extent of suppression is thought to be a measure of fear elicited by the CS Kamin et al ‘63 – rats lever pressing on VI schedule - CS tone  shock US in shuttle box for a number of groups until successful avoidance on 1, 3, 9 or 27 consecutive trials - then back to lever pressing – tone presented - lower values – greater disruption - suppression for gps that successfully avoided 1, 3 & 9 shocks, but with more training response suppression declined - suggests that fear decreases during extended avoidance training – however the decrease in fear is not accompanied by decrease in avoidance! - If fear reduction is necessary to reinforce avoidance behaviour how can the behaviour persist in the face of declining CS elicited fear? – Some suggestion that both the CS & the background acquire fear, but as training continues get discriminative & fear to background  more than fear to CS. So CS termination could continue to be fear reducing Asymptotic Avoidance Performance - Two process theory makes specific predictions about the nature of performance - predicts that the strength of the avoidance response will fluctuate in cycles - extinction of conditioned fear by successful avoidance leads to less reinforcement for reduction of fear – the avoidance response will cease to occur in time to prevent the US. - When shock occurs CS is paired with US again an reinstates fear – avoidance reconditioned - so avoidance should go through cycles of extinction & reacquisition - usually avoidance does not fluctuate in cycles - avoidance is highly resistant to extinction ie hundreds of trials of avoidance after shock off - persistence of avoidance is difficult to explain with 2 factor theory - recent empirical findings suggest a clue if focus on the role of response feedback stimuli - avoidance response followed by a period free from shock, there are feedback stimuli as a result of avoidance - these include – change in location, stimuli from making response such as touching & manipulating lever (tactile) – also proprioceptive feedback - because there is no shock these are correlated negatively with shock – this can lead to the development of conditioned inhibition. - so, response cues can become conditioned inhibitors of fear - After extensive avoidance training subjects experience 2 conditioned stimuli on each trial 1) fear eliciting warning stimulus (CS+) followed by 2) fear inhibiting feedback cues from the avoidance response (CS‑) Therefore asymptotic avoidance trials should not be viewed as extinction trials with CS+ alone, but as trials where CS+ is followed by fear-inhibitory response feedback cues. - Data indicates that the presentation of fear-inhibiting stimuli following a CS+ can block extinction of CS+ - Because the inhibitory stimulus is a signal for the absence of shock, occurrence of the CS- makes the absence of the shock on avoidance trials fully expected – this protects the CS+ from any changes in associative strength. – alternative way to think about this – the absence of shock is attributed to CS‑, the absence of shock is attributed to CS-, therefore reevaluation of CS+ unnecessary – so response feedback cues prevent CS+ extinction so CS+ still elicits fear and motivates avoidance. EXTINCTION OF AVOIDANCE THROUGH REPSONSE BLOCKING & CS ALONE EXPOSURE - avoidance persists for along time - are there procedures that can lead to rapid extinction? - flooding or response prevention – present CS+ in avoidance situation, but prevent subject from making response – thus exposed to CS without being allowed to terminate it “flooded” with CS - important variable is duration of forced exposure to CS. - Schiff, Smith & Prochaska ‘72 - rats – CS tone, one way shock avoidance - after 10 successful avoidance trials a barrier blocked off the safe compartment and ratio received various lengths of CS exposure without shock - 1, 5 or 12 blocked trials, CS for 1, 5, 10, 50 or 120 sec - Then barrier removed and subjects tested for extinction - rat in box, CS presented until animals crossed to safe side – NO SHOCKS – subjects tested until they took 120 seconds or longer to cross over - blocked exposure to CS facilitated extinction - effect determined by the total deviation of exposure to the CS - increases in the total duration of blocked exposure resulted in more rapid extinction. - Two process view predicts flooding produces avoidance extinction through extinction of fear to the CS. - however conditioned suppression experiments have found in some situations avoidance extinguishes more rapidly than fear, while in other situations fear extinguishes more rapidly than avoidance. - probably extinction of fear is only 1 factor responsible for effectives of flooding - another variable is being prevented from making the avoidance response. - Katzen & Kerman ‘ 74 first trained rats to avoid shock in shuttle box then 50 extinction trials - pairs of rats – 1 member – shuttle response not blocked so rat could turn off CS by crossing over - other subject ‘yoked’ so received same duration CS but shuttle response blocked by barrier - 3rd group control – no extinction trials during this phase – all groups then got standard extinction - control gp greatest # of crossings - fewest crossings with subjects with blocked responses. Two groups with identical CS exposure performed very differently - so response blocking can facilitate extinction – so more than just Pavlovian extinction of Cs going on - Perhaps subjects learn a response that is incompatable with the avoidance behaviour during blocked exposure to CS & this contributes to the observed loss of avoidance responding. Non-Discriminated (Free operant) avoidance - could animals learn to avoid if no warning signal? - SIDMAN ’53 (Non-Discriminated, Free Operant or SIDMAN AVOIDANCE) – shock is scheduled without warning – ie every 10 sec – a behaviour is specified an avoidance response (eg lever press) – each response prevents delivery of shock for a set time (eg 30 sec). Animals will learn to avoid shock under these conditions even if no warning. - 2 intervals – the S-S interval (shock-shock) and the R-S interval (response-shock) – this is the period of safety after a response - this avoidance allows responses to occur at any time - another diff from signaled avoidance – response there only effective if during CS – here a response anytime will reset the R-S interval – if R-S interval 30 sec - shock rescheduled 30 sec after each response – by always responding right before R-S interval over the safety period can be indefinitely prolonged - these studies involve longer training – animal receives more shocks – even after extensive training animals don’t avoid all shocks - procedures sensitive to individual differences - rate of responding is controlled by the value of the S-S and R-S intervals - the more frequently shocks occur without responding (S-S interval) – the more likely the animal is to learn avoidance - increasing the periods of safety (R-S interval) also promotes avoidance behaviour - the relative values of S-S and R-S are also important (ie R-S can’t be shorter than S-S) - this task challenges the two process theory because there is no explicit CS to elicit conditioned fear and it is not clear how avoidance reduce- attempt to use 2-process theory – S-S and R-S are short and fixed,  they are predictable - suggests that animals learn to respond to the passage of time as a signal for shock. “Temporal conditioning” - assume the passage of time after last shock (in S-S interval) or last response (in R-S interval) becomes conditioned to elicit fear. Since the timing starts over after each occurrence of the avoidance response the response removes the fear eliciting temporal cues – so time cues take on the role of the CS - this predicts subjects will do most of their responding at end of R-S interval when fear is high. - Results obtained are consistent with this prediction - BUT many animals successfully avoid a great many shocks without distributing their responses in this manner - also avoidance behaviour has been successfully conditioned with variable interval S-S an R-S interval - so perhaps 2 process theory is not a good or complete explanation of non-discriminated avoidance. Delay - time interval between response and punisher - increasing the interval results in less suppression of behaviour Schedules of Punishment - same as schedules of reinforcement – have large affect on suppression of behaviour - so higher the ratio, lower the suppression Effects of punishment on schedules of Positive Reinforcement - FI or VI schedule of positive reinforcement – punishment produces a decrease in overall responding but no change in response pattern – VI – stable rate, FI scallop pattern - FR – punishment increases length of post-reinforcement pause but has little effect on ratio run - shock delivery early in ratio run increases the post-reinforcement pause more than shock delivered near the completion of the ratio. In addition, punishment has less effect on instrumental responses that produce more frequent positive reinforcement. Availability of Alternative Responses - in many experiments the punished response is the only one to get positive reinforcement such as food - by decreasing rate of responding subject also decreases the food it gets – therefore conflict between suppressing behaviour to avoid punishment and responding to obtain positive reinforcement - the availability of an alternative source of reinforcement greatly increases the suppression of responding produced by punishment. - Herman & Azrin ’64 – adult (humans) males - response levers - each VI to produce a cigarette – after behaviour occurring at a stable rate responses on 1 lever resulted in an obnoxious noise – for 1 gp only 1 lever available during this phase, in another both levers available – 1 punished - when punished response was the only way to obtain cigarettes punishment produced moderate suppression - when the alternative lever was available, responding on the punished lever ceased completely - thus availability of an alternative response for obtaining positive reinforcement greatly increased the suppressive effects of punishment. Effects of a Discriminative Stimulus Discriminative Punishment – if responding is punished in the presence of a discriminative stimulus but is not punished when the stimulus is absent – here the suppressive effects of punishment will become limited to the presence of the discriminative stimulus. Dinsmoor ’52 – rats, food at VI 2 min, after training 5 min punishment alternated with 5 min no punishment. During punishment the lights in the chamber were turned off and each lever press resulted in shock. During safe period the lights were on & no shocks delivered – food was delivered during all times - rats quickly learned to restrict responding to the safe periods – when the lights were off responding was suppressed. Real World Discriminative Stimuli - strict teacher or parent - police cars Stimulus Control - similarity of situations and generalization of punishment - Honig & Sliuka (’64) – punishment of key-pecking in pigeons – during training key light varied 490-610 nm - pecking reinforced to all wavelengths – once pecking was equal on all key wavelengths they selectively punished pecking to 550 nm with shock - punishment was effective from the outset - got generalization gradient centered around the punished wavelength – as training continued the generalization gradient sharpened considerably. - if subjects find that responding to similar situations is safe – then responding only suppressed in a situation where it is actually punished. - ie teaching child not to cross street – need it to generalize! Punishment to signal Positive reinforcement - punishment does not always decrease behaviour sometimes people seem to seek out punishment - punishment may become a signal or discriminative stimulus for the availability of positive reinforcement - if this occurs punishment will increase rather than decrease responding. - pigeons – peck key for food reinforcement VI schedule then when trained – each response punished with mild shock sufficient to reduce response rate to 50% - then periods when punishment was in effect were alternated with periods of no punishment – the pecking response was only reinforced with food during the punishment periods. No other signal to tell birds about punishment or food. The only way to tell if food would be delivered was to see if punished. - higher rate of pecking occurred during punishment periods than during safe periods. - punishment became an SD for food reinforcement Self Punitive Behaviour - ie making punishment an SD for positive reinforcement - self punitive behaviour can also result from prior escape training – called Vicious-Circle behaviour – rats given escape training in which entire runway electrified – have to run length of it to reach safe goal box – after subjects have learned this an extinction procedure begun – control – no shock experimental group – final third of runway is electrified – Thus experimental groups encounter shock if they run during extinction, but control groups receive no punishment. - punishment increases resistance to extinction of the running response in the experimental subjects - instead of suppressing behaviour, punishment of conditioned escape behaviour facilitates responding. - some innate defensive responses appear to be relatively resistant to punishment - Punishment also leads to a reallocation of responding – suppression of a punished response leads to increase in expression of other responses. - there are few systematic theories of punishment - Conditional Emotional Response theory - Estes ’44 – based on the observation that a conditioned stimulus that has been paired with shock will suppress the performance of food reinforced instrumental behaviour - condition to lever press – associate CS with shock - see effects of CS presentation on lever pressing - interpretation was that CS produces competing responses incompatible with lever pressing - Estes suggests that stimuli experienced just prior to punishment serve the function of signaling punishment – so if lever press right before shock then stimuli associated with lever pressing become associated with shock. These cues will acquire conditioned aversive properties & will elicit conditioned emotional responses that are incompatible with the punished behaviour. - can explain why more intense/longer shocks produce more response suppression – more vigorous conditioned emotional responses – why is contingency important? Stimuli associated with response are more closely related to performance of behaviour so conditioned emotional response likely to interfere with responding Estes ’69 – re thought problem – new account of much of conditioned suppression – paraphrased in motivational terms – shock conditioned stimuli disrupt food – reinforced responding by evoking an emotional or motivational state incompatible with the motivation to maintain the food – reinforced behaviour - shock conditioned stimulus inhibits the motivation to respond based on positive reinforcement Avoidance Theory of Punishment Dinsmoor ’54, ’77 – Stimuli that set the occasion for the instrumental response become conditioned by the aversive stimulus when the response is punished – so stimuli acquire aversive properties - subjects learn to escape from the conditioned aversive stimuli related to the punished response by engaging in a behaviour incompatible with the punished response – performance of the alternate activity results in suppression of the punished behaviour – so don’t weaken the punished response – rather strengthen competing responses. One strength of this theory is that both avoidance and punishment can be understood in the same theoretical framework - problem is that all of the difficulties in explaining avoidance now occur for punishment as well - another difficulty is that the stimuli that acquire conditioned aversive properties are not under the direct control of the experimenter so are difficult to investigate Punishment and negative law of effect - Thorndike – (1911) – punishment opposite to positive reinf. - Thorndike – (1932) – failed to find supporting evidence but other researchers have retained notion - Premack – positive reinforcement when able to engage in valued activity is contingent on prior performance of lower value activity - punishment reverses this – a low valued activity (shock) occurs contingent on the performance of a higher-valued behaviour - undergoing shock can punish lever pressing - running wheel with drinking tube – rats thirsty – so drinking more likely than running – drinking was followed by forced running – drinking was suppressed! - the same contingency can produce opposite outcomes – depending on the relative value of running and drinking – when running more likely (rats not thirsty) then running positively reinforced drinking! So same thing could be positive reinforcer or punishment – depending on the state of the organism - only difference is in punishment animal is forced to engage in lower-valued activity Punishment in Humans - evidence suggests many similarities between effects of punishment on animals and effects on humans - Bucher & Lovaas ’68 – electric shock to treat self destructive behaviour in autistic children - autistic children – stereotyped behaviour, some engage in terrible self destructive behaviour - eg John – physical damage so severe had to be restrained for 24 hrs a day! – how to eliminate the behaviour – one way might be to reinforce other behaviours incompatible with self injury & ignore self-injury – John so severe this was not an option - used punishment instead – taken to specific room – restraints removed – shock each time he hit himself - during 15 baseline sessions no punishment – average of 250 self-hits per session - when punishment introduced – behaviour disappeared almost immediately – there were a number of experimenters

14. Counterarguments • Almost NO evidence that Sr- is more effective than Sr+ • Rewarded behavior also is likely to ext when reinforcer is removed • Avoidance is one of the most resistent to ext schedules

15. Key features of effective punishment • Intensity • Response-dependency and schedule • Immediacy • Consistency • Availability of alternative behavior • Co-occurrence with reinforcement • Existence of conditioned punishers

16. Intensity • Start off at the highest acceptable severity • More humane in long run? • Starting with mild shock and increasing intensity works very poorly! • However, starting intense and then reducing intensity • suppression continues resulting in more suppression than the mild shock would have produced on its own.

17. Avoidance - organisms responses prevent the occurrence of an aversive stimulus. - eg moving around a dark room - hold hands out front - feel with feet - the avoidance response is followed by nothing - no particular pleasure, you simply don’t get hurt - how can the absence of something provide reinforcement - 1st avoidance experiments by Bechterev 1913 - human subjects – finger on a metal plate – warnings – CS followed by shock (US) – subjects quickly lift finger off plate upon being shocked – learned to withdraw finger upon presentation of warning. - actually not formal Classical condit since delivery of US depends on subjects behaviour - starting in the 30’s avoidance was studied in detail - Guinea Pigs – rotating wheel - Tone CS – shock US - shock stimulated running on the wheel - CC group – shock always 2 sec after start of tone - avoidance group – if wheel moved during CS prior to shock then no shock delivery - so the performance of the avoidance group was superior to the classical group - what can explain this facilitation of performance? If viewed from a classical conditioning paradigm the avoidance trials are like extinction! And should have attenuated the development of the conditioned response. So if pairings are important the results should be opposite! Therefore the results require more than a classical conditioning explanation. Discriminated avoidance – signaled avoidance - discrete trials, each initiated by CS - the events that occur next depend on subject - if respond before shock, CS terminated & avoidance - if respond after shock, CS+ shock stay on until escape - common apparatus – shuttle box – 2 compartments, low barrier – animal shuttles back & forth between two sides of box – “shuttle avoidance” (animal can be shocked on either side of box) - a variation on this is one – way avoidance’ where 1 side of box is always the start side and one side the ‘safe side’ so here there is a ‘shock compartment’ and a ‘safe compartment’. This is usually learned more rapidly than shuttle avoidance. Two process Theory – Mowrer ’47 and Miller ‘51 - a long standing theory – no longer reviewed as a complete explanation of avoidance - 2 mechanisms 1) classical conditioning process activated by pairings of the warning stimulus (CS) and the aversive event (US) when subject fails to make avoidance response. Through association with US, CS comes to elicit fear. 2) Fear is an arousing state that motivates an organism – it is also aversive, so reduction in fear can provide negative reinforcement - since fear is elicited by CS, termination of CS results in reduction of fear! So, the 2nd component is instrumental reinforcement of the avoidance response through fear reduction. - the two processes, classical & instrumental conditioning are interdependent. Classical conditioning must occur 1st, after this the instrumental conditioning creates extinction trials for classical conditioning procedure. - This theory predicts a constant interplay between the two processes - it explains avoidance behaviour in terms of escape from conditioned fear rather than prevention of shock - Thus the instrumental response is reinforced by a tangible event (fear reduction) rather than merely the absence of something (aversive stimulation). Acquired Drive Experiments - attempt to demonstrate 2 diff processes in avoidance 1) condition fear to a CS with a “pure” classical conditioning procedure 2) animals periodically exposed to fear-eliciting CS and allowed to perform an instrumental response to terminate it (thereby reducing far) (No shocks in this phase) - If the 2 process theory is correct then subjects should be able to learn phase 2 – this is called an acquired drive study because the drive to perform the instrumental response (fear) is learned through classical conditioning and is not innate Brown & Jacobs ’49 – rats in shuttle box – confined to one side – light/tone CS + shock – control gp – no checks - phase 2 – subject place on one side of box, CS on until crossed to other side (no shocks, one-way procedure) - would rats learn to cross if only reinforcement was turning off CS? - two groups had similar latencies at the start of training. - As training progressed the shock conditioned animals learned to cross faster -  termination of a fear condition stimulus is sufficient to provide reinforcement for an instrumental report. These results provide strong support for the two process theory! Independent measurement of Fear during Acquisition of avoidance behaviour. - if 2 process theory correct then conditioning of avoidance and conditioning of fear should proceed together - however, conditional fear and avoidance responding are not always highly correlated - animals become less fearful as they learn the avoidance response. - look at conditioned suppression as a measure of amount of fear - animals learn to bar press for food, then a shock conditioned CS presented – measure suppression of lever press behaviour – extent of suppression is thought to be a measure of fear elicited by the CS Kamin et al ‘63 – rats lever pressing on VI schedule - CS tone  shock US in shuttle box for a number of groups until successful avoidance on 1, 3, 9 or 27 consecutive trials - then back to lever pressing – tone presented - lower values – greater disruption - suppression for gps that successfully avoided 1, 3 & 9 shocks, but with more training response suppression declined - suggests that fear decreases during extended avoidance training – however the decrease in fear is not accompanied by decrease in avoidance! - If fear reduction is necessary to reinforce avoidance behaviour how can the behaviour persist in the face of declining CS elicited fear? – Some suggestion that both the CS & the background acquire fear, but as training continues get discriminative & fear to background  more than fear to CS. So CS termination could continue to be fear reducing Asymptotic Avoidance Performance - Two process theory makes specific predictions about the nature of performance - predicts that the strength of the avoidance response will fluctuate in cycles - extinction of conditioned fear by successful avoidance leads to less reinforcement for reduction of fear – the avoidance response will cease to occur in time to prevent the US. - When shock occurs CS is paired with US again an reinstates fear – avoidance reconditioned - so avoidance should go through cycles of extinction & reacquisition - usually avoidance does not fluctuate in cycles - avoidance is highly resistant to extinction ie hundreds of trials of avoidance after shock off - persistence of avoidance is difficult to explain with 2 factor theory - recent empirical findings suggest a clue if focus on the role of response feedback stimuli - avoidance response followed by a period free from shock, there are feedback stimuli as a result of avoidance - these include – change in location, stimuli from making response such as touching & manipulating lever (tactile) – also proprioceptive feedback - because there is no shock these are correlated negatively with shock – this can lead to the development of conditioned inhibition. - so, response cues can become conditioned inhibitors of fear - After extensive avoidance training subjects experience 2 conditioned stimuli on each trial 1) fear eliciting warning stimulus (CS+) followed by 2) fear inhibiting feedback cues from the avoidance response (CS‑) Therefore asymptotic avoidance trials should not be viewed as extinction trials with CS+ alone, but as trials where CS+ is followed by fear-inhibitory response feedback cues. - Data indicates that the presentation of fear-inhibiting stimuli following a CS+ can block extinction of CS+ - Because the inhibitory stimulus is a signal for the absence of shock, occurrence of the CS- makes the absence of the shock on avoidance trials fully expected – this protects the CS+ from any changes in associative strength. – alternative way to think about this – the absence of shock is attributed to CS‑, the absence of shock is attributed to CS-, therefore reevaluation of CS+ unnecessary – so response feedback cues prevent CS+ extinction so CS+ still elicits fear and motivates avoidance. EXTINCTION OF AVOIDANCE THROUGH REPSONSE BLOCKING & CS ALONE EXPOSURE - avoidance persists for along time - are there procedures that can lead to rapid extinction? - flooding or response prevention – present CS+ in avoidance situation, but prevent subject from making response – thus exposed to CS without being allowed to terminate it “flooded” with CS - important variable is duration of forced exposure to CS. - Schiff, Smith & Prochaska ‘72 - rats – CS tone, one way shock avoidance - after 10 successful avoidance trials a barrier blocked off the safe compartment and ratio received various lengths of CS exposure without shock - 1, 5 or 12 blocked trials, CS for 1, 5, 10, 50 or 120 sec - Then barrier removed and subjects tested for extinction - rat in box, CS presented until animals crossed to safe side – NO SHOCKS – subjects tested until they took 120 seconds or longer to cross over - blocked exposure to CS facilitated extinction - effect determined by the total deviation of exposure to the CS - increases in the total duration of blocked exposure resulted in more rapid extinction. - Two process view predicts flooding produces avoidance extinction through extinction of fear to the CS. - however conditioned suppression experiments have found in some situations avoidance extinguishes more rapidly than fear, while in other situations fear extinguishes more rapidly than avoidance. - probably extinction of fear is only 1 factor responsible for effectives of flooding - another variable is being prevented from making the avoidance response. - Katzen & Kerman ‘ 74 first trained rats to avoid shock in shuttle box then 50 extinction trials - pairs of rats – 1 member – shuttle response not blocked so rat could turn off CS by crossing over - other subject ‘yoked’ so received same duration CS but shuttle response blocked by barrier - 3rd group control – no extinction trials during this phase – all groups then got standard extinction - control gp greatest # of crossings - fewest crossings with subjects with blocked responses. Two groups with identical CS exposure performed very differently - so response blocking can facilitate extinction – so more than just Pavlovian extinction of Cs going on - Perhaps subjects learn a response that is incompatable with the avoidance behaviour during blocked exposure to CS & this contributes to the observed loss of avoidance responding. Non-Discriminated (Free operant) avoidance - could animals learn to avoid if no warning signal? - SIDMAN ’53 (Non-Discriminated, Free Operant or SIDMAN AVOIDANCE) – shock is scheduled without warning – ie every 10 sec – a behaviour is specified an avoidance response (eg lever press) – each response prevents delivery of shock for a set time (eg 30 sec). Animals will learn to avoid shock under these conditions even if no warning. - 2 intervals – the S-S interval (shock-shock) and the R-S interval (response-shock) – this is the period of safety after a response - this avoidance allows responses to occur at any time - another diff from signaled avoidance – response there only effective if during CS – here a response anytime will reset the R-S interval – if R-S interval 30 sec - shock rescheduled 30 sec after each response – by always responding right before R-S interval over the safety period can be indefinitely prolonged - these studies involve longer training – animal receives more shocks – even after extensive training animals don’t avoid all shocks - procedures sensitive to individual differences - rate of responding is controlled by the value of the S-S and R-S intervals - the more frequently shocks occur without responding (S-S interval) – the more likely the animal is to learn avoidance - increasing the periods of safety (R-S interval) also promotes avoidance behaviour - the relative values of S-S and R-S are also important (ie R-S can’t be shorter than S-S) - this task challenges the two process theory because there is no explicit CS to elicit conditioned fear and it is not clear how avoidance reduce- attempt to use 2-process theory – S-S and R-S are short and fixed,  they are predictable - suggests that animals learn to respond to the passage of time as a signal for shock. “Temporal conditioning” - assume the passage of time after last shock (in S-S interval) or last response (in R-S interval) becomes conditioned to elicit fear. Since the timing starts over after each occurrence of the avoidance response the response removes the fear eliciting temporal cues – so time cues take on the role of the CS - this predicts subjects will do most of their responding at end of R-S interval when fear is high. - Results obtained are consistent with this prediction - BUT many animals successfully avoid a great many shocks without distributing their responses in this manner - also avoidance behaviour has been successfully conditioned with variable interval S-S an R-S interval - so perhaps 2 process theory is not a good or complete explanation of non-discriminated avoidance.

18. Response Dependency • Contingency • although response independent aversive stimulation can result in some general suppression of behavior, significantly more suppression of behavior occurs if the aversive stimulus is produced by the instrumental response • rats VI 1 min food 3 groups • never shocked • brief shock every 2 min unrelated to bar press • brief shock about every 2 min contingent on lever press

21. Availability of Alternative Responses • Often, punished response is the only one to get positive reinforcement • By decreasing responding, subject also decreases reward earned • conflict between suppressing behaviour to avoid punishment and responding to obtain Sr+ • Availability of alternative source of reinforcement greatly increases suppression

22. Azrin and Herman • Adult humans, responding on VI for cigs • Next, obnoxious noise • 1 group only 1 lever available during this phase • another group, 2nd lever available, only 1 punished • When punished response was the only way to obtain cigarettes punishment produced moderate suppression • When alternative lever available, ceased responding on the punished lever

24. When should punishment be used • First off, it is never useful to use it alone • Even if it needs to be first strategy, once initial suppression begins, it is crucial to engender and reinforce other behavior • That said, it is most necessary if… • Frequent negative behavior, nothing to reinforce • Intense, dangerous negative behaviors

25. Factors that limit effectiveness of punishment • Discriminative punishment • Concurrently available reinforcement • Punishment signals Reinforcement • Elicitation of aggression and General suppression of behavior • Related to Functional Class? • Unauthorized behavior • Breakfast in bed

26. Discriminative Punishment • If responding is punished in the presence of a discriminative stimulus but is not punished when the stimulus is absent – here the suppressive effects of punishment will become limited to the presence of the discriminative stimulus.

27. Dinsmoor 1952 • Food at VI 2 min • 5m punishment alternated w/ 5m no punishment • During punishment the lights in the chamber turned off, each lever press resulted in shock • During safe period the lights were on & no shocks delivered, food delivered during all times • rats quickly learned to restrict responding to the safe periods • when the lights were off responding was suppressed.

28. Punishment signals Sr+ • Punishment does not always decrease • Punishment may be Sd for availability of Sr+ • if so, punishment increases responding

29. How does punishment affect the punisher? • Harris et al., 1991 • Found that individuals given opportunity to use a wide variety of aversive procedures performed better and reported greater job satisfaction than individuals who were unable to use stronger aversives.

30. Other strategies for reducing behavior • Removal of reinforcement (time out) • Satiation • Remove or alter Sd for misbehavior