Preference pulses and the win-stay, fix-and-sample model of choice

Yosuke Hachiga, Takayuki Sakagami, Alan Silberberg

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Two groups of six rats each were trained to respond to two levers for a food reinforcer. One group was trained on concurrent variable-ratio 20 extinction schedules of reinforcement. The second group was trained on a concurrent variable-interval 27-s extinction schedule. In both groups, lever-schedule assignments changed randomly following reinforcement; a light cued the lever providing the next reinforcer. In the next condition, the light cue was removed and reinforcer assignment strictly alternated between levers. The next two conditions redetermined, in order, the first two conditions. Preference pulses, defined as a tendency for relative response rate to decline to the just-reinforced alternative with time since reinforcement, only appeared during the extinction schedule. Although the pulse's functional form was well described by a reinforcer-induction equation, there was a large residual between actual data and a pulse-as-artifact simulation (McLean, Grace, Pitts, & Hughes, 2014) used to discern reinforcer-dependent contributions to pulsing. However, if that simulation was modified to include a win-stay tendency (a propensity to stay on the just-reinforced alternative), the residual was greatly reduced. Additional modifications of the parameter values of the pulse-as-artifact simulation enabled it to accommodate the present results as well as those it originally accommodated. In its revised form, this simulation was used to create a model that describes response runs to the preferred alternative as terminating probabilistically, and runs to the unpreferred alternative as punctate with occasional perseverative response runs. After reinforcement, choices are modeled as returning briefly to the lever location that had been just reinforced. This win-stay propensity is hypothesized as due to reinforcer induction.

Original languageEnglish
Pages (from-to)274-295
Number of pages22
JournalJournal of the Experimental Analysis of Behavior
Volume104
Issue number3
DOIs
Publication statusPublished - 2015 Nov 1

Fingerprint

Appointments and Schedules
Artifacts
Light
Reinforcement Schedule
Cues
Food
Reinforcement (Psychology)
Psychological Extinction

Keywords

  • bar press
  • choice
  • fix-and-sample
  • preference pulse
  • rats
  • win-stay

ASJC Scopus subject areas

  • Experimental and Cognitive Psychology
  • Behavioral Neuroscience

Cite this

Preference pulses and the win-stay, fix-and-sample model of choice. / Hachiga, Yosuke; Sakagami, Takayuki; Silberberg, Alan.

In: Journal of the Experimental Analysis of Behavior, Vol. 104, No. 3, 01.11.2015, p. 274-295.

Research output: Contribution to journalArticle

Hachiga, Yosuke ; Sakagami, Takayuki ; Silberberg, Alan. / Preference pulses and the win-stay, fix-and-sample model of choice. In: Journal of the Experimental Analysis of Behavior. 2015 ; Vol. 104, No. 3. pp. 274-295.
@article{89df6e669f7e4cc4a1d1e23dee3669ec,
title = "Preference pulses and the win-stay, fix-and-sample model of choice",
abstract = "Two groups of six rats each were trained to respond to two levers for a food reinforcer. One group was trained on concurrent variable-ratio 20 extinction schedules of reinforcement. The second group was trained on a concurrent variable-interval 27-s extinction schedule. In both groups, lever-schedule assignments changed randomly following reinforcement; a light cued the lever providing the next reinforcer. In the next condition, the light cue was removed and reinforcer assignment strictly alternated between levers. The next two conditions redetermined, in order, the first two conditions. Preference pulses, defined as a tendency for relative response rate to decline to the just-reinforced alternative with time since reinforcement, only appeared during the extinction schedule. Although the pulse's functional form was well described by a reinforcer-induction equation, there was a large residual between actual data and a pulse-as-artifact simulation (McLean, Grace, Pitts, & Hughes, 2014) used to discern reinforcer-dependent contributions to pulsing. However, if that simulation was modified to include a win-stay tendency (a propensity to stay on the just-reinforced alternative), the residual was greatly reduced. Additional modifications of the parameter values of the pulse-as-artifact simulation enabled it to accommodate the present results as well as those it originally accommodated. In its revised form, this simulation was used to create a model that describes response runs to the preferred alternative as terminating probabilistically, and runs to the unpreferred alternative as punctate with occasional perseverative response runs. After reinforcement, choices are modeled as returning briefly to the lever location that had been just reinforced. This win-stay propensity is hypothesized as due to reinforcer induction.",
keywords = "bar press, choice, fix-and-sample, preference pulse, rats, win-stay",
author = "Yosuke Hachiga and Takayuki Sakagami and Alan Silberberg",
year = "2015",
month = "11",
day = "1",
doi = "10.1002/jeab.170",
language = "English",
volume = "104",
pages = "274--295",
journal = "Journal of the Experimental Analysis of Behavior",
issn = "0022-5002",
publisher = "Society for the Experimental Analysis of Behavior Inc.",
number = "3",

}

TY - JOUR

T1 - Preference pulses and the win-stay, fix-and-sample model of choice

AU - Hachiga, Yosuke

AU - Sakagami, Takayuki

AU - Silberberg, Alan

PY - 2015/11/1

Y1 - 2015/11/1

N2 - Two groups of six rats each were trained to respond to two levers for a food reinforcer. One group was trained on concurrent variable-ratio 20 extinction schedules of reinforcement. The second group was trained on a concurrent variable-interval 27-s extinction schedule. In both groups, lever-schedule assignments changed randomly following reinforcement; a light cued the lever providing the next reinforcer. In the next condition, the light cue was removed and reinforcer assignment strictly alternated between levers. The next two conditions redetermined, in order, the first two conditions. Preference pulses, defined as a tendency for relative response rate to decline to the just-reinforced alternative with time since reinforcement, only appeared during the extinction schedule. Although the pulse's functional form was well described by a reinforcer-induction equation, there was a large residual between actual data and a pulse-as-artifact simulation (McLean, Grace, Pitts, & Hughes, 2014) used to discern reinforcer-dependent contributions to pulsing. However, if that simulation was modified to include a win-stay tendency (a propensity to stay on the just-reinforced alternative), the residual was greatly reduced. Additional modifications of the parameter values of the pulse-as-artifact simulation enabled it to accommodate the present results as well as those it originally accommodated. In its revised form, this simulation was used to create a model that describes response runs to the preferred alternative as terminating probabilistically, and runs to the unpreferred alternative as punctate with occasional perseverative response runs. After reinforcement, choices are modeled as returning briefly to the lever location that had been just reinforced. This win-stay propensity is hypothesized as due to reinforcer induction.

AB - Two groups of six rats each were trained to respond to two levers for a food reinforcer. One group was trained on concurrent variable-ratio 20 extinction schedules of reinforcement. The second group was trained on a concurrent variable-interval 27-s extinction schedule. In both groups, lever-schedule assignments changed randomly following reinforcement; a light cued the lever providing the next reinforcer. In the next condition, the light cue was removed and reinforcer assignment strictly alternated between levers. The next two conditions redetermined, in order, the first two conditions. Preference pulses, defined as a tendency for relative response rate to decline to the just-reinforced alternative with time since reinforcement, only appeared during the extinction schedule. Although the pulse's functional form was well described by a reinforcer-induction equation, there was a large residual between actual data and a pulse-as-artifact simulation (McLean, Grace, Pitts, & Hughes, 2014) used to discern reinforcer-dependent contributions to pulsing. However, if that simulation was modified to include a win-stay tendency (a propensity to stay on the just-reinforced alternative), the residual was greatly reduced. Additional modifications of the parameter values of the pulse-as-artifact simulation enabled it to accommodate the present results as well as those it originally accommodated. In its revised form, this simulation was used to create a model that describes response runs to the preferred alternative as terminating probabilistically, and runs to the unpreferred alternative as punctate with occasional perseverative response runs. After reinforcement, choices are modeled as returning briefly to the lever location that had been just reinforced. This win-stay propensity is hypothesized as due to reinforcer induction.

KW - bar press

KW - choice

KW - fix-and-sample

KW - preference pulse

KW - rats

KW - win-stay

UR - http://www.scopus.com/inward/record.url?scp=84988836343&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84988836343&partnerID=8YFLogxK

U2 - 10.1002/jeab.170

DO - 10.1002/jeab.170

M3 - Article

VL - 104

SP - 274

EP - 295

JO - Journal of the Experimental Analysis of Behavior

JF - Journal of the Experimental Analysis of Behavior

SN - 0022-5002

IS - 3

ER -