Reinforcement? Time-based contingencies control local choice

Abstract

In three experiments, pigeons worked on two-key concurrent variable-interval (VI) VI schedules in which the local food ratio changed systematically according to time since the last food delivery from any key. In the experiment reported in Chapter 3, the relative frequency of food deliveries for responding on the two alternatives reversed at a fixed time after the most recent food delivery from any key. Across conditions, the food-ratio reversal time was varied from 10 s to 30 s, and the overall reinforcer rate was varied from 1.33 to 4 per minute. In all conditions, immediate post-food choice was toward the locally-richer key, regardless of the last-food location – food deliveries acted as time markers, signaling subsequent local food ratios. Unlike the local food ratio, which changed in a stepwise fashion, local choice changed progressively across the inter-food interval, often reversing before the food ratio reversed. When the ratio reversal occurred further into the inter-food interval, choice reversed earlier relative to the arranged reversal time. The overall rate of food delivery had little effect on choice. In the experiment reported in Chapter 4, local food-rate ratios also reversed at a fixed time after each food delivery, but the locally-richer key, and/or the time since the most recent food delivery, was signaled by a colored key-light presentation that lasted either 5 s, 10 s or for the duration of the inter-food interval. The strongest control by the local food ratios was observed when the stimulus was present throughout the inter-food interval. Under these conditions, local choice approximated local food ratios throughout the inter-food interval. Stimulus presentations that signaled the locally-richer key and whether or not the food ratio had reversed had a similar effect on choice to stimulus presentations that only signaled whether the food ratio had reversed. Brief stimulus presentations appeared to act as time markers, shifting choice toward the locally-richer key. Thus, deviations from the local food ratio in the experiment reported in Chapter 3 were the result of a failure to discriminate the time elapsed since the last food delivery, rather than to a failure to discriminate the local food ratio itself. The experiment reported in Chapter 5 investigated the ability of food deliveries to signal progressive changes in the local food ratio across the inter-food interval. In Phase 1, each food delivery was equally likely to be arranged for a left-key or right-key response. Conditions were arranged such that the next food delivery was likely to occur either sooner on the left (or right) key, or sooner on the just-productive (or not-just-productive) key. In Phase 2, similar contingencies were arranged, but the last-food location was signaled by a red key-light. Preference was jointly controlled by the likely time and location of the next food delivery. When local food ratios changed differentially with respect to time since the last food delivery, but not with respect to the location of that food delivery, local choice followed the local food ratio throughout the inter-food interval. When local food ratios were differential with respect to time since the last food delivery and the location of that food delivery, local choice followed the local food ratio only briefly before stabilizing within the inter-food interval at indifference. When the last-food location was signaled, local choice in these conditions followed the local food ratio throughout the inter-food interval, suggesting that the complexity of the signaling stimulus moderates the degree to which local food ratios control local choice. In all three experiments, a small effect of the most recent food delivery on choice was observed. Although the pattern of local choice across the inter-food interval did not change according to the location of the last food delivery, choice after a left-key food delivery was displaced toward the left key, and choice after a right-key food delivery was displaced toward the right key. This effect of last-food location was largest when the likely availability of food on a key was differential with respect to the location of the last food delivery, but was still observed when the likely time and location of food was independent of the last-food location, suggesting a small response-strengthening effect of food deliveries. This effect of last-food location was cumulative across sequences of same-key food deliveries. Discontinuation food deliveries had larger effects on choice than did continuation food deliveries, suggesting that the response-strengthening effect of a food delivery may depend on what that food delivery signals in the context of other recent food deliveries. In all three experiments, local choice was described well by a model that assumed that errors in discriminating the local food ratio arose as a result of a failure to discriminate the response that produced a food delivery, and as a result of a progressive failure to discriminate the time within the inter-food interval as this time increased. The model assumed that log response ratios matched food ratios that were calculated by reallocating a small number of food deliveries obtained on each key to the other key, then by redistributing the resulting food ratios in each time bin across surrounding time bins with time since the most recent time marker (a food delivery or key-light stimulus change) determining the standard deviation of the redistribution. When no stimulus change occurred within the inter-food interval, a single coefficient of variation was used. When stimulus changes did occur during the inter-food interval, two coefficients of variation were used – one for redistributions before the first stimulus change, and a separate coefficient of variation for redistributions after the first stimulus change. Thus, local choice approximated the arranged local food ratio to the extent that the arranged local food ratio obtained at a time was discriminable to the animal. When contingencies of reinforcement are differential with respect to time, as in the present experiments and in experiments designed to investigate timing behavior, discrimination of the arranged contingencies cannot be assumed to be perfect. Deviations in local choice from the arranged local food ratio thus cannot be attributed solely to a failure to discriminate elapsed time. Control by time since an event depends on the animal’s ability to time, and hence on the animal’s ability to discriminate the likely availability of reinforcement for a response.