Static and dynamic models of concurrent variable-interval schedule performance

Abstract

Elucidation of the determinants of behaviour in situations involving choice remains a central task in psychology. This thesis addressed itself to a subset of the range of problems within this area, namely that of modelling choice behaviour in experiments involving concurrent variableinterval schedules of reinforcement. Both non-dynamic and dynamic models were developed using data obtained from the various experiments conducted. Pigeons served as subjects in all the studies. A non-linear parameter-free algebraic model was developed to predict the rate of transition between alternative choices. The model was shown to apply to a large body of the literature where both transition and reinforcement contingencies were manipulated. It was argued that following each transition, the organism makes a given number of constrained responses (i.e., highly dependent in the Markovian sense) which are then followed by unconstrained responses that are determined by a zeroth-order Markov process. Experiments were performed to assess the combined influence on response rate of both the force required to effect a response and the rate of reinforcement for these responses. It was found that force and reinforcement rate combine multiplicatively to determine response rate. These variables were shown to be orthogonal with respect to their effects on response rate. A model was developed which encompassed these findings. The experimental techniques, analytic methodologies and theoretical framework of engineering system identification were employed to identify the transfer function relating session by session response-rate ratio (output) changes to unpredictable dynamic changes (session by session) in the reinforcement-rate ratio (input). In initial experiments impulse and step inputs were used and it was found that a logarithmic transformation of both the input and the output data linearised the dynamics. However these transient inputs were problematic because of the stochastic nature of the pigeons' behaviour. Therefore stochastic identification techniques involving random inputs (pseudo-random binary sequences) were used and proved most powerful in delineating the dynamics. Following linearisation, the session by session performance was modelled by a second-order critically damped linear transfer function, with a gain of less than unity and a time constant in the order of two hours, which made highly accurate predictions of the output to arbitrary unpredictable manipulation of the input. Prediction of minute by minute performance was shown to require a more complicated non-linear dynamic model. Finally the various results in this thesis were explained by an adaptive control model incorporating investigative and control components. Subunity gain in the overall system transfer function was proposed as a necessary condition for keeping the investigative component functionally operative.