A Machine Capable of Being Conditioned
Joseph R. Cautela; Harutune Mikaelian
The American Journal of Psychology, Vol. 76, No. 1. (Mar., 1963), pp. 128-134.
RH2009-Its interesting to note that in the introduction is says "type of conditioned reflex machine developed by Slukin, who tells of a machine that was conditioned to respond to a whistle when paired to a light source. A description of the machine was not given." However, in the referenced book by Slukin "Mind and Machines" 1960, the chapter bibliography gives Grey Walter's book "The Living Brain" as a reference. So much for looking up cited references.
APPARATUS
A MACHINE CAPABLE OF BEING CONDITIONED
By JOSEPH R. CAUTELA, Boston College, and HARUTUNE MIKAELIAN, Brandeis UniversityUnder the rubric of Cybernetics, Weiner includes such areas of study as the electrical engineering theory of the transmission of messages, the study of messages as a means of controlling machinery and society, and the development of computing machines and other such automata. More specifically, it is the purpose of this paper to discuss the development of a machine that can be controlled and also the implications and heuristic value of such a machine.
The construction of the first fully electronic calculator at the University of Pennsylvania in 1945, and the later development of other calculators paved the way for the solving of problems heretofore too time-consuming or too difficult to solve. Another outgrowth of the development of such machines has been the increased speculation concerning the similarities between man and machine. One viewpoint is that many fruitful hypotheses can be generated about the structure and behavior of man because sometimes the behavior of machines can be shown to be similar to that of man, and that similar behavior may be determined by similar mechanism. The use of machine models in the investigation of man has one very distinct advantage over the use of lower animals; the construction and operation of the machine is known to a much greater degree than that of an animal. Also much more manipulation is possible with the machine.
Most of the speculation has developed out of the use of the digital electronic computer; the main assumption is that the covert behavior (thinking) of the human involved in mathematical computation is similar to the process used by the machine in the same computations. More recently there has been some speculation and investigation of automata whose problem-solving behavior is not essentially of the mathematical nature but concerned with other types of adaptive behavior. One type of such automata is Ashby's "Homeostat" which simulates the human organism in seeking and maintaining homeostatic equilibrium.' Another such machine is the maze-solving machine developed by Shannon. The kind of machine that is the concern of this paper is the type of conditioned reflex machine developed by Sluckin, who tells of a machine that was conditioned to respond to a whistle when paired to a light source.' A description of the machine was not given. Earlier attempts to develop learning machines that exhibit associative learning were of the one-trial learning type, hence no statistical data concerning the course of acquisition and extinction were presented! We believed, therefore, that it would be worthwhile to build a machine that can provide data. Besides many other uses of building such a machine, we think it will make a contribution by providing statistical data, which have, as far as we know, never been reported by any investigator. It must be borne in mind that the conditioned reflex-machine differs from the maze-solving and chess-playing machines developed in this country. These machines are problem-solving, whereas the conditioned reflex machine behaves in a manner corresponding to the Pavlovian conditioning paradigm of straight classical conditioning.6 If a neutral stimulus (which never alone elicits Response A)' is paired with a stimulus that always elicits Response A and, after a number of pairings, the heretofore neutral stimulus elicits Response A, conditioning is said to have taken place.7 In the typical Pavlovian conditioning paradigm, the bell (neutral stimulus-NS) is paid with food (unconditioned stimulus-US). After a number of trials the animal salivates (conditioned response-CR) at the sound of the bell (conditioned stimulus-CS) as well as salivating (unconditioned response-UR) at the presentation of the food. Extinction of the response takes place if the CS is not followed by the US for a number of trials. Spontaneous recovery occurs if the animal is allowed to rest awhile, after extinction, and then gives the CR when the CS is presented again.The Machine
A machine that fulfills the above conditions is described here. It (see Figs 1 and 2) is mounted on three wheels. The US is an activator-an extension with a button at (9)-which is connected to the apparatus and makes the machine move on its wheels (UR) when the button is pressed. The neutral stimulus is a beam of light focused on the front of the apparatus. If the light is paired with the US for a number of times, it (now the CS) will make the machine move (CR) before the presentation of the US.
After a number of presentations of the light without the US, the CR is not given and the machine is extinguished. If the machine is allowed to 'rest,' for a few minutes (3 or 4), and the light is again presented, the machine will move (CR). In other words, spontaneous recovery takes place.
An examination of Table I[not reproduced here] reveals that the number of trials to acquisition (the first time the CR equals the speed of the UR) and the number of trials to extinction (no perceptible movement) varies from session to session as it does in organisms. The machine does not reach acquisition at the different training sessions in the same number of trials because of certain variabilities. Conditioning of the machine is determined by a potentiometer whose shaft rotates at every impulse, thus decreasing resistance to current flow to the motor. This rotation, however, is not segmental, i.e. if the shaft receives a strong push, it rotates more than if it receives a weak push. The more impulses (US) the greater over-all extent of shaft-rotation. The extent of shaft-rotation at each impulse is, however, the critical determinant for the number of impulses required for conditioning. There are several variables which in turn determine this. First, the power of the battery, which is continually decreasing; the solenoid which transmits its translational movement into rotation of the shaft is pulled in by varying strengths. Secondly, the magnetic characteristics of the solenoid, the degree to which it holds its magnetism, until the impulse arrives is a critical factor, the solenoid pulls in strongly when initially the impulses reach it relatively frequently. After the fifth or sixth impulse, however, there is a rise in the temperature of the solenoid which once more changes the characteristics of the magnetic core and thus its pull. Again, the temperature-rise is a function of the impulse-frequency and cooling characteristics of the solenoid. The combination of the above mentioned factors accounts for most of the variability. The machine operates on a series of relays and memory-systems. There are two stages built in the machine. The first stage accomplishes the initial conditioning, and the second accomplishes spontaneous recovery. The machine does not show retention from one training session to another because it is so built that, upon completion of the spontaneous recovery-cycle, it resets itself to its original condition. That is, there is built in the machine a mechanism which, upon completion of spontaneous recovery, erases the 'traces' of conditioning and resets it to its initial preconditioned state.
Pressing the push button closes a simple series circuit with the motor thus making the machine move. Synchronizing this with a flash of light permits the photocontrolled memory-system to transform this impulse into a certain value of resistance. When this resistance is lowered beyond a certain value, enough current passes through to energize the motor every time a flash of light is presented; then the machine has been conditioned. At this stage, however, if the light is presented alone, the process is reversed in the memory-system. Now a certain value of resistance is added instead of being subtracted, so that after a while the resistance increases to the point where no current passes through when a flash of light is presented (extinction). The value of the resistance that is added in each step when light alone is presented is one-fifth that of the resistance which is subtracted when the light and push button and light to condition the machine, then it would take 25 presentations of light (conditioned stimulus) for extinction. When complete extinction has taken place, the delaying relay is energized which closes after about 5 min., and transfers the operation to the second stage for spontaneous recovery. This consists of a stepping relay, which is energized by a flash of light connecting a set of batteries in series to the motor. When the last poles of the stepping relay is reached, the operation is transferred to the first stage, and the cycle can be repeated.
The machine at present exhibits only classical conditioning, reinforcement, extinction, and spontaneous recovery. The addition of external inhibition was deemed necessary, and the unit for accomplishing that has just been finished. As yet, however, it has not been added to the machine, as other additions also are being planned. When added, external inhibition will take place if, during the process of conditioning, a loud voice or intense vibrations are presented. The unit is made up of a sound-actuated relay which, when energized, will temporarily shut off the conditioning system.
Another addition that is being planned is the substitution of a sound-stimulus for the push button as the unconditional stimulus. When this is done, the machine will always respond if a certain frequency of sound is presented. Stimulus-generalization could also be added, but this would require bulky instrumentation, and since the machine operates purely by batteries, it was decided to leave out stimulus-generalization, but instead add another, and far more interesting, characteristic, maze-solving. As yet the design for maze-solving has not been completed. There are several problems to be surmounted; paramount among them is the bulk of instrumentation, as it is intended to keep the machine as small and light as possible and yet still be powered by batteries.Comparison of machine and animal.
As far as the experimenter is concerned there is no essential difference between conditioning an animal and conditioning a machine in the sense that E presents stimuli, observes responses, and attempts to establish a relationship between them. In both types of Ss (machine and animal), the receptor apparatus receives particular stimulation from an external environment and this stimulation is transformed to control entropy (measure of disorganization) through feedback. In both cases, responses are made on the basis of past experiences and the Ss exhibit variability in their responses (2.e. the variability in responses for the different series of acquisition and extinction in our machine).
For our purposes, the value of the machine is related more to the differences between it and organisms than with the similarities.) One important difference involved in conditioning a machine is that much more is known about the nature of the principles operating the machine, and also of its construction than in the case of animals. Another important difference is that, in general, we have a more precise knowledge of the external stimuli influencing the behavior of the machine. The machines are not so sensitive to these and similar variables that influence organisms. Related to this difference is the factor of greater control of the past history of the machine in comparison to that of the animal. Another interesting difference is that there is a greater possibility of altering the machine to investigate different variables and to alter the conditioning process.
Value of the machine. The differences between conditioning an animal and a machine point to some interesting advantages of the machine.
Spence has emphasized that in other sciences, analysis and investigation of simple situations under artificially controlled conditions quite remote from reality have played a decisive part in the successful development of comprehensive systems of knowledge He then makes a plea for the study of simpler types of learning with the use of lower animals. The development of the conditioning machine takes us one step further in control and simplicity and one step closer to the frictionless rolling balls and the perfect vacuum of the physicist.
With the notable exception of Ashby's speculations, the machine models of the human brain have so far been developed largely on the basis of the digital computerlo The use of the conditioning machine as a model would seem more appropriate since organisms are more likely to form associations between sensory impressions and responses than performing mathematical manipulations. We know what is built into the machine to enable it to be conditioned; perhaps by analogy we can infer something about the structure and function of organisms which make these organisms capable of being conditioned. Of course, we must distinguish between a machine that can only imitate the overt human behavior from one that also works internally on the same principles as the brain.
It is also possible that with the adding of more and more conditioning phenomena, such as generalization, discrimination, and external inhibition, the machine will elicit all the known conditioning behavior of organisms, perhaps new laws can be discovered concerning the behavior of organisms by using only machines. This possibility is not as remote as it at first seems, for it is quite likely that if any conditioning procedure is described adequately in operational terms, then it can be built into a machine (this of course would include instrumental conditioning). MacKay, in discussing the possibility of imitating human behavior with the use of an artificial mechanism, puts it another way: "As soon as the required behavior has been specified by an adequate statistical test, a simple logical process can convert the test into a description of at least one possible mechanism that will meet it."ll Also on this point, Ashby states that the essential difference between the brain and any machine yet made is that the brain makes extensive use of principles hitherto made little used in machines.12 He further states that by the use of these principles (and he claims to know what these are) a machine's behavior may be made as adaptive as we please.
Once most of human conditioning behavior is built into the machine, hypotheses concerning the function of the brain and its relationship between external events and observable behavior can be formulated and tested on human Ss. The use of the machine model might lead us to look for facts which might not otherwise have been considered with the use of another model or conceptual scheme. Then, too, there is always the chance that perhaps some mistake in wiring or other defect in the apparatus will lead to some behavior or some hypotheses previously unnoticed or unformulated. Finally, the machine can be used very effectively, in the classroom, for demonstrating conditioning phenomena. The machine has the advantage of being able to elicit more conditioning behavior in a shorter period of time than with the use of an animal. The machine could be so constructed as to exhibit acquisition, extinction, spontaneous recovery, generalization, and external inhibition, all in one class period. The greater control over the machine will enable students to observe the conditioning process more closely than would be possible with the use of animals. For some small psychology departments that do not have animals, the machine would be of especial value. For other departments, the machines could be used in conjunction with animals; the machine could be used at first to demonstrate in a more detailed manner the conditioning process then a similar demonstration could follow with a use of an animal.