Accurate two-dimensional eye tracker using first and fourth Purkinje images.

Although a number of devices are currently in use for monitoring eye position, none is both accurate and convenient to use. Methods based on the use of contact lenses can provide high accuracy but have obvious inconveniences. Other techniques-e.g., skin-mounted electrodes, or eyeglass-mounted photoelectric pickups-are relatively convenient, but eye position can be measured to an accuracy of no better than about 0.5' to 1°. A novel eye-tracking instrument has been developed that makes use of two Purkinje images. The instrument operates in the infrared, so that it does not interfere with normal vision; it requires no attachments to the eye; it has a sensitivity and accuracy of about 1 min of arc, and operates over a two-dimensional visual field of 10' to 20' in diameter. The basic principle of the instrument is described, and operating records are shown.

Index Headings: Vision; Optical system; Infrared. This paper describes a method of eye tracking that is based on the use of a pair of eye reflections that move similarly under translation but differentially under rotation. By monitoring the spatial separation of these two images, eye rotation can be measured accurately without confusion by translation. Similarly, eye translation can be measured accurately without confusion by eye rotation.

I. DOUBLE-PURKINJE-IMAGE METHOD OF EYE TRACKING
The virtual image formed by the light reflected from the front of the cornea is referred to as the first Purkinje image, as shown in Fig. 1. A second Purkinje image, formed by reflection from the rear surface of the cornea, is almost exactly coincident with the first Purkinje image.
The light that is not reflected from either of these surfaces passes through the cornea, passing in turn through the aqueous humor and then the lens of the eye. The third Purkinje image, a virtual image formed by light reflected from the front surface of the lens, is much larger and more diffuse than the others and is formed in a plane far removed from the plane of the other images. The fourth Purkinje image is formed by light reflected from the rear surface of the lens, at its interface with the vitreous humor that fills the bulk of the eyeball. This rear surface of the lens acts as a concave mirror, forming a real image of the source.
The fourth Purkinje image is almost the same size, and is formed in almost exactly the same plane, as the first Purkinje image. However, because the change of index of refraction at the back of the lens is much less than at the air-cornea interface, the intensity of the fourth Purkinje image is less than 1% of that of the first Purkinje image.
If the eye undergoes translation-e.g., due to a lateral head movement-the first and fourth Purkinje images move together through exactly the same distance. If the eye rotates, however, the two images move through different distances and thus change their separation. The physical separation of these two images, therefore, yields a measure of the angular orientation of the eye that is uncontaminated by translational movements.
The instrument described here provides a pair of continuous electrical signals whose magnitudes are proportional to the horizontal and vertical components of separation between the first and fourth Purkinje images of an input pattern of light. This, in effect, provides continuous monitoring of the angular position of the eye, or of the point in space at which the gaze is directed. Errors caused by translation are substantially eliminated; the accuracy is limited, in principle, by the quantum fluctuations in the verylow-intensity fourth Purkinje image.

II. MOVEMENT OF THE PURIINJE IMAGES
Let us consider the positions of the first and fourth Purkinje images in the eye. The first Purkinje image is formed by light reflected from the front surface of the cornea, which has about 7.8-mm radius of curvature. For a distant source, the (virtual) corneal image would be in the plane indicated by the solid dot in Fig. 2(a), i.e., at a distance r/2=3.9 mm from the front surface of the cornea. The fourth Purkinje image is formed by light that is refracted by the cornea and the lens, reflected from the rear surface of the lens, and then refracted again by the lens and cornea. A single mirror that would form the identical real image is shown by the heavy line in Fig. 2(a). The radius of this mirror, for relaxed accommodation, is about 5.8 mm and its center C 4 is close to the corneal surface.' The position of the (real) fourth Purkinje image is shown by the open dot in Fig. 2(a), and we see that the planes of the first and fourth images are almost identical. (For about 9 diopters of accommodation, i.e., a normal eye focused on a target 11.1 cm from the eye, the equivalent mirror changes to about 5.3-mm radius, with its center shifted slightly farther from the corneal surface; the image plane, however, remains almost constant.) For purposes of explanation, it is convenient to use the approximation that the equivalent mirror for the fourth Purkinje image has the same curvature as the cornea and that they are separated by exactly their radius of curvature. This configuration resembles a clam-shell arrangement, as shown in Fig. 2(b), C, being the center of curvature for the cornea (first Purkinje image) and C 4 the center of curvature for the fourth Prukinje image.
From Fig. 2(b), we see that the distance that each image moves as a consequence of eye rotation is directly proportional to the distance from the 'center of rotation to the center of curvature of the surface that forms it. These distances are approximately 6 and 13 mm for the first and fourth images, respectively. Thus, when the eye rotates through an angle A, with respect to the input axis, the two images, as viewed from the input axis, will move in the same direction and will separate by a distance S-7 sinA. (1) With respect to eye space, however, i.e., as viewed from the optic axis of the eye, 2 the images actually move in opposite directions. This is because one image is in front of its center of curvature and the other is behind its center of curvature. If the eye undergoes translation, both images move through the same distance and direction as the eye. If the eye rotates, however, the two images change their separation. The change of separation between these two images yields a measure of the angular rotation of the eye, and the measure is uncontaminated by lateral movements. The basic action of the eye tracker is to monitor continuously the separation of these two images.

m. DESCRIPTION OF THE
EYE-TRACKER SYSTEM The basic optical system is shown in Fig. 3. Light source S is a GE DFW 500-W projection lamp. Its multiple filaments form a 7-mm square area and it contains a built-in reflector. (Although rated at 120 V, it is operated at only 70 V in this application.) The light source is imaged by lens LI onto stop Si, which is a circular aperture 3.4 mm in diameter. Stop SI is in the focal plane of lens L 2 and is imaged in the plane of the pupil of the eye, which is located in the focal plane of lens L 3 . The image of SI at the eye is approximately 8.5 mm (magnified in the ratio of the focal lengths of lenses L 2 and L 3 , which are 60 and 150 mm, respectively). Stop S 2 , which contains a 2.2-cm round hole, is located in the focal plane of lens L 3 and therefore appears to the eye at optical infinity. All of the light that appears to emanate from stop S 2 passes through the image of SI formed at the eye. This input light also passes through a filter F with a pass band between 0.8 and 1.1 ium, and is chopped at 1200 Hz by a chopper wheel, CW, which contains alternating open and closed sectors.  Mirror M is pivoted at its center and is driven in altitude and azimuth by two separate motors, so as to maintain the first Purkinje image centered on the stationary photocell Pi. The control signals M. and M, that drive these two motors are derived from signals from the four sectors of Pi, arranged so that P, functions simultaneously both as a horzontally oriented split-field cell and as a vertically oriented split-field cell, as shown in Fig. 4. The position of photocell P 4 is controlled by two separate motors, P. and P, so as to keep the photocell centered on the fourth Purkinje image. The control signals that drive these two motors are derived from the four quadrants of P 4 in a similar manner to that of photocell Pi, shown also in Fig. 4. In sum, mirror M is servo-controlled to maintain the first Purkinje image stationary on photocell P 1 , which is spatially fixed, and photocell P 4 is servo-controlled in two dimensions to track any movement of the fourth Purkinje image that remains after mirror M has removed the movement component that is shared with the first Purkinje image. The position of P. 1 thus indicates the distance between the first and fourth Purkinje images, which is a measure of the angular position of the eye.
If the eye is translated, mirror M is automatically repositioned so as to maintain the first Purkinje image centered on P 1 . This same movement properly repositions the fourth Purkinje image as well, and no movement of P. is required. In other words, there is no change of output. However, if the eye rotates, the images move differentially and the output signal changes accordingly. (A direct measure of eye translation can be obtained by subtracting the two-dimensional movement of P 4 from the movement of the twodimensional mirror.) The four servo motors used in the current instrument are Pye-Ling type V-47 vibration generators; two of the motors drive mirror M and two drive photocell P 4 . In order to obtain good high-frequency response, each motor control loop incorporates an accurate motion sensor, MS, shown in Fig. 5, which has a position accuracy better than 1 gm. Signals from these motion sensors (Hewlett-Packard DCDT displacement transducers) are fed back to form high-gain internal loops within each major drive loop. All four drive circuits are flat to about 100 Hz (corner frequency).
The subject is positioned by means of a standard chin-rest, forehead-rest combination, or a dentalimpression plate. He views the visual target through the dichroic mirror, DC, which transmits visible light and reflects the infrared. The angle between the eye axis and input axis is labeled A in Fig. 3. The angle between the light-collecting axis and the input axis is labeled 0. In the current version of the instrument, the collecting axis is displaced horizontally from the input axis by about 300, i.e., 0=30' in Fig. 3.
The instrument has a tracking range of the form sketched in Fig. 3. As the eye rotates toward the input axis (i.e., toward A), the angular separation A decreases and the first and fourth Purkinje images move closer together. (At A=0, the two images would be superimposed and therefore inseparable.) Tracking in this direction is eventually limited when light from the very-much-brighter first Purkinje image enters photocell P 4 (through the aperture A 4 ), and also when light from the third Purkinje image falls on P 4 . (The effects of the third Purkinje image are discussed in Appendix A.) As the eye moves away from the input axis (i.e., increasing A), the horizontal separation of the images increases. Tracking in this direction is eventually limited when the fourth Purkinje image is cut off by the pupil of the eye.
Vertical rotation of the eye, i.e., in the direction C-D in Fig. 3, results in corresponding vertical separation of the first and fourth images and is again limited by occultation by the pupil. In other words, the range of the instrument is primarily limited by the eye pupil on the boundaries away from the input axis, and by confusion from the first and third Purkinje images with the fourth Purkinje image in the direction of the input axis. We have not yet had extensive experience with adjustment of the instrument for maximum range, although it is quite easy to obtain an operating field greater than 10° in diameter.
In order not to lose tracking during an eye blink, the gain of each motor driver is substantially reduced during the blink period. As a result, each servo-control tends to return slowly to some central position. If the eye has not wandered to a completely different gaze position during closure then, when the eye is reopened, the servo controls quickly recapture their respective Purkinje images and tracking is resumed. Blinks are detected by monitoring the sum of all quadrants of P 1 , i.e., signal (a'+b'+c'+d'). This sum signal is nominally independent of eye position and therefore remains constant throughout operation except during an eye blink, when it might become larger or smaller than normal, depending on the condition of light reflection from the eyelid. A blink is signaled whenever this sum signal becomes greater or less than the limiting values, Si. and Smax.

IV. PERFORMANCE
The primary goal of this project was to build an instrument that would monitor eye rotation without error caused by translations of the eye. Figure 6 shows the horizontal motion of the eye during steady fixationj and during two horizontal rotations of 30 each. The top record is the horizontal motion of the two-dimensional mirror (driven from the first-Purkinje-image cell) and the bottom record is the horizontal movement of the fourth-Purkinje-image photocell. The first record is identical with that obtained from a typical corneal image tracker (i.e., it contains the usual wandering baseline that results from translation-induced errors), whereas the signal from the fourth cell is translation insensitive, as explained earlier.
While the record in Fig. 6 was being taken, the head was held rigidly by a tight-fitting dental-impression plate with the extra support of a forehead rest. (Translation-induced effects may be attributable to movements of the eye within its socket.) To test for sensitivity to 924 Vol. 63 translation of the eye, the dental plate and headrest were translated horizontally by 0.5 mm at the point marked T in the record and returned at point T'. As can be seen, the upper record deflected strongly, whereas the lower record is almost completely insensitive to even such a large movement. We see then that the fourth-Purkinje output provides a measure of eye rotation that is uncontaminated by translation.
Conversely, by subtracting this fourth-Purkinje output from the first-Purkinje output (which contains rotation plus translation signals), we can obtain a measure of eye translation, undisturbed by rotations. during steady fixation-the subject fixed each spot for approximately 2-3 s. The ragged traces on the horizontal axis, left of center, are due to eye blinks, which the subject was instructed to make at this location as he traced out each successive square. The usable range of the present instrument is better than 100 in diameter. Determining the ultimate range will require a greatdeal more-detailed design study. Design for improved linearity over the field will also require more-detailed design study.

B. Over-All Accuracy
Appendix B contains a description of a set of measurements showing that an earlier version of the double-Purkinje-image tracker tracked the eye with an error of no more than one minute of arc over the range of movements that occur during attempted fixation. We have not yet measured the accuracy of this version of the instrument but, because the system described here contains significant improvements over the one already tested, it is reasonable to suppose that its accuracy will be at least as good.

C. Response to Saccades
Figure 8(a) shows horizontal and vertical eyemovement components during steady fixation. Note that the peak-to-peak noise level is on the order of one minute of arc. Figure 8(b) is an eye-movement record with the subject tracing out the corners of a 0.50 square. translated but move differently when the eye rotates. Therefore, the spatial separation of these images gives an accurate measure of eye rotation undisturbed by eye translations. (An instrument that monitors only the first Purkinje image-i.e., a corneal tracker-is limited in absolute accuracy to about 0.5-1°, at best.) The light used to form these Purkinje images is in the near infrared and is invisible to the subject. The instrument r is very easy to use and align. A standard chin-rest and forehead-rest combination is adequate for alignment of the subject. It takes only a few minutes to align the instrument to a new subject. This eye-tracker system operates by measuring the w the basic difference of position between the first and fourth tved in sub-Purkinje images. However, a third Purkinje image is here is to also present, and disturbing light from this image each fourth could fall onto the first and fourth photocells. It is frresponding important, therefore, to understand how the third rsignals can Purkinje image moves with respect to the other two iected to an images. of the fed- Figure 9(a) shows the configuration of all of these final signal images, if we assume that the eye is rotated so that the eye axis is aligned with the collecting axis. This angle of the eye axis corresponds to position b in the range sketch of Fig. 3. To simplify the drawing, we have assumed that the source is simply a point at infinity, scribed that resulting in the pattern of point images labeled 1, 3, ing accuracy and 4 in Fig. 9(a). The centers of curvature and ,hieved with equivalent mirror surfaces are labeled C 1 , C 3 , and C 4 and instrument 1st, 3rd, and 4th, respectively. Figure 9(b) shows the e first and condition when the eye is rotated so that its axis is in the plane halfway between the input and collecting axes, correof the eye. sponding to intercept position a in Fig. 3. ight that is In Fig. 9(a), the third Purkinje image is behind the a; this is the first Purkinje image and slightly to the side away from tracker described in the body of the paper were first tested in a simpler one-dimensional version, which was / 3used by Dr. Michael Katcher, in his dissertation A C 3 research, to stabilize images on the retina. 4 As part of \c that study, he measured how well the composite 300 system stabilized the image, using a modification of a technique developed by Barlow. 5 Because the eye tracker must be at least as accurate as the complete system, the accuracy of stabilization may be taken as a conservative estimate of the accuracy of the eye ASt tracker itself.
The essence of this technique is to compare the

927
channel maxwellian-view stimulating system. Mirror MR, which occupies the position normally occupied by the eye of the observer, is mounted on a high-quality galvanometer and imaged by means of lenses L 1 and L 2 in the plane of the subject's pupil. Rotation of mirror MR causes the image to shift laterally with respect to the eye axis. Lenses L 1 and L 2 are separated by the sum of their focal lengths so that the target plane of the maxwellian-view stimulating system is imaged in the focal plane of lens L 2 . This optical system, then, images the light source in the plane of the subject's pupil and presents the target to him at optical infinity.
The output signal from the eye tracker is amplified and used to drive the mirror galvanometer. Whenever the subject's eye moves, the voltage at the output of the eye tracker changes by an amount proportional to the magnitude of that eye movement. This voltage change causes the mirror to rotate by the amount necessary to stabilize the image on the retina. Because the eye movements being tracked in the study were small (no more than 30' of arc), the front of the eye never moved by more than about 0.1 mm and the image of the rotating mirror remained well centered in the subject's pupil.
With this composite optical-electronic system, the following procedure was used to measure stabilization accuracy. With the subject carefully fixating a point source, a bright vertical slit (4'X 12' of arc) was flashed just below his fixation point. Seven seconds later a pair of parallel vertical slits (both 4'X 12' and separated by 4') was flashed. If there were no errors of tracking or stabilization, the afterimages of the three slits would form the pattern shown in the upper portion of the inset in Fig. 10. If there were a tracking or stabilization error, the afterimages would be misaligned, as shown in the lower portion of the inset.
To evaluate the disparity within the afterimage pattern, the subject was provided with a device containing a pair of fixed lines and an adjustable line identical in retinal-image size with those presented in the display flashes. After each pair of flashes, the subject moved off the bite bar and observed the afterimage against a blank field that was illuminated by a flickering light. After examining the afterimage in this manner, he adjusted the position of the single line until the relationship between the single and double lines in the device matched the afterimage. [The subject's ability to make this type of adjustment was measured independently by offsetting the single and double lines in the stimulating system by a fixed amount (e.g., 1' of arc) and flashing the single and double lines simultaneously. The subject then adjusted his device to match the afterimage; the adjustment was accurate to within 0.15 '.] Stabilization measurements were made under two different conditions. In one, the subject remained on the bite bar and maintained fixation during the 7-s interval between the two flashes; in the other condition, he got off and then back on the bite bar between the two flashes. The stabilization errors were approximately normally distributed in both conditions. The standard deviations were 1.1' of arc for the first condition and 1.2' for the second.
The results compare favorably with those obtained using other types of stabilization. Barlow 5 reported that the standard deviation of stabilization errors for a "tightly fitting" scleral contact lens, with optics mounted directly on the contact lens, was from about 3' to 5'. The standard deviation for a small cup applied to the cornea by suction and similarly carrying the optics for stabilization was 0.6' to 0.7'. Riggs and Schick 7 determined that the standard deviation for a tightly fitting contact lens with a small plane mirror mounted directly on it was 0.4'. Thus, the Purkinjeimage tracker is probably more accurate than most of the contact-lens eye-tracking and stabilization devices that have been used in the past, and only slightly less accurate than the best ones.