Eye gaze interaction: how gaze becomes computer input

The short answer

Eye gaze interaction turns where you look on a screen into a pointer position. A camera, usually helped by infrared light in dedicated eye trackers, watches the eyes and estimates the point of regard. Software then turns that estimate into actions such as selecting, scrolling or typing. Gaze is good at pointing but poor at clicking, because people look at things without meaning to select them. Every practical gaze system therefore adds a separate way to confirm a selection, such as holding the gaze, blinking, pressing a switch or saying a command. The main limit is precision. A standard webcam can estimate gaze, but it is generally less accurate and less stable than purpose-built eye-tracking hardware. That affects how small on-screen targets can be and how reliable the experience feels.

A note on evidence: no specific product data, accuracy figures or test results were supplied for this guide. What follows draws on widely documented principles of how gaze tracking works. It does not compare particular devices. Where a decision depends on a specific tracker, webcam or app, check the manufacturer's current documentation.

How a camera estimates where you are looking

Most dedicated eye trackers shine near-infrared light toward the eyes. The light is invisible to the user and produces a small reflection, called a glint, on the surface of the cornea. A sensor records both the pupil and the glint. As the eye rotates, the position of the pupil centre changes relative to the glint, and software maps that relationship to a point on the screen. This approach is commonly called pupil centre corneal reflection.

Webcam-based gaze estimation works differently. An ordinary webcam captures visible light, so it usually has no controlled glint to measure against. Instead, software locates the face, eyes and irises in the image and infers gaze direction from their appearance and position. Machine-learning models are often used for this step. Because it depends on room lighting, image resolution and head position, this method tends to be coarser and more sensitive to conditions than infrared tracking.

Both approaches usually need calibration. The user looks at a series of points on the screen so the software can learn how their eyes relate to screen positions. Glasses, contact lenses, eyelid shape, strong backlight and head movement can all affect how well calibration holds.

Turning a glance into a click

The core design problem is often called the Midas touch problem. If every glance triggered an action, the user could not look at anything safely. Gaze systems separate looking from selecting in a few common ways:

• Dwell: holding the gaze on a target for a set time triggers it. The time is usually adjustable, trading speed against accidental selections.
• Blink or wink: a deliberate eye closure acts as the click. This can be tiring, and it can be confused with natural blinking.
• Switch: a physical button, head switch or other assistive switch confirms what the gaze is pointing at.
• Voice: a spoken word confirms the selection or names the action to take.
• Gaze gestures: deliberate eye movements, such as glancing off-screen in a particular direction, map to commands.

Interfaces built for gaze usually use large targets, generous spacing and visual feedback that shows where the system thinks you are looking. Some also use zoom or two-step selection to make up for limited precision.

Dedicated eye tracker versus webcam

Aspect | Dedicated eye tracker | Standard webcam

Light source | Built-in near-infrared illumination | Ambient visible light

Typical method | Pupil and corneal-reflection geometry | Appearance-based estimation from the face and eye image

Sensitivity to room lighting | Generally lower | Generally higher

Precision | Generally suited to smaller targets | Generally coarser; works best with large targets

Typical mounting | Below the display, angled up at the eyes | Usually on top of the display

Hardware cost | Additional device | Often already present

The table describes general tendencies, not measured comparisons. Actual accuracy varies by device, software, calibration and environment, and no figures were available to quantify it here.

Who uses gaze interaction

The most established use is accessibility. People who cannot reliably use a keyboard, mouse or touchscreen may use gaze to control a computer. This includes some people with motor neurone disease (ALS), spinal cord injury or cerebral palsy. Gaze also drives augmentative and alternative communication (AAC) software, where the user selects letters, words or symbols and the device speaks them aloud. Some operating systems now include built-in eye-control features, either for compatible external trackers or using a device's front camera. Support varies by platform and version. Outside accessibility, gaze tracking appears in research, usability testing, gaming and some headset-based interfaces.

What this means for speech and webcam audio

Gaze and voice often appear together, but they do different jobs. Gaze points; voice can confirm or command. A spoken confirmation is one way around the Midas touch problem: the user looks at a target and says a short word to activate it. This only works if the microphone picks up those short commands reliably, so speech capture becomes part of the input system rather than an extra.

• Microphone placement: a dedicated eye tracker usually sits below the screen, while a webcam and its built-in microphone usually sit on top. Make sure neither device blocks the other's view or sound path, and that the user's head position for good tracking still leaves them within comfortable range of the microphone.
• Short-command reliability: confirmation words are brief and may be spoken quietly. Choose the input device in the speech software deliberately, rather than relying on the system default. Test with the user's natural speaking volume and the room's usual background noise.
• Camera sharing: if the webcam is also used for gaze estimation, some systems allow only one application to use the camera at a time. A video call may then conflict with the gaze software. Whether the webcam's microphone stays available is a separate audio-device setting. Behaviour depends on the operating system and apps, so test the full combination before relying on it.
• AAC output is not capture: when gaze-driven AAC software speaks for the user, that voice is synthesized output. In a video call, it usually needs to reach other participants as audio from the device, not through a microphone held up to a speaker. Check how the AAC software and the conferencing app handle that routing.
• Gaze does not improve audio: tracking eye position has no effect on microphone pickup or voice quality. Evaluate the microphone on its own merits.

Gaps and next steps

No device specifications, accuracy measurements, compatibility lists or prices were available for this guide. It cannot recommend a specific tracker or webcam or say how accurate a given setup will be. To move forward, confirm three things with the vendors involved:

• Which gaze software supports your camera or tracker.
• Whether that software accepts voice or switch confirmation.
• How it coexists with your conferencing and AAC apps on your operating system.