Computer interaction using gaze

The short answer

Gaze interaction means controlling a computer with your eyes. An eye tracker estimates where you are looking on the screen, and software turns that point into a cursor position or a selection. It is mainly used as an accessibility method for people who cannot reliably use a mouse, keyboard or touchscreen, and it can also work alongside other inputs. The main limit is that looking at something is not the same as wanting to act on it. Every gaze system needs a separate way to confirm a choice, and that is where voice, switches or timed dwelling come in.

No specific product specifications, accuracy figures, prices or test results were available for this guide. It covers general, well-established principles only. Check the documentation for any tracker, webcam or software before you rely on it.

How gaze interaction works

A gaze system pairs a camera with software that finds your eyes in the image and estimates where your gaze lands on the screen. Before use, a calibration step usually asks you to look at a series of on-screen targets so the software can map eye positions to screen positions for your face and seating position. After that, the estimated gaze point can move a pointer, highlight on-screen keys, or trigger interface elements designed for eye control.

Dedicated eye trackers typically use their own infrared illumination and sensors built for this task. General webcam-based eye tracking relies on an ordinary visible-light camera. It is generally less precise and more sensitive to lighting and head movement, so it suits larger targets and coarser control better than fine pointing.

How it differs from other ways of controlling a computer

Method | What it reads | How a selection is made | Practical note

Gaze interaction | Where the eyes are looking | Dwell time, blink, switch press or spoken command | Fast for pointing, but needs a deliberate confirmation step

Head tracking | Movement of the head or a marker on it | Dwell, switch or other confirmation | Uses neck movement rather than eye movement

Mouse or touchpad | Hand movement | Click | Precise, but requires reliable fine motor control

Voice control | Spoken words | The spoken command itself | Depends heavily on microphone quality and room noise

Webcam eye-contact correction | The image of your eyes during a call | None; it does not control the computer | Alters how you appear to others and is not an input method

The last row is a common source of confusion. Some conferencing and camera software adjusts the video so you appear to look into the lens. That feature changes the picture other people see. It does not let you operate the computer with your eyes.

Why confirmation matters: the Midas touch problem

Eyes move constantly and often land on things without any intent to act. If every glance triggered a click, the interface would be unusable. Designers call this the Midas touch problem. Common solutions include:

• Dwell selection: holding your gaze on a target for a set time activates it.
• Blink or deliberate eye gesture: a specific eye action confirms the choice.
• External switch: a button, sip-and-puff device or other switch confirms what the eyes point at.
• Voice confirmation: a spoken word such as a click command acts on the gazed-at target.

What this means for capturing speech

When gaze handles pointing and voice handles commands, the microphone becomes part of the control system rather than just a call accessory. A missed or misheard word means a missed action. That raises the bar for audio input compared with ordinary video calls.

• Webcam microphones sit at monitor distance, so they pick up more room sound and echo than a microphone close to the mouth. Fans, other people talking and keyboard noise can interfere with recognition.
• The ideal camera position for eye tracking, usually centered and with a clear view of the face, may not be the best place to capture speech. If recognition is unreliable, a separate headset or desk microphone lets each device be placed for its own job.
• Check which audio input your operating system and voice software are actually using. A computer with both a webcam microphone and a built-in laptop microphone may default to the wrong one.
• If the person using gaze control also uses a speech-generating device, the audio path matters for conferencing too. Confirm that synthesized speech reaches the call clearly and is not captured second-hand through a distant webcam microphone.

There is no general rule for how much a webcam microphone degrades voice-command accuracy. It depends on the specific hardware, the room and the recognition software. Test in the actual setting with the actual commands before relying on it.

Factors that commonly affect gaze accuracy

• Lighting: strong backlight, glare or very dim rooms can make the eyes harder to detect, especially with ordinary webcams.
• Glasses and reflections: lenses can reflect light and interfere with tracking. Recalibrating while wearing them usually helps.
• Head movement and posture: moving outside the position used during calibration can shift the estimated gaze point.
• Camera placement: a camera that is off-center or too far from the screen makes the mapping harder.
• Target size: larger on-screen targets are easier to select reliably, particularly with webcam-based tracking.

Choosing a next step

If gaze is the primary way someone will use a computer, a dedicated eye tracker supported by their operating system or communication software is generally the more dependable route. An occupational therapist or assistive technology specialist can help match the hardware and selection method to the person's needs. If you only want to try gaze control, webcam-based software is a low-cost way to learn the concept, with lower precision expected. In either case, decide early how selections will be confirmed. If voice is part of that plan, choose and test the microphone as carefully as the camera.

Evidence gaps

This guide does not include measured accuracy, latency, compatibility lists, microphone performance data or prices for any product, because no verified evidence of that kind was supplied. Treat it as an orientation to how gaze interaction works and how it relates to speech capture, not as a product comparison.