Gaze control in VR apps

The short answer

Gaze control in VR lets you point at things by looking at them instead of aiming a hand controller. Apps use it in two different ways. Head gaze follows the direction your headset is facing. Eye tracking follows where your eyes are actually looking. Eye tracking only works on headsets with built-in eye-tracking sensors, and only in apps that support it. Looking at something usually just targets it. To select it, you typically hold your gaze for a moment, press a button, make a gesture or say a command.

A note on scope: no specific headset specifications, app lists, measurements or test results were available for this article. What follows explains the general principles of gaze interaction. It does not compare particular products or measure accuracy.

Head gaze versus eye tracking

Both methods are called gaze control, but they behave differently. The difference affects comfort, precision and which hardware you need.

Aspect | Head gaze | Eye tracking

What it follows | The direction the headset faces, usually shown as a dot or reticle in the center of view | Where the eyes are pointed, measured by sensors inside the headset

Hardware needed | Any headset with head tracking | A headset with dedicated eye-tracking sensors

How targeting feels | You turn your head to move the pointer | You glance at an item, with little or no head movement

Calibration | Generally none | Usually a short calibration step so the system can map eye position to the scene

App support | Common, especially in menus and simpler apps | Depends on both the headset and the app adding support

How gaze-based selection works

Looking is a natural way to point, but a poor way to click, because people look at many things they don't mean to activate. Interaction designers call this the "Midas touch" problem. Gaze-based VR apps get around it by pairing gaze with a separate confirmation step. The common patterns are:

• Dwell selection: keep looking at an item until a timer, often shown as a filling ring, completes. No hands are needed, but it can feel slow and may trigger by accident.
• Gaze plus button: look to target, then press a controller or headset button to confirm.
• Gaze plus gesture: look to target, then confirm with a hand-tracked gesture such as a pinch.
• Gaze plus voice: look to target, then speak a command to act on it.

Why gaze control matters for accessibility

Gaze input can make VR usable for people who have difficulty holding controllers or making precise hand movements. Dwell selection and gaze-plus-voice combinations can remove the need for hands entirely. How useful this is in practice depends on whether a given app offers these modes. Users who find extended head turning tiring may prefer eye tracking over head gaze where it is available.

Limitations to keep in mind

• Accidental activation: gaze alone cannot tell glancing apart from choosing, which is why apps add a confirmation step.
• Fatigue: long sessions of head-gaze pointing can strain the neck, and long dwell timers slow interaction down.
• Calibration and fit: eye tracking generally depends on calibration and how the headset sits on your face. Glasses or a shifted headset may affect it.
• Privacy: eye-tracking data can reveal what holds your attention. Check how the headset and each app handle that data before enabling it.

Where speech capture fits in

Voice matters for gaze control when an app pairs gaze targeting with spoken commands, or when you talk to others in social and meeting apps. In a standalone headset, speech is normally picked up by the headset's built-in microphone, not by a webcam. A webcam microphone is generally not part of a VR setup. With a headset tethered to a computer, the computer may offer several audio inputs: the headset mic, a webcam mic or a separate microphone.

If voice commands are not recognized reliably, these steps are a reasonable place to start:

• Confirm which microphone is selected as the input device, both in the operating system or headset settings and inside the VR app, since some apps choose their own input.
• Prefer a microphone close to your mouth, such as the headset mic, over a webcam mic across the room. Distance increases room echo and background noise relative to your voice.
• Record a short test clip, or use the app's voice test if it has one, to check that your speech is clear and at a steady level before relying on spoken commands.

No evidence was available on how well particular headset or webcam microphones perform with gaze-plus-voice controls, so no specific device can be recommended here. Compare devices using your own test recordings in the app you plan to use.