Eye tracking is the next frontier of human-computer interaction

Eyes are the windows to the soul, and perhaps an extra pair of hands too. arosoft/Shutterstock

Eye tracking devices sound a lot more like expensive pieces of scientific research equipment than joysticks – yet if the latest announcements about the latest Assassin’s Creed game are anything to go by, eye tracking will become a commonplace feature of how we interact with computers, and particularly games.

Eye trackers provide computers with a user’s gaze position in real time by tracking the position of their pupils. The trackers can either be worn directly on the user’s face, like glasses, or placed in front of them, such as beneath a computer monitor for example.

Eye trackers are usually composed of cameras and infrared lights to illuminate the eyes. Although it’s invisible to the human eye, the cameras can use infrared light to generate a grayscale image in which the pupil is easily recognisable. From the position of the pupil in the image, the eye tracker’s software can work out where the user’s gaze is directed – whether that’s on a computer screen or looking out into the world.

But what’s the use? Well, our eyes can reveal a lot about a person’s intentions, thoughts and actions, as they are good indicators of what we’re interested in. In our interactions with others we often subconsciously pick up on cues that the eyes give away. So it’s possible to gather this unconscious information and use it in order to get a better understanding of what the user is thinking, their interests and habits, or to enhance the interaction between them and the computer they’re using.

Practical uses outside the lab

There are lots of useful applications. For example, in marketing and usability studies, eye trackers are commonly used to study the impact of an advertising campaign or the design of a website. For people who cannot use their arms or are completely paralysed, eye tracking can be used to operate a computer or speech synthesiser: eye-based applications allow them to move a mouse cursor and spell out sentences using only their eyes.

Other more futuristic-sounding applications have been explored, such as appliances that listen to your commands when you look at them: imagine speaking “on” and “off” commands to your lamp, your hi-fi system or your television, which until you looked at them had been in standby. Other examples include automatic scrolling when you have reached the bottom of a screen of text, or automatic pausing of a movie if you look away.

While there are uses for eye tracking in industry and among researchers, firms are now looking seriously at how to make them useful for the general public. Tobii – the same firm that brought us pizza ordering by mind control – recently launched a consumer-priced remote eye tracker, the Tobii EyeX (US$139) with the aim of encouraging games developers to build eye tracking support into their products. For comparison, research lab-grade eye trackers cost around US$20,000.

Another large eye tracking company, SMI, has announced a partnership with Sony to integrate eye-tracking into games for the PlayStation 4.

Interactivity at the cutting edge

There’s a lot of potential for eye tracking in video games. For example, in the popular first-person view (“3D shooter”) style of games, eye tracking can be used to automatically pan the screen to where the player is looking, replacing a task usually performed by the mouse. The eyes can be used to target weapons, too.

One of the most interesting applications is interaction with game characters. When using eye tracking video game characters can be made to react to the player’s gaze the same way a human would. Imagine entering a shop and letting your eyes rest on a sword you find interesting: the merchant could tell you directly about this item, making the interaction that bit more real. Or a character might get upset if, instead of looking at him while he’s talking, you eyes rest on his wife. The eyes are very powerful means of nonverbal communication. Implementing human-like reactions in virtual characters could mean a whole new level of immersion in video games.

Beyond games, there is another range of applications where eye tracking is becoming a hot topic: smart glasses. Because of its shape, a lot of people think Google Glass also tracks the eyes, but it doesn’t. But it wouldn’t be surprising to see the next generation of smart glasses including eye tracking capabilities. This could provide further ways of interacting with the head-up display projected onto the glasses, adding automatic scrolling and navigation that leaves the wearer’s hands free instead of having to use the manual control.

There’s already an eye tracking upgrade for the Oculus Rift virtual reality headset. If users are willing to wear something on their heads, why not add an eye tracker too and enhance interaction using all that information that’s being given away by the eyes? Using the eyes as a tool opens up the possibility for more natural, subtle interaction.