Why do we look where we look? That is the question that UT Computer Science professor Dr. Dana Ballard is trying to answer. Through his research in virtual reality he has begun to solve this inquiry.

Recently, UT Computer Science alumni filled the auditorium to hear Dr. Ballard’s presentation of his research on virtual reality and it’s effect on human vision and motor control.

“We try to get answers on how brains are behaving in the world. To figure out these answers what we like to do is change things,” Dr. Ballard said. “We have found that you have to change things rather quickly on a 100 millisecond time scale. In real reality that’s hard but in virtual reality it’s very easy.”

The human eye makes such rapid gaze changes that it achieves velocities up to 700 degrees per second between making 150,000 fixations per day. Dr. Ballard’s research explores the idea that instead of the visual scene causing a shift in gaze, it is the brain’s cognitive agenda. This result suggests that the location that the eye chooses to focus on is based on the value of furthering this agenda.

Dr. Ballard began his presentation by illustrating how the visual scene causes a shift in gaze through the process of making a peanut butter and jelly sandwich. As the knife moves from the jelly to spread on the bread, the focus of the eye is constantly shifting based on movement without much reliance from the brain. In the video below you can see that the round yellow cursor represents where the eye is currently focusing. When it moves across the screen, that indicates that the eye is shifting its gaze and focusing on another object.

To illustrate shifts in sight based on a cognitive agenda, a  car is shown driving down the street on the GDC Auditorium projector screen. With every turn and motion of the vehicle the perception of view changes as the brain adjusts.  This explores the concept of the brain having a cognitive agenda in that it knows it must drive a vehicle and get to where it’s going.

Dr. Ballard's work has led to an increased understanding of the role of behavior in vision, and led to the extension of his research in regard to human’s natural behavior in virtual reality environments.