Eye movements on a gaze-contingent display that locally modifies spectral energy

Michael Dorr, Karl Gegenfurtner, and Erhardt Barth

We develop gaze-guidance systems that help users to deploy optimal scanpath patterns. We build gaze-contingent displays that modify videos in real time such as to change the saliency distribution.

For a possible saliency measure, we used a large data set of eye movement recordings on high-resolution videos of natural scenes to compare image patches that were fixated often with those that were not attended. We found that the average local spectral energy at fixated locations is about 50% higher.

We therefore implemented a Laplacian pyramid capable of a real-time modification of selected frequency bands of an image sequence.

In three randomly chosen quadrants of the subject's field of view, candidate points had their local spectral energy reduced to that of non-salient image patches, i.e. their saliency was decreased. Graphics updates had a latency of less than 30 ms; the set of modified locations was changed only during saccades, so that the onset of the modification typically was rendered invisible by saccadic suppression.

We recorded eye movements of 12 subjects watching 6 high-resolution movies of 20 s duration each on our gaze-contingent display; previously recorded gaze data from 54 subjects on the same (but unmodified) movies served as a baseline.

On average, the modified quadrants did not get fixated less often, contrary to our hypothesis. However, we found that two effects canceled out each other: saliency suppression, e.g. the reduction in local spectral energy, repulsed many saccades (see saccade rates below); but graphics updates occasionally were too slow so that modification onsets "popped out" and attracted saccades.

Due to the reduction in saliency, the overall rate of saccades decreased significantly with respect to the baseline (1.9 vs. 2.4 saccades/second).

In conclusion, gaze guidance seems to be possible in principle, but needs even faster graphics and more complex modifications.


A video recording of this talk can be found here.