Speaker/Affiliation: Emo Todorov Ph.D. Assistant
Professor Department of Cognitive Science University of California San D

iego

When/Location: 4/16/07 12:00 SEA 4.244

Host: Dana H. B

allard

Title of Talk: ''Optimality of Activity Sensing''

Rece

ption with Refreshments at 11:30 AM

Abstract: Active sensing via exp

loratory actions is an important yet little-understood aspect of motor beha

vior. It includes eye movements exploratory finger movements whisker and

ear movements vocalizations used for echo-location command signals sent t

o muscle spindles via gamma motoneurons. Such actions do not normally have

direct consequences in terms of rewards or punishments. Instead they enhanc

e the flow and quality of sensory information and thereby contribute indir

ectly by improving the feedback control of %93regular%94 actions. This co

ntribution can be substantial - as for example in driving where the wrong

pattern of eye movements can have grave consequences.

Our goal i

s to develop a computational theory of active sensing in the broader contex

t of sensorimotor integration. We consider active sensing to be a special c

ase of stochastic optimal control combined with Bayesian inference. This ge

neral framework is becoming the theoretical framework of choice for studyin

g perception and action. A growing body of evidence supports the view that

sensory systems integrate all sources of information in a statistically-opt

imal manner and send the resulting estimates to motor systems which apply
feedback control laws optimized to yield the best possible performance.

Despite the mathematical coherence of this framework the sensory and

motor aspects of sensorimotor integration are often studied separately res

ulting in a gap which prevents the theory from reaching its full potential.
This is partly because the information processing which the theory calls f

or is often beyond the reach of existing algorithms forcing researchers to
over-simplify their problems. In particular existing models of motor cont

rol assume that motor commands are generated on the basis of point estimate

s and ignore the uncertainty associated with those estimates. In special c

ircumstances - involving linear dynamics Gaussian noise and quadratic perf

ormance criteria - the optimal way to act is indeed independent of estimati

on uncertainty. But in general uncertainty is likely to matter. We need for

mal models which make such effects explicit. Active sensing is an ideal can

didate in that regard because reducing uncertainty is the only purpose of

exploratory actions.

We have developed two formal models in which op

timal control laws for active sensing can be efficiently approximated and t

heir predictions compared to experimental data. The first model is a model

of eye-hand coordination in the context of manual tasks involving multiple

objects. Key to the model is the fall-off of visual acuity with distance aw

ay from the fixation point. We represent this with state-dependent sensory

noise. Different patterns of eye movements cause different patterns of unce

rtainty in the estimates of object positions and thereby affect the accura

cy of the concurrent hand movements. The model yields concrete predictions

which are in close agreement with data from eye-hand coordination experimen

ts. In these experiments we manipulated the visual feedback available to th

e subject in ways suggested by the model and observed changes in eye-hand
coordination as predicted by the model.

The second model addresse

s the tradeoff between exploration and exploitation. It applies to experime

nts where hand movements are mapped to screen cursor movements via a well-d

efined but unknown to the subject mapping. Even though the subject has a si

mple task - to track a moving target on the screen - this task imposes conf

licting demands on the hand movement system: it requires both tracking (exp

loitation) as well as probing and learning the unknown mapping (exploration

). We represent the online learning process in innovations form which allo

ws us to transform the partially-observed system into a fully-observed syst

em whose augmented state incorporates the uncertainty about the hand-cursor
mapping. The solution to the resulting stochastic optimal control problem

is a control law which incorporates both exploration and exploitation and

achieves an optimal tradeoff between the two. Although this tradeoff has re

ceived a lot of attenion in Reinforcement Learning it has previously been

resolved heuristically.