Research Interests

Organisms learn to make inferences, choose actions, and gather information using highly restricted and noisy sensory channels in an uncertain and rapidly changing environment.  My interest is in using modern methods from machine learning and probability theory to solve real-world problems in computer vision and robotics to help bring a theoretical understanding of this process in the brain.  I have placed particular emphasis on real-time performance and robustness in unconstrained situations in order to prevent the use of “trick solutions” that may work for simple problems but don’t work in practice and may not lead us toward understanding real systems.

My research can be broken down into several areas:

  1. •Probabilistic generative models for real-time computer vision

  2. •Discovery of object categories

  3. •Automatic facial expression analysis

  4. •Computational models of cognitive development and developmental robotics

  5. •Human-Robot and Human-Computer Interaction

SBFs for Discovery and Real-Time Detection of Object Categories

My dissertation began with the problem of learning to detect and localize object categories from training images whose labels indicate only the presence or absence (not the location) of objects of interest.  This led to a new generative modeling framework called “Segmental Boltzmann Fields” (SBFs), which has achieved state-of-the-art performance in learning to detect generic object categories (such as “airplanes”, or “bonsai trees”) at video frame-rates.

Examples of results on Caltech-101 images:

Examples of results on GENKI face images:

We then developed interactive infant robot equipped with an SBF as its “visual cortex”.  Using simple auditory contingencies as the only cue to determine when the visual field probably contained or did not contain a caregiver, the infant robot was able to autonomously learn an accurate “person” visual category from only a few minutes worth of experience interacting with caregivers.  Remarkably, though it had only seen images of the real world, the infant robot’s preferences for face-like line-drawings closely mirrored the preferences of 40 minute-old neonates to identical stimuli. (Nick Butko contributed significantly to this project. See the BEV page he put together for more info).

Research 1

Research 1