There is a sign-up schedule for this event that can be
found at http://www.cs.utexas.edu/department/w

ebevent/utcs/events/cgi/list_events.cgi

Type of Talk: UTCS Colloq

uium/AI

Speaker/Affiliation: David Forsyth/UIUC

Date/Time: Frid

ay, March 5, 2010, 11:00 a.m.

Location: ACES 2.302

Host: Kri

sten Grauman

Talk Title: Words and Pictures

Talk Abstract:

<

p>Many pictures appear with text nearby. The relations between
this te

xt and the pictures are complicated and interesting.
There are several
reasons to study these relations. We could
try and predict word annot

ations from pictures. A good
solution to this problem would make it po

ssible to search for
unlabelled pictures using keywords - we just labe

l the
pictures automatically. Images that have multiple labels are
cues to what objects co-occur and what do not; but they are
also cu

es to what image features are stable and what are not.
Linguistic phen

omena can give strong information about what
it means to do object rec

ognition. For example, sentences
written about pictures can reveal sp

atial relations between
objects. As another example, adjectives in ca

ptions can help
focus search for training data. As yet another example

,
captions sketch the major interesting phenomena in pictures,

with reference to a shared discourse between the viewer and
the captio

n writer which reveals information about the
picture. A picture titled
"This man died in a fire" would
show a man, but a pictur

e titled "A man died in this fire"
would show a fire. Info

rmation of this form might tell us
what is worth recognizing and label

ling in pictures.

I will review a large set of methods for reaso

ning about
relations between words and pictures, placing a special
emphasis on methods that link linguistic phenomena to image
observa

tions.