UTCS AI Colloquia - Mirella Lapata, Professor, School of Informatics, University of Edinburgh, "Grounded Models of Semantic Representation" ACE 2.402

Contact Name: 
Karl Pichotta
ACE 2.402
Mar 1, 2013 11:00am - 12:00pm

Signup Schedule: http://apps.cs.utexas.edu/talkschedules/cgi/list_events.cgi

Talk Audience: UTCS Faculty, Grads, Undergrads, Other Interested Parties

Host:  Ray Mooney

Talk Abstract: A popular tradition of studying semantic representation has been driven by the assumption that word meaning can be learned from the linguistic environment, as approximated by naturally occurring corpora. However, ample evidence suggests that language is grounded in perception and action. In this talk we focus on grounded models of word meaning and discuss how these can be formulated based on linguistic and visual data.

A major question in developing such models is the provenance of the visual modality. We investigate whether it can be approximated by feature norms (i.e., attributes native speakers consider important in describing the meaning of a word), image labels (i.e., keywords describing the objects depicted in an image) or automatically computed visual attributes. A second question concerns the mechanisms by which the two modalities can be integrated. We present a comparative study of models that create a bimodal meaning either by abstracting the two modalities into a joint semantic space or simply by concatenating them. Experimental results suggest that textual data can indeed be used for approximating visual information, and that joint models are superior.

Speaker Bio: Mirella Lapata is a Professor at the School of Informatics at the University of Edinburgh. She hold an MLT from the Language Technologies Institute at Carnegie Mellon University, and a PhD in Natural Language Processing from the University of Edinburgh. Her research interests include statistical natural language processing, with an emphasis on unsupervised methods, mathematical programming, and generation applications. She serves as an associate editor of the Journal of Artificial Intelligence Research (JAIR) and is an action editor for Transactions of the Association for Computational Linguistics (TACL). She is the first recipient (2009) of the British Computer Society and Information Retrieval Specialist Group (BCS/IRSG) Karen Sparck Jones award. She has also received best paper awards in leading NLP conferences and financial support from the EPSRC (the UK Engineering and Physical Sciences Research Council).