Research from UT professors and TRIPODS members Alex Dimakis and Eric Price shows that it is possible to learn a deep generative model that dreams images of human faces (right panel), trained by observing only occluded images (left panel). The middle panel shows a previous approach for solving this problem, that fails. [Figure from: AmbientGAN: Generative models from lossy measurements, by A. Bora, E. Price and A.G. Dimakis, ICLR 2018.]
Advances in machine learning are announced every day, but efforts to fundamentally rethink the core algorithms of AI are rare.
The University of Texas at Austin announced this week that it has received a three-year, $1.5 million National Science Foundation TRIPODS (Transdisciplinary Research in Principles of Data Science) award to establish a new institute on the foundations of data science. The institute will coordinate foundational research in AI and data science across several university departments, launch a large-scale workshop and signature seminar series, and provide seed funding for a number of graduate and post-doctoral fellowships in artificial intelligence and machine learning.
The grant is co-led by Adam Klivans (Computer Science), Sujay Sanghavi (Electrical and Computer Engineering), Purnamrita Sarkar (Statistics and Data Science), and Rachel Ward (Math/Oden Institute), and will involve additional faculty from these departments.
“We are extremely excited to have successfully brought together experts from across four departments for this venture,” Sanghavi said. “Over the years UT has managed to amass significant talent in this important area, but it remains somewhat fragmented across organizational units. This TRIPODS institute will help centralize UT’s expertise in this area via a series of joint initiatives.”
The award is the first phase of a multi-step nationwide process to build larger institutes for machine learning and AI, as part of NSF’s Harnessing the Data Revolution initiative.
The impact of artificial intelligence and machine learning has exploded in business, science, and society, but many important fundamental questions remain. The Texas institute aims to address these along three themes: developing an algorithmic theory of deep learning; making machine learning robust; and improving graph-based applications. By looking at foundational approaches to analysis and design, the institute will form a central hub for theoretical machine learning and data science research.
Current machine learning methods produce models that are enormous and very hard to analyze. Developing a new science to better understand, and hence better train, such models is the first thrust area of this award.
“Training a machine learning model is computationally expensive and deciding how to set parameters is more of an art than a science,” Klivans explained. “There are no good mathematical underpinnings for how to set the dials and levers. For all we know, we are leaving a lot of performance on the table.”
The second thrust is robustness, both to errors in the data and to deliberate adversarial manipulations. “The current model of relying on vast amounts of spotless training data is untenable in most applications. As people try to use machine learning models for high-stakes applications, this problem will be exacerbated by attempts to attack its reliability,” said Sanghavi.
The UT team will design systems that are less-easily deceived. “Developing robust machine learning models will be key to their wide deployment,” said Klivans.
The final research thrust will explore graphs — a mathematical way of capturing interactions between individual units or nodes — and apply new approaches to questions in neuroscience, including searching for signs of Alzheimer’s, ADHD, or autism based on changes in brain connectivity.
“A main focus will be on developing provable, scalable, and robust methods for large biological networks,” said Purnamrita Sarkar. “A very important part of developing a large-scale AI or ML systems is to quantify and estimate uncertainty, because without that, one can in fact make disastrous decisions in this era of big data.”
UT has one of the most esteemed and widely-published faculty in the area of AI and machine learning. The award will give UT researchers time and space to work together on some of the most pressing fundamental problems in the field, and in the process make large gains in AI performance. The TRIPODS faculty will leverage the supercomputers at the Texas Advanced Computing Center (TACC) for building algorithms at scale.
Beyond the work of the individual researchers, the new institute will engage undergraduates, especially women and underrepresented minorities, in research; support the undergraduate Machine Learning and Data Science club; contribute to a new portfolio program in machine learning; and generally, create a more fertile environment for AI research on campus.
“The PIs have a strong track record for engaging women and minorities in research. For example, the very successful Rising Stars Workshop for women graduates in computational and data sciences was successfully organized by one of the PIs at the Oden institute last year,” said Sarkar. “The new institute will contribute funding and participants to the upcoming 2020 Rising Stars conference as well as future years.”
Successful TRIPODS projects will be eligible to apply for Phase II of the project which will provide $10 million awards to a handful of institutes.
“Machine learning is changing the world and has become a core tool across all the sciences. I think future progress in science will be dependent on how well we as a community do machine learning," Klivans said. "This grant positions UT to take the lead on these challenges.”