FoCS Corporate Connection - "Scaling @ Twitter: Killing the Fail Whale" with Gary Tsang

Contact Name: 
Jenna Whitney
GDC 6.302
Sep 24, 2013 5:00pm - 6:30pm

Speaker: Gary Tsang

Synopsis: Twitter will be giving a talk that covers the technical challenges of scaling Twitter and how we have overcome those challenges in our current architecture.

Abstract: Three years ago, when the 2010 World Cup put Twitter squarely in the center of a real-time, global conversation, the influx of Tweets—from every shot on goal, penalty kick, and yellow or red card—repeatedly took its toll and made Twitter unavailable for short periods of time. Twitter engineers worked throughout the nights during this time, desperately trying to find, and implement, order-of-magnitudes of efficiency gains. Unfortunately, those gains were quickly swamped by Twitter’s rapid growth, and engineering had started to run out of low-hanging fruit to fix.

After that experience, we determined we needed to step back and re-architect the site to support the continued growth of Twitter. Since then we’ve worked hard to make sure that the service is resilient to the world’s impulses. We’re now able to withstand events like Castle in the Sky viewings, the Super Bowl, and the global New Year’s Eve celebration. To highlight how far we've come, the Castle in the Sky viewings produced a new tweet-per-second record of 143,199 tps. This is in stark contrast to the 2010 World Cup when we were struggling with huge, at the time, tweets-per-second spikes of 3k.

This re-architecture has not only made the service more resilient when traffic spikes to record highs, but also provides a more flexible platform on which to build more features faster. These include synchronizing direct messages across devices, Twitter cards that allow Tweets to become richer and contain more content, and a rich search experience that includes stories and users.

Speaker Bio: Gary Tsang is a UT Austin Alum and Infrastructure Engineer in Twitter's San Francisco office. His work with the Timeline Service Infrastructure team is focused on scaling the Twitter platform. Projects include, tweet migration and products such as the home timeline, @connect, and conversations.

We will be taking resumes.

FREE SWAG and food available!