Personalization at Scale: Modeling and Infrastructure at Twitter

NYC Machine Learning
NYC Machine Learning
Public group
Location image of event venue


Twitter users create a lot of data: hundreds of million of tweets per day, thousands of tweets per second, and an order of magnitude more measurable user actions like impressions, clicks, video plays, and profile views. Thus, many parts of the company employ ML in various ways, from the Ads team to live video projects.

The Timelines Quality presentation will talk about how over the past year Twitter has moved from a Home Timeline product almost exclusively showing reverse-chronologically ordered tweets of the people you follow, to a quality-based product showing the best tweets for each user first. This has involved building a large-scale personalization model, aggregating user features, and training these features on downstream tasks -- both short-term tasks like maximizing engagement, and longer-term tasks like user satisfaction.

The Twitter Cortex presentation will focus on a related problem -- predicting engagement for a piece of Twitter content given a specific user. Nikolai Yakovenko will show how his team learns embeddings for tweets, for tweet producers and for consumers, while also predicting the expected engagement for a {tweet, producer, consumer} tuple, all in a single deep neural net model. He does so by starting with building blocks like word vectors to represent the text, sequence based models like LSTM to represent sentence structure, and graph-based methods like Node2Vec embeddings to represent users based on random walks through the Twitter follow graph.

It would be hard to use deep learning on practical data problems unless these networks could be trained at scale. A member of the Cortex infrastructure team will briefly show how they scaled a neural network to power Timelines Quality model training, at a speed and quality on par with sparse logistical regression and gradient boosted decision trees.

DeepBird is Twitter's new end-to-end solutions for Deep Learning across the company. Most DL tools focus only on the modeling part of the problem; they have little to say about models as software artifacts that integrate with a bigger infrastructure, and generally make few concessions to non-expert users. DeepBird is a layer on top of Torch that provides a language-agnostic model description language, and a set of high-level primitives that minimize friction across the end-to-end lifecycle of model development and deployment.


Nikolai Yakovenko is an engineer on the Cortex Core team, focused on deep learning modeling of Twitter users and content. He has an MS from Columbia University, where he studied theoretical cryptography. Prior to Twitter, Nikolai worked on search ranking and spelling correction at Google, as well as founding two semi-successful technology startups. Nikolai is a competitive poker player, qualifying for several World Series of Poker final tables. In his spare time, he built a deep learning poker bot that competes in the Annual Computer Poker Competition.

Kovas Boguta is a senior software engineer at Twitter Cortex, working on the DeepBird platform. Previously he was the head of analytics at Weebly, founder of a YCombinator-backed startup doing Twitter visualization, and member of the Special Projects office at Wolfram Research, working on Wolfram Alpha and Mathematica.