49th #ebaytechtalk : Vespa — the open-source big data serving engine

Offline and stream processing of big data sets can be done with tools such as Hadoop, Spark and Storm, but what if you need to process big data at the time a user is making a request? This talk introduces Vespa - the open-source big data serving engine. Vespa allows you to search, organize and evaluate machine-learned models from tools such as TensorFlow over large, evolving data sets with latencies in the tens of milliseconds. Vespa is behind the recommendation, ad targeting and search engines at Yahoo, where it handles billions of daily queries over billions of documents, and was recently open-sourced at http://vespa.ai.

Jon Bratseth is a distinguished architect at Oath (formerly Yahoo), and the architect and one of the main contributors to Vespa, the open big data serving engine. Jon has 20 years experience as an architect and programmer on large distributed systems.