Data Engineer Things Meetup at Ebay
Details
Join the next DET Bay Area Meetup on Wed, May 28th at eBay's Headquarters for learning and networking!
---
Talk #1: Graph Data Engineering for Complex Low-Latency Analytics & AI at Scale
Speaker: Sahana Debnath, Senior Data Engineer at eBay
Abstract:
Graph-based analytics revolutionizes data analysis by leveraging graph theory to model and explore complex relationships within data. Unlike traditional methods that rely on tables or linear lists, graph-based analytics utilizes nodes and edges to represent entities and their interconnections, offering a more intuitive and flexible approach to handling intricate, interconnected datasets.
eBay's pioneering xID graph-based architecture exemplifies the transformative power of transitioning from linear models to graph representations. Operating at an impressive scale—1.7 billion unique customer account nodes, 15 billion vertices, and 24 billion edges—this architecture efficiently handles over 200 million calls daily, enabling sophisticated applications such as identity resolution, account linking, fraud detection, personalization, and an expanding array of AI-driven use cases.
In this presentation, we will delve into the architecture and data engineering processes that underpin eBay's innovative and scalable graph system. Key topics include the design enabling historical relationship tracking, akin to 'time travel'; the config-based graph ETL processes facilitating real-time and batch data loading; and insights gained from building complex enterprise graphs at scale.
Join us to explore how advanced data engineering can propel analytics and AI into new frontiers through graph data engineering architecture.
Talk #2: Inside the next-gen Presto C++ engine
Speaker: Aditi Pandit, Principal Engineer at IBM (Ahana)
Abstract:
Presto (https://prestodb.io/) is a popular open source SQL query engine for high performance analytics in the Open Data Lakehouse. Originally developed at Meta, Presto has been adopted by some of the largest data-driven companies in the world including Uber, ByteDance, Alibaba and Bolt. Today it’s available to run on your own or through managed services like IBM watsonx.data and AWS Athena.
Presto is fast, reliable and efficient at scale. The latest innovation in Presto is a state of the art C++ native query execution engine that replaces the old Java execution engine. The native engine uses Velox (https://velox-lib.io/) which is another open source project from Meta that is a library of composable data processing primitives to use in query engines. Deployments with the new Presto native engine show massive price performance improvements with fleet sizes shrinking to almost 1/3rd of their Java cluster counterparts, leading to enormous cost savings.
The Presto Native engine project began in 2020. Since then, it has matured into production use at Meta, Uber, and in IBM watsonx.data.
This talk gives an in-depth look at this journey, covering:
Production experiences and learnings from the teams
Benchmarking results from TPC-DS workloads
New lakehouse capabilities enabled by the native engine
Beyond the product features, we will highlight how the open source community shaped this innovation and the benefits of building technology like this openly across many companies.
---
Location:
eBay Global Headquarters
2025 Hamilton Ave
San Jose, CA 95125
This meetup will be held at the Main Street visitor welcome center. Please see the map HERE.
---
Interested in giving a talk at a future DET meetup? Submit your talk idea HERE.
By attending this meetup, you agree to abide by our Code of Conduct. Failure to comply with our CoC may result in removal from current and future DET events.
---
Data Engineer Things is a global online data engineering community for data professionals to connect and learn. Join us to grow together!
Data Engineer Things Meetup at Ebay