Uber x Apache Pinot Meetup

We welcome you to join us for an in-person meetup hosted by Uber and the Apache Pinot community. We will kick off with opening remarks by Shanshan Song, Senior Director of Engineering, Storage, Search and Data at Uber followed by a series of three interesting talks by the speakers from Uber, StarTree, and LinkedIn.

Event Details

• This Meetup is a co-hosted by Uber and StarTree
• This Meetup is an in-person event only
• Registration is required for the Meetup. Please RSVP & answer the questions (full name & email address)
• Event location details will be emailed a few days before the event to those who have registered for the event and provided an email address (not located at pin location)

Event Agenda

• 5:00 PM - Networking and Snacks
• 5:30 PM - Opening Remarks
• 5:40 PM - Talk 1 - Uber
• 6:10 PM - Talk 2 - StarTree
• 6:40 PM - Talk 3 - LinkedIn
• 7:10 PM - Closing Remarks

Presentation Info

• Talk 1: Powering Real-Time Observability: Distributed Tracing on Apache Pinot at Uber
  Modern distributed systems generate vast amounts of tracing data, crucial for observability and diagnostics. At Uber, we leverage Jaeger for tracing and have integrated Apache Pinot as its high-performance backend for both indexing and primary storage. This talk will showcase how we utilize Pinot to ingest and query massive volumes of trace data in real time, delivering low-latency analytics at a highly efficient cost. Join us for a demo and deep dive into our architecture, highlighting how Pinot's capabilities supercharge our tracing system.
  - Speaker Bios:
  Xin Gao is a Staff Software Engineer in Uber's Real-Time Analytics (Pinot) team. He has played a key role in integrating Apache Pinot with Uber's large-scale logging and tracing observability systems. Xin specializes in enhancing Pinot's real-time ingestion performance, focusing on latency reduction, storage efficiency, and developing features like multi-topic ingestion to meet versatile user needs.
  Chen Xu is a Staff Software Engineer in Uber's Observability team, focusing on distributed tracing. He plays a crucial role in evolving storage solutions and enhancing query capabilities, significantly improving real-time troubleshooting for the company.
• Talk 2: Pre-Aggregation in Apache Pinot
  Apache Pinot is a high-performance real-time OLAP database designed for ultra-fast analytics at scale. To accelerate data filtering, it employs various indexing techniques, including sorted index, inverted index, and range index. However, even with efficient filtering, query performance can be significantly impacted by the volume of data that needs to be processed. In this talk, we will explore the different types of pre-aggregation strategies in Apache Pinot that reduce the amount of data processed, thereby enhancing query performance. Join us to learn how these techniques can optimize your data analytics workflows.
  - Speaker Bio:
  Xiaotian (Jackie) Jiang is a Founding Engineer & Chief Architect at StarTree, PMC for Apache Pinot. He is profoundly passionate about real-time user-facing analytics, constantly striving to enhance its value for businesses. He relishes the opportunity to solve complex challenges associated with achieving low latency and high throughput in analytics, ensuring seamless performance and insightful user experiences.
• Talk 3: Supporting predictable latencies during data refresh and host restarts
  After pinot-server restarts or refreshes segments, Pinot experiences cold page-cache effects that lead to high tail latencies and query timeouts as data pages are not effectively pre-loaded into memory. We propose a page-cache warmup feature that issues selective "warmup" queries—sourced from an offline flow —during startup and refresh to preload pages with actual queries before serving traffic. Rollout at LinkedIn showed significant P95/P99 latency reductions and fewer timeouts with warmup enabled, improving SLO compliance during restarts and refreshes
  - Speaker Bio: Praveen is a Systems Software Engineer in the Data Infrastructure team at LinkedIn, working on Pinot. He plays a crucial role in enhancing the platform's scalability, efficiency, and privacy features to support LinkedIn's vast user base.

More Info on Pinot

• Apache Pinot Website