About us
The Real-Time Analytics meetup covers a range of topics around building Real Time Analytics systems; including use cases, technical deep dives, and best practices.
Interested in speaking, organizing, or volunteering? Contact community@startree.ai
This meetup is organized by the founders of StarTree and original creators of Apache Pinot:
Apache Pinot is a realtime distributed OLAP datastore, used to deliver scalable real time analytics with low latency. It can ingest data from batch data sources (S3, HDFS, Azure Data Lake, Google Cloud Storage) as well as streaming sources (such as Kafka). Pinot is used extensively at LinkedIn and Uber to power many analytical applications such as Who Viewed My Profile, Ad Analytics, Talent Analytics, Uber Eats and many more serving 200k+ queries per second while ingesting 1Million+ events per second.
Resources
> • What is Apache Pinot? https://www.startree.ai/what-is-apache-pinot
> • Launching At LinkedIn: The Story of Apache Pinot: https://www.startree.ai/blog/launching-at-linkedin-the-story-of-apache-pinot
> • For more info on Apache Pinot go to dev.startree.ai
> •Our community is active on slack! To join our slack, go to stree.ai/slack
Upcoming events
1
- Network event
Webinar: Full-Text Search on Apache Iceberg w/ Pinot and Lucene
·OnlineOnline3 attendees from 10 groupsTo attend, register here.
While Data Lakehouses like Apache Iceberg provide massive, cost-effective scalability, they are fundamentally designed as scan-heavy engines.
They lack the sub-second, "needle-in-a-haystack" search capabilities provided by inverted indices found in traditional search engines.
This session explores how Apache Pinot fills this gap by integrating Apache Lucene segments directly into its distributed serving layer while maintaining the source of truth in Iceberg's Parquet format.
We will conduct a technical deep-dive into:
- Segment-to-Parquet Virtualization: Pinot’s segment abstraction onto remote Iceberg/Parquet files without data duplication or heavy re-ingestion.
- Hybrid Index Pinning: The mechanics of pinning Lucene Inverted and Text Indexes to local NVMe storage on Pinot servers while leaving the raw data blobs on S3.
- Lucene I/O Orchestration: How the Pinot optimizes query plans to minimize S3 "Time to First Byte" by leveraging metadata-heavy index structures.
Past events
28
