Webinar: Full-Text Search on Apache Iceberg w/ Pinot and Lucene

To attend, register here.

While Data Lakehouses like Apache Iceberg provide massive, cost-effective scalability, they are fundamentally designed as scan-heavy engines.

They lack the sub-second, "needle-in-a-haystack" search capabilities provided by inverted indices found in traditional search engines.

This session explores how Apache Pinot fills this gap by integrating Apache Lucene segments directly into its distributed serving layer while maintaining the source of truth in Iceberg's Parquet format.

We will conduct a technical deep-dive into:

• Segment-to-Parquet Virtualization: Pinot’s segment abstraction onto remote Iceberg/Parquet files without data duplication or heavy re-ingestion.
• Hybrid Index Pinning: The mechanics of pinning Lucene Inverted and Text Indexes to local NVMe storage on Pinot servers while leaving the raw data blobs on S3.
• Lucene I/O Orchestration: How the Pinot optimizes query plans to minimize S3 "Time to First Byte" by leveraging metadata-heavy index structures.