Real Time Analytics Meetup New York

To attend, register here.

Your lakehouse already holds the data. Iceberg and Delta solved openness, durability, and cost. So why does every workload with a real performance SLA still live somewhere else?
The moment a query is tied to a customer experience, a revenue event, or an on-call engineer at 2am, best-effort latency stops being acceptable. Traditional lakehouse query engines scan too much data to hold sub-second responses, so teams do the only thing available to them: they copy the data out. Into Elasticsearch for log search. Into a time-series store for metrics. Into a key-value store or a serving warehouse for the customer-facing dashboard. Into a vector database for similarity search. Each copy adds pipeline complexity, a second infrastructure bill, and one more reason the lakehouse is not actually the source of truth.

That tradeoff was an artifact of how query engines read Parquet, and it no longer holds. StarTree brings page-level fetching and the full index portfolio of Apache Pinot (inverted, text, JSON, star-tree, and vector) directly to Iceberg and Delta tables. The data stays in open format, in your object storage, under your catalog. The sub-second SLA gets enforced at the query layer.

In this session we walk through six workloads that have historically been evicted from the data lake, and show what changes when they can run in place:

1. Customer-facing analytics
   Embedded dashboards and usage analytics serving thousands of concurrent end users, with no reverse ETL and no parallel serving layer.
2. Real-time and historical in one query path
   Streaming ingest and lakehouse tables queried together, so freshness and context arrive in the same result set instead of being reconciled by the application.
3. Interactive analytics
   Sequences of dozens of ad hoc questions across high-cardinality dimensions, held at sub-second latency without pre-aggregating every path in advance.
4. Log and event analytics
   Lucene-style text search and JSON indexing on data already sitting in Iceberg, which is what has kept log analytics off the lake and inside a duplicated Elasticsearch cluster.
5. Observability
   High-cardinality time-series queries with native PromQL, joined against logs and events, at retention windows that are cost-prohibitive in a purpose-built observability backend.
6. Vector similarity search
   ANN search over embeddings stored alongside the structured columns they belong to, so filters, aggregations, and semantic retrieval resolve in a single query. This is the pattern agent-facing and RAG workloads need, and it is the newest reason teams were about to stand up yet another database.

What you will learn

• The engineering breakthrough behind sub-second queries on Parquet.
• Real-world examples of these use cases in action.
• How to prepare your lakehouse for agentic query patterns.

Who should attend: Data platform leaders, data engineers, and architects running Iceberg or Delta who are maintaining one or more purpose-built databases downstream of it.