Streaming + Lakehouse fusion

We’re bringing the best of Bengaluru Streams and Lakehouse Days together for an exclusive half-day session on data streaming, lakehouse architecture, and the future of real-time analytics.

A curated, upscale gathering hosted by Platformatory and e6Data, sponsored by Confluent. We’re keeping the guest list limited to ensure the conversations stay sharp and meaningful.

Talk Sessions:
1. Reimagining Ingestion for Iceberg – Shubham Baldava, CTO @ Datazip
Shubham will share his learnings from building OLake, an open-source high-performance ingestion tool for Apache Iceberg. The session covers challenges with scaling Iceberg ingestion and practical approaches like parallel historical loading, lightweight Golang-based ELT, schema evolution, partitioning, dead-letter queues, and file optimization. Attendees will gain insights into achieving speed, scale, and cost-efficiency with Iceberg ingestion.

2. Building a Stream Processing Engine from First Principles – Mrinal Paliwal & Arush Bansal, E6data
This talk walks through the journey of designing and building a stream processing engine from scratch. The speakers will explain the fundamentals of handling unbounded data, ensuring high availability, and creating a flexible streaming pipeline framework. Key concepts like sources, operators, sinks, channels, and advanced techniques such as operator chaining, checkpointing, and watermarking will be discussed.

3. Iceberg Origins, Internals & Production Challenges – Shabeeb, Senior Software Engineer @ Confluent
Shabeeb will dive into Apache Iceberg’s origins, core internals, and how it addresses limitations of traditional Hive tables. The session covers metadata structures, schema evolution, time travel, and supporting both batch and streaming workloads. Real-world production challenges such as metadata growth, small file handling, compactions, and concurrent writes will be explored. The talk also highlights the Kafka → Iceberg pipeline and introduces TableFlow for simplifying Iceberg operations.

4. Solving for Kafka in Private Cloud – Lightning Round, led by Avinash Upadhyaya @ Platformatory

Running unmanaged Kafka at scale isn’t for the faint of heart — it demands serious chops in certificates, Java/JDK, complex networking (especially on Kubernetes), and security (mTLS, OAUTHBEARER, Kerberos, LDAP, RBAC). Add to that Kafka Connect, Schema Registry, Flink, DR strategy, and right-sizing — and you’ve got a real engineering puzzle. In this talk, Avinash will unpack the internals of how and why we built a platform to make Kafka on private cloud not just possible, but practical.

👉 RSVP early—spots are limited.