Name: Streaming + Lakehouse fusion
Start: 2025-09-27T10:30:00+05:30
End: 2025-09-27T13:30:00+05:30
Location: Accel LaunchPad

We’re bringing the best of **Bengaluru Streams** and **Lakehouse Days** together for an exclusive half-day session on **data streaming, lakehouse architecture, and the future of real-time analytics**.

A **curated, upscale gathering** hosted by **Platformatory** and **e6Data**, sponsored by **Confluent**. We’re keeping the guest list limited to ensure the conversations stay sharp and meaningful.

**Talk Sessions:**
**1\. Reimagining Ingestion for Iceberg** – *Shubham Baldava, CTO @ Datazip*
Shubham will share his learnings from building OLake, an open-source high-performance ingestion tool for Apache Iceberg. The session covers challenges with scaling Iceberg ingestion and practical approaches like parallel historical loading, lightweight Golang-based ELT, schema evolution, partitioning, dead-letter queues, and file optimization. Attendees will gain insights into achieving speed, scale, and cost-efficiency with Iceberg ingestion.

**2\. Building a Stream Processing Engine from First Principles** – *Mrinal Paliwal & Arush Bansal, E6data*
This talk walks through the journey of designing and building a stream processing engine from scratch. The speakers will explain the fundamentals of handling unbounded data, ensuring high availability, and creating a flexible streaming pipeline framework. Key concepts like sources, operators, sinks, channels, and advanced techniques such as operator chaining, checkpointing, and watermarking will be discussed.

**3\. Iceberg Origins\, Internals & Production Challenges** – *Shabeeb, Senior Software Engineer @ Confluent*
Shabeeb will dive into Apache Iceberg’s origins, core internals, and how it addresses limitations of traditional Hive tables. The session covers metadata structures, schema evolution, time travel, and supporting both batch and streaming workloads. Real-world production challenges such as metadata growth, small file handling, compactions, and concurrent writes will be explored. The talk also highlights the Kafka → Iceberg pipeline and introduces TableFlow for simplifying Iceberg operations.

**4\. Solving for Kafka in Private Cloud** – *Lightning Round, led by Avinash Upadhyaya @ Platformatory*

Running unmanaged Kafka at scale isn’t for the faint of heart — it demands serious chops in certificates, Java/JDK, complex networking (especially on Kubernetes), and security (mTLS, OAUTHBEARER, Kerberos, LDAP, RBAC). Add to that Kafka Connect, Schema Registry, Flink, DR strategy, and right-sizing — and you’ve got a real engineering puzzle. In this talk, Avinash will unpack the internals of how and why we built a platform to make Kafka on private cloud not just possible, but practical.

👉 **RSVP early—spots are limited.**

Akansha Deshmukh

Avinash Upadhyaya

Bangalore Streams

Technology

Database Professionals

New Technology

Data Management

Big Data

Data Analytics

Data Visualization

Data Mining

Data Science

Technology Startups

Apache Kafka

Apache Druid

Real-Time

Stream Processing

Distributed Systems

Messaging Systems

Streaming + Lakehouse fusion 

Accel LaunchPad

Share

Bangalore Streams

Streaming + Lakehouse fusion

Bangalore Streams

Details

Members are also interested in