Stream Processing Meetup hosted by Snowflake

This event will be hosted at Berlin Snowflake office, Leipziger Platz 18, 5th floor, All hands area. Important: to RSVP for this event, please register via the location URL (displayed after registering on Meetups)

• 5:30 pm - Registration & Networking
• 6:20 pm - Introduction by Héctor Ríos (Ververica)
• 6:30 pm - Streaming Analytics with Snowflake Dynamic Tables by Leon Papke & Vlad Lifliand (Snowflake)

Streaming data pipelines remain challenging and expensive to build and maintain, despite significant advancements in stronger consistency, event time semantics, and SQL support over the last decade. Persistent obstacles continue to hinder usability, such as the need for manual incrementalization and the lack of enterprise-grade operational features (e.g., granular access control, disaster recovery). While the rise of incremental view maintenance (IVM) as a way to integrate streaming with databases has been a huge step forward, transaction isolation in the presence of IVM remains underspecified. Meanwhile, most streaming systems optimize for latencies of 100 milliseconds to 3 seconds, whereas many practical use cases are well-served by latencies ranging from seconds to tens of minutes.
In this talk, we present delayed view semantics, a conceptual foundation that bridges the semantic gap between streaming and databases, and introduce Dynamic Tables, Snowflake's declarative streaming transformation primitive designed to simplify analytical stream processing.

• 7 pm - Efficient Incremental View Maintenance with Stateful Stream Processing Engines by Fabian Hueske (Confluent)

Materialized views (MVs) speed up analytical queries by precomputing and storing query results. However, when their base tables are modified, MVs can become stale and need to be updated to maintain correctness. While a full recomputation of an MV is simple, it can be inefficient for large datasets. Incremental view maintenance (IVM), on the other hand, offers a more efficient alternative. IVM captures base table changes since the last update, computes the necessary modifications to the MV, and applies them.
In this talk, we explore how incremental view maintenance can be effectively implemented using stateful stream processing engines like Apache Flink, providing a solution for low-latency MV updates.

• 8 pm - Networking
• 9 pm Closing