Swiss Streaming Meetup #5 Kafka

On 01. April 2021, we are organizing the 5th Meetup of our Messaging & Streaming Community.

Program:

• 17:30 - Opening speech and a short introduction of the participants by the organizers
• 17:35 - 18:00 - 1. Talk - Ant Kutschera from Mobiliar (DE)
• 18:00 - 18:25 - 2. Talk - Nicolas Frankel from Hazelcast (ENG)
• 18:25 - 18:40 Q&As

See below more details about the speakers and their talks

*1. Talk * "Kafka Streams - Using KTables to track the State of a Billing Process by Ant Kutschera from Mobiliar

Im Kontext eines Fakturierungssystems in einer Microservice Landschaft, wo der globaler Zustand überwacht werden muss, um sicherzustellen dass alle zu fakturierenden Verträge tatsächlich fakturiert werden, stellt sich die Frage ob der Zustand im zentralen Oracle Datenbank gehalten werden muss, oder ob es mit Kafka eine andere Möglichkeit gibt, in dem man das out-of-the-box “KTables” Feature verwendet und somit Kafka als Datenbank braucht.

Bio:
Ant Kutschera, an independent & freelance software architect and developer with over 20 years of experience in IT.

2. Talk A Change-Data-Capture use-case: designing an evergreen cache
by Nicolas Frankel from Hazelcast

When one’s app is challenged with poor performances, it’s easy to set up a cache in front of one’s SQL database. It doesn’t fix the root cause (e.g. bad schema design, bad SQL query, etc.) but it gets the job done. If the app is the only component that writes to the underlying database, it’s a no-brainer to update the cache accordingly, so the cache is always up-to-date with the data in the database.

Things start to go sour when the app is not the only component writing to the DB. Among other sources of writes, there are batches, other apps (shared databases exist unfortunately), etc. One might think about a couple of ways to keep data in sync i.e. polling the DB every now and then, DB triggers, etc. Unfortunately, they all have issues that make them unreliable and/or fragile.

You might have read about Change-Data-Capture before. It’s been described by Martin Kleppmann as turning the database inside out: it means the DB can send change events (SELECT, DELETE and UPDATE) that one can register to. Just opposite to Event Sourcing that aggregates events to produce state, CDC is about getting events out of states. Once CDC is implemented, one can subscribe to its events and update the cache accordingly. However, CDC is quite in its early stage, and implementations are quite specific.

In this talk, I’ll describe an easy-to-setup architecture that leverages CDC to have an evergreen cache.

Bio:
Developer Advocate with 15+ years experience consulting for many different customers, in a wide range of contexts (such as telecoms, banking, insurances, large retail and public sector). Usually working on Java/Java EE and Spring technologies, but with focused interests like Rich Internet Applications, Testing, CI/CD and DevOps. Currently working for Hazelcast. Also double as a trainer and triples as a book author.