Meetup passé

Kafka at scale @ Criteo ; Microservices :CQRS, Kafka Stream and BPMN @ QuickSign

Ce Meetup est passé

244 y sont allés

Image du lieu de l'événement

Détails

Bonjour et bonne année 2019 à tous !

Nous avons le plaisir de vous recevoir à Criteo le mardi 15 janvier prochain pour 2 talks retours d'expériences :

- How Criteo is managing one of the largest Kafka Infrastructure in Europe (for 4 years and it still rocks!) - Criteo (Ricardo Paiva)
- Do microservices dream about CQRS ES, Kafka Stream and BPMN ? - QuickSign (Cédric Vidal)

Un grand merci à Criteo Labs qui nous accueille pour ce meetup et sponsorise le buffet !

Agenda :

18h30 : Accueil

19h00 : How Criteo is managing one of the largest Kafka Infrastructure in Europe (for 4 years and it still rocks!) - Ricardo Paiva - Senior devops lead - Criteo

Criteo is a worldwide online advertising platform, building the next generation of digital advertising technologies. We manage billions of ad impressions, each one automatically personalized for its audience. This is truly big data and machine learning without the hyperbole!

We have been using Kafka in production for log aggregation and streaming for 4 years. Our data platform is spread over 8 data centers around the world, collecting and processing 7 million msg/sec and generating more than 150 TB of data each day.

This talk is a deep dive into our critical data pipeline, the architectural choices we made, and tools we developed to maintain scalability, reliability, and performance. Take a look at our journey so far and the challenges we are facing now.

20h00 : Do microservices dream about CQRS ES, Kafka Stream and BPMN ?- Cédric Vidal - QuickSign CTO

Like many other software vendors Quicksign has chosen to migrate its SAAS platform from an on-premise monolith to a microservices platform in the Cloud.

Using a modern tech stack including Kafka, Kubernetes, the Camunda BPMN engine and some in-house development we have built a Kappa CQRS-ES multi-tenant platform able to execute BPMN workflows that can discover and consume any microservice registered in a Kubernetes backed extended registry that allows content type and protocol negotiation.

Our platform leverages Kafka Streams to dynamically generate from a reactive stream of workflow events a bespoke REST-API (HATEOAS) representation so as to ease our customers’ integration effort and allow fast iterations.

We propose to briefly introduce the audience to BPMN and then expose the rationale that led us to build this architecture, its advantages in terms of agility and scalability.

21h - 22h - Networking & Apéro

Keep Calm and Stream your Data !