48th meetup - Streaming architectures

Companies are getting up to speed with their big data efforts. However, next to volume, they also see added value in speedier data. But how to fit that into the architecture?

1/ Capturing, storing and anonymising CDRs (Matej Pazdic & Pedro Santos, Voxbone)

At Voxbone we are dealing with millions of Call Detail Records (CDR) per day generated by our customers. With the growth of our customer base, we needed a faster way to store and process this data to enable personalised data access to fulfil the company’s specific business needs. We will show how we leveraged Kafka and Kafka Streams to build a custom solution for Voxbone that is able to cope with different use cases such as rating CDRs, comply with the GDPR directive, and data dashboarding powered by the ELK stack.

2/ Pragmatic Streaming, beaming to BigQuery

At Vente-Exclusive.com we've taken a pragmatic approach to streaming. Each micro service streams streams data over Pub/Sub to our data lake. We'll go over the mechanics and see how BigQuery, BigTable and Elastic fits into the picture. You also gain some inside on how our data lake looks like and how we handle streaming data in BigQuery.

Alex Van Boxel, Big Data Architect
Bob De Schutter, Product Manager

-BD.be

19h30 pizza and beers, thanks to Voxbone!
20h00 Storing and processing CDRs (Voxbone)
20h45 Pragmatic Streaming, beaming to BigQuery (Alex Van Boxel)
21h30 networking