Leveraging Apache Kafka for Web Crawling and Data Processing

Apache Kafka has become the de-facto standard for distributed message passing, processing and storage of data. At Fluquid we are using Kafka for decoupling and connecting multiple applications asynchronously and for storing data in the context of web crawling and big data processing.

In this talk, Johannes Ahlmann is going to explain using Kafka for web crawling and big data processing.