Data Processing and SecOps at Scale

Hi Everyone! Were back with our first meetup of the year and have the CTO from Humio joining us to discuss data processing and SecOps. As always the venue and beer/pizza is kindly provided by OpenCredo!

Plan!

6.30pm - Arrive - Beers, Pizza and Socialising

7pm - Talk - Kresten Thorup - CTO @ Humio - Data Processing and SecOps at Scale (A Humio Case Study)

Humio is a log aggregation and data processing system designed for sec-ops and dev-ops users. Fundamentally designed as a streaming timeseries-text data engine, it is perfectly suited for high-volume log processing.

In this talk, we will walk through how one of our customers use Humio as a central component in their security and incident response infrastructure, doing live-processing of live logs from some 30.000 desktop PCs to identify malware and bad actors in a changing environment.

This case study will explore the impact of processing data where the vast majority of the stored data is never actually directly retrieved, instead operating with most of the data being processed in-flight on arrival. This affords a data processing architecture that combines stream processing of live data and aggressive compression and time-only indexing of stored data. We explore how this trade off provides the ability to vastly outperform indexing-heavy solutions in both cpu and disk capacity load.

8pm More beer pizza etc and to the pub!

Hope to see you all there.