Infrastructure & Monitoring at CERN scale

DevOps Lisbon
DevOps Lisbon
Public group

Needs a location

Details

CERN is the European Organization for Nuclear Research, where physicists and engineers probe the fundamental structure of the universe. To achieve its goals it has always required a large amount of computing capacity, with its infrastructure evolving over time from large mainframes to the datacenters of today.

It hosts the Large Hadron Collider, a 27km particle accelerator where two beams of protons collide millions of times per second generating 100s of PetaBytes of data. The monitoring pipeline collects, transports and processes several terabytes of metrics and logs per day from more than 50k hosts of the CERN data centres and world-wide grid distributed services.

Big Data, Big Compute, Big Infra, Big Metrics. Don't miss this meetup, it's going to be... well, BIG!

As always, you can expect interesting chats with other members of the community, beers and something to bite after the talks!

The event starts at 19:00 and the talk will start at 19:15 sharp, please be on time.

We will have our usual book raffle among those who arrive on time ;)

=========

TALK 1: "Computing and Operations at CERN: From Physical HW to Virtualization and Containers"
Speaker: Ricardo Rocha, Software Engineer at CERN

In this talk, we describe and cover the challenges of running the infrastructure required to store and analyse 100s of PetaBytes of data, and how we manage 1000s of servers totalling more than 300k cores and offering over 400PBs of storage. We will cover the compute and networking infrastructure running on OpenStack as well as the required configuration management services for automation. And we will finish with the current move towards a containerized infrastructure where Docker and Kubernetes play a key role.

Ricardo is a software engineer at CERN currently part of the CERN cloud team, focusing primarily on networking and container based deployments. Previously he helped develop and deploy several components of the Worldwide LHC Computing Grid, a network of ~200 collaborating sites around the world helping to analyze the Large Hadron Collider data. He has a computing degree from FEUP (Faculdade Engenharia da Universidade do Porto), joining CERN as part of his final project focusing on Grid Computing. Ricardo has presented his and his teams work in different international conferences - Computing for High Energy Physics (CHEP), IEEE NSS/MIC, IEEE MSST, DockerCon, Kubecon and multiple OpenStack summits.

=========

TALK 2: "Monitoring CERN Data Centre and WLCG Experiments"
Speaker: Diogo Nicolau, Software Engineer at CERN

This talk discusses the monitoring architecture, the challenges encountered in operating and scaling a pipeline to handle billions of events and presents how users benefit from a central monitoring service for processing and analysis of monitoring data.

Diogo is a Software Engineer with experience in designing and operating complex data pipelines. He earned his Masters in Computer Science at Instituto Superior Técnico with focus on intelligent decision systems. While yet studying he had the chance to explore several startups where he helped building from recommender systems to marketing newsletters. He joined the CERN IT Department to work on the evolution of monitoring infrastructure towards modern and scalable pipelines for data ingestion and stream processing. In his spare time, as a true millennial, he enjoys travelling and finding new places to eat as much as watching Netflix.

=========

IMPORTANT NOTICE: the talks will be recorded and published in the DevOps Lisbon YouTube channel. By signing up for this event you explicitly agree that your image might appear in the video recordings. Nevertheless, you can always get in touch with the organisation ([masked]) to have it removed.