Taming a Supercomputer with Elastic Observability through Kubernetes and GitOps
Dettagli
Do you want to learn more about how Swiss National Supercomputing Centre (CSCS) is monitoring their supercomputer with Elastic and how Elastic stores your Observability data under the hood? This meetup is for you.
This meetup will be hosted by CSCS in Lugano, Switzerland. Save the date, more details coming soon.
Schedule
- 17:30 Doors open, welcome, Networking
- 18:00 Intro
- 18:05 EMOI: CSCS extensible monitoring and observability platform by Massimo Benini @CSCS
- 18:35 A look under the hood on how Elastic stores your Observability data by Nicolas Ruflin @Elastic
- 19:00 Tour of the CSCS machine room
- 19:30 Snacks, Drinks
Talks
EMOI: CSCS extensible monitoring and observability platform
The Swiss National Supercomputing Centre (CSCS) is enhancing its computational capabilities through the expansion of the Alps architecture, a Cray HPE EX system equipped with GH200 modules, in addition to the pre-existing nodes of a diverse combination of CPUs and GPUs. CSCS has developed an Extensible Monitoring and Observability Infrastructure (EMOI), designed to manage the substantial data influx and provide insightful analysis of the infrastructure`s behavior. This presentation presents the architecture and capabilities of EMOI at CSCS, emphasizing its scalability and adaptability to handle the increasing volume of monitoring data generated by the Alps infrastructure. We detail the integration of the Cray System Management (CSM) and Cray System Monitoring Application (SMA) within EMOI. The presentation describes our hardware infrastructure, leveraging Kubernetes for dynamic data collection and analysis tools deployment, and outlines our GitOps strategy for efficient service management. We also explore the distinctions in data models across various node architectures within the Alps system, focusing on power consumption data and its relevance concerning global supercomputing challenges.
A look under the hood on how Elastic stores your Observability data by Nicolas Ruflin @Elastic
Elasticsearch has in the past introduced Time Series Data Streams (TSDS) to have a better way to store your metrics data. While TSDS is continuously improved, there is an ongoing effort around introducing a potential LogsDB. In this talk, I plan to dive deeper into what the changes are made on the Elasticsearch side related to these new features which are "just" a setting and how this applies to your Observability data.
