Modern Linux Performance Tools for C++

On this meeting, Sasha Goldshtein (CTO of Sela Group) will give a 2-part talk and short lab about modern Linux performance tools for C++.

Agenda:

17:15 - 17:30: Gathering
17:30 - 18:00: Part I
18:00 - 18:15: Hands-on lab
18:15 - 18:25: Break
18:25 - 19:00: Part II and wrap-up

Notes:

1. If you'd like to participate in the lab, please bring a laptop and some way to get online (there's no wifi in the auditorium). Having Linux is not required, VMs on Amazon EC2 will be available.
2. If we find a sponsor for pizza we'll gather earlier. Your help in finding a sponsor is appreciated!

Talk details:

Linux performance monitoring has gone through multiple phases of evolution, from very simple instrumentation-based profilers all the way to an in-kernel virtual machine with modern BPF technology. In this talk, we will explore modern Linux performance tools for continuously monitoring system performance, and then dive deep into CPU utilization, disk requests, network accesses, and blocked threads. We will use Linux perf and use its output to generate flame graphs, explore the perf-tools scripts based on perf and ftrace, and finally explore BPF and the BCC tool collection, which enables powerful low-overhead tracing tools for systems level software as well as high-level languages. At the end of the talk, you'll be equipped with a collection of tools and techniques for diagnosing high utilization, I/O issues, blocked time, C++ memory leaks, and many other problems. And, as a bonus, you will practice some of these scenarios first-hand using a virtual lab: bring only a laptop with a modern web browser and (optional) SSH client.

High-level outline:
• Landscape of modern Linux performance tools
• Performance data sources: tracepoints, PMU events, kprobes/uprobes, USDT
• Recording and visualizing CPU activity with perf
• Hands-on lab: CPU profiling with perf and flame graphs
• ftrace- and perf-based tools for system activity monitoring
• BPF and BCC
• Blocked time analysis with offcputime
• Tracing I/O activity (biolatency, fileslower, tcp* tools)
• Hands-on lab: experimenting with BCC tools