- What We Got Wrong: Lessons from the Birth of Microservices at Google
Hello NYC Metrics and Monitoring! The folks at Lightstep are hosting a Meetup on microservices nightmares and wanted to extend the invitation to the NYC Metrics and Monitoring group. Hope you can make it! Join A Lively Discussion Google deserves a lot of credit for imagining (and popularizing) what we now call "microservice architectures." That said, hindsight is 20/20, and many of the mistakes we made at Google are being recreated by the rest of the industry today. What did we get wrong about microservices at Google, and how can we apply those lessons today? Speaker Spoons (Daniel Spoonhower) is CTO and Co-founder at LightStep, where he’s building performance management tools for modern software systems. Previously, Spoons spent almost six years at Google where he worked on developer tools as part of both Google’s internal infrastructure and Cloud Platform teams. He has published papers on the performance of parallel programs, garbage collection, and real-time programming. He has a PhD in programming languages from Carnegie Mellon University but still hasn’t found one he loves. This free event is being held at: The Cellar at District Social 252 West 37th Street, New York, NY 10018
- The Modern, Open Source Metrics Stack, told in 4 Acts
The metrics landscape is constantly evolving. Teams are always evaluating new technologies, scaling established technologies, and finding the best ways for them to work together cohesively. Our night will consist of 4 talks focusing on these challenges, what technologies they've used, are using, and are interested in. Do you know what your application is doing? Applications based on microservices are notoriously difficult to debug. We show how Jaeger (a CNCF project) provides a lot of visibility into the inner working of distributed applications, suitable for root cause analysis, performance optimization, and service dependency analysis. Yuri Shkuro @ Uber A sneak peak at Grafana v5 Matt Toback, VP Customer Experience @ Grafana Labs. How we do monitoring on our Android Driver App Leigh Douglas, Jeff Hu @ Uber M3DB - Uber's open source time series database A high level overview of Uber's metrics needs and why we decided to build our own time series database. We will also briefly outline M3DB's clustering, consistency and replication strategies, it's custom compression scheme as well as the file system layout. Martin Mao @ Uber
- TimescaleDB and Prometheus
Prometheus has become an open-source standard in collecting and monitoring metrics, giving software developer teams greater clarity on complex IT operations. And while Prometheus has its own time-series storage subsystem specifically for metrics monitoring, a time-series database like TimescaleDB allows you to ask more complex questions of your data. In this talk, Ajay will describe TimescaleDB and its use as a somewhat heretical backend for Prometheus. Engineered up from PostgreSQL, TimescaleDB is the only relational, row-oriented time-series database available that supports both full SQL (e.g., JOINs, window functions) and specialized time-centric functions. For our own internal monitoring needs we hoped to use Prometheus, but realized it lacked a proper SQL storage backend. So, we set to work on natively supporting a Prometheus data type. Mat will demonstrate this data type, which allows for the storage of data in Prometheus' format to be transparently stored in relational format in TimescaleDB for the user. Ultimately, your relational metadata and time-series metrics are then fully available to be indexed and queried using SQL, while the most recent metrics are monitored through the full functionality of Prometheus. We then show how these metrics are beautifully visualized using Grafana. About Ajay: Ajayis the CEO and Co-founder of TimescaleDB, an open-source time-series database packaged as an extension of PostgreSQL. Ajay's previous startup, communication data analysis company Sensobi, was acquired in 2011 by GroupMe/Skype/Microsoft. Ajay led the mobile team at GroupMe, which grew to millions of daily users and billions of monthly messages over a short period of time. His past experience includes roles at Microsoft, Citigroup, and several startups. He holds Bachelors, Masters degrees from MIT in Computer Science, and an MBA from the MIT Sloan School of Management. About Mat: Mat has been working on data infrastructure in both academia (Princeton, PhD) and industry. As one of TimescaleDB's core architects he works on performance, scalability, and query power. Previously, he attended Stuyvesant, The Cooper Union, and Princeton.
- Ntop, a Web-based Network Traffic Monitoring Application
Ntopng is a web-based, open source, realtime traffic monitoring application for Windows and Unix systems. It is suitable for passively monitoring traffic and characterize it using nDPI (an open source DPI library developed and maintained by ntop). ntopng can be used as a passive network traffic sensor or as a collector if NetFlow/IPFIX flows generated by routers and switches. Monitoring data can be accessed using the ntopng web GUI, or ntopng can be used as a data source and export it on a SQL database, BigData system such as ElasticSearch, or time series database. Finally, ntopng can also be used inline for enforcing network policies by dropping selected application protocols, prioritizing traffic, as well protecting selected users from inappropriate content. About Simone Simone Mainardi received his BSc, MSc and PhD degrees in Computer Science from the University of Pisa, Faculty of Information Engineering. He worked as a research associate both at the University of Pisa and at the Institute for Informatics and Telematics (IIT) of the Italian National Research Council (CNR). He is now with ntop as a Senior Data Scientist. He is interested in computer networking, parallel and distributed algorithms, Internet measurements and data analysis.
- Monitoring Services @ Squarespace
Squarespace's custom-built service framework, Tyson, has been in development for a few years. Tyson provides a lot of functionality out-of-the-box to service developers, including integration with the company's monitoring and alerting frameworks. In this presentation, Franklin Angulo, will go through the development lifecycle and provide in-depth demos for these integrations. We will begin by declaring a new service endpoint and having stubs generated automatically. Upon deploy, the automatic sensing of new endpoints will kick in and new metrics will be emitted to Graphite, logs will be shipped to ELK, and Grafana dashboards will be updated. We will also be able to define an alert in Sensu based on the new metrics.
- Building a Snap Telemetry Plugin
Snap is an open telemetry framework by Intel designed to simplify the collection, processing and publishing of system data through a single API. In an environment with huge amounts of data, Snap simplifies the gathering of measurements. It can dynamically load new collector plugins and new measurements without having to stop and restart. This means you can collect new metrics in real-time when you need them. Written in Go, it is highly performant and inherently scalable with its built-in clustering mechanism. Staples are on the forefront of introducing Snap into their stack and joining us will be Jacob Lisi, a software developer on the Staples SRE Instrumentation team. Jacob will show us how to build a Snap plugin, live at the podium, without a net. Anything can go wrong! Danger! Intrigue! Telemetry! Snapping turtles! We'll give you the whole seat, but you'll only need the edge.
- Graphite for Power Users
It's time again for NYC Metrics and Monitoring; this time, an evening of Graphite for Power Users. Starting the evening will be Allan Liu from Digital Ocean to talk about their new open source project Vulcan, an API-compatible alternative to Prometheus. Then, Jason Dixon (@obfuscurity) will be joined by Ben Burry from Etsy and Nikita Ostrovsky from PulsePoint to chat about their experience in implementation, scaling, and where they see their own monitoring stack evolving over the coming year. Following the discussion, Jason will hold on-stage office hours with meetup members about their own Graphite questions. To participate, please enter your name and question here: https://raintank.typeform.com/to/q13fzE About Jason Jason Dixon is a leading authority on metrics, monitoring, and infrastructure and author of the O'Reilly book Monitoring with Graphite. He is the creator of the worldwide Monitorama conference series and has recently joined raintank, where he is pushing forward the open source Graphite project every day.
- An Evening of Alerting (ft. Prometheus)
Welcome to the first NYC Metrics and Monitoring meetup in lower Manahattan. We're lovers of all things metrics and monitoring, organized by the team behind Grafana. This meetup will highlight the modern day monitoring & alerting stack and illustrate a couple different approaches. 6:30pm - 7:00pm: Food, drink, meet each other. 7:00pm - 7:15pm: Intro & The Human Side of Alerting 7:30pm - 8:00pm: *Alerting at Soundcloud* with Tobias Schmidt. Tobias is a Production Engineer at Soundcloud and a core member of the Prometheus project. Tobias will talk about the alerting stack at Soundcloud and share best practices he's learned over time. Space provided by the very generous Packet, premium bare metal servers and container hosting.