A talk on anomaly detection by Toufic Boubez, CTO of Metafor Software.
Signal from Noise: Making Sense of Metrics
Have you ever wondered why you can't get anything useful out of your data with exponential smoothing? Tried Holt-Winters and still getting tons of false alarms? You’re not alone. Analyzing the mountains of data that you’re collecting and extracting signal from that noise is not easy, so most IT ops teams only keep an eye on a small fraction of the metrics they collect, or they run some simplistic analytics that don’t generate any useful information. The choice of what analytic method to use ranges from simple statistical analysis to sophisticated machine learning techniques. And one algorithm doesn’t fit all data types.
While the more advanced algorithms require math nerds with PhD’s to develop, there are some basic statistical methods that anyone can implement and they can provide surprisingly valuable insights. In this talk, we will revisit the history of some of the techniques people have used, and why they mostly don’t work. We will then explore a few statistical techniques that are appropriate for the most common distributions out of data centers and discuss their pros and cons. Yes, there will be math in this talk but you will be able to follow along. The goal is for you to walk away with at least one technique that you can apply right away in your monitoring environment to improve the signal to noise ratio and get information out of your data.
About Toufic Boubez
Toufic has been passionate about machine learning for over 20 years. Prior to Metafor, he was the founder and CTO of Layer 7 Technologies, a leader in API security and management and recently acquired by CA. Prior to Layer 7, Toufic was the founding CTO of Saffron Technology, a big data analytics company specializing in associative memory technology. Toufic is also a well-known SOA and Web Services pioneer and was Chief Architect for Web Services at IBM’s Software Group, where he founded the first SOA team and drove IBM’s early XML and Web Services strategies. He was the co-editor of the W3C WS-Policy specification, and the co-author of the OASIS WS-Trust, WS-SecureConversation, and WS-Federation submissions. He has also participated on the OASIS WS-Security, SAML and UDDI Technical Committees. He is the author of many publications, articles and several books, including “Building Web Services with Java”, and is one of the co-authors of the SOA Manifesto. Toufic holds a Master of Electrical Engineering degree from McGill University and a Ph.D. in Biomedical Engineering from Rutgers University.
• 6:00PM Doors are open, feel free to mingle
• 6:30 Presentation start
• ~8:00 Off to a nearby restaurant (Mr Brownstones) for food, drinks, and breakout discussions
By transit there a number of high frequency buses (check Google Maps or the Translink site for your particular case) that will get you there. For the drivers, there is a fair bit of street parking (free and pay) in the area, especially after 6.
How to Contact Us / Re Comments
Please note any comments you add to this event (below) will be e-mailed to all members of the group. We're trying to avoid spamming the list, so please do not use comments for jokes, job postings, requests for help programming something or anything else off topic. If you have questions or need to contact us, use the 'contact us' link on the left. Thanks!