Taming Big Data DevOps with Automation for Machine Learning

We are pleased to host Muji Qadri, Sr. Solution Engineer from Unravel, to present on automation for big data in machine learning.

Organizations are being disrupted by the convergence of Machine Learning/AI, Cloud and Big Data. Organizations typically end up with a stack of tools in their Big Data ecosystem, some of these are cloud storage S3/ADLS, Hive, Impala, Spark, LLAP, Kafka etc. Hundreds of applications, queries and data streams depend on the stack to be available and scalable. Any disruption in these services or running non-optimally impacts users across the organization.

We will explore how to move from Observability (Logging, Monitoring) to DevOps for Big Data. Admins need help to identify patterns in the workloads, tune the hundreds of knobs in configs to enable frictionless multi-tenant environment and automate actions to enable self-service, both for developers and admins. This frees up valuable time of admins, who today are scrambling to find root-cause for every failed app which they have very little knowledge of; to planning for new releases/apps, and enabling organizations to delivery faster ROI.

Muji Qadri is a senior solution engineer at Unravel. He is passionate about performance tuning, scaling distributed systems and using data to solve customer problems. For the past 20 years, Muji has been working in the Big Data space for Financial Services, Telco, Retail and Government entities in different roles ranging from Pre-Sales, Engineering/Architecture & Project Management.