Open Data Platform and Data Observability by Acceldata

Hello everyone! Excited to invite you all to our 5th In-person meetup on July 11th at 11:00 AM, hosted by Acceldata in Bangalore. We’d love to see you there—come join us!
**********
📍 Venue:
Acceldata, Urban Vault, 1090/A, 18th Cross Rd, Sector 3, HSR Layout, Bengaluru- 560102
**********
***IMPORTANT***
Please ensure you obtain a “Visitor Badge” by presenting a government-issued ID (e.g., Aadhar Card, Driving License) at the Reception desk on the Ground Floor.
**********
Agenda:

• 11:00 - 11:10: Welcome
• 11:10 - 11:50: Vivek Pemawat, Principal Engineer, Acceldata
• 11:50 - 12:30: Snehasish Roy, Technical Lead, Phonepe
• 12:30 - 12:40: Break
• 12:40 - 13:20: Gowtham Sadasivam, Senior Staff Engineer, Acceldata
• 13:20 - 14:20: Lunch / Networking

****************
💡 Speaker: Vivek Pemawat, Principal Engineer, Acceldata
Talk: AI-Powered Automated Quality Engineering for Open Data Platform at Scale
Abstract:
The Open Data Platform is a complex ecosystem comprising more than 40 interconnected repositories built around core Hadoop technologies, including Apache Hadoop, Apache Hive, Apache HBase, Apache Ranger, and Apache Ambari. Maintaining quality, stability, and security across such a distributed platform presents significant engineering challenges, particularly when managing large-scale CVE remediation, dependency upgrades, and continuous feature delivery.

To address these challenges, we built an AI-powered, fully automated quality engineering framework that combines CI-based merge gating, nightly end-to-end validation, and intelligent test optimization. Every code change undergoes automated compilation, build verification, dependency checks, and quality validations before it can be merged, ensuring that unstable code never reaches shared integration branches. This significantly reduces build failures and improves development efficiency.
A dedicated nightly validation environment continuously deploys and tests the complete Hadoop ecosystem. The pipeline performs end-to-end functional validation, cluster health verification, upgrade testing, security validation, and cross-component integration checks across all platform services. This nightly quality gate provides continuous confidence that the platform remains stable despite ongoing development and security updates.
A key differentiator is the use of AI-driven test selection. By analyzing code changes and identifying impacted components, AI automatically selects the most relevant test suites instead of executing the entire regression pack. This reduces execution time, lowers infrastructure costs, accelerates feedback cycles, and enables faster release decisions while maintaining high quality standards.
The outcome is a scalable and intelligent quality engineering platform that automates security validation, accelerates CVE remediation, reduces manual effort, and improves release reliability across more than 40 repositories. By combining AI, automation, and continuous validation, the Open Data Platform has transformed quality assurance into a proactive, data-driven, and highly efficient engineering practice.
-------------------------------------------------
💡 Speaker: Snehasish Roy, Technical Lead, Phonepe
Talk: Clockwork: Building a Distributed Job Scheduler for 2 Billion Daily Callbacks
Abstract:
Have you ever had an alarm fail to wake you up, causing a ripple effect of chaos in your morning? At PhonePe, we understand the criticality of such 'alarms' in our digital ecosystem.
Take, for instance, our daily Merchant Settlements process. A merchant receives multiple transactions during the day. At the end of the day, we want to ensure the final amount gets credited to their account. A potential delay in this routine job being executed means a merchant not receiving their earnings on time, shaking their trust as a PhonePe Customer.

At PhonePe, we face the colossal task of managing over 2 billion daily callbacks. The ability to handle over 100,000 job schedules per second with single-digit millisecond latency is not just a goal; it's a necessity. At p99, our system ensures that there's no lag in job execution, which in the worst case is capped at 1 minute. Follow along to learn how we've made this possible.

In this talk, we will take a look at the internals of Clockwork – the system that powers job scheduling across various teams at PhonePe and enables clients to easily onboard and schedule future jobs without the need for heavy lifting on their own.
---------------------------------------------
💡 Speaker: Gowtham Sadasivam, Senior Staff Engineer, Acceldata
Talk: Local-First AI: Choosing Models, Running Inference, and Sandboxing Agent Code
Abstract:
Local-First AI: Choosing Models, Running Inference, and Sandboxing Agent Code
Running AI locally isn't just about privacy or cost — it's about control. This talk walks through the practical decisions involved in running a fully local LLM stack on your laptop or your own server.
We'll start with choosing the right open-weight model and understanding the GGUF format, then dig into quantization tradeoffs (like Q4_K_M) and how settings such as ngl, cache-type-k, and cache-type-v affect speed, memory usage, and output quality.
If we have enough time, we'll move to running AI agents safely: using Docker-based sandboxes via MCP so that any AI harness can write and execute code in an isolated, controlled environment — no cloud execution required.
By the end, you'll have a clear mental model for picking the right model for your hardware, tuning inference for your use case, and letting AI agents run code without giving them the keys to your machine.