DET Bay Area Meetup at Altimate AI
Details
Join the next DET Bay Area Meetup on Wed, Oct 1 at Altimate AI for an evening of learning and networking!
Big thank you to Altimate AI for sponsoring the event.
---
Talk #1: Charting New Territory: LinkedIn’s Early Bet on Flink Batch for Large-Scale Workloads
Speakers: Venkata Sowrirajan and Archit Goyal (Software Engineers at LinkedIn)
As one of the earliest adopters of Flink Batch, LinkedIn has taken a bold step toward redefining large-scale batch processing. This talk shares how we built a production-grade Flink Batch platform from the ground up—covering architectural decisions, platform engineering challenges, and lessons learned while scaling it across mission-critical workflows. If you're considering Flink beyond streaming, this is your inside look at what it takes to run Flink Batch reliably at scale.
About the Speakers
Venkat Sowrirajan is a Staff Software Engineer at LinkedIn specializing in building scalable batch processing infrastructures. Currently, he leads the development of LinkedIn’s Flink Batch platform, enhancing its large-scale data capabilities. Previously, Venkat contributed significantly to Apache Spark , tackling shuffle infrastructure challenges to meet LinkedIn’s massive data scale. With experience at startups like Qubole and MapR, Venkat has a deep passion for distributed systems and has contributed to open-source projects including Flink, Spark, and Iceberg.
Archit Goyal is a Sr. Software Engineer at LinkedIn and one of the founding members of LinkedIn’s Flink Batch team. He brings experience in platformization and productionization, with a strong focus on making Flink Batch a reliable and scalable solution for large-scale data processing.
Talk #2: Multi-dimensional Anomaly Detection in Petabyte-scale Time Series Data
Speaker: Suvodeep Pyne
Modern systems produce metrics across regions, devices, versions, and partners. Aggregate views often hide the real issues. This talk explains how to solve multi-dimensional anomaly detection at petabyte scale. We will cover query-level optimizations such as bucketing and indexes to build efficient monitoring. We will also show how to optimize searching the high dimensional space to ensure teams monitor what truly matters. The result is higher precision and recall, faster detection and resolution, and reduced alert fatigue. We will further explore how AI enhances root cause analysis through LLM-assisted triage, anomaly-to-event correlation, and lightweight causal scoring, supported by feedback loops that improve with every confirmed RCA.
About the Speaker
Suvodeep Pyne is a technologist and engineering leader with over 15 years of experience, holding multiple patents in scalable systems and real-time analytics. He earned his B.Tech from IIT Kharagpur and a Master’s degree from Texas A&M University. As a Founding Engineer at StarTree, he is focused on advancing observability through AI-powered analytics and storage innovations in Apache Pinot. He also played a key role in building and scaling StarTree ThirdEye, a real-time anomaly detection platform on top of Apache Pinot, driving enterprise adoption and revenue growth.
Previously, Suvodeep co-founded Quasix, where he built Realia, a B2B educational platform and LMS serving 100K+ users across 2,000 institutes. He has also contributed to global organizations such as LinkedIn, where he scaled data and ML systems and contributed to open-source projects like Azkaban and Cubert, and NVIDIA, where he worked on GPU drivers that power high-performance computing and graphics platforms. His expertise spans distributed systems, machine learning, and large-scale data infrastructure where he holds 3 patents. Suvodeep is an IETE & BCS Fellow, IEEE Senior Member, and serves as a reviewer for major tech venues including IEEE TNNLS, IEEE IOT, CIKM and ACM TKDD. Outside of work, he is a self-taught pianist who enjoys hiking on weekends.
---
Location:
Plug and Play Tech Center
440 N Wolfe Rd, Sunnyvale, CA 94085
---
Interested in giving a talk at a future DET meetup? Submit your talk idea HERE.
By attending this meetup, you agree to abide by our Code of Conduct. Failure to comply with our CoC may result in removal from current and future DET events.
---
Data Engineer Things is a global online data engineering community for data professionals to connect and learn. Join us to grow together!
DET Bay Area Meetup at Altimate AI