#49 Large Scale Production Engineering India (LSPE-IN) meet up

Event Agenda and Speaker info

10:45 - Registration

11:00 - Tea

11:15 - Opening and keynote by LinkedIn India Leadership

11:30 - Data Ingestion at Scale @ LinkedIn : Bhupendra Kumar Jain, LinkedIn

In this talk Bhupi will navigate through various data ingestion pipelines enabling continuous stream of data available into offline data lake at LinkedIn. The talk will deep dive into the current data ingestion ecosystem at LinkedIn , the various challenges around operating at exabyte scale of data and ensuring data quality & compliance. It also talks about our plan to revamp the data ingestion ecosystem to meet the ever growing data needs.
Bhupendra Kumar Jain (aka Bhupi) is a Sr Staff & Tech lead for offline data ingestion and data compliance at LinkedIn. He leads the low latency ingestion for offline data lake, scaling it to guarantee member trust by keeping the data secure & compliant. Prior to LI he was the lead architect in the Big data platform team (HDFS, Hbase, Yarn) and developed a scalable key value store and related cloud offerings

12:15 - Scaling up Database in AWS - Design strategies to serve million queries and perform seamless upgrades : Niranjan R and Sandipan Pratihar, AWS India

1:00 - Lunch

2:00 - Re-engineer CI/CD Pipeline For Multi-arch Environments : Santosh Kumar, Specialist Solution Architect, AWS India

Reducing infrastructure cost in public cloud is every cloud user's priority. After his talk on value proposition of AWS Graviton last year, Santosh will walk us through how to design a multi-architecture CI/CD pipeline to make a web scale product ready for deployment to a lower cost Graviton based virtual machine infrastructure. He will end his talk with a easy to follow demo.

Santosh is long time Amazonian with 10+ years in industry. In his current role at AWS, he helps users with scalable production architecture design. In his previous role he was helping AWS customers in troubleshooting critical production issues as a senior GSE engineer.

2:45 - Moving towards real time data ingestion using dataflow and cloudrun : Ojasv Singh and Harshit Agarwal, Zeotap

In the modern data ecosystem, orchestrating the ingestion of data from multiple sources that transmit at irregular intervals poses intricate challenges. Failure at any stage or single record in data could mean failure of the entire file and reprocessing overhead of even earlier processed data. Recently, at Zeotap, we have revamped our architecture to move away from batch based processing to provision GCP cloud assets on demand to ingest large volumes of data after converting batch data to individual events. This made our system near real time reducing SLA from 48 hrs to 4 hrs.To be announced

3:30 - Open forum

4:00 Onwards - Tea, snacks and networking.