Using Spark for Batch & Streaming Use Cases at Capital One

Speaker:

Kaushik Das, Capital One

Abstract:

Kaushik will be leading a discussion about a Capital One platform called "Second Look" that leverages Spark's core and streaming capabilities. He'll describe the platform's distributed/decoupled architecture, dynamic rules, and throttling capabilities. He'll highlight some key lessons learned, with a special focus and demo of end-to-end test automation.

Speaker Bio:

Kaushik has been in the IT industry for 19 years. He has had many varied roles - delving into biometrics, application security, APIs, big data and code generation strategies. In his most recent role with Capital One, he''s had a chance to dive deep into Apache Spark, with a special focus on automating end-to-end data pipeline testing - making that testing quick, easy and air-tight.