Spark ML at scale@ Experticity, Building continuous applications with SnappyData

Thank you Experticity for hosting and SnappyData for feeding us!

Agenda

6:00-6:30 food/drink/social!

6:30 logistics

6:35 ML based Recommender systems using Spark at scale at Experticity

7:05 Building continuous Spark applications with SnappyData

7:35 Q/A and closing

ML based Recommender systems using Spark at scale at Experticity - Vikrant

Experticity is a passion based social network of experts who engage with brands they love and with each other by sharing their interests and activities. These experts are often the people that everyday consumers seek advice from when making considered purchases. This rich network is a highly effective marketing channel for aspirational brands to inform them and their networks about their product innovations and uniqueness. At Experticity, we use the power of data and the recommendation engine that leverages it to match the most relevant content to the experts and match the most relevant experts to the brands who want to target them. We provide highly scalable, reliable and personalized experiences for our experts and brand clients. We leverage spark extensively for our big data pipelines, machine learning capabilities, steaming and much more to come. In this talk, we will provide an overview of the business problem, nature and scale of data that we process, the various processing flows and the results that we drive as a result of it.

Speaker:
My name is Vikrant. I work in Experticity as Senior Software Engineer for Platform team. In my role, I heavily use Spark, EMR and other Machine Learning capabilities. In my earlier role, I got a chance to work for a company that deals with NLP. Prior to that, I worked at Microsoft for more than a decade. I am super excited to meet everyone and be part of this amazing journey of what I call “Making the world a simpler, clearer and predictable place with the muscle of Big Data and the brain of AI”

Building continuous Spark applications with SnappyData - Suds

With support for structured streaming, Apache Spark moved one step closer to being able to support continuous applications. Continuous applications are characterized by their ability to work very natively with incoming streams and help build applications that are reactive and data driven. Given that Spark is a compute only platform, things that become very important when you are building applications, like, consistency, persistence, fast recoverability and concurrency are delegated to a data store that interfaces with Spark via connectors. The connector model helps bridge some of these capabilities but comes with significant downsides. In this talk, we show how to build true continuous applications using SnappyData, an integrated compute+data platform based on and 100% compatible with Apache Spark, that allow enterprises working in verticals such as IoT, Finance and Healthcare to present a single programming model and interface to application developers, to build powerful end user applications that are pro-active, intelligent and context aware

Speaker:
Suds Menon is responsible for engineering, marketing, venture funding and juggles all operational aspects of SnappyData. At Pivotal, Suds was the Head of Products for all the Real Time and Big Data products at Pivotal and was named as a member of the technical leadership team. Prior to that he led GemFire engineering for 8+ years, both at Pivotal and VMware, overseeing the transformation of the product from a disruptive product to a mainstream, widely used and supported product within the industry. He holds several patents in the area of distributed systems. He is a frequent speaker at in-memory and distributed data conferences. He holds a Bachelors of Engg in Computers(Pune University) and an MBA (Babson) in Entrepreneurship