Powering Near Real Time decisioning with Impala

Topic Title: Powering Near Real Time decisioning with Impala

Abstract:

In the advertising industry decisions regarding ad space bidding are done in a very dynamic fashion. A large set of variables will influence this decision, hence it is very important to access and analyze the data in near real time fashion. Bidtellect offers an innovative native marketing platform, connecting advertisers and publishers to deliver optimized and insightful advertisement at scale.

In this session will focus on sharing our experiences in moving away from a SaaS solution for analytics to a Hadoop/Impala based solution. While cost and flexibility of the solution are a factor, our biggest gain was in building a system that can support analytics across our complete data set and achieve that in a near real time fashion. We will cover lessons learned with fast ingestion of large data sets, using Impala to power an efficient dashboard for reporting and ad hoc analysis.

The topics include :

Designing the system for fast ingestionLessons learned on Impala tuning, sizing Building real time analytics on top of Hadoop

Take-away:

The Audience will learn about architecting a system for fast ingestion and building performant queries on Impala across large scale data sets. we will share key details on Impala tuning and some hard decisions we had to make to ensure our queries execute .

Speaker:

Shekhar is a passionate technologist, with a drive to solve large scale and complex problems. He has the ability to get into minute details helping solve and implement solutions. At the same time Shekhar has a holistic approach to architecting large scale systems and product lines. With years of experience in big data, he has the expertise to design and implement an ecosystem around Big data.

Shekhar Vemuri - CTO at Clairvoyant, www.clairvoyantsoft.com

http://clairvoyant.solutions/wp-content/uploads/2016/10/team5.jpg