I am happy to tell you that we will have a great talk on this Meetup with Henning Kropp.
Data Reliability Challenges with Spark
Current Data Lake projects are facing enormous issues over generating business value. According to Gartner, more than 65% of the projects are failing. The most common reasons for projects to fail are centered around data reliability and performance issues resulting in delays, complexity, and errors.
Delta is the next-generation analytics engine as part of the Databricks Runtime tackling some of the most challenging issues with Spark today. Delta provides ACID, Data Versioning, and Schema Enforcement on top of Apache Parquet. In this talk, we will discuss the current challenges and give a live demo of Delta.
Henning is a Solutions Architect for Databricks, the company founded by the creators of Apache Spark. In his previous roles, Henning worked as an Architect, Data Engineer, and Data Scientist at various different companies like Hortonworks, SAP, and GfK.