Rise of the Lakehouse with Delta Lake
Details
ABSTRACT:
Delta Lake is the open source, transactional storage layer for tabular data, which forms the foundation of the Lakehouse, offering the best of data lake and data warehouses. Delta Lake is built on the Delta transactional protocol and it’s the most widely used lakehouse format in the world. In this presentation, we will talk about the internal workings of Delta, some of its recent innovations, and future work.
SPEAKER(S) BIOS:
Speaker #1: Vijayan Prabhakaran, Senior Staff Engineer, Databricks
LinkedIn: https://www.linkedin.com/in/vijayan-prabhakaran-73314263/
Vijayan is an engineer at Databricks, where he leads various efforts on making Delta simpler and faster to use. He has over 2 decades of experience in building file and storage systems. He has published over 50 papers in top-tier research conferences, has over 10 patents, and holds a Ph.D in Computer Science.
Speaker #2: Karthik Ramasamy, Head of Streaming, Databricks
LinkedIn: https://www.linkedin.com/in/kramasamy/
Karthik Ramasamy is the Head of Streaming at Databricks. Before joining Databricks, he was a Senior Director of Engineering, managing the Pulsar team at Splunk. Before Splunk, he was the co-founder and CEO of Streamlio that focused on building next-generation event processing infrastructure using Apache Pulsar and led the acquisition of Streamlio by Splunk. Before Streamlio, he was the engineering manager and technical lead for real-time infrastructure at Twitter where he co-created Twitter Heron, which was open sourced and used by several companies. He has two decades of experience working with companies such as Teradata, Greenplum and Juniper in their rapid growth stages building parallel databases, big data infrastructure and networking. He co-founded Locomatix, a company that specializes in real-time streaming processing on Hadoop and Cassandra using SQL, which was acquired by Twitter. Karthik has a Ph.D. in computer science from the University of Wisconsin, Madison, with a focus on big data and databases. During his college tenure, several of the research projects he participated in were later spun off as a company acquired by Teradata. Karthik is the author of several publications, patents and a popular book, Network Routing: Algorithms, Protocols and Architectures.
Swags and Food are sponsored by Databricks.