Deep dive Avro and Parquet - Read Avro/Write Parquet using Kafka and Spark

Agenda

1. Avro and Parquet - When and Why to use which format?
2. Data modeling - Avro and Parquet schema
3. Workshop

• Read Avro input from Kafka
• Transform data in Spark
• Write data frame to Parquet
• Read back from Parquet

Speakers

Timothy Spann - Sr. Solutions Architect, airisDATA
Srinivas Daruna - Data Engineer, airisDATA