Past Meetup

Deep dive Avro and Parquet - Read Avro/Write Parquet using Kafka and Spark

This Meetup is past

102 people went

Details

Agenda

1) Avro and Parquet - When and Why to use which format?
2) Data modeling - Avro and Parquet schema
3) Workshop
- Read Avro input from Kafka
- Transform data in Spark
- Write data frame to Parquet
- Read back from Parquet

Speakers

Timothy Spann - Sr. Solutions Architect, airisDATA
Srinivas Daruna - Data Engineer, airisDATA