Skip to content

Deep dive Avro and Parquet - Read Avro/Write Parquet using Kafka and Spark

Photo of Prasad Sripathi
Hosted By
Prasad S.
Deep dive Avro and Parquet - Read Avro/Write Parquet using Kafka and Spark

Details

Agenda

  1. Avro and Parquet - When and Why to use which format?
  2. Data modeling - Avro and Parquet schema
  3. Workshop
  • Read Avro input from Kafka
  • Transform data in Spark
  • Write data frame to Parquet
  • Read back from Parquet

Speakers

Timothy Spann - Sr. Solutions Architect, airisDATA
Srinivas Daruna - Data Engineer, airisDATA

Photo of NJ Generative AI group
NJ Generative AI
See more events
NJ Big Data and Hadoop Meetup
3525 Quakerbridge Rd #1400 IBIS Office Plaza, Suite 1400 · Hamilton Township, NJ