From 0 to Dashboard in 20 mins

BACK IN PERSON!!
Level 28, 161 Castlereagh St, Sydney, 2000

Join us for pizza and meet like minded data people!

Learn how to build a modern data lakehouse architecture leveraging open-source technologies like Apache Nifi, Spark, Hive and Icebergo.

You will see real-time data ingestion at scale using Spark SQL and see ACID transactions and time-travel queries (data versioning) for ETL and streaming workloads.

Slides, demos and Q&A will help you understand the concepts required to build a modern multi-analytics data lakehouse architecture.

Motivation:
Whether you are new to the field of data analytics and data science, this workshop will help you understand tools required to build petabyte scale data pipelines easily and efficiently.

Who this is for?
Solution Architects
Data engineers
Data scientists

Agenda
1. Fundamentals of a modern hybrid multi function data platform
2. Building data ingestion pipelines at scale

• Introduction to Apache Nifi & serverless flows
• Monitoring flows

3. Data engineering

• Orchestration automation
• Pipeline monitoring and visual troubleshooting

4. Data warehousing & Visualisation

• Introduction to Apache Iceberg as a table format
• ACID transactions
• Time travel
• Data versioning
• Dashboarding