Perth Data Engineering - May 2024 Edition

Hello Perth Data Engineers,
We're excited to host our May Perth Data Engineering meetup of 2024!
🍕 🍺 Food and drinks will be provided! Please RSVP your attendance.
Thank you Clickhouse for sponsoring the event!

Agenda:
5:00pm - 5:30pm: Doors open, networking
5:30pm - 6:00pm: Talk #1 – Why is ClickHouse a step-wise change for Real-Time OLAP use cases, and how have the big data technology trends brought us to where we are today
6:00pm - 6:30pm: Talk #2 - Why is building reusable pipelines important. What is the cost of building custom pipelines? How can we ensure maximum reusability of pipelines by a simple mindset change
6:30pm - 7:30pm: Networking, doors close

[📢 Talk #1] – Why is ClickHouse a step-wise change for Real-Time OLAP use cases, and how have the big data technology trends brought us to where we are today.
Speaker: Lee Mc Donald, Principal Consultant, Data Vanguards

Talk summary: Lee will explore how ClickHouse is transforming the building of large-scale streaming and event-based data pipelines. His talk will highlight emerging architectural trends, demonstrate ClickHouse's impact on data pipeline design, and discuss its strengths and unique capabilities suited for near real-time data challenges. Lee will also summarise this talk with how and why he is helping customers migrate their ElasticSearch/OpenSearch use cases to ClickHouse.

Speaker summary:
Lee McDonald leads DataVanguards, a data strategy and engineering firm transforming big ideas into tangible data products. Among working with many global clients, Lee Partners with HyperSec.io, a pioneer in cybersecurity and an early adopter of ClickHouse, DataVanguards is at the forefront of the big data revolution. HyperSec.io has been evangelising and pushing the boundaries of big data technology for decades.

[📢 Talk #2] - Why is building reusable pipelines important. What is the cost of building custom pipelines? How can we ensure maximum reusability of pipelines by a simple mindset change.

Talk summary: Sujal will introduce the YODA (You will Do it Again) principle to underscore the importance of reusability and strategic planning in data engineering. How we can accelerate the creation of repeatable, pattern-based pipelines using proactive strategies, rather than waiting for evolution. Compare the impact of choosing between existing technology and new tools on operational adaptability and support. This presentation aims to shift traditional pipeline construction towards a framework that prioritizes long-term efficiency and cost-effectiveness in standardization.

Speaker summary:
Sujal is a Data Engineer at Data Vanguards, a data strategy and engineering firm transforming big ideas into tangible data products. Sujal is also the founder of Data Profession Academy where I help people start their career in Data Analytics by learning the right skills. My passion lies in building data pipelines that can stand the test of time.

💬 Join our Slack Group here:[ https://goo.gl/forms/DVNazDmNBg1FFm2X2](https://goo.gl/forms/DVNazDmNBg1FFm2X2)
💬 Submit a talk here:
https://dataengconf.com.au/forms/meetup-talk