Perth Data Engineering - April 2024 Edition

Hello Perth Data Engineers,
We're excited to host our April Perth Data Engineering meetup of 2024!
🍕 🍺 Food and drinks will be provided! Please RSVP your attendance.
Thank you AWS User Group (Perth / Boorloo) for sponsoring the venue!

Agenda:
5:00pm - 5:30pm: Doors open, networking
5:30pm - 6:00pm: Talk #1 – The State of Apache Airflow
6:00pm - 6:30pm: Talk #2 - Application Development in the Era of LLM
6:30pm - 7:30pm: Networking, doors close

[📢 Talk #1] – The State of Apache Airflow
Speaker: Principal Developer Advocate, Amazon

Talk summary: Apache Airflow is a popular workflow orchestration tool that has a thriving and welcoming community, that is driving constant innovation and new releases. I will walk you through some of the new things you can expect to find in Apache Airflow, as well as covering use cases, and some advanced topics to help you supercharge how you use Apache Airflow. I will be looking at Managed Workflows for Apache Airflow, a managed Apache Airflow service from AWS that helps you get started with Airflow quickly.

Speaker summary:
Ricardo is a principal developer advocate at Amazon Web Services; he works with builders, technology leaders and enterprise executives to help them transform their businesses. Prior to joining AWS Ricardo worked for one of the largest professional services firms. He has over twenty years of experience leading open source, emerging technology and innovation programmes. He has been working with cloud technologies since 2008. Ricardo is passionate about cloud, innovation and open source and is excited about how cloud accelerates and amplifies customers’ ability to innovate.

[📢 Talk #2] - Application Development in the Era of LLM
Speaker: LLM Researcher (PhD) at UWA

Talk summary: LLMs are revolutionising industries, but their real-world applications are still unfolding. As a researcher and developer, I'm at the forefront of this, creating an experimental cloud data analyst leveraging LLM technology. Our tool transform business queries into SQL or vector-based searches, extracting insights from our data lake, which houses Perth Crime Data, ABS 2021 Census Data, and school information. Results are presented visually or answered directly. Our demo at map.findperth.com showcases this. It features a GPT-4 backed analyst generating map reports and a chat feature for interactive queries about Perth Crime data. While promising, our prototype isn't market-ready. We're keen to share our development challenges and insights, aiming to spark discussions about the future of cloud application development.

Speaker summary:
Pascal Sun is a PhD Candidate at the University of Western Australia's NLP-TLP Group. His research focuses on spatiotemporal knowledge graph question answering. Prior to this, Pascal spent five years as a full-stack developer, where he successfully developed several products for global markets. Seeking greater challenges and recognizing the imminent impact of AI, he commenced his PhD journey in late 2022. Pascal is passionate about harnessing AI to simplify everyday life. In the era of generative AI, Pascal is committed to exploring how generative AI and Large Language Models (LLMs) can be employed to unlock the potential within our data, bridging the gap between complex datasets and everyday individuals.

💬 Join our Slack Group here:[ https://goo.gl/forms/DVNazDmNBg1FFm2X2](https://goo.gl/forms/DVNazDmNBg1FFm2X2)
💬 Submit a talk here:
https://dataengconf.com.au/forms/meetup-talk