What we're about

Anyone who is interested in the movement of data and getting data from one place to another. We will be talking about cloud based and open sourced solutions - sharing our knowledge on how we've managed to achieve real-time streaming and the challenges we've faced along the way

We have our own slack group and website which you can find out more details about here: https://sydneydataengineers.github.io/

Upcoming events (4)

Special Edition, Data Engineering on AWS

Amazon Web Services

AWS have kindly invited us to their Park St offices for a special edition of the Sydney Data Engineering meetup. 🏠Venue Host: AWS 🍕Food and Beverage sponsor: AWS -------------------------------- Speakers: 🎤Chris Horder - Solutions Architect, AWS Stream processing in 2019 Explore the various options for streaming data on AWS, such as Amazon Kinesis and Amazon Managed Streaming for Kafka, and the various options for processing streams of data such as Apache Spark, Apache Flink, AWS Lambda, and Amazon Kinesis Analytics for Java. Let's explore what an architecture for processing Australia's new Open Banking data format at 60,000 transactions per second could look like. 🎤Melody Yang - Big Data & Analytics Solution Architect, AWS Codeless ETL - ARC Framework ARC is a Spark-based data ingestion and process framework, which simplifies and accelerates ETL work load at scale. In this session, we will focus on a few data engineering problems that ARC has solved. From a user perspective, I will also touch on the business value and outcomes by leveraging the codeless nature of the framework. Melody is a data enthusiast - passionate about Data Engineering and Data DevOps. A pioneer of cloud-native Data Lake. Love developing and implementing architectural concepts, practical data patterns and frameworks. 🎤Najah Naaji - Solutions Architect, AWS Open Distro for Elasticsearch Elasticsearch has become an essential technology for log analytics and search, fuelled by the freedom of open source. In this talk, we will discuss what Open Distro for Elasticsearch offers and explore functionalities like SQL, Security, Event Monitoring & Alerting, Deep Performance Analysis with Open Distro for Elasticsearch and Kibana. Najah is a member of AWS Data and Analytics Technical Field Community and has been working with Data and Analytics for over 14 years. He has been with AWS for over 5 years and before joining AWS he was working in the Telecommunications industry. He is an AGSM alumni and holds a Bachelors and Masters in Engineering. He considers himself as a Big&Fast data enthusiast, loves to travel and read books in his spare time. ~~~~~~~~~~~~~~~~~~~~~~~~ Bring along some great questions for our speakers!

October Edition, Sydney Data Engineering meetup

Canva have kindly invited us to their new Surry Hills offices - please note the new location! 🏠Host: Canva 🍕Food and Drink: Canva Speakers: 🎤 Michael Compton, Senior Engineering Manager at Rokt Idempotent, non-linear data structures and multiple dispatch inspired stream processing. An overview of how we developed a bespoke stream processing architecture on elastic compute and key value data stores to process dynamic client events allow us to drive timely business outcomes and reporting. Michael leads a team of full stack data architects designing and building data solutions across a wide range of business functions. Further speaker details to follow!

November Edition, Sydney Data Engineering meetup

ThoughtWorks Australia

Thoughtworks have kindly invited us to their Carrington St offices - speaker details to be announced closer to the date!

End of Year Special Edition, Sydney Data Engineering meetup

Atlassian have kindly invited us to George St offices - we will be doing a special format for this one - stay tuned for details!

Past events (18)

September Edition, Sydney Data Engineering meetup

SafetyCulture

Photos (85)