• Data Engineering Melbourne Meetup - July(ish) 2019

    Hi folks, due to last minute virus acquisition, I've had to postpone the meetup. The good news is that gives us more time to find awesome talks. Let me know if you have sometime that you'd like to present! Let's do some more data engineering talks, networking and knowledge sharing! We still have room for more talks, so let me know if you've got something to talk about. Talks can run anywhere from 5-30 mins. Let me know if you think you've got something to talk about - all skill levels welcome to present something - just about everything is new to someone! We will also have time after the talks for a community noticeboard where folks will have around 60 seconds to pitch their wares - if you've got an awesome data job going or you are available to fill an awesome job or even that you've found a great open source tool that we should all be contributing to, this will be the time to let us know! Speakers: Andrew Jones - Observability for everyone Have you ever tried to debug a production outage, when your system comprises apps your team has written, third-party apps your team runs, with logs going into some system, application performance metrics going into another system, and cloud platform metrics going somewhere else? Did you find yourself switching tabs, trying to correlate metrics with logs and alerts and finding yourself in a huge tangle? It is a nightmare. In the data world, we talk about aggregating all our data so we can derive new insights quickly, but what about our operational data? Observability is your ability to be able to ask questions of your system without having to write new code, or grab new data. When you’ve got an observable system, it feels like you have debugging superpowers, but can be challenging to even know where to start. In this talk, Andrew will talk about what monitoring and logging are not sufficient anymore (if they ever were), observability basics, and demo an observability architecture that you can use to start your observability journey today.

    11
  • Data Engineering Melbourne Meetup - June 2019

    ThoughtWorks

    Let's do some more data engineering talks, networking and knowledge sharing! We will also have time after the talks for a community noticeboard where folks will have around 60 seconds to pitch their wares - if you've got an awesome data job going or you are available to fill an awesome job or even that you've found a great open source tool that we should all be contributing to, this will be the time to let us know! Speakers: Ryan Collingwood Sharing some of my experiences working at the intersection of Business Analysis and Data Analysis. Reviewing some lessons learned that will be of interest to those who find themselves somewhere between the business and the technology spaces.

    4
  • Data Engineering Melbourne Meetup - May 2019

    ThoughtWorks

    Let's do some more data engineering talks, networking and knowledge sharing! We will have time after the talks for a community noticeboard where folks will have around 60 seconds to pitch their wares - if you've got an awesome data job going or you are available to fill an awesome job or even that you've found a great open source tool that we should all be contributing to, this will be the time to let us know! Speakers: Balkan Misirli - An intro to Google Cloud Data Fusion: a managed CDAP tool that lets you do data exploration/wrangling and build data pipelines without coding, and runs those jobs on the Google Cloud backend. (https://cloud.google.com/data-fusion/) Adebayo Akinlalu - Getting Started with Apache Airflow: Simple orchestration system for data practitioners The talk will provide introduction of Apache Airflow, its architecture, how it works, basic installation, why it should be used and live demo if time permits

    3
  • Data Engineering Melbourne Meetup - March 2019

    ThoughtWorks

    Let's do some more data engineering talks, networking and knowledge sharing! We don't have any talks booked in yet, so let me know if you've got something to talk about. Talks can run anywhere from 5-30 mins. Let me know if you think you've got something to talk about - all skill levels welcome to present something - just about everything is new to someone! We will also have time after the talks for a community noticeboard where folks will have around 60 seconds to pitch their wares - if you've got an awesome data job going or you are available to fill an awesome job or even that you've found a great open source tool that we should all be contributing to, this will be the time to let us know! Speakers: Harmeet Kaur Sokhi - Big Query, what's in a name? Interacting with big data has been complex, expensive, slow and inefficient. Big Query changes all that. If data interests you and you are looking for simpler ways to play with enormous amount of structured and unstructured data, your search ends here. Big Query is amazing and has a lot to offer. For this session we have handpicked some specific and cool features of Google Cloud Platform's enterprise data-warehousing solution. We will try to show you how easily you can start using it which will enable you to play with big data just like using an excel workbook but without worrying about storage and speed.

    7
  • Data Engineering Melbourne Meetup - February 2019

    ThoughtWorks

    Let's do some more data engineering talks, networking and knowledge sharing! We don't have any talks booked in yet, so let me know if you've got something to talk about. Talks can run anywhere from 5-30 mins. Let me know if you think you've got something to talk about - all skill levels welcome to present something - just about everything is new to someone! We will also have time after the talks for a community noticeboard where folks will have around 60 seconds to pitch their wares - if you've got an awesome data job going or you are available to fill an awesome job or even that you've found a great open source tool that we should all be contributing to, this will be the time to let us know! Speakers: Will Gauvin: Astronomy and Machine Learning - Classifying Galaxy Morphologies The field of astronomy has a huge big data problem to the point that professional astronomers have turned to machine learning techniques and using citizen science projects, like Galaxy Zoo, to build training sets. In this meetup presentation, we will scratch just the surface of Astronomy and Machine Learning by looking at galaxy morphology classifications using the Sloan Digital Sky Survey data.

    12
  • Data Engineering Melbourne Meetup - Welcome to 2019

    ThoughtWorks

    Data Engineers of Melbourne unite! Let's kick off the new year with some great data engineering talk, networking and knowledge sharing! We don't have any talks booked in yet, so let me know if you've got something to talk about. Talks can run anywhere from 5-30 mins. Let me know if you think you've got something to talk about - all skill levels welcome to present something - just about everything is new to someone! We will also have time after the talks for a community noticeboard where folks will have around 60 seconds to pitch their wares - if you've got an awesome data job going or you are available to fill an awesome job or even that you've found a great open source tool that we should all be contributing to, this will be the time to let us know! Speakers: Rizvi Rahim - Azure vs AWS: The Devil is in the Details On paper, Azure and AWS look quite similar. However, the many problems (and advantages) of Azure were only discovered after implementing big-data projects on each of these clouds. This technical talk goes through the oddities and idiosyncrasies of Azure when compared to AWS, and the few cases where it shines. We'll go through challenges faced when using services like Azure HDInsight (Hadoop), Data Factory (ETL), Blob Storage, File Shares, Disk Storage, Active Directory and more.

    13
  • Data Engineering Melbourne Meetup Round 2

    ThoughtWorks

    Data Engineers of Melbourne unite! Let's keep the momentum rolling with a second event, and close the year with a bang! We're going to a have a talk on graph data stores, and we'll have a go at a data engineering meetup flavour of open spaces (see https://www.devopsdays.org/open-space-format/) to get some really interesting conversations happening! We will also have time after the talks for a community noticeboard where folks will have around 60 seconds to pitch their wares - if you've got an awesome data job going or you are available to fill an awesome job or even that you've found a great open source tool that we should all be contributing to, this will be the time to let us know! Speaker: Timothy Findlay: Introduction to Graph Stores Timothy will introduce some of the basic concepts of using a graph database and some distinct advantages to using traditional relational databases. He will show a live demo ingesting a generated dataset and exploring way to access and use that data.

    2
  • Data Engineering Melbourne Meetup Kickoff!

    ThoughtWorks

    Data Engineers of Melbourne unite! This meetup will launch our new community. I hope to create a space where we can meet other data engineers, share what we're working on and learning and have some fun. We will also have time after the talks for a community noticeboard where folks will have around 60 seconds to pitch their wares - if you've got an awesome data job going or you are available to fill an awesome job or even that you've found a great open source tool that we should all be contributing to, this will be the time to let us know! We've got two talks confirmed. We could still fit in a lightning talk or two, so let me know if you have an idea! Confirmed talks: Reza Yousefzadeh: Continuous Deployment for Spark Applications at Seek In this presentation, I'm going to talk about how we are treating our spark applications like any other software and go through our CI/CD pipeline. I'll show examples of unit tests as well as integration tests that enable us to push the updates to production with confidence. Andrew Jones: The accidental data engineer In which I will discuss my journey starting a long time ago as a newbie front end engineer, through infrastructure and cloud specialist to data engineer, all the things I've learnt on the way and what skills I think are required to make a great data engineer.

    5