Past Meetup

August 2018 Meetup

This Meetup is past

93 people went

Zendesk @ Haw Par Techno Centre

401 Commonwealth Drive, #07-01 · 149598

How to find us

Take the elevator form Lobby B, which is located at the centre of the building.

Location image of event venue

Details

For the August meetup, our friends at Zendesk are kindly hosting us, again!

Agenda:

7pm: Meet & Greet

7:30pm: "Introduction to Apache Airflow – Programmatically Manage Your Workflows for Data Engineering" by Xiaodong Deng

Apache Airflow was started in 2014 at Airbnb, then became an Apache incubator project in 2016.

Mainly built with Python, Airflow is a platform to programmatically author, schedule and monitor workflows. Workflows can be defined as directed acyclic graphs (DAG) and configured as Python scripts. It supports complex task dependency management, distributed task execution (like Celery + RabbitMQ), and integration with different technology stacks/tools (RDBMS, AWS, Google Cloud, HDFS, Spark, etc.). Friendly UI is also provided for monitoring and configuration.

Currently Airflow is adopted by many companies, including Airbnb, Glassdoor, Google, HBO, ING, Quora, Yahoo!, and Zendesk. Alibaba also built a workflow scheduling system named "Maat" on top of Airflow.

Xiaodong DENG is a data engineer at DBS Bank. He had experience working on data science & engineering in research sector and insurance industry. Trained in mathematics background, Xiaodong believes that solid engineering implementation is the key to converting theory to reality (http://XD-DENG.com).

8pm: "Dota Flask - A Dota 2 recommendation engine written in Flask" by Wang Eric GaoXiang

In this talk, we will explore how to build a recommendation engine using OpenDota API and Python Flask. We will also discuss about best practice in writing Python Flask application as well as how to use Docker with Python Flask. At last, Eric will share the project roadmap for an open source project with the audience, and if you are interested in helping the project, we can discuss more about it.

Eric started programming with Python at RP & NUS. He joined HPE Singapore Lab as a full stack developer and gained exposure to microservices, devops and big data technology. He worked at DBS as a data engineer and today is with Silot.ai as a senior software developer to explore fintech and blockchain solutions.