Data engineering: Micro-services with Apache Kafka and reporting with Spark

Hi Sydney!
We are pleased to invite you to the first edition of the Tech Share meetup.

This first edition will revolve around Data engineering. We are very happy to welcome Matt Howlett, engineer at Confluent, who will delve into about Apache Kafka's ins and outs then explain us how to lean ont it to build a robust micro-services architecture.
The second talk will be given by Arthur Baudry and Nicolas Guignard, and revolve around how they use Apache Spark and AWS EMR and S3 to generate financial reports.

Agenda:
• 6:00pm : Meetup start, welcome attendees
• 6:20pm : 1st talk: A Tour of Apache Kafka
• 7:05pm : 15 min recess
• 7:20pm : 2nd talk: Sustainable enterprise-wide agile reporting with Spark on AWS
• 8:00pm : Networking time
• 9:00pm : Meetup close

First talk: A Tour of Apache Kafka

Apache Kafka is a scalable streaming platform that forms a key part of the infrastructure at many companies including Netflix, Walmart, Uber, Goldman Sachs and LinkedIn. In this talk, Matt will give a technical overview of Kafka and how it differs from traditional messaging systems. He will then walk through some typical use cases including how Kafka can be used as a backbone for building fault-tolerant micro-services, how it can be used to connect applications and data stores together at scale, and how to process large quantities of data in real time using the Kafka Streams API and KSQL.

Second talk: Sustainable enterprise-wide agile reporting with Spark on AWS

In this talk, Arthur and Nicolas will give us an overview of how they used Apache Spark on AWS EMR to generate financial reports for a large ASX50 company out of a datalake on top of AWS S3.
They will detail the way they built a framework to enable data layer consumption throughout the company and how they drove the product development using Agile methodologies to collaborate with the business as well as CI/CD techniques to ensure software quality and deployment safety.

Bios

Matt Howlett: As an early employee at Confluent, Matt has worked on many of Confluent's open source and enterprise products including Confluent Control Center, Schema Registry, Rest Proxy and Apache Kafka client libraries in various languages. Prior to joining Confluent, Matt spent many years developing materials tracking and optimization systems for large mining companies in Australia.

Arthur Baudry: Arthur is a data engineer with a strong interest in data processing and predictive analytics. He's overseen numerous projects for large companies such as Airbus, Total and Orange administering data platforms, developing frameworks or delivering architecture solutions.
His focus is on making software projects on distributed platforms and tools an easy task using CI/CD, automated testing, software craftsmanship and agile methodologies.

Nicolas Guignard: Passionate about IT, he forged his experience from startups to multinationals. He has a broad technical culture. He has worked on different application architectures especially in the Java world (JEE, Web, Scala) but not only, and has a good knowledge of all the associated software components such as front-end, real-time application using Kafka, or Big data analysis using Spark.
In addition to its obvious interest for software development, Nicolas is an adept of agile project management and software craftsmanship practices.

Thanks to our partner ING for hosting us!

Call for speakers: If you wish to talk and share about technology also feel free to drop us a little note and will be more than happy to talk with you :) Next sessions will certainly revolve around the Blockchain and cryptocurrencies in February 2018