Skip to content

Apache Beam meetup 7 at Datatonic: Beam at Lyft + datalake using Beam + schemas

Photo of Matthias Baetens
Hosted By
Matthias B.
Apache Beam meetup 7 at Datatonic: Beam at Lyft + datalake using Beam + schemas

Details

We want to invite you to join us for the 7th Beam meet up in London. With a lot of data people in London around the time of Strata Data Conference, this is the perfect opportunity to learn about the latest and greatest and meet people from all over the world.

We will have 3 speakers: 2 use-cases: one from Datatonic and one from Lyft and a technical deep dive by Google. We hope to be able to welcome you at Datatonic this time (http://datatonic.com) offices!

=======
Agenda

18:30 - Registrations, pizza and drinks.

18:45 - kick-off

19:00 - 1st talk: Beam at Lyft.

19:30 - 2nd talk: Building a data lake using Beam at LKQ by Datatonic.

19:50 - 3rd talk: Beam in-depth: schema support in Beam by Google

=====
Talks

1st talk

For the first talk, we welcome Thomas Weise (https://www.linkedin.com/in/thomas-weise-0b57a63). Thomas is Software Engineer, Streaming Platform at Lyft. He is also a PMC member for Apache Beam and Apache Flink and contributor to several more of the ASF ecosystem projects. His talk will be about dynamic pricing at Lyft with a combination of various data sources, machine learning models, and streaming infrastructure for low latency, reliability and scalability. Dynamic pricing allows Lyft to quickly adapt to real world changes and be fair to drivers (by say raising rates when there's a lot of demand) and fair to passengers (by let’s say offering to return 10 mins later for a cheaper rate). The streaming platform powers this use case by bringing together the best of two worlds using Apache Beam; ML algorithms in Python and Apache Flink as the streaming engine. Learn how the pricing legacy infrastructure was migrated to become the first production ready deployment on the new portable Flink runner and how Beam's portability framework enables the execution of Python code on Flink.

2nd talk

Joe Cullen (https://www.linkedin.com/in/joseph-cullen-97a8727a), a Data Engineer at Datatonic, a team of problem solvers working on cutting-edge Machine Learning solutions in the fields of Media, Telecom, and Retail will talk about their journey with LKQ, building reusable Beam pipelines to ingest CSV data onto Google Cloud Platform with the goal to build a data lake to facilitate machine learning and analytics.

3rd talk

Reuven Lax (https://www.linkedin.com/in/reuven-lax-a82818/), senior software engineer at Google, will present one of the features he has been working on extensively: schema support and how it can be useful for you!

==================
Who should attend

Everyone interested in Data Engineering, Data Science and Machine Learning, who wants to learn about one of the newer and exciting Apache projects focused on batch & stream processing of data. We try to cover both business value as well as digging deeper technically.

=========
Sponsors

Thanks to Datatonic (datatonic.com) for providing the space. Thanks to Google (about.google) for sponsoring food & drinks at this meet up.

Photo of London Apache Beam Meetup group
London Apache Beam Meetup
See more events
Level39
One Canada Square, Canary Wharf, E14 5AB · London