In-Person! Apache Flink® Meetup Bay Area (December 2023)

Hello Friends! Please join us for an IN-PERSON Apache Kafka® meetup on Thursday, December 7th starting at 5:30pm hosted by our friends at Palo Alto Networks!

📍Venue:
Palo Alto Networks
​​3000 Tannery Way
Santa Clara 95054

*There is ample street parking and visitor parking around campus.

We will meet in Room is SC1.1.404 Multi-purpose Room 1, located in building #1 first floor

Please note, an NDA must be signed by all participants upon arrival to the meetup.
Campus Map

***
🗓 Agenda:

• 5:30pm: Doors open
• 5:30pm: Doors Open, Networking, Pizza and Drinks
• 6:00pm -6:45pm: Flink’s Pluggable Failure Handling: deal with streaming errors the smart way!,Panagiotis Garefalakis, Senior Software Engineer II, Confluent
• 6:50pm-7:35pm: Building Fully Managed Service for Beam Jobs with Flink on Kubernetes, Talat Uyarer, Senior Principal Software Engineer, Palo Alto Networks Cortex Data Lake Team
• 7:35pm-8:00pm: More networking, Q&A

***
💡 Speaker One:
Panagiotis Garefalakis, Software Engineer, Confluent

Title of Talk:
Flink’s Pluggable Failure Handling: deal with streaming errors the smart way!

Abstract:
Apache Flink applications aren’t free from failures, thus it’s important to know how to prepare for and handle failures in your dataflows. Thankfully, Flink employs restart and failover strategies that define the behavior of the pipeline when encountering exceptions to minimize disruptions.
In this talk, we’ll explore the types of production failures and the unique impact that each has on your pipeline’s SLAs. We’ll then see how FLIP-304 and the new Pluggable Failure Handling Interface enables users to implement custom failure handlers using Flink’s generic plugin framework. Throughout, we’ll introduce use cases like: classifying failures (e.g., User or System), emitting custom metrics (e.g., application or platform), exposing to downstream consumers (e.g., notification systems), and implementing custom failover/restart strategies. Finally, as part of the live demo, users will learn how to implement simple failure Classifiers and expose their metadata as part of Flink's web interface.

Bio:
Panagiotis Garefalakis is a Software Engineer at Confluent where he is part of the Flink runtime team. He’s spent the last several years working on open-source big data systems and holds a Ph.D. in Computer Science from Imperial College London where he was affiliated with the Large-Scale Data & Systems (LSDS) group. His interests lie within the broad area of systems including large-scale distributed systems, big data processing, resource management and streaming.

***
💡 Speaker Two:
Talat Uyarer, Senior Principal Software Engineer, Palo Alto Networks

Title of Talk: Building Fully Managed Service for Beam Jobs with Flink on Kubernetes

Abstract: At Palo Alto Networks, We were using Beam on Dataflow for 10K+ jobs. Beam has a good abstraction run on multiple runners. For multi Cloud Provider use case We developed a fully managed stream processing platform on Flink running on K8s to power thousands of stream processing pipelines in production without changing our business code. This platform is the backbone for other infra systems like Real Time Analytics and Log processing to handle 10 Million rps.We had provided a rich authoring and testing environment which allows users to create, test, and deploy their streaming jobs in a self-serve fashion within minutes with Dataflow. Now we provide similar functionality by building a Beam Flink based platform on Kubernetes. Users can focus on their business logic, leaving the Beam platform to take care of management aspects such as resource provisioning, auto-scaling, job monitoring, alerting, failure recovery and much more on multi cloud platforms. In this talk, we will introduce the overall platform architecture, highlight the unique value propositions that it brings to stream processing at Palo Alto Networks and share the experiences and lessons we have learned while creating Beam Kubernetes based platform

Bio: Talat is a Senior Principal Software Engineer at Palo Alto Networks Cortex Data Lake Team working on building a streaming data platform using Apache Kafka, Apache Beam and Dataflow to secure Palo Alto networks customers. Previously he has worked on several data projects in Turkey. He is an Apache Member and active committer.

***
DISCLAIMER
BY ATTENDING THIS EVENT IN PERSON, you acknowledge that risk includes possible exposure to and illness from infectious diseases including COVID-19, and accept responsibility for this, if it occurs.
NOTE: We are unable to cater for any attendees under the age of 21.
***