Let's meet again and talk about Apache Spark on Kubernetes
New Spark version is getting closer to use all possible native Kubernetes scheduler features. Latest Spark comes with support of client-mode in Kubernetes, which enables us to use Spark Notebooks and work in interactive mode. In this talk we will learn how Spark is using Kubernetes to spin up Spark executors and coordinate Spark jobs with a driver process. It will be a short demo having Spark Job running in Google Kubernetes Engine and Apache Zeppelin notebook in the client-mode. We will look at two scenarios: driver inside the K8s cluster, a driver outside the cluster. We will also talk about new K8s-Spark integration features and future plans of Spark community with regards to K8s.
Alexey Novakov is an experienced Scala/Java Developer and Data Engineer. He’s been working at product-based and IT-consultancy companies for more than 10 years. He’s taught Java through online courses for several years. At the moment, he is discovering world of Kubernetes and Cloud Native applications.
Stay for discussions, drinks and pizza afterwards (sponsored by codecentric AG).
17:30 – 18:00 Welcome to the codecentric office
18:00 – 18:05 Opening
18:05 – 19:00 Talk by Alexey Novakov
19:00 – 20:30 Networking & Pizza