Professional ML Platforms require solid infrastructure. Setting up such infrastructure is a difficult task and an interesting use case for Cloud Native technologies. The talks will give an update on what to use today.

Agenda

18:30: Doors open. Have a snack, grab a drink
19:00: Talks (abstracts below)

• The case for a common Metadata Layer for Machine Learning Platforms
• Building ML Pipelines with DCOS
• ML pipelines with Big Data
  21:00: Have more drinks and snacks, and get in touch with the speakers (and other attendees)

Talks

*** The case for a common Metadata Layer for Machine Learning Platforms (Jörg Schad, ArangoDB) ***

With the rapid and recent rise of data science, the Machine Learning Platforms being built are becoming more complex. For example, consider the various Kubeflow components: Distributed Training, Jupyter Notebooks, CI/CD, Hyperparameter Optimization, Feature store, and more. Each of these components is producing metadata: Different (versions) Datasets, different versions a of a jupyter notebooks, different training parameters, test/training accuracy, different features, model serving statistics, and many more.
For production use it is critical to have a common view across all these metadata as we have to ask questions such as: Which jupyter notebook has been used to build Model xyz currently running in production? If there is new data for a given dataset, which models (currently serving in production) have to be updated?
In this talk, we look at existing implementations, in particular MLMD as part of the TensorFlow ecosystem. Further, propose a first draft of a (MLMD compatible) universal Metadata API. We demo the first implementation of this API using ArangoDB.

Jörg is Head of Machine Learning at ArangoDB. In a previous life, he has worked on or built machine learning pipelines in healthcare, distributed systems at Mesosphere, and in-memory databases. He received his Ph.D. for research around distributed databases and data analytics.