In-Memory Computing and Massively Parallel Hyperparameter Tuning

This month we present 2 speakers: Ameet Talwalkar and Akmal Chaudhri who will discuss Massively Parallel Hyperparameter Tuning and Machine and Deep Learning with In-Memory Computing respectively.

Massively Parallel Hyperparameter Tuning

Ameet Talwalkar, Carnegie Mellon University Assistant Professor of Machine Learning, Chief Scientist at Determined AI, and leading expert in the area of AutoML. Ameet will focus on the problem of massively parallel hyperparameter optimization.

Modern learning models are characterized by large hyperparameter spaces. In order to adequately explore these large spaces, we must evaluate a large number of configurations, typically orders of magnitude more configurations than available parallel workers. Given the growing costs of model training, we would ideally like to perform this search in roughly the same wall-clock time needed to train a single model. We tackle this challenge by introducing ASHA, a simple and robust hyperparameter tuning algorithm with solid theoretical underpinnings that exploits parallelism and aggressive early-stopping. Our extensive empirical results show that ASHA outperforms state-of-the-art hyperparameter tuning methods; scales linearly with the number of workers in distributed settings; converges to a high quality configuration in half the time taken by Vizier (Google's internal hyperparameter tuning service) in an experiment with 500 workers; and competes favorable with specialized neural architecture search methods on standard benchmarks.

Ameet's Bio: Ameet Talwalkar is an assistant professor in the Machine Learning Department at Carnegie Mellon University, and co-founder and Chief Scientist at Determined AI. His primary interests are in the field of statistical machine learning, including problems at the intersection of systems and learning. His current work is motivated by the goal of democratizing machine learning, with a focus on topics related to the scalability, automation, and interpretability of learning algorithms and systems. He led the initial development of the MLlib project in Apache Spark, is a co-author of the graduate-level textbook 'Foundations of Machine Learning' (2012, MIT Press), and created an award-winning edX MOOC about distributed machine learning.

Machine and Deep Learning with In-Memory Computing

Apache Ignite, an open source memory-centric distributed database, caching, and processing platform used for transactional, analytical and streaming workloads -- delivering in-memory speeds at petabyte scale. https://ignite.apache.org

In this talk, Akmal Chaudhri, technology evangelist at GridGain Systems, will explain these new capabilities and how to use them in Apache Ignite.

This talk will provide:
An overview of the ML and DL algorithms and how they work
Examples of how to implement each ML and DL algorithm
Tips and tricks for getting the most performance out of ML and DL

Next, Akmal will unveil GridGain's deep learning capability, which is part of the GridGain Continuous Learning Framework. (GridGain was built upon Apache Ignite). This framework enables real-time model training and can improve models and outcomes as events happen. The deep learning capability is built on the Apache Ignite multilayer perceptron neural network which is included in GridGain. The neural network is optimized for massively parallel processing (MPP), allowing the system to run each algorithm locally against the data residing on each node of the GridGain cluster.

He'll also demonstrate how models can be continuously retrained as events happen to help improve decisions and outcomes with GridGain deep learning. The GridGain in-memory computing platform supports hybrid transactional/analytical processing (HTAP), allowing any analytics or deep learning to be run in-place in memory within a transaction or interaction.