Democratizing Deep Learning with DeepHyper

This month we are thrilled to have Prasanna Balaprakash from the Argonne National Laboratory speak with us on Democratizing Deep Learning with DeepHyper. Prasanna's research interests span the areas of artificial intelligence, machine learning, optimization, and high-performance computing. His team focuses on advanced research areas in AI for Science at Argonne including scalable automated machine learning, meta-learning, reinforcement learning, geometric machine learning, and neuromorphic computing.

Scientific data sets are diverse and often require data-set-specific deep neural network (DNN) models. Nevertheless, designing high-performing DNN architecture for a given data set is an expert-driven, time-consuming, trial-and-error manual task. To that end, we have developed DeepHyper [1], a software package that uses scalable neural architecture and hyperparameter search to automate the design and development of DNN models for scientific and engineering applications. In this talk, we will focus on two new algorithmic components that we developed recently. The first is DeepHyper/AgEBO [2] that seeks to reduce the overall computation time by combining Aging Evolution (AE) to search over neural architectures and asynchronous Bayesian optimization (BO) to tune hyperparameters of data-parallel training. The second is DeepHyper/AutoDEUQ [3], an automated approach for generating an ensemble of deep neural networks and using them for estimating aleatoric (data) and epistemic (model) uncertainties.

[1] https://deephyper.readthedocs.io/en/latest/
[2] R. Egele, P. Balaprakash, I. Guyon, V. Vishwanath, F. Xia, R. Stevens, Z. Liu. AgEBO-Tabular: Joint neural architecture and hyperparameter search with autotuned data-parallel training for tabular data. In SC21: International Conference for High Performance Computing, Networking, Storage and Analysis, 2021.
[3] R. Egele, R. Maulik, K. Raghavan, P. Balaprakash, B. Lusch. AutoDEUQ: Automated Deep Ensemble with Uncertainty Quantification, (in review), 2021.