MLAI Meetup : Mixed Precision DL, Cost sensitive learning (& Burgers to Mars)

Title: Faster Deep Learning Training with Automatic Mixed Precision

Abstract: Automatic Mixed Precision is an easy to use method for improving training performance up to 3x by utilizing NVIDIA Tensor Cores. We will review the theory behind mixed precision using Tensor Cores, show how to use it in model training scripts, and highlight some real world performance improvements observed by customers. Compared to single precision, mixed precision offers many benefits: 2x better use of the available DRAM bandwidth, smaller memory footprints which allow larger batch sizes or network architectures with more parameters to fit in GPU memory, and allow usage of Volta's Tensor Cores to boost raw math throughput by up to 8x.

Bio: Maggie Zhang joined NVIDIA in 2017 and she is currently working on deep learning frameworks. She got her PhD in Computer Science & Engineering from the University of New South Wales in 2013. Her research background includes GPU/CPU heterogeneous computing, compiler optimization, computer architecture, and deep learning.

Title: Cost-Sensitive Learning

Abstract: Classification is a staple of ML. But how well do standard approaches deal with real world class and cost imbalances, which may be substantial? I’ll share techniques and tools based on sample distributions, probability thresholds for expected costs, and direct learning to minimise cost, illustrated with synthetic examples and practical experience. Don’t Panic - it will be a Hitchhikers Guide to the Galaxy of techniques, from Golgafrincham leaf currency to Hotblack Desiato spending a year dead for tax purposes. No Vogon poetry, I promise.

Bio: David heads the ThoughtWorks Australia data practice. He is an experienced leader of strategy, delivery, and organisational design initiatives, and an accomplished technologist experienced in research and new product prototyping, especially in mathematically and data-intensive applications. David works with leaders and teams to identify the characteristics of successful technology research and delivery in their unique context and helps them to understand and adapt their organisations accordingly. David is also a strong technical leader able to augment the skill set of research and delivery teams.

Copy/Paste
Arushi Sinha will talk about implementing the Copy Paste technique for Synthetic Image Data generation during her internship at Silverpond.

⚡⚡⚡Lightning Bonus : From Burgers To Mars: A preview of Paul McLaughlin’s forthcoming “Burgers To Mars” talk that will detail the creation of a prototype robot he built, which he used for a 21 Day Challenge whereby the only thing he was permitted to eat was food the robot prepared for him. ⚡⚡⚡