Towards the Limits of Binary Neural Networks - Series of Work
Details
This talk covers the recent advances in binary neural networks (BNNs). With the weights and activations being binarized to -1 and 1, BNNs enjoy high compression and acceleration ratio but also encounter severe accuracy drop. In this lecture, we will start from the background and common practice of network binarization, and then dive deep into a widely used BNN model, "Bi-Real Net" (ECCV2018). Further, we introduce two recent works, done in Carnegie Mellon University: "MobileNet via Evolution-based Searching" (CVPR2020), which use neural architecture search for finding the most suitable architecture for binarization, and "ReActNet: Towards Precise BNNs with Generalized Activation Functions" (ECCV2020), which achieves 69.4% top-1 accuracy on ImageNet dataset, for the first time binary neural network reaches the real-valued ResNet-level accuracy.
Talk is based on the speaker's papers:
Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithm (ECCV2018)
Paper: https://arxiv.org/abs/1808.00278
Code: https://github.com/liuzechun/Bi-Real-net
Binarizing MobileNet via Evolution-based Searching (CVPR2020)
Paper: https://arxiv.org/abs/2005.06305
ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functions (ECCV2020)
Paper: https://arxiv.org/abs/2003.03488
Code: https://github.com/liuzechun/ReActNet
Presenter BIO:
Papers’ author Zechun Liu is a Ph.D. student at Hong Kong University of Science and Technology, supervised by Kwang-Ting Cheng. She is also a visiting scholar at Carnegie Mellon University. Her research interest focuses on neural network compression, including network quantization, channel pruning, neural architecture search, etc.
Zechun Liu’s homepage: https://zechunliu.com/
** ** Please register through the zoom link right after your RSVP. We will send the links to the zoom event via email only to those who have registered through zoom. ** **
Find us at:
All lectures are uploaded to our Youtube channel ➜ https://www.youtube.com/channel/UCHObHaxTXKFyI_EI8HiQ5xw
Newsletter for updates about more events ➜ http://eepurl.com/gJ1t-D
Sub-reddit for discussions ➜ https://www.reddit.com/r/2D3DAI/
Discord server for, well, discord ➜ https://discord.gg/MZuWSjF
Blog ➜ https://2d3d.ai
AI Consultancy -> https://abelians.com