What we're about
Upcoming events (1)
Our May 23 event will feature a live-streamed presentation by Mark Kurtz, machine learning lead at Neural Magic. If you're in the Boston area, register via the Boston NLP meetup to attend in person in Cambridge.
In the presentation Optimizing Transformer Models for Performance, Mark Kurtz will walk through the current state-of-the-art research for compressing transformers resulting in smaller and faster models that are just as accurate. The techniques include structured pruning, distillation, unstructured pruning, and quantization. After a quick background over the techniques, he will walk through newly published research which expands on these techniques: effects of optimizations on models of different sizes, results of applying optimizations for pretraining vs fine-tuning, and the surprising ways that sparse networks outperform dense for out-of-domain transfer and zero shot use cases. After this, he will end with a demo showing the results in action and how you can leverage and build upon the research, opening up for Q&A afterwards.
Mark Kurtz leads ML and engineering teams in the design, development, and implementation of cutting-edge technologies and products. With over 12 years of experience in software engineering and machine learning, he is experienced in building and leading teams for both closed source and open source software solutions. He is currently the Director of Machine Learning at Neural Magic managing teams to lower the cost, improve the performance, and increase the adoption of deep learning technologies through SOTA research and engineering. He is an active GitHub contributor, blogger, and researcher with published papers in top ML conferences. Mark holds a Master's degree in robotics engineering from Washington University in St Louis.
By responding here, you acknowledge and consent to our Code of Conduct: We seek to provide a respectful, friendly, professional experience for everyone regardless of gender, sexual orientation, physical appearance, disability, age, race, and religion. We do not tolerate behavior that is harassing or degrading to any individual, in any form. Participants are responsible for knowing and abiding by these standards. We encourage all attendees to assist in creating a welcoming, safe, and respectful experience.