[PDG 478] einspace: NAS from Fundamental Operations

Link to article: https://arxiv.org/pdf/2405.20838
Title: einspace: Searching for Neural Architectures from Fundamental Operations
Content: This paper introduces einspace, a much more expressive NAS search space built from a parameterized probabilistic context-free grammar, enabling search over fundamentally diverse architectural building blocks such as convolutions, attention, and other operations rather than only small variations of a fixed design. On the Unseen NAS benchmarks, the authors show that this richer space can discover competitive architectures from scratch and substantially improve strong baselines when search is initialized from them, arguing that expressive search spaces and smart initialization are key for more transformative NAS.
Slack link: ml-ka.slack.com, channel: #pdg. Please join us -- if you cannot join, please message us here or to mlpaperdiscussiongroupka@gmail.com.

In the Paper Discussion Group (PDG) we discuss recent and fundamental papers in the area of machine learning on a weekly basis. If you are interested, please read the paper beforehand and join us for the discussion. If you have not fully understood the paper, you can still participate – everyone is welcome! You can join the discussion or simply listen in. The discussion is in German or English depending on the participants.