PTO IR: The Compiler Should Place Your NPU Barriers (And Can Beat You At It)
Details
Giacomo Castiglioni from the Huawei Computing Systems Lab is joining us to talk about their work on compiling for heterogeneous asynchronous execution units on the Huawei Ascend NPU.
As usual there will be pizza, snacks, beer, and soft drinks after the event for participants, sponsored by Huawei.
Title: PTO IR: The Compiler Should Place Your NPU Barriers (And Can Beat You At It)
Abstract: Ascend NPUs expose several asynchronous execution units: cube, vector, scalar, and memory-transfer engines. Kernel authors must coordinate these units manually using explicit event flags and barriers. For large kernels, this process is challenging and error-prone. When dependencies are not explicitly represented in the IR, compilers must recover them through costly, conservative analysis. PTO IR separates scheduling from synchronization. Authors specify the schedule, tiling, unit assignments, and n-buffering structure in a kernel dialect; the compiler manages synchronization. Dependencies are represented as explicit edges in the IR, allowing the compiler to check their consistency with the types of the values they carry, insert set/wait pairs automatically, and detect synchronization errors at compile time. Set/wait pairs are connected by linear type tokens to ensure correctness by construction. Event IDs are inserted as virtual IDs and are allocated through interference-graph coloring following register allocation techniques. Code generation takes place entirely in MLIR, through register allocation and machine-instruction emission. This preserves dependency information through instruction emission, avoiding the need for the production compiler to partially reconstruct it at the LLVM IR level. We evaluate PTO IR on Gated DeltaNet (linear attention) and FlashAttention layers compiled end to end, reporting per-stage performance against PyTorch implementations and handwritten C++ kernels. The generated kernels match or outperform the handwritten baselines. We conclude with lessons learned along the way and discuss how explicit dependency edges could support automatic schedule selection in future work.
Location: The event takes place in room G51 in the CAB building of ETH Zurich's central campus.



