Guest Lectures: Multithreaded scheduling (Kiran) and Cassette.jl (Jarrett)

Agenda

• Food
• Fast, efficient scheduler for parallel programming
• Cassette.jl: Overdub Your Julia Code

Fast, efficient scheduler for parallel programming

Parallel programming is essential for high performance on current processor and system architectures. Many libraries include carefully tuned parallel implementations of their functionality, but using these in a parallel program requires effective nested parallelism. We show that existing threading models such as fork-join (OpenMP) and work-stealing (Cilk/TBB/Go) fail to nest parallel code effectively, making these highly tuned parallel libraries near useless to parallel programs; a state of affairs we do not wish to inflict on Julia. A decade-old idea, parallel depth-first scheduling, offers hope. We show the promise of this approach and report on our implementation of PDF scheduling in Julia.

Speaker:
Kiran Pamnany is a research scientist in Intel’s Parallel Computing Lab. His research interests include programming models, high performance computing and compilers. He is the author of Julia’s existing experimental threading runtime as well as Julia’s forthcoming parallel task runtime, and is one of the people responsible for Julia joining the petaflop club with Celeste. He has a PhD in computer science from Brown University.

Cassette.jl: Overdub Your Julia Code

Julia's popularity is, in large part, due to its "hackability." The bulk of the standard library is written in Julia itself, and new language features are often prototyped in external Julia packages rather than requiring deep rewrites of Base code.

However, Julia's most powerful component remains inaccessible to external package developers: the compiler itself. If we really want to drive this language to it's full potential, that's going to need to change.

Enter Cassette, a tool for injecting user-defined code transformation passes into Julia’s JIT compilation cycle, enabling normal Julia packages to analyze, optimize, and modify Cassette-unaware Julia programs via both low-level IR transformation and high-level "contextual dispatch." This latter feature allows users to safely and quickly overload existing Julia methods with context-specific behaviors without ever needing to handle Julia’s IR directly.

Importantly, Cassette requires no manual source annotation or refactoring of target code. Cassette even works in the presence of structural and/or dispatch type constraints. This renders traditionally “invasive” techniques, like automatic differentiation, applicable to previously inapplicable Julia programs.

Downstream applications for Cassette include dynamic pass injection features can be used to perform dynamic code analysis (e.g. profiling, rr-style debugging, etc.), compilation to new hardware/software backends, automatic differentiation, interval constraint programming, automatic parallelization/rescheduling, automatic memoization, and more.

Speaker:
Jarrett Revels is a research software developer at MIT. In 2014, he tried out Julia to speed up some few-body entanglement spectra calculations, and - to the dismay of his advisor - was hooked instantly. Since then, he's authored and contributed to many packages across the Julia ecosystem, including several popular automatic differentiation and performance analysis tools.

How to get here

The Kiva Conference room is on the 4th floor of the Dreyfoos Wing in the Stata Center (Building 32). Enter Building 32 at the front entrance and proceed straight ahead; there will be elevators to the right. Take the elevators to the 4th floor; exit to the left and then turn right at the end of the elevator bank. At the end of the short corridor cross the R&D Dining room. The Kiva Conference Room is to the right.

If you are visiting from outside MIT, please note that the doors lock promptly at 6pm. We will meet as a group by the Dreyfoos elevators and walk up together.