Principles of Programming Seminar (PoP) - Stephen Mell
September 8, 2026 3:00PM—4:00PM
Location:
In Person
-
ASA Conference Room, Gates Hillman 6115
Speaker:
STEPHEN MELL,
Postdoctoral Fellow, Computer Science Department, Carnegie Mellon University
https://mell.ac/
AI systems have shown impressive performance across a wide range of tasks. Part of this has been driven by a shift toward compound AI systems, composed of one or more AI models and programmatic components. Latency is a key concern for these systems, and there are many opportunities to parallelize and stream model calls. However, manually doing so greatly increases code complexity, intertwining high-level algorithms with low-level performance optimizations. In this work, we propose providing parallelization and streaming automatically for high-level, sequential code via an approach called opportunistic evaluation. First, we develop its theory, introducing a novel calculus, λO, providing sound, out-of-order execution of general-purpose programs that make external calls, such as those to AI models. We then describe a language, Opal, which offers automatic parallelization and streaming, and we demonstrate its ability to express and optimize a wide range of compound AI systems, improving performance by up to 12.7×. Next, we build a system, PopPy, which automatically parallelizes Python programs, and we show that it can express 5 compound AI systems from the literature with minimal modifications, yielding speedups of up to 6.4×. Finally, we build an extensible language, Quasar, for code generated by AI models and show that it can provide latency, security, and reliability improvements, without accuracy degradation. Together, these show how opportunistic evaluation can provide significant benefits to compound AI systems written in general-purpose programming languages.
—
Stephen Mell recently joined CMU CSD as a postdoc, advised by Zhihao Jia. Previously, he completed his PhD at the University of Pennsylvania, advised by Osbert Bastani and Steve Zdancewic. His interests are broadly at the intersection of programming languages, machine learning, and systems.
The talk is roughly based on the following papers:
https://arxiv.org/abs/2405.11361 | https://arxiv.org/abs/2605.18697 | https://arxiv.org/abs/2506.12202
For More Information:
mstanle2@andrew.cmu.edu