Recently, there has been a lot of chatter around harness engineering. This, in part, stems from the fact the unit of work has evolved from simple prompts to complex, high-level tasks (as exemplified by agentic autoresearch), which then need to be broken down into smaller tasks and then mapped to multi-agent workflows. Optimization of model harnesses has started to gain traction. There are opportunities galore - say, how to construct a task graph from a complex high level specification, how to determine task granularity so as to not overrun the context window, how to mix-and-match smaller and larger models, how to “optimally” map a task graph to a multi-turn multi-agent workflow. Interestingly, there are a lot of parallels between problems that have been addressed in the realm of compiler-driven program optimization - e.g., auto-parallelization, auto-vectorization and profile-guided optimization - and optimization of agentic execution. To this end, I penned down the parallel along the different axes. Feedback welcome!

Thanks Vivek and Milind for the feedback on the early versions of the writeup.


Download Paper