No Retraining, No Recompiling: Unlocking Hidden Headroom
TL;DR
Weight-space editing makes a trained network better without retraining it, much as binary optimization has long made compiled programs faster without recompiling them.
- [Weight space] A large language model (LLM) edited 0.016% of another model’s weights, with no training, and that model’s score on MATH-500, a competition-math benchmark, rose from 7.9% to 48.1%.
- [Binary] Optimizers such as BOLT (Binary Optimization and Layout Tool) rewrite a finished executable using a profile of how it runs, and gain mostly through code layout. In a recent experiment, BOLT made the C++ compiler in GCC 16.2 (the GNU Compiler Collection) build the LLVM compiler infrastructure 15% faster, with byte-identical output, while executing about 1% fewer instructions (Appendix A).
Click on the animation to zoom in.
Both edit the finished artifact rather than its source, which neither recovers; interpretability researchers even describe a network’s parameters as “in some sense a binary computer program”. A weight edit is worked out from the current weights and from how the network behaves on chosen inputs; BOLT works from the binary’s disassembly and a runtime profile, and reorders the code to cut misses in the instruction cache and the instruction translation lookaside buffer (iTLB).
They also share their limits. Both pay off only when one localized bottleneck holds back performance the artifact already has: a repetition loop1 in the edited model, or frequently run code scattered across a binary too large for the instruction cache and iTLB. Neither supplies what is missing: a search of the same kind barely moved the score of Gemma 3 270M, a model that rarely loops, and BOLT gains mainly by reordering existing code. New capability still comes from retraining or recompiling.
Both are also fragile outside narrow settings. A weight edit can break a model: in one study, a single edit was sometimes enough to wreck a model’s performance across benchmarks. Binary rewriting in general is no safer: when 10 general-purpose rewriters were asked to insert extra code into test programs, even the best produced a correctly running program only 70% of the time, and half of the tools never did. Rewriting, the study’s authors concluded, can be made to work for particular kinds of programs but not for all. BOLT stays practical by asking the build for extra information, such as symbol tables and relocation records, so that it can move code safely.
| Dimension | Weight-space editing | Binary optimization (BOLT) |
|---|---|---|
| What is edited | Weights of a trained network | Machine code of a linked executable |
| The missing “source” | Training data, objective and training code | Source code, with its types and high-level control flow |
| Guidance signal | The network’s outputs and activations on chosen inputs | A profile of real runs, from hardware samples or exact counts |
| Edits that work | Removing one internal direction, tied to looping or to refusal | Reordering basic blocks and functions; separating frequently run code from rarely run code |
| Reported gain | MATH-500 from 7.9% to 48.1% once the looping direction was removed | LLVM built 15% faster (Appendix A); up to 20% faster Clang and GCC in the original BOLT paper |
| Adds new capability? | Not on its own: barely any gain on Gemma 3 270M, which rarely loops | No: the same instructions run in a better order (Appendix A) |
| Understanding the artifact | Finding the directions or neurons behind a behaviour, which may span several directions | Disassembling functions and rebuilding their control flow, which relies on heuristics |
| Reliability | One edit can wreck a model | BOLT skips code it cannot analyze with confidence; general-purpose rewriters often break the programs they modify |
The analogy breaks on correctness. A binary optimizer must preserve behaviour, and the original binary is the reference to test it against: the optimized compiler in Appendix A produced byte-identical object files. A weight edit changes behaviour on purpose, so there is no reference to test against, and side effects surface only in evaluations beyond the target task (Section 1.2).
The two also differ in scope. A binary optimizer has one job, making a program faster, and never needs to know what the program is for: whether it plays Go or chess, BOLT only needs to know which parts run most. Weight edits serve many goals, from removing a loop to updating a stored fact or stripping out refusal (Appendix B), and the broader idea of weight-space programming also reads what a small network was trained to do and writes new networks from scratch (Section 1). On the binary side, those two jobs belong to reverse engineering and to compilers, not to optimization.
1. Weight-space Programming
Almost every network gets its weights from training, though not all: networks have been trained to write other networks’ weights, and programs have been compiled into transformer weights. Weight-space editing changes a trained network’s weights directly, without retraining it, and most methods derive the change from other fine-tuned models or from the network’s own activations (Appendix B).
The newest variant hands the edit to a general-purpose LLM, which decides it from a goal stated in plain language. A post by MIT researchers calls the broader idea weight-space programming: the LLM inspects a network’s weights, as numbers in its prompt or through Python, and writes new values. Its experiments span three settings, and only the second is editing in the sense that parallels binary optimization:
- [Reading] Infer what a network was trained to do from its weights alone.
- [Fixing] Change a trained network to meet a specification.
- [Writing] Build a working network from all-zero weights, without training.
In eight experiments, on networks ranging from small game-playing policies to a 0.8-billion-parameter language model, OpenAI’s GPT-6 Astra read, fixed or wrote weights directly, without gradient descent. In that language model, Qwen3.5-0.8B, it subtracted one direction linked to repetitive looping, the edit behind the MATH-500 gain.
How much feedback the LLM gets depends on the size of the network. For networks small enough to fit in the prompt, it read the weights as text and returned its answer or new weights in a single reply, with no tools and without running the network. Larger networks came with a Python sandbox and a budget of forward passes, runs of the network on chosen inputs, which the authors varied to measure what feedback buys.
1.1 Can You Just Edit Weights to Make a Model Better at Math?
The evidence says no: the edit removed an obstacle rather than adding mathematical skill.
- [It was a loop fix] Qwen’s own model card warns that the model is prone to thinking loops, as the post notes. Before the edit, 93% of its MATH-500 answers ran out of output tokens unfinished; after it, 33% did.
- [Instruction following rose too] On IFEval (Instruction-Following Evaluation), which checks whether answers obey verifiable instructions such as “write in more than 400 words”, the score rose from 21% to 62%. Better math would not explain that; removing a general failure would.
- [No loop, no gain] Gemma 3 270M rarely loops, and the same search barely moved its score: from 6.8% to at most 7.7%.
- [Independent replication agrees] A reader who reproduced the released edit, in a comment on alphaXiv, found accuracy rising from 11% to 62% as unfinished answers fell from 89% to 18%.
Weight edits can add skill only by importing it from a model trained for it. A reasoning vector, the weight difference between two copies of a model fine-tuned on the same data, one with reinforcement learning and one without, raised a compatible 1.5-billion-parameter model’s grade-school math accuracy by 2.6 points. The skill came from training; the edit only moved it.
1.2 Limitations
The post drew quick pushback, much of it reflected in the table below. Critics noted that reading raw weights without tools was shown only on small networks, while the Qwen fix relied on Python and practice runs [1]; that naming the games reduces to simple statistics over input channels, and the loop fix follows a familiar recipe for removing a direction [2]; that the task prompt itself said fixing repetition counted [3]; and that small networks may not predict how large ones behave [4]. Others asked whether the gains extend beyond math [5], warned that the work could accelerate risky AI progress [6], and debated whether weights resemble a binary or source code [7]. Two follow-ups tested the claims: a local open model read three of four games from raw weights [8], and a reader’s reproduction of the released Qwen edit, posted on alphaXiv, confirmed its gain [9].
| Limitation | Description |
|---|---|
| Elicitation, not new capability | Gains appear only where a localized obstacle, such as a loop, hides an existing skill (Section 1.1), and loop edits cannot supply missing knowledge. |
| Weights-only reading at small scale | Reading raw weights without tools worked only on small networks; with two larger language models, the LLM could not tell which had been fine-tuned to be sycophantic until it inspected their attention patterns. |
| A guided search | The 48.1% came from a prompt that named repetition as a failure worth fixing; a prompt without that hint scored the same 14.1% as the hinted one on a smaller test budget, with no result reported at the budget that reached 48.1%. |
| Harder at scale | Loop fixes that took two neurons in the smallest Gemma 4 model took over a thousand in the 31-billion-parameter one, and refusal, once traced to a single direction, spans several. |
| Collateral damage | Removing the refusal direction lowers scores on a truthfulness benchmark, and fact edits bleed into other stored facts, including earlier edits. |
| Edits do not compose | Fact edits applied one after another erode a model gradually, then abruptly: with one popular method, a 6-billion-parameter model broke down within about 1,500 edits. |
| Dual use | The same kind of edit strips safety training: removing one direction disables refusal in 13 chat models. |
2. Binary Optimization
Binary optimizers rework compiled code, most often its layout, guided by a profile of the program on representative inputs. BOLT starts from the finished executable, which makes it the closest analogue to weight-space editing. It disassembles each function, rebuilds its control-flow graph and attaches the profile, then rewrites the layout: it orders basic blocks so that the hot paths, those run most often, execute straight through, places hot functions that call each other side by side, and moves rarely run blocks to a separate section. The hot code then fits in fewer cache lines and memory pages.
The profile is BOLT’s only feedback. It comes either from hardware samples of the running program or from exact branch counts collected by a copy of the program with counting code added. Profile-guided compilation uses such a profile too, but must map each sample back onto source code, and after inlining and other compiler transformations that mapping is only approximate. BOLT applies the profile to the exact machine code that ran, so it finds gains even in programs already compiled with a profile.
Profile-guided layout predates BOLT, and newer tools apply it at other stages (Appendix C). Propeller, for example, has the compiler emit each basic block as a separately movable piece and lets the linker arrange the pieces by profile, instead of rewriting the finished binary.
2.1 Where Does BOLT’s Speedup Come From?
Not from doing less work. Like the Qwen edit, BOLT removed an obstacle rather than adding capability; the experiment in Appendix A shows how:
- [Nearly the same work] The optimized compiler executed about 1% fewer instructions but completed 13% more of them per clock cycle.
- [Fewer stalls fetching code] Misses in the iTLB fell by half and in the instruction cache by 41%, because nearly all of the compiler’s time now falls in 8 MiB of packed hot code instead of being spread over 30 MiB.
- [Gains depend on the program] BOLT helps most on large programs whose hot code overflows the instruction cache and iTLB; reported gains range from 1–5% for CPython to up to 20% for Clang and GCC.
A faster algorithm still has to come from the source code: BOLT preserves what the program computes and changes mainly where its code sits.
2.2 Limitations
| Limitation | Description |
|---|---|
| Layout, not new algorithms | Code reordering is BOLT’s primary optimization and the source of most of its gains, so the instructions themselves change little. |
| Missing metadata | BOLT needs the binary’s symbol table, which release builds often strip, and for full gains the relocation records that linkers discard by default; without those records it cannot reorder functions. |
| Profile staleness | A profile describes one version of the code: once the source changes, BOLT drops the profile data that no longer matches and loses much of its gain, and stale-profile matching recovers most, but not all, of it. |
| Scale and build integration | Rewriting a whole binary at once strains memory on very large programs and fits poorly into distributed builds, which later work such as Lightning BOLT addresses. |
| Heuristic control-flow recovery | Disassembly relies on heuristics, so BOLT skips any function whose control-flow graph it cannot rebuild with confidence. |
| Toolchain fragility | Hand-written assembly has tripped up BOLT, and new compiler output can too; CPython marks its BOLT build experimental because success depends on toolchain, flags and processor. |
Notes
- 1 A repetition loop is a failure in which the model keeps regenerating the same text or reasoning step until it reaches its output-length limit. ↩︎
Appendix A: The BOLT Experiment (GCC 16.2 Building LLVM)
This appendix describes the experiment behind the BOLT numbers in the TL;DR and Section 2.1: GCC 16.2’s C++ compiler, optimized with BOLT, timed against the original while building LLVM.
A.1 Setup
Figure A1. The experiment’s workflow and the time each stage took.
Click on the animation to zoom in.
The experiment compares two builds of the same compiler on one cloud virtual machine (VM):
- [A] GCC 16.2, built from its release source.
- [B] The same GCC 16.2 install, except that its C++ compiler,
cc1plus, is replaced by a copy rewritten by BOLT.
The run took about 7 hours, unattended, in three stages:
- [Build A] A standard build of GCC 16.2.0 from source, linked with
-Wl,-qso that relocation records survive and BOLT can move functions, and with GCC’s own hot/cold function splitting turned off, as BOLT’s documentation requires for GCC 8 and later. - [Profile, then build B] A copy of
cc1pluswith BOLT’s counting code added compiled GCC’s own sources, one compiler run per source file (the 1,139 runs in Figure A1), each recording exact branch counts. Counting replaced the usual hardware sampling because the VM does not expose the processor’s branch history, the last branch record (LBR).llvm-boltfrom the LLVM 23.1.4 release then used the merged counts to reorder basic blocks and functions and to split hot code from cold. - [Time] Five pairs of clean builds of Clang and LLVM 23.1.4, alternating A and B, each measured with
perf statand each running eight parallel jobs, one per virtual CPU. The profile came from compiling GCC and the timing from compiling LLVM, so the profile was tested on code it never saw. For the heatmaps, each compiler also built two of LLVM’s libraries,LLVMSupportandLLVMCore, underperf record.
The VM was a Google Cloud c4-standard-8: an Intel Xeon Platinum 8581C with 4 cores × 2 hyperthreads, 30 GB of memory, and a 32 KiB level-1 instruction cache and 2 MiB level-2 cache per core.
A.2 Results
The BOLT-optimized compiler built LLVM 14.75% faster (95% confidence interval, 14.50–15%): a 34.8-minute build took 30.3 minutes. Every pair gained between 14.6% and 15.1%, and the two compilers produced byte-identical output: all 2,672 object files of the first pair matched. Hardware counters, recorded over each whole build, show where the gain came from (Figure A2).
Figure A2. Front-end misses fell most while the work barely changed: mean change from A to B in each perf stat count over the five pairs, with 95% confidence intervals as whiskers. A 12.85% drop in wall time is a 14.75% speedup.
A.3 Where the Time Went
A Linux executable keeps its machine code in a section named .text. BOLT does not compact that section in place: it keeps the original, renamed .bolt.org.text, for the functions it did not move; writes the optimized hot code into a new .text section at a higher address; and puts the cold blocks it split off into .text.cold. There is no .hot.text; the new .text is BOLT’s hot section. Figure A3 maps the cycle samples taken, while each compiler (original and BOLT-optimized) built LLVMSupport and LLVMCore, onto these sections, one 4 KB page at a time: before BOLT, the compiler’s time spread across the whole 29.6 MiB .text; after it, 99.6% falls in the new 8.1 MiB .text.
Figure A3. Share of cycle samples per 4 KB page of code, on a log scale from icy blue (few samples) to hot red (many); light gray pages have no samples. Each panel’s labels sum to just under 100%: the rest, 0.24% in A and 0.23% in B, falls in the .plt, the stubs through which the compiler calls shared libraries. The panels come from separate runs, so their totals need not match.
Packing the hot code shrinks how much of it the processor must keep in its caches and iTLB. Table A1 counts the 4 KB pages and 64-byte cache lines that hold a given share of the compiler’s time. Pages shrank by half or more, in line with the halving of iTLB misses; cache lines shrank by a quarter to a third, alongside the 41% drop in instruction-cache misses.
Two moves produce this packing. Splitting moved cold blocks out of the profiled functions, more than halving their hot code; ordering placed the hottest functions together, so the 100 hottest now span less than a tenth of the address range they used to.
| Code holding the compiler’s time | A original | B BOLT | Change |
|---|---|---|---|
| 4 KB pages for 50% / 99% of samples | 159 / 2,133 | 55 / 977 | −65% / −54% |
| 4 KB pages touched at all | 3,300 | 1,658 | −50% |
| 64 B cache lines for 50% / 99% of samples | 1,510 / 32,076 | 998 / 23,648 | −34% / −26% |
Table A1. Code that holds the compiler’s time, before and after BOLT.
A.4 The Profile Carried Over to New Code
The profile came from GCC compiling itself, yet it fit LLVM well: only 0.15% of the compiler’s time on LLVM fell in code that BOLT left in place or marked cold. BOLT’s own statistics predicted 1.2% fewer instructions on the profiled run, and the LLVM build measured 1.1%.
Appendix B: Prior Work on Weight Editing
None of these methods retrains the model being edited. They differ in where the edit comes from: other fine-tuned models, the model’s own activations, or a program compiled into weights.
| Work | Idea | Key result |
|---|---|---|
| Editing Models with Task Arithmetic | A task vector, fine-tuned minus pre-trained weights, is added to teach a task, negated to unlearn it, or combined by analogy. | Negation lowers accuracy on the target task with little change on control tasks; adding vectors builds a multi-task model. |
| Reasoning Vectors: Transferring Chain-of-Thought Capabilities via Task Arithmetic | Moves reasoning learned by reinforcement learning to another model as a weight difference (Section 1.1). | For a 1.5B model, +2.6 points on GSM8K (Grade School Math 8K) and +10.2 on BigBenchHard; +4.9 and +12.3 with a step-by-step prompt. |
| Evolutionary Optimization of Model Merging Recipes | Evolutionary search over ways to merge existing models’ weights and layers, with no gradient training. | A merged model scored 52.0% on the Japanese part of Multilingual Grade School Math (MGSM-JA), where no source model scored above 30%. |
| Locating and Editing Factual Associations in GPT | Factual recall localizes to mid-layer multilayer-perceptron (MLP) modules, where Rank-One Model Editing (ROME) rewrites a single fact. | Edits generalize to paraphrases while leaving unrelated subjects unchanged. |
| Mass-Editing Memory in a Transformer (MEMIT) | Spreads updates over several layers to change many facts at once. | Thousands of edits in GPT-J (6B) and GPT-NeoX (20B), orders of magnitude beyond earlier methods. |
| Refusal in Language Models Is Mediated by a Single Direction | A refusal direction, found by contrasting activations on harmful and harmless prompts, is projected out of the weights, a step practitioners call abliteration. | Refusal is disabled across 13 chat models of up to 72B parameters, with most benchmarks changing by under 1%. |
| When Can One Neuron Fix Repetition Loops in LLMs? | Locates Gemma 4 enumeration loops with ablation and attribution, then edits the responsible neurons. | Static edits remove the loops; the residual failure is a knowledge gap that editing cannot fill. |
| Tracr: Compiled Transformers as a Laboratory for Interpretability | Compiles programs written in RASP (Restricted Access Sequence Processing) into transformer weights. | Models whose algorithm is known exactly, but too sparse and inefficient to scale to full language models. |
Appendix C: Prior Work on Binary Optimization
Profile-guided code layout predates BOLT; later systems lay code out at link time instead of rewriting the binary, or re-optimize it while the program runs.
| Work | Idea | Key result |
|---|---|---|
| Profile Guided Code Positioning | Orders procedures in the linker and basic blocks in the compiler by execution profile, to cut instruction-cache misses. | Established profile-guided code layout; the profile-guided function ordering in LLVM’s linker (LLD) is a variant of it. |
| Spike: An Optimizer for Alpha/NT Executables | Instruments executables and their dynamic-link libraries (DLLs), then applies profile-directed code layout and hot/cold optimization. | 5–20% lower execution time across a wide set of applications. |
| Dynamo: A Transparent Dynamic Optimization System | Interprets a running binary, selects hot instruction traces, optimizes and caches them, with no recompilation. | SpecInt95 benchmark programs compiled with -O ran roughly as fast as their -O4 builds. |
| OCOLOS: Online COde Layout OptimizationS | Applies BOLT-style layout inside a running process and swaps in the new code online, so the profile cannot go stale. | Up to 1.41× faster on MySQL, 2.20× on Verilator and 1.14× on a Clang build. |
| Propeller: A Profile Guided, Relinking Optimizer for Warehouse-Scale Applications | Has the compiler emit basic blocks as separately movable pieces and relinks them in profile order, instead of disassembling and rewriting the binary; this fits distributed builds. | Matches BOLT’s speedups with 30–70% less memory and runs on tens of millions of production cores. |

