You can't retrain the model
The model isn't the part you get to change. The scaffolding around it is, and it's just text files you own. Here's what happens when you let it rewrite itself.
There's a paper going around about recursive harness self-improvement. The idea in one line: you can't touch the weights, but the scaffolding around the model is yours, so improve that instead, over and over, using what went wrong last time as the input. They ran it across thirty research tasks. Cheaper agents started beating expensive ones, and inference cost dropped by more than half. The gain didn't come from more thinking. It came from better information moving between the pieces.
I read it and recognized my own repo.
The scaffolding is already text
Nothing about that is exotic once you notice what a harness actually is here.
It's CLAUDE.md. It's the files in context/. It's the workflow doc that says
branch first, build before commit, ask before you push anything anywhere. Plain
markdown, version controlled, sitting three inches from the code.
Which means the loop the paper describes is available to anyone with a text editor and a bad afternoon.
Every rule in there is a scar
Read this project's CLAUDE.md and you're really reading an incident log.
There's a line that says never create a tailwind.config.js. There's a warning
that metadata routes need force-static or the export build dies. There's a
whole section marked as superseding the older context docs, because those docs
describe a Prisma backend and Server Actions that this site has never had, and
something confidently built against them once.
None of that was foresight. Each line is the residue of a session that went sideways, written down so the next session skips the discovery. That's the recursion, minus the formalism: the harness gets edited by the thing the harness is for.
Cheap in a way that surprised me
The paper's cost result reads like an optimization footnote. In practice it's the whole point.
A sharp line in the instructions file doesn't just save you a correction. It deletes an entire loop: the wrong approach, the failing build, the reading of the error, the backtrack, the retry. Ten minutes and a lot of tokens, gone, replaced by one sentence that was already in the window before the work started. I have gotten more out of a well-placed warning than out of any model upgrade.
The part nobody warns you about
Harnesses rot, and they rot by growing.
The context/ docs in this repo drifted far enough from reality that
CLAUDE.md needed a section explicitly overriding them. That's the failure mode
of a self-improving scaffold: every incident wants to add a rule, nothing wants
to remove one, and eventually you're spending half the window on instructions
that describe a version of the project that no longer exists.
So the loop has a second half people skip. Add the rule when it bites you. Delete it when it stops being true. A harness you never prune is a harness slowly turning back into noise.
What it actually feels like
You don't get a better model on a Tuesday because you asked nicely.
You get a better harness the moment you notice something, and you get it by writing one sentence in a file that will still be there next month. The model forgets. The file doesn't. That asymmetry is most of the job now, and the strange part is how ordinary the work is: notice, write it down, prune later.