The Plugin That Turns Your Claude Code into a Continuously Self-Improving Monster

Something goes wrong in a session, so you write a rule. Next week the rule gets buried under the other rules, so you write a sharper one. A month later your CLAUDE.md is a page of capital letters, begging Claude to “PLEASE NEVER EVER DO X WITHOUT FIRST DOING Y” and Claude still does the thing. My global config grew exactly like that, and I’ve watched a lot of other people’s grow the same way: one person I read is at 38 memory files and 18 recorded correction files, another has six postmortems and a five-gate review system, and their complaint is the same as everyone’s. There’s a paper on it now. Across 1,867 repositories, agent instruction files more than triple over their lifetime, they gain about five net instructions per commit, and the older a rule gets, the less likely anyone ever deletes it.

What I run instead is a plugin that watches my sessions, works out what the workspace is short of, proposes one change at a time for my approval, and then, the next time it runs, checks whether the last thing it talked me into actually helped. If it didn’t, it says so and offers to roll it back. It’s called Quartermaster. It has already shown up twice in this series as the thing that installs and configures the other plugins, so this time it gets its own story.

It lives in the Eigenwise Toolshed, my plugin marketplace, free and MIT-licensed like everything else in there, and it’s the piece that decides what each workspace should grow next.

Rule 41 Makes Rules 1 Through 40 Worse

Claude reads your CLAUDE.md every session. Anthropic’s own docs say so, and then add that you should keep it under 200 lines, because longer files “consume more context and reduce adherence”, and that you should review it periodically to remove outdated or conflicting instructions. So the file gets read. It’s one page in a book that keeps getting longer, and every rule you add makes the ones before it a little quieter. Rule 41 costs you rules 1 through 40.

Skills have the same problem one layer down. Claude Code loads a listing of every skill’s name and description so it knows what’s available, and that listing gets a budget of 1% of the context window. When you overflow it, the descriptions get cut starting with the skills you use least. Those stop matching your requests, so you use them even less, so they get cut harder. A skill can go invisible without ever throwing an error, and the only notice is a line in a debug log.

Everyone’s answer to both is more of the same: more rules, then hooks to enforce the rules, then memory files to remember why the hooks exist. The “review periodically” line in the docs is the whole maintenance story, and it’s a chore with no tool attached. Nothing prunes on evidence, and nothing checks whether rule 17 has done anything since June.

What It Actually Does

The loop is mostly counting, and the counting is cheap. When a session ends, a hook reads the transcript that just closed and writes six integers into a local state file: prompts, tool calls, tool errors, permission denials, interrupts, corrections. No text, no model call. Once 4 sessions have piled up unreviewed, or 6 friction events have, Claude gets a nudge at the start of your next session, and at the next real pause it offers you an optimization round. Say no and it clears the backlog and waits. Say yes and the actual work starts.

The work is a script. It reads your last 30 days or 40 sessions, whichever is smaller, into one bounded aggregate: what each session was for, which parts of the tree the work landed in, the commands you keep running, the friction counts, and at most a dozen clipped quotes of you correcting Claude. Raw transcripts never enter the model’s context. On my notes vault a month is 127 MB of JSONL, and the aggregate that comes out the other end is 31 KB. It also flags which sessions a human actually typed into, because with a ticket board spawning executors all day those sessions outnumber mine by a wide margin, and without the flag the pass would report the automation’s goals back to me as my own.

session ends ──▶ tally (6 integers, local)
                    │
     4 sessions or 6 friction events
                    ▼
nudge at start ──▶ offer at the next pause (once)
                    │ yes
                    ▼
mine 30d/40 sessions ──▶ bounded aggregate (no transcripts)
                    ▼
verify last round ──▶ keep / improve / roll back
                    ▼
≤ 7 findings, one at a time, each needs your yes
                    ▼
ledger (applied AND rejected) ──▶ next round reads it

Then it proposes. At most 7 findings, best first, each one shown as the exact command or diff, and each one waits for a yes or a no. The no is recorded too. Rejected findings go in a ledger with a fingerprint, and the next round drops anything you already turned down. On my machine that ledger holds 90 applied decisions and 13 rejected ones right now.

The round on this vault on Monday is a fair sample. It found that I’d written three throwaway shell loops in one afternoon to generate article images, so the image skill got a batch mode. It found I’d added new style rules with no way to check the 73 existing articles against them, so a sweep script now does that (69 of them flagged something, most already published, which was a fun morning). And it found me hand-typing lines into a change log with a strict grammar, so a helper now validates them first. I took all three, in about ten minutes. Two of them improved something that already existed, which is on purpose: it’s told to fix a capability that misfires before proposing a new one beside it.

The Most Valuable Gaps Never Raise an Error

The first version of this plugin was a friction hunt: count how often Claude got denied, corrected, or interrupted, then fix those. It worked, and it’s now the last thing on the list, because fixing what went wrong only gets you back to the speed you already expected. A capability you never had moves that baseline.

And the best capabilities leave no trace. Take a goal like “make the ingest reliable”. You can’t close it without a way to check it, and a check that doesn’t exist yet never fails, because it never runs. The need for it shows up exactly once, so every threshold of the form “did this happen three times” skips right past it. A pass that only counts pain is structurally blind to the best thing it could find.

So the pass reads purpose first, weighing sessions by effort rather than by count. It looks for a goal set and never met, or restated across sessions, or a property that keeps being asserted (“should be correct”, “fast enough”) and never demonstrated. An unmet goal is the single best lead in the whole aggregate, and the fix for it is usually an instrument, built as a committed skill so the number can be re-run later. It also has to say whether a gap is attested by your history or inferred by the model, because only the attested kind gets to outrank a cheap fix it’s certain about. And a healthy project with no stated standard gets told there’s nothing to build, which is a valid outcome. A pass that invents work to look useful is a pass you learn to skip.

The Next Pass Grades the Last One

Claude Code already looks at itself: /insights reports on your recent sessions, /doctor finds unused skills, /fewer-permission-prompts proposes an allowlist, and every one of them ends at the suggestion. I looked again this week and couldn’t find one, native or otherwise, that re-measures after you apply a change, and none that rolls a change back. That’s the seam Quartermaster sits in, and the verify step is the whole reason I trust the rest of it.

Every applied decision names the signal it was meant to move: denials, interrupts, corrections, tool errors, or total friction for a capability nothing counts. On the next round the script compares the sessions before the decision with the sessions after it, refuses to call a verdict on too few sessions, and then says one of three things out loud: keep, improve, or roll back. In a repo where I open and close a session per ticket, that verdict arrives within days. In this vault, where I keep one session open for a week at a time, it takes a month, because a week-long session is one data point.

The reason it measures instead of asking me is that I can’t feel the answer. In METR’s randomized study of experienced open-source developers, people forecast a 24% speedup, reported a 20% speedup, and measured 19% slower. The one person I’ve seen actually test their CLAUDE.md changes properly, eight candidates optimized against real PR tasks and then a clean holdout, watched the winning version regress on the holdout. Their line was that the rule was philosophically right and empirically bad, and that editing by vibes, they’d have shipped it. Everybody else is editing by vibes, including the people with the most elaborate setups, and several of them say so.

A config change that can be rolled back on evidence is a change you’re allowed to try. I wrote a whole piece earlier this year about the pitch where the right setup is what’s being sold, so buy mine and the gains appear, and how that turns every disappointment into proof you needed more of the product. A self-improving workspace is exactly that shape of product, and the verify step is what makes mine a different one: it tells you when a change didn’t work, and takes it back.

It Works Where There Are No Tests

The same loop runs in every workspace I have, and it grows something different in each one. The code repos grow permissions, build chores, and small instruments: a check that the docs still describe the code, a report of what a session changed. A research vault grew rules about evidence tiers and about which model gets which kind of question, because I kept re-deciding both by hand. A personal journal grew a privacy rule and a script that checks entries for contradictions, since the point of a journal is seeing patterns and nothing was checking that the entries agreed with each other. A brand-new project gets the whole plugin set plus a few craft rules on day one. And this vault, where I write, grew the three things above. Same plugin everywhere, and after a month no two workspaces have grown the same thing.

That’s the test I’d put to any “self-improving” setup. After a month, does workspace A look different from workspace B? If every project converges on the same toolkit, the thing was accumulating, and the whole point is fitting. It’s also why I’d rather run a loop than install a list of twenty recommended skills: the list is somebody else’s month.

Who gets left out otherwise matters too. Every enforcement answer I’ve seen assumes a test suite and CI, and the people using Claude Code for a vault, a research project, consulting, or a novel have none of that. A loop built on session history works for them, because a writing project has standards it can’t check and chores it does by hand just as much as a repo does. Only the instruments differ. For a brand-new project, the setup skill runs the same idea across your whole history at once and outfits the workspace from what you keep needing, each item approved.

What to Expect Day to Day

When a round is due, the Stop hook holds Claude’s final reply long enough to make the offer, once per session and never twice inside a day. It does that on purpose, because nudges that can be ignored get ignored. Everything else runs on your machine: no model in any hook, transcripts reduced to counts and clipped quotes before anything reads them, and the only network call is a check for plugin updates. And since anything the loop writes into a workspace lands in context on every later session looking exactly as true as everything else, rollback is a first-class verdict, which is why I’m comfortable letting this run for a long time rather than for a few exciting weeks.

Where to Get It

Three commands in a Claude Code session:

/plugin marketplace add Eigenwise/eigenwise-toolshed
/plugin install quartermaster@eigenwise-toolshed --scope project
/quartermaster:setup

Then work normally, and say yes when it asks for a round. I install per project or local and keep the machine-wide level as close to empty as I can, because a rule that’s right for one repo is noise in the next one. The source and README are in the Toolshed repo. Give it a month, then look at what it grew. If it’s something you’d never have thought to build, I want to hear about it, on the repo or on the Discord.

If you’ve read the Sidequest tour, you saw a section called “It Builds Itself Now”, and the two are worth telling apart. Sidequest improves the plugin opportunistically, whenever an executor trips over friction mid-ticket. Quartermaster improves your workspace deliberately, on a counted cadence, and checks its own work next round. The origin story, the codebase-mapper piece, and the model-gateway one (where Quartermaster does the installing) fill in the rest of the Toolshed.

If you only take one idea from this: close the loop on the rules you already have before you add another one.

Everything in the Toolshed is free and MIT-licensed, and it stays that way. If it saves you time, a coffee on Ko-fi or a GitHub sponsorship is what keeps me building and maintaining it.

And if you want this inside your own company, that’s what I do for a living. I built Atomic Agents and everything in this series, and I work with anyone who wants their AI spend to actually pay off: a solo developer, a team of five, or an engineering org that bought the licenses and is still waiting for the productivity. I train teams on AI-assisted development, build the plugins, hooks, and routing that make Claude Code productive on your own codebase and your own model budget, and audit a setup that isn’t delivering and tell you why. A one-day workshop or a standing engagement, whatever fits. Reach out and tell me what you’re stuck on.