The Great Skill Shitpile Cleanup Theory

I had Fable 5.2 refactor 16 of my most bloated Hermes skills. They used about 60% fewer tokens on average and still worked on GPT 6.2 SOL and Grok 4.7.

Translated by AI

Originally published on 微信公众号

When powerful models like Fable 5.2 and Gemini 4 carbon come out, most people start by building a complicated frontend or a little game to see how far they can push them. For me, the most valuable first step is to use them to refactor the workflows I already have.

Yesterday, I asked Fable 5.2 to go through all the skills I’d accumulated in Hermes and suggest improvements. If you’ve spent enough time tinkering with agent systems, you probably have a pile of your own. The scan surprised me:

Serious token bloat: the top ten skills on the cleanup list averaged more than 10,000 tokens. The worst was over 30,000.

Defensive negatives everywhere: the skill at the top of the list had 293 instances of things like “don’t do X” and “never output Y.”

From first principles, these skill shitpiles are technical debt from the era of weaker models.

When the models weren’t smart enough, we’d add another prohibition every time we hit a bug. Over time, each skill became a bloated manual for keeping the model from screwing up. Once the underlying model gets much better, those hundreds of negative rules don’t just eat up context. They get in the way of what the model can do.

It’s like managing a top performer. Spell out a few hundred things they can’t do and you’ll limit what they can deliver. Tell them clearly what you want them to accomplish and they’ll do better work.

So I handed Fable 5.2 sixteen of my most-used, most bloated skills to refactor. The results were pretty clear:

  • Smaller: Average token use fell by about 60%.
  • A change in approach: Hundreds of negative instructions dropped to single digits. Nearly all were rewritten to say clearly what the skill should accomplish.
  • Better in use: Responses were noticeably faster. The output didn’t get worse; it got better and more consistent.

One detail from the test: Fable 5.2 did the refactoring, but GPT 6.2 SOL was what I actually ran in Hermes. I then switched to Grok 4.7, and all the slimmed-down skills still ran smoothly. To me, that suggests replacing defensive rules with clear, positive instructions isn’t a trick for one model. It’s a general principle for working with stronger models.

Writing this left me with a more interesting thought:

If models no longer need so much hand-holding, do some of the skills we spent so much time writing even need to exist anymore? What if the mountain of skills and prompts we’ve written over the past few years has already made it into the new models’ training data and become part of their intuition? 🐶🤔

Search