Skill drift in Claude Code: why your SKILL.md goes out of date
By Mike van BeaumondAI Engineer · company brain & automationpublished on
9 min read
Quick Take
Skill drift ontstaat niet doordat een SKILL.md kapot gaat, maar doordat niemand hem actief bijhoudt terwijl het project eromheen verandert. Een onderhoudsloop met scoring en een diff-review maakt zichtbaar welke regel een probleem had laten glippen.
Quick Take: Skill drift doesn't happen because a SKILL.md breaks, it happens because no one actively maintains it while the project around it changes. The file keeps working, it just becomes less and less relevant.
If you have written Claude Code skills, or any form of agent instruction file, you know they slowly drift away from reality. The skill that ran code review flawlessly six months ago now quietly misses the exact check that matters today. Not because anyone made a mistake, but because nobody audits a growing stack of Markdown files under deadline pressure.
That problem is called skill drift, and it is one of the quietest failure modes in agentic development. The skill still technically works, it just loses touch with how the project actually runs today. Why you keep the skill about the how and your company knowledge separate, read What are Claude skills?.
What is skill drift exactly?
Direct answer: Skill drift is the gradual ageing of an instruction file while the underlying project, process or policy keeps changing, without anyone updating the file.
You start with skills for code review, tests, migrations or API conventions. In the beginning everything works. The agent follows the instructions, the output is usable, so you keep expanding the library.
Then the project moves on and the skill files do not. A skill that was indispensable a few months ago now misses a critical check. That doesn't come from a deliberate decision, simply from the fact that manually auditing a growing stack of instruction files is exactly the kind of task nobody picks up when a deadline looms.
Why is this more dangerous than a normal bug?
A normal bug breaks visibly. Skill drift breaks invisibly. The agent keeps producing answers, the output looks plausible, and nobody notices a check is missing until the problem hits production.
- An authorisation check that was never written into the skill is never enforced, even when the project now handles more sensitive data.
- A test standard that is outdated lets the agent skip exactly the test cases that matter now.
- Nobody audits a skill file structurally, because it feels like documentation maintenance, which rarely tops a priority list.
Quick Take: The danger of skill drift isn't the mistake itself, it is the invisibility. A missing instruction doesn't look like a bug, it looks like silence.
How did teams handle this before, and how now?
Before
Someone noticed a missed check only after an incident, and then had to manually figure out which skill rule was missing. Recovery was reactive and usually cost more time than the original maintenance would have.
Now
A standing review loop scores every skill run and automatically signals when an instruction lags behind practice. The difference isn't extra tooling, it's that the check is baked into the workflow itself.
How do you build a maintenance loop for your skills?
n8n recently demonstrated a concrete approach built around cognee's memory layer that tackles the problem structurally instead of ignoring it until it hurts. The idea: run a skill against a real task, score the result, and if the score drops below a threshold, the system proposes a sharper rewrite including reasoning and a confidence score before anything is applied.
What makes this work is memory. Instead of treating every skill run as an isolated moment, a persistent memory layer connects earlier runs and skills, so every improvement is grounded in real history rather than a one-off guess.
The steps of the loop:
- Ingest: the current SKILL.md is loaded as the baseline for a review.
- Review: the skill runs against a real task and gets a score per dimension, such as coverage and correctness.
- Threshold check: if the run scores below the set bar the improvement path starts. If it scores above it, the loop stops here.
- Proposal: the system proposes a rewrite without applying it yet.
- Diff review: you see exactly what changes, including the reasoning and a confidence score, before approving anything.
- Apply: only after your approval is the change actually written back into the skill.
The most important design principle: a single weak run can never silently rewrite your instructions. There is always an approval moment between proposal and apply.
Quick Take: The power of a loop like this isn't auto-fixing skills, it is making visible exactly which rule would have let a problem slip through, with a diff you can review in seconds.
How do you apply this without building a whole new system?
In my experience this doesn't have to be a full cognee implementation to deliver value. The core principle, periodically scoring and consciously reviewing, can start much simpler inside your own Claude Code workflow.
A prompt to trigger this manually, before a team is ready for a fully automated loop:
Review this SKILL.md against the last three tasks it was used for.
For each section, indicate:
1. Which instruction still holds
2. Which instruction is missing given recent project changes
3. A concrete proposal for the missing rule, with reasoning
Return the answer as a table, not as prose.For teams who want to automate this structurally, including scoring, thresholds and an approval step, we build that loop visually in n8n: from trigger to diff to Slack notification, without having to maintain a pile of one-off scripts.
What does this maintenance loop not solve?
Reality check: an automated review loop is not a replacement for content ownership. A few things we always flag before a team starts.
- The system signals that a skill is falling short, but doesn't decide who is responsible for final approval. That stays human work.
- A proposed rewrite is only as good as the task it was tested against. Test against an unrepresentative task and you get an unrepresentative proposal.
- It doesn't solve an organisational problem if nobody is given time to actually review proposals. The bottleneck then sits at prioritisation, not at tech.
- It doesn't replace onboarding. A team that doesn't understand why a skill exists won't learn that by approving a diff.
My take: start with one skill your team truly relies on, for example code review, and build the habit of periodic scoring before you tackle the whole library. Want to know where your team has the most drift? Score your five most-used skills first. More on how I work lives on about Mike.
Bottom line
Bottom line: a SKILL.md isn't a document you write once and forget. It is just as maintenance-sensitive as the code it describes, only without the compiler errors regular code would give you.
No bullshit, just results: teams that take skill drift seriously don't try to build the perfect skill in one go. They build a habit of regular checking, reviewing a diff, and consciously approving before a change goes live.
Sound familiar? Let's talk.
In 30 minutes we look together at which team your brain starts with. No obligation.
Frequently asked questions
No. Any instruction file for an AI agent, whether a system prompt, a SKILL.md or an internal guideline, can age the moment the underlying process changes and nobody actively updates the file.
No. Start with the skill that is used most often or carries the highest risk, such as a security or authorisation check. Expand only once that first loop has proven its value.
Yes. A manual periodic review with Claude Code, like the prompt example above, already catches a large share of the drift. Automation with scoring and an approval gate becomes most valuable once you have many skills or a larger team.
Preferably the person with the content knowledge of why the original rule existed, not automatically the person who happens to have built the workflow.
That depends on the number of skills, your existing tooling and whether you choose a manual or automated approach. We prefer to discuss this concretely in a free intro call, so you don't get a ballpark that doesn't fit your situation.