← Back to writing

Tool Schema Drift Agentic

By TaeHo Kim (Hannune)  ·  Published July 21, 2026

Tool Schema Drift Agentic

The most common agentic system failure I encounter is not a bad prompt. It is not a context overflow. It is a tool that changed without its registration changing.

Most teams maintain tool schemas in a registry separate from the implementation. The `search_entities` function gets its description written once at setup and rarely revisited. Six months later, the function has a new required parameter and returns a different response shape, but the registry still describes the old interface. The model calls the tool with the old parameter pattern and gets back a response it was not told to expect.

This fails silently. The agent sees a response, generates text from it, and moves on. There is no exception. The output just drifts.

Three checks that catch this before it reaches production:

Response-side schema validation, not just call-side. Most frameworks validate that the model produced a valid tool call format. Fewer validate that the tool returned a response matching what the model was told to expect. Both directions matter.

Version the tool description alongside the implementation. If a tool changes its interface, the old description version stays in the registry for agents that depended on it. Breaking changes get a new tool name, not a silent update to the existing one.

Canary evals that cover the full call-response cycle, not just output quality. A single eval prompt that triggers the tool is enough. If the tool response shape changes, the eval breaks before production does.

The agent is not wrong when it mishandles an unexpected tool response. It was not told the tool changed. The registration is the contract. Treating it as one means versioning it like one.

Building something similar?

Get in touch →