A tool returns green. Status code 200. No errors. The agent moves on.
Something just went wrong, and nobody noticed.
Here’s the uncomfortable part: the tool call succeeded. It ran. It produced output. The status is green. The problem is that the tool’s definition of success and the agent’s definition of success are not the same thing, and most tool interfaces have no way to express the difference.
When a search tool returns three results for a query that has fifty relevant answers, the tool succeeded. When a file-scanning tool checks eight of twelve directories because the other four timed out silently, the tool succeeded. When a code analyzer flags three issues but the actual bug lives in a pattern it was never trained to recognize, the tool succeeded.
The tool reported what it did. It did not report what it didn’t.
The Receipt Is Not the Observation
There is a structural gap between a tool’s execution receipt and the agent’s observation needs. The receipt says: I ran, I finished, here is my output. The observation needs the shape of the space actually covered, plus the part of the question the output does not address.
Most tool interfaces are designed for the receipt. They are not designed for the observation.
Consider a vulnerability scanner. It finishes its run and returns a clean bill of health. The agent trusts the result. What the scanner’s success report does not tell you: the scanner’s rule set covers CVE-2023 patterns, not CVE-2024 patterns, and the vulnerability in question lives in a novel class of exploit the scanner has never seen. The scanner is not lying. It is telling the truth about what it checked. The problem is that “what it checked” is a proper subset of “what the agent needed checked,” and the interface has no field for that gap.
The failure lives in interface design, not in the tool itself.
The Completeness Problem
The issue maps directly to a problem in formal verification: negative claims require coverage certificates. If I claim “there are no vulnerabilities in this code,” the claim is only meaningful alongside a specification of what was tested, how thoroughly, and where the testing stopped. Without the coverage certificate, the negative claim is just silence wearing a lab coat.
Tool calls in agent systems face the same problem. Every tool that produces a result is implicitly making a claim about the scope of that result. Most tools omit the scope specification entirely.
Tool contracts need to be honest before tools need to be smarter.
Three States, Not Two
Current tool interfaces are binary: success or failure. A green status or a red status. Nothing in between. But there is a third state that agents need: completed with incomplete coverage.
Here is what a three-state tool interface looks like:
Full coverage: The tool’s declared scope was exhausted. Every path the tool knows how to check was checked. The result is reliable within the tool’s known domain.
Partial coverage: The tool’s declared scope was not fully reached. Some paths were checked; others were not. The result is reliable for the checked portion and unknown for the rest. The tool must specify which portion.
Unknown coverage: The tool ran, but it cannot determine what fraction of the question its output addresses. This happens when the tool’s internal state is opaque to the caller: a black-box model, an external API with no coverage metadata, a tool that does not track its own boundaries.
The critical design choice: partial coverage is not a failure. It is a different kind of success. The tool tells the agent exactly where the boundary lies, and the agent decides whether to seek another tool, accept the gap, or escalate to a human.
What the Contract Should Look Like
A tool call response should include at minimum:
-
What was done. The standard output field. No change here.
-
What was not done. This is the new field. If the tool searched three of five data sources, list the two it skipped and why. If the tool analyzed eight of twelve directories, name the four and the reason (timeout, permission, format incompatibility). If the tool’s model has never seen the pattern in question, say so.
-
Coverage confidence. A statement about the tool’s own assessment of how much of the declared scope was actually reached. Not a percentage. Percentages create the illusion of precision where none exists. A qualitative assessment: all declared paths checked, some paths not reached, or scope boundary unknown.
-
Failure mode taxonomy. When something goes wrong, what kind of wrong was it? Did the tool encounter an error (red status), or did it encounter a gap (partial coverage)? These require different responses. An error means retry or fix. A gap means supplement or escalate.
This is not optional metadata. It is what separates an agent that reasons about tool output from one that merely trusts or distrusts it.
The Epistemic Debt of Missing Coverage
When a tool omits its coverage boundary, the agent inherits a debt it did not take out. The agent has to decide: do I trust this result, or do I run another tool to verify? Without coverage metadata, every tool call forces the agent to make this choice blindly.
Over time, agents develop a habit of running redundant tool calls. Not because they need to, but because they cannot tell if a single call was sufficient. This is the coordination tax of agentic workflows in miniature: each missing coverage statement generates a retry, and each retry generates another missing coverage statement, and the loop compounds.
The solution is structural, not behavioral. Fix the tool contract, and the agent stops needing to hedge.
The Hardest Part: Absence-Producing Tools
Some tools are designed to produce absence as output. A security scanner that finds nothing. A test suite that passes every test. A monitoring check that returns all green.
These are the most dangerous tool calls in an agent system, because the output is the absence of evidence, and the absence of evidence is not evidence of absence. The tool needs to certify that the evidence space was fully covered.
An absence-producing tool without a coverage certificate produces silence. Silence is not a result.
The Agent Hiring Metaphor
Think of it this way: if you hired a contractor to inspect a building, and they came back and said “I inspected it,” you would ask: which floors? which rooms? did you check the basement? what did you skip? If they say “I didn’t skip anything,” you would ask: what does “anything” include?
We accept this questioning for contractors because we understand that inspection has a scope, and scope has boundaries. We do not accept it for tools because tool interfaces have trained us to think of a green status as “the job was done.”
The job was not done. The tool ran. Those are different things. An agent system needs interfaces that reflect the difference.
The fix is not to distrust tools. It is to demand that tools tell the truth about the shape of the work they actually completed. A tool that reports its own gaps is honest. In an agent system, honesty about gaps beats false completeness every time.