We removed 80% of our agent’s tools

Link to headingWhat is d0

If v0 is our AI for building UI, d0 is our AI for understanding data.

d0 translates natural language questions into SQL queries against our analytics infrastructure, letting anyone on the team get answers without writing code or waiting on the data team.

When d0 works well, it democratizes data access across the company. When it breaks, people lose trust and go back to pinging analysts in Slack. We need d0 to be fast, accurate, and reliable.

Link to headingGetting out of the model's way

Looking back, we were solving problems the model could handle on its own. We assumed it would get lost in complex schemas, make bad joins, or hallucinate table names. So we built guardrails. We pre-filtered context, constrained its options, and wrapped every interaction in validation logic. We were doing the model’s thinking for it:

• Built multiple specialized tools (schema lookup, query validation, error recovery, etc.)

• Added heavy prompt engineering to constrain reasoning

• Utilized careful context management to avoid overwhelming the model

• Wrote hand-coded retrieval to surface “relevant” schema information and dimensional attributes

Every edge case meant another patch, and every model update meant re-calibrating our constraints. We were spending more time maintaining the scaffolding than improving the agent.

1
import { ToolLoopAgent } from 'ai';
2
import { GetEntityJoins, LoadCatalog, /*...*/ } from '@/lib/tools'
3
const agent = new ToolLoopAgent({
4
  model: "anthropic/claude-opus-4.5",
5
  instructions: "",
6
  tools: {
7
      GetEntityJoins, LoadCatalog, RecallContext, LoadEntityDetails, 
8
      SearchCatalog, ClarifyIntent, SearchSchema, GenerateAnalysisPlan, 
9
      FinalizeQueryPlan, FinalizeNoData, JoinPathFinder, SyntaxValidator, 
10
      FinalizeBuild, ExecuteSQL, FormatResults, VisualizeData, ExplainResults
11
    },
12
});

Link to headingA new idea, what if we just… stopped?

We realized we were fighting gravity. Constraining the model’s reasoning. Summarizing information that it could read on its own. Building tools to protect it from complexity that it could handle.

So we stopped. The hypothesis was, what if we just give Claude access to the raw Cube DSL files and let it cook? What if bash is all you need? Models are getting smarter and context windows are getting larger, so maybe the best agent architecture is almost no architecture at all.

Link to headingv2: The file system is the agent

The new stack:

• Model: Claude Opus 4.5 via the AI SDK

• Execution: Vercel Sandbox for context exploration

• Routing: Vercel Gateway for request handling and observability

• Server: Next.js API route using Vercel Slack Bolt

• Data layer: Cube semantic layer as a directory of YAML, Markdown, and JSON files

The file system agent now browses our semantic layer the way a human analyst would. It reads files, greps for patterns, builds mental models, and writes SQL using standard Unix tools like grep, cat, find, and ls.

This works because the semantic layer is already great documentation. The files contain dimension definitions, measure calculations, and join relationships. We were building tools to summarize what was already legible. Claude just needed access to read it directly.

1
import { Sandbox } from "@vercel/sandbox";
2
import { files } from './semantic-catalog'
3
import { tool, ToolLoopAgent } from "ai";
4
import { ExecuteSQL } from "@/lib/tools";}
5

6
const sandbox = await Sandbox.create();
7
await sandbox.writeFiles(files);
8

9
const executeCommandTool(sandbox: Sandbox) {
10
  return tool({
11
    /* ... */
12
    execute: async ({ command }) => {
13
      const result = await sandbox.exec(command);
14
      return { /* */ };
15
    }
16
  })
17
}
18

19
const agent = new ToolLoopAgent({
20
  model: "anthropic/claude-opus-4.5",
21
  instructions: "",
22
  tools: {
23
    ExecuteCommand: executeCommandTool(sandbox),
24
    ExecuteSQL,
25
   },
26
})

Link to heading3.5x faster, 37% fewer tokens, 100% success rate

We benchmarked the old architecture against the new file system approach across 5 representative queries.

The file system agent won every comparison. The old architecture’s worst case took 724 seconds, 100 steps, and 145,463 tokens before failing. The file system agent completed the same query in 141 seconds with 19 steps and 67,483 tokens, and it actually succeeded.

The qualitative shift matters just as much. The agent catches edge cases we never anticipated and explains its reasoning in ways we can follow.

Link to headingLessons learned

Don’t fight gravity. File systems are an incredibly powerful abstraction. Grep is 50 years old and still does exactly what we need. We were building custom tools for what Unix already solves.

We were constraining reasoning because we didn’t trust the model to reason. With Opus 4.5, that constraint became a liability. The model makes better choices when we stop making choices for it.

This only worked because our semantic layer was already good documentation. The YAML files are well-structured, consistently named, and contain clear definitions. If your data layer is a mess of legacy naming conventions and undocumented joins, giving Claude raw file access won’t save you. You’ll just get faster bad queries.

Addition by subtraction is real. The best agents might be the ones with the fewest tools. Every tool is a choice you’re making for the model. Sometimes the model makes better choices.

Link to headingWhat this means for agent builders

The temptation is always to account for every possibility. Resist it. Start with the simplest possible architecture. Model + file system + goal. Add complexity only when you’ve proven it’s necessary.

But simple architecture isn’t enough on its own. The model needs good context to work with. Invest in documentation, clear naming, and well-structured data. That foundation matters more than clever tooling.

Models are improving faster than your tooling can keep up. Build for the model that you’ll have in six months, not for the one that you have today.

If you’re building agents, we’d love to hear what you’re learning.

Learn more about building agents with a filesystem.