A code generator can return many files from one tool call. That reduces the number of tool operations an agent has to request. Whether it also reduces model round trips depends on how the agent batches those operations.
I built a small harness to measure that distinction, and a second one: how much model output a generator needs when its input arrives as inline JSON or as a file produced by a short transformer script. The token totals below come from recorded claude -p result events. Message and tool counts are independently reconstructed from the original streams.
The setup
| Scenario | Generate a 71-entity domain model (aggregates, value objects, enums, services) as TypeScript from a JSON spec |
| Model / runtime | Claude Opus 4.7 through claude -p; Sonnet 4.6 for the cross-model rows |
| Server | @metaengine/mcp-server through npx, registered with claude mcp add |
| Judge | tsc --strict, plus a structural check for expected entities, kinds and selected members; no runtime behavior check |
| Prepared generation | A separate warm-up session writes a brief, which is included in the generation session's prompt |
| Date | 2026-04-26, one machine |
| n | 5 runs per cell; tables show the mean and [min–max], including failed judgments |
The baseline prompt requires the agent to use Write for each file. It still allows multiple tool calls in one assistant response. The assisted prompts prescribe one generation call, plus the reads and verification needed around it. These are comparisons of specified workflows, not an unrestricted competition between agents.
A message is not a tool call. Here, “assistant messages” counts distinct assistant message IDs in the stream. One can contain many tool requests. The recorded CLI num_turns happens to equal tool calls plus one in these runs; it does not measure model round trips. The message count describes visible responses, without claiming to count provider retries hidden from the trace.
Manual vs assisted
TypeScript, Opus 4.7, prepared generation, two runs in parallel.
Open the corrected measurements beside the table. ts-multilang/summary.md reports this experiment; each session also has a hash identifying its original stream.
| Arm | Assistant messages | Output tokens | Prompt-cache reads | Pass |
|---|---|---|---|---|
Manual — the agent writes each file with Write |
76.4 [73–82] | 21,007 [19,621–22,922] | 4.80M [4.43M–5.37M] | 5/5 |
| Assisted — one generation call, spec inline | 5.0 [5–5] | 18,872 [16,940–20,817] | 231k [227k–237k] | 4/5 |
| Warm-up session, counted separately | 8.2 [5–10] | 12,436 [11,274–14,776] | 317k [177k–395k] | — |
In this particular cell, Opus requested each tool operation in a separate response. The manual arm therefore carried a growing conversation through many more model responses. The assisted arm needed fewer responses and recorded fewer cache-read tokens. For a first use, add the warm-up: the assisted workflow emitted 31,308 output tokens on average, compared with the baseline's 21,007.
Java's baseline also used one operation per response: 76.6 [73–82] assistant messages against 5.0 [5–5]. Python batched some writes: 22.6 [6–74] messages against 6.6 [5–12]. A file-per-call tool does not force a model response per file.
Inline spec vs transformer + file
Same TypeScript task and Opus model, with five runs in parallel in each of three separate batches. These cells have different concurrency and warm-up briefs from the first table, so their absolute numbers should not be interchanged.
“Tool-input characters” counts a consistent JSON serialization of all tool arguments in the session. It is a content-size measure, not a tokenizer result or a count of network bytes.
| Arm | Assistant messages | Output tokens | Tool-input characters | Session duration | Pass |
|---|---|---|---|---|---|
| Inline JSON in the generation call | 5.0 [5–5] | 17,913 [16,154–21,021] | 34,689 | 146s [130–179] | 5/5 |
| Heredoc writes the JSON file | 6.4 [6–7] | 20,217 [19,234–22,377] | 45,587 | 175s [150–200] | 4/5 |
| Transformer writes the JSON file | 6.8 [6–10] | 4,691 [3,247–5,742] | 5,337 | 70s [54–94] | 4/5 |
The transformer arm emitted 74% fewer output tokens than inline and took about half the session duration in this sample. Moving literal JSON into a Bash heredoc did not produce that saving: the model still had to emit the JSON as tool input. A short program can instead read the source spec and produce the larger generator spec outside the model's output channel.
This is a plausible explanation supported by the smaller recorded tool inputs, not a complete causal decomposition. The prompts and generated programs differ, and the traces do not supply an exact split of output tokens into prose, tool arguments and thinking. Assistant-event usage cannot be treated as visible-text tokens.
The technique needs a spec that can be derived from a smaller source the agent already has. Hand-authoring every detail leaves less to transform. Tool arguments count toward generated output in Claude's API usage model; subscription billing is a separate matter. Claude tool-use pricing.
Does it survive a model swap?
TypeScript again, Sonnet 4.6, prepared generation, serial runs. The manual row comes from the inline comparison batch.
| Arm | Assistant messages | Output tokens | Pass |
|---|---|---|---|
Manual (Write per file) |
3.2 [3–4] | 16,280 [15,258–18,568] | 5/5 |
| Assisted, inline | 5.0 [5–5] | 21,299 [17,474–31,142] | 3/5 |
| Assisted, transformer + file | 7.0 [7–7] | 5,275 [4,075–6,530] | 5/5 |
Sonnet usually put all 71 Write requests into one assistant message. In the first baseline run, three messages cover reading the source, requesting the writes, and saying DONE. There are still 72 tool calls, but there is no 73-response model loop.
Across the two Sonnet batches, nine of ten baselines used only three or four assistant messages; the remaining run used 73. Their lower cache reads coincide with batched writes. These traces do not establish a cache-policy anomaly. The second batch's manual baseline is reported separately in the repository: 17.0 [3–73] messages and 16,936 [15,803–18,298] output tokens.
The transformer workflow still emitted fewer output tokens in both models. A reduction in model responses did not survive the swap: Sonnet's inline-batch baseline used fewer responses than either assisted arm. Tool operation count, response count and output volume answer different questions.
What this suggests for tool design
- Support multiple artifacts per call. It reduces required tool operations, and can shorten a workflow that otherwise uses sequential responses.
- Accept file input when a compact transformation can produce it.
load_spec_from_filegives the agent a way to run that transformation outside its output channel. - Measure the trace as well as the summary. Count responses and tool calls separately. Keep pass rates beside resource measurements.
What this setup does not show
Five runs per cell expose useful behavior, but give little precision about reliability or performance across workloads. The observed ranges are descriptive, not confidence intervals. A 68-entity modular-monolith variant adds one other shape; 200- or 1,000-entity specs are unmeasured. The judge checks compilation and selected structure, not equivalent runtime behavior. Failed judgments remain in the means, and the experiment does not measure the cost of repairing them.
Concurrency, prompts, warm-up briefs and provider load all affect comparisons. The prepared brief is a real input, not a simulation of a model trained on the API. New runs will use a changed service and tool environment; recomputing these historical measurements is distinct from repeating the experiment today.
Research note
A smaller initial tool description with detailed material loaded when needed may reduce onboarding context. That is a possible follow-up experiment. The runs above do not compare disclosure strategies, and none of their results depends on this idea.