diff --git a/doc/OBSERVATIONS.md b/doc/OBSERVATIONS.md index 55e938e..78ad001 100644 --- a/doc/OBSERVATIONS.md +++ b/doc/OBSERVATIONS.md @@ -40,3 +40,5 @@ ``` This enables prompt pipelines where each stage transforms, filters, or augments the previous stage's output — all with standard shell syntax. + +**Tool schema complexity and model capability** — The execute package originally exposed a single `execute_code` tool with a unified schema covering inline steps, stdin-chained pipelines, named tool scripts, parallel fan-out, elevated execution, and sandbox selection. Larger models with strong reasoning handled this without difficulty. Smaller models struggled: common failure modes included emitting both `steps` and `pipe` in the same call, confusing the two execution modes, and ignoring named tool scripts in favour of reimplementing them inline. The tool was therefore split into three purpose-specific primitives — `execute_code` (inline code, `steps` key), `call_tool` (named scripts, `calls` key), and `pipe` (stdin-chained pipeline, `stages` key) — giving smaller models a narrow, unambiguous contract per tool. The `{parallel: [...]}` fan-out option is retained inside `execute_code` and `pipe` stages. The trade-off: a minor ergonomic regression for large models that handled the unified schema well, offset by significantly more reliable behaviour from smaller models.