From abdc4f2146f6c61b0034ba21b9ef853d81cbcca4 Mon Sep 17 00:00:00 2001 From: Ollie Agent Date: Wed, 29 Jul 2026 00:33:06 +0200 Subject: [PATCH] rewrite lazy loading observation: error rates improved after facade layer --- doc/OBSERVATIONS.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/OBSERVATIONS.md b/doc/OBSERVATIONS.md index 39b88f1..1bed306 100644 --- a/doc/OBSERVATIONS.md +++ b/doc/OBSERVATIONS.md @@ -8,7 +8,7 @@ **Lazy tool loading** — After transitioning from `execute_code`/`call_tool`/`pipe` to a unified `shell` primitive, ollie adopted a lazy loading architecture. The system has 7 primitives: `shell` (sandboxed), `tool_load`/`tool_list`/`tool_active`, `skill_load`/`skill_list`/`skill_active`, plus associated 9P filesystem endpoints. Tools are executable scripts discovered from `OLLIE_TOOLS_PATH`; skills are `SKILL.md` directories discovered from `OLLIE_SKILLS_PATH`. Both use session-scoped registries with per-session load tracking. The agent autonomously loads tools and skills as needed — no explicit user permission required. -**Why lazy loading works** — Smaller/cheaper models that struggled with the `execute_code`/`call_tool`/`pipe` abstraction benefit from having tools as first-class callable functions with JSON schemas. Larger models that reasoned well with the old approach are unaffected. The trial-and-error overhead of wrong tool names, incorrect argument formats, and failed invocations — which consistently plagued smaller models — is eliminated when tools appear as native function calls. This compromise preserves the simplicity of a small core while supporting a rich ecosystem of specialized capabilities. +**Error rates significantly improved after the facade layer** — The original architecture exposed `execute_code`/`call_tool`/`pipe` as the only primitives. Larger models with strong reasoning handled this well. Smaller and cheaper models consistently failed: wrong tool names, incorrect argument formats, confused execution modes, ignoring named tool scripts in favour of reimplementing them inline. The facade layer — `tool_load`, `tool_list`, `tool_active` — promotes a tool script to a first-class callable function with a JSON schema. Error rates dropped dramatically because the model no longer needs to construct the correct invocation string; it calls a native function with typed arguments. The tool scripts themselves are unchanged — they are still executable files on disk. The facade is purely a contract layer between the agent and the tool ecosystem. This preserves the simplicity of a small core while supporting a rich ecosystem of specialized capabilities, and it costs nothing for models that already reasoned well through the abstraction. **Parallel execution** — `execute_code` accepts a `steps` array of pipeline stages. A `{parallel: [...]}` stage fans out concurrently within the pipeline. Sequential stages chain stdout → stdin. All stages accept inline code and named tool scripts interchangeably.