# Architectural Evolution ## AnviLLM: Research That Led to Ollie Before Ollie, there was [`anvillm`](https://git.lneely.de/lkn/anvillm): the working laboratory in which the filesystem-as-agent idea first became concrete. Beginning on February 9, 2026, AnviLLM orchestrated Claude Code, Kiro, and Ollama sessions in `tmux`, exposing session state, control, output, and inter-agent communication through a 9P namespace. It answered the first question: **can independent agent CLIs be made to cooperate through ordinary file operations?** Yes—with caveats. AnviLLM was deliberately exploratory. Its history records the path from a monolithic client to the `anvilsrv` daemon and Assist client; from polling and process inspection to hooks, events, and streaming reads; from ad-hoc prompts to fire-and-forget workflows, roles, a supervisor, and a conductor; and from simple messages to mailboxes, beads, and cross-backend coordination. It also tried several frontends—Acme, Emacs, a web interface, and a curses TUI—and proved that a 9P surface lets each of them remain replaceable. That experiment left durable lessons. A filesystem is a powerful integration surface: `read`, `write`, pipes, and blocking streams compose without a special-purpose orchestration protocol. But wrapping external CLIs makes the runtime depend on backend-specific hooks and fragile process heuristics; even “running” versus “idle” cannot be inferred uniformly. A 9P socket is not an authentication system: Unix ownership and permissions must carry the boundary. State, recovery, cancellation, and message delivery must be explicit rather than inferred from terminal behavior. And orchestration belongs outside the agent core, where scripts and clients can evolve independently. Ollie is the continuation of that research. It keeps AnviLLM's strongest result—the 9P namespace as the API—while moving the agent loop, backends, tools, sandbox, persistence, and context management into a runtime designed for direct control. AnviLLM established what the filesystem surface could unlock; its limitations clarified what the runtime must own. This log begins with that lineage. ## How the Ollie system-of-systems emerged and evolved. This is a study log. It records what was built, what was killed, and why — in the order it happened. The dead ends are as instructive as the survivors: the rejected ideas at the end teach more about building agent systems than the architecture that remains. The document evolves with the author's understanding; when past decisions turn out to be wrong, they get recorded here, not hidden. ## Timeline ```mermaid gantt title Ollie Evolution dateFormat YYYY-MM-DD axisFormat %b %d section Foundation Monorepo + submodules :done, 2026-04-11, 14d 9P filesystem :done, 2026-04-14, 28d section Core Primitives Session lifecycle (session/) :done, 2026-04-20, 30d Tool definitions :done, 2026-05-01, 20d Skills system :done, 2026-05-10, 14d section Execution execute_code / shell :done, 2026-05-08, 20d Batch jobs (b/ → session/bfg) :done, 2026-05-15, 25d Sandbox (native Landlock) :done, 2026-05-20, 40d section Multi-Agent Subagent spawn :done, 2026-05-25, 15d Cascade orchestrator :done, 2026-06-05, 20d JIT agent generation :done, 2026-07-10, 10d section Frontends TUI + Emacs (ellie) :done, 2026-04-15, 45d Acme (Plan 9) :done, 2026-05-20, 30d KDE/Plasma :done, 2026-06-10, 40d section Control Planes D-Bus adapter (ollied) :done, 2026-06-25, 20d D-Bus embedded in srv :done, 2026-07-15, 10d D-Bus removed :done, 2026-07-31, 1d section Late Evolution Remote execution :done, 2026-07-05, 15d Bypass broker :done, 2026-07-10, 15d Tool registry (lazy) :done, 2026-07-20, 10d FUSE → 9P client :done, 2026-07-25, 5d The Great Flattening :done, 2026-07-29, 2d 9P Declarative EDSL :done, 2026-08-02, 1d Zero built-in tools :done, 2026-08-02, 2d EDSL extraction + cleanup :done, 2026-08-03, 2d Good idea fairy cleanup :done, 2026-08-05, 1d toolsrv 9P + process isolation :done, 2026-08-10, 2d Meta-only tool definitions :done, 2026-08-11, 2d Bypass via 9P :done, 2026-08-11, 1d fs/ flattening :done, 2026-08-11, 1d Feed + observer agents :done, 2026-08-13, 1d BlockOnce/Stream refactor :done, 2026-08-13, 1d Sub-agents via 9P rdwr :done, 2026-08-14, 1d Code intelligence tools :done, 2026-08-15, 1d Go file tools + cleanup :done, 2026-08-16, 1d OptMem persistent memory :done, 2026-08-16, 1d Markdown parsing + output cap :done, 2026-08-16, 1d Goals + conductor workflow :done, 2026-08-17, 1d Split session/agent indexes :done, 2026-08-17, 1d Agent peers + consensus :done, 2026-08-19, 1d Embedding-guided discovery :done, 2026-08-20, 1d One-tool bootstrap discovery :done, 2026-08-21, 1d Explicit tool loading :done, 2026-08-21, 1d Streaming rdwr + event filters :done, 2026-08-22, 1d Bypass coordination + state UI :done, 2026-10-06, 1d JSONL chat log format :done, 2026-10-07, 1d Live chat stream (chat.raw) :done, 2026-10-10, 1d Tool/bypass text visibility :done, 2026-10-10, 1d ``` ## Current Size (Oct 6) | Component | Lines | Notes | |-----------|------:|-------| | Go core (excluding backends) | ~22,000 | Agent, session, tools, 9P | | Compiled tools (tools/) | ~3,000 | Code intel, file, LSP, web | | KDE integration (kde/) | ~15,700 | GUI, Kate, KRunner | | Script tools (data/tools/) | ~2,600 | Python/Bash tools | | CLI scripts (data/scripts/) | ~960 | `o` wrapper and helpers | | **Core runtime** | **~25,000** | Go core + compiled tools | | **With KDE** | **~40,700** | Core + KDE integration | Agent profiles: 14. The interesting logic is ~25K of core runtime plus ~15.7K of KDE integration. Backend adapters (~8.3K), prompts, skills, and docs are excluded from this count. ## Phase 1: Monorepo Bootstrap (Apr 11) Started as independent git repos unified under a monorepo with submodules. Initial components: - **agent** — agent loop (Go): backend dispatch, tool execution, session state - **fs**** — 9P file server exposing agent sessions as a synthetic filesystem - **tui** — terminal UI (later removed) - **el** — Emacs integration (ellie.el) Build system was `mkfile` (Plan 9 make), later `Makefile`, finally `justfile`. ## Phase 2: The 9P Decision (Apr 14–28) The defining architectural choice: **every agent primitive is a file**. Sessions are directories. Sending a prompt is writing to `session/{sname}/prompt`. Reading state is `cat session/{sname}/state`. This made the control plane protocol-agnostic — any program that can read/write files can be a frontend. Key files crystallized: `prompt`, `chat`, `state`, `ctl`, `cfg`, `statewait`. The `session/new` file (write key=value pairs, read back session ID) became the session factory. ## Phase 3: Tool System (Apr 28 – May 10) Tools evolved through several incarnations: 1. **MCP servers** (denote-mcp, 9beads-mcp) — external processes, heavyweight 2. **Shell/Python scripts** in `~/.config/ollie/tools/` — lightweight, sandboxed 3. **execute_code** as the single built-in tool — all scripts invoked through it 4. **call_tool/pipe** — named tool dispatch and cross-tool pipelines 5. **Tool registry** (final form) — dynamic lazy-loading via `tool_list`/`tool_load`/`tool_active`; `execute_code` renamed to `shell`; tools hot-reloadable without server restart The file tools (`file_read`, `file_write`, `file_edit`, `file_grep`, `file_glob`) were extracted into standalone Python scripts. Memory (`memory_remember`, `memory_recall`) and reasoning (`reasoning_think`) followed the same pattern. The memory tools (`memory_remember`, `memory_recall`) now call OptMem directly. OptMem owns the persistent B-tree at `$XDG_DATA_HOME/ollie/optmem` (default: `~/.local/share/ollie/optmem`); Ollie does not maintain a parallel memory directory or memory-file format. This keeps reads and writes on the same store and avoids competing writers. MCP servers were removed early. Simple scripts won over complex daemons; the OptMem-backed tools retain the lightweight tool interface without duplicating storage. ## Phase 4: Planning System Churn (May – Jul) Planning went through the most iterations of any subsystem: 1. `pl/` directory namespace with flat file-per-task 2. `plan_create` / `plan_complete` tools 3. **Beads** integration (external issue tracker) — made optional, then dropped 4. `task_add` / `task_check` tools 5. Inline markdown plans forbidden 6. **Final form**: a single `session/{sname}/plan` file (markdown checklist), written by the agent, persisted across context compaction The lesson: planning needed to be simple, agent-controlled, and not bureaucratic. ## Phase 5: Multi-Agent Coordination (May 25 – Jun 15) Three patterns emerged: - **subagent_spawn** — fire-and-forget session creation, parent never blocks - **subagent_generate** — JIT agent config generation (role, constraints, tools) - **cascade** — script-driven fan-out with `-max-workers`, `-retries`, optional synthesis step ```mermaid flowchart TB UP["User prompt"] PARENT["Parent Session"] W1["Worker 1"] W2["Worker 2"] W3["Worker 3"] CASCADE["u/cascade\n(spawn + throttle)"] CW1["Cascade Worker"] CW2["Cascade Worker"] UP --> PARENT PARENT -- subagent_spawn --> W1 PARENT -- subagent_spawn --> W2 PARENT -- subagent_spawn --> W3 W1 --> PARENT W2 --> PARENT W3 --> PARENT PARENT -- cascade --> CASCADE CASCADE --> CW1 CASCADE --> CW2 ``` Inter-agent communication uses the filesystem: agents write to each other's `prompt` file. ## Phase 6: Frontend Proliferation ```mermaid flowchart TB subgraph Core["agent.Agent"] AG["Agent Engine"] end subgraph Surfaces["Integration Surfaces"] P9["9P Filesystem\n(session/ namespace)"] DB["D-Bus\n(org.ollie.SessionManager)"] end subgraph Frontends SH["s/sh (terminal)"] ACME["acme (Plan 9)"] EL["ellie (Emacs)"] KG["KDE GUI"] KP["KDE Plasmoid"] KK["Kate Plugin"] KR["KRunner"] HTTP["curl / scripts"] end AG --> P9 AG --> DB SH --> P9 ACME --> P9 EL --> P9 HTTP --> P9 KG --> DB KP --> DB KK --> DB KR --> DB WEB --> DB ``` - **s/sh** — shell script frontend (bash, briefly rc, back to bash) - **acme** — Plan 9 editor integration with workspace-scoped navigator sessions - **ellie.el** — Emacs with ghost-text completion - **ollie-kde** — full Plasma integration: plasmoid, KRunner, Kate plugin, standalone GUI, GUI automation tools - **TUI** — removed (Jul) in favor of s/sh and richer GUIs ## Phase 7: Security Model (May – Jul) ```mermaid flowchart LR subgraph Permissions["9P Permissions"] OWNER["Owner → rw"] AGENT["Agent → restricted"] PEERS["Peers → read-only"] end subgraph Execution["Execution Sandbox"] AGT["Agent"] SANDBOX["native Landlock sandbox\n(restricted fs)"] TOOLS["Tool definitions"] BYPASS["x/bypass\n(socket broker)"] PRIV["Privileged Action"] end AGT --> SANDBOX --> TOOLS TOOLS -- needs escape --> ELEV --> PRIV ``` Evolution: 1. No sandboxing initially 2. YAML-based native Landlock sandbox configs for execute_code 3. Per-session 9P identity — agents can't read other sessions' tools 4. Bypass broker: socket-based, user-confirmed privilege escalation 5. SSH agent proxy added then removed (too much attack surface) 6. Final: integrated bypass broker replaces separate superpowerd adapter ## Phase 8: D-Bus as Second Control Plane (Jun 25 – Jul) Originally everything was 9P-only. D-Bus was added for desktop integration: 1. Separate `ollie-dbus` daemon (ollied) 2. Embedded directly into `olliesrv` (9P server got `-no9p` flag) 3. KDE uses D-Bus exclusively; acme/sh/el use 9P 4. **9P and D-Bus do not share sessions** — independent stores, same core ## Phase 9: Remote Execution (Jul 5–19) Split-brain architecture: orchestration stays local, code execution on a remote machine via SSH. - `ollie-remote` binary auto-deployed via SSH bootstrap - Tools embedded in remote binary - Streaming output notifications back to local session - Session persistence across remote restarts ## Phase 10: Tool Registry (Jul 20–29) The final major architectural shift — from "agent knows all tools upfront" to lazy discovery: 1. Tools embed their own `ollie:prompt` metadata blocks 2. `tool_list` — discover available tools and descriptions 3. `tool_load` — promote a tool to a native callable 4. `tool_active` — introspect loaded tools 5. Skills get the same treatment: `skill_list`, `skill_load`, `skill_active` This solved system prompt bloat — only load what's needed per task. ## Phase 11: The Great Flattening (Jul 29–30) Largest single-day structural change: **−4,573 lines net** across the codebase. Eliminated accumulated abstractions that no longer served a purpose. ### Core (−4,400 lines) - **Killed `agent.Core` interface** — concrete `*agent.Agent` used directly. Nobody else implemented Core; the interface just added indirection. - **Killed `Dispatcher` indirection** in toolsrv — `toolsrv.Server` called directly. - **Merged `execute/` into `tools/`** — separate package for 3 functions was pointless overhead. - **Renamed for clarity**: `agent.Session` → `agent.History`, `session.Core` → `session.Session`, `NewAgentCore` → `New`. - **Extracted `agent/` package** from the monolithic session package. - **Threaded context.Context** properly through daemon → session → agent (replaced ad-hoc interrupt mechanisms with cancellation). - Dead code removal: `GlobalToolNames`, `ExtractReturnSchema`. ### 9P server (−214 lines) - **Killed `mgr/` package entirely** — replaced `Manager` struct with package functions in `**fs**/`. The `*fs.Tree` IS the session collection; `rootState` (unexported) lives in `tree.Data`. CRUD via `NewRoot`, `Lookup`, `Create`, `Kill`, `Rename`, `Shutdown`. - **Fixed infinite recursion** in `session/{sname}/agent/` directory — `pathType()` didn't recognize agent subdirectories, causing walk to succeed at any depth. - **Fixed phantom session descent** — walk into non-existent sessions now fails immediately. - **Fixed `/agents` empty read** — `makeStat` wasn't computing Length for root files, FUSE kernel saw 0 bytes and never issued a read. - **Fixed `session.All()`** — was looking at `root.Children()` (empty) instead of `rootState.sessions`. Broke D-Bus ListSessions and GUI session restore. ### Frontends - **acme**: paths updated `s/` → `session/`, agent files route through `session/{sname}/agent/{aname}/`. - **ellie.el**: same path migration, added `ellie--agent-dir` for agent ID discovery. - **KDE GUI**: no changes needed (D-Bus operates on Go objects, not paths). ### New filesystem layout ``` session/ ├── new write key=value to create session ├── idx session index (tab-separated) ├── {sname}/ │ ├── plan session-scoped markdown checklist │ ├── env session environment │ └── agent/ │ └── {aname}/ │ ├── cfg session configuration (key=value) │ ├── ctl control commands │ ├── state current state │ ├── statewait blocks until state changes │ ├── chat conversation log │ ├── prompt submit prompts │ ├── prompt.prev last submitted prompt │ ├── offset byte offset after last user prompt │ ├── fifo.in queue a prompt (FIFO input) │ ├── fifo.out pop queued prompt (FIFO output) │ ├── context rendered context window │ ├── systemprompt rendered system prompt │ ├── usage token stats │ ├── cost cost estimate │ ├── ctxsz context size │ ├── models available models │ ├── tools tool management │ ├── toolsrv.loaded currently loaded tools │ ├── toolsrv.rev revision counter │ ├── tail exec helper │ └── proc/ detached process output ``` ### Architecture after (single Go module, no submodules for core/9p) ``` *fs.Tree (root) ← IS the session collection └── .Data = *rootState ← sessions map, config, nextUID └── sessions[id] = *Session ← owns Agent, context, cancel Package functions (no Manager): session.NewRoot(cfg) → *fs.Tree session.Lookup(tree, id) → *Session session.All(tree) → []*Session session.CreateFromRoot(tree, args) → (id, error) session.KillFromRoot(tree, id) session.RenameFromRoot(tree, old, new) session.Shutdown(tree) ``` ### SLOC after flattening | Component | Lines | |-----------|-------| | agent + backend + toolsrv + session | 10,437 | | **fs** + cmd/olliesrv | 7,925 | | kde gui | 12,331 | | **Total** | **32,538** | Zero dead exported functions remain (verified via LSP + grep across all repos). ## Phase 12: Second Flattening & .meta Sidecar (Jul 30) Another aggressive structural pass. Eliminated the entire script namespace layer (s/, u/, x/), flattened `contrib/` into `data/`, and replaced header-comment-based tool metadata with JSON sidecar files. ### Scripts removed (−1,100 lines) - **s/{sh,b,bfg,bbg,ls,kill,cleanup}** — shell-based session frontends replaced by `ollie-9p rdwr generate/complete/route` and GUI frontends - **u/{optimize,complete,cascade,escalate}** — utility scripts replaced by `generate`, `complete`, `route` files in the 9P namespace - **x/{bd,prime,freeloader,task}** — internal plumbing scripts; PATH prepend (`prependOlliePath`) removed from toolsrv - 9P session root trimmed to just `new` and `idx`; script-serving code removed - `install-scripts` target now empty ### Tool metadata: .meta sidecar files Header comment parsing (`ollie:prompt`, `ollie:tier`, `ollie:parallel read`, `args_json:`) **eliminated entirely**. All tool metadata now lives in a JSON sidecar file alongside the executable: ``` tools/ file_edit ← executable (any language) file_edit.meta ← JSON metadata ``` ```json { "description": "Replace text in a file.", "prompt": "## file_edit\n\n...", "args": {"type":"object", ...}, "tier": "cold", "readOnly": true } ``` This decouples metadata from implementation language — compiled Go binaries, Python scripts, and bash tools all use the same discovery mechanism. ### LSP tools ported to Go The Python LSP bridge (`_lib/lsp/`) replaced with a pure Go implementation: ``` tools/ ├── builtin/ ← in-process handlers (shell, reasoning, tool/skill registry) └── lsp/ ← shared LSP client library └── cmd/ ← individual binaries (lsp_definition, lsp_hover, ...) ``` Architecture: each LSP tool binary embeds a bridge daemon (started on first invocation via `--bridge` flag, persists via Unix socket, auto-exits after 5min idle). No Python. No `_lib`. No external dependencies beyond the LSP servers themselves (gopls, clangd, intelephense). ### Repo reorganization - `contrib/{prompts,scripts,services,tools,skills,agents}` → `data/` - `contrib/elisp` stays (Emacs frontend contribution) - `tools/builtin/` — built-in tool handlers (moved from `tools/`, removed in Phase 18) - `tools/lsp/` — Go LSP implementation (replaced `data/tools/_lib/lsp/`) ### Conditional variants for network transparency Tools can declare multiple variants gated by host conditions (`binary`, `file`, `os`, `arch`, `env`, `nenv`). First match determines the tool's schema and executable. Enables one `.meta` to work across heterogeneous hosts — the contract adapts to capabilities. Critical for `ollie-remote` deployments where the remote host may have different tools, package managers, or init systems. ## Principles That Emerged 1. **Filesystem-as-API** — everything is read/write on synthetic files. No custom protocols needed. Monorepo coordinates versions. 3. **Scripts over servers** — MCP servers removed early. Simple scripts won. 4. **Progressive disclosure** — lazy tool/skill loading keeps system prompts small until complexity is needed. 5. **Plan simplicity** — after 5 iterations, planning settled on one markdown checklist file per session. 6. **Single control plane — 9P for everything. Streaming via blocking reads. No polling. 7. **Security by default** — sandboxed execution with explicit bypass, per-session identity. ## Phase 13: 9P Streaming & D-Bus Removal (Jul 31) The D-Bus adapter (`org.ollie.SessionManager`) is deleted. All clients now use 9P exclusively via blocking reads for natural streaming. Key changes: - **`chat` file**: blocking read that delivers tokens as the agent produces them. Per-fid offset. Never EOF (blocks between turns). EOF only on kill. - **`log` file**: replaces old `chat`. Returns last 64KB (sliding window). Non-blocking, tail-able via Qid.Vers. - **`kill`/`.` ctl command**: kills session → EOF on chat readers. - **KDE GUI**: rewritten from scratch. Uses `9p` (plan9port) via QProcess. Streaming via `readyReadStandardOutput`. Plain text tail (8KB). No ChatBlockModel, no D-Bus, no ThemeManager. ~250 lines total. - **Kate plugin**: all D-Bus calls replaced with `9p` subprocess calls. Streaming chat + statewait via persistent QProcess. - **KRunner**: uses `9p` for session listing and one-shot generation. - **Acme frontend**: uses 9fans.net/go plan9/client directly. Blocking read loop on `chat`. No FUSE mount. - **Plasmoid, tray**: deleted (not useful enough to maintain). - **dbus/ package**: deleted (-771 lines from server). The only remaining godbus usage: `org.freedesktop.Notifications` for bypass prompts (desktop integration, not ollie protocol). Lessons: - 9P's request-response model gives natural streaming for free. Server delays Rread until data arrives. No polling needed. - FUSE mounts do NOT support blocking reads (return EOF immediately). Clients must use the 9P protocol directly. - Never bind a growing string to a QTextArea with word-wrap. QTextDocument relayout is O(n) on the full content. - The `loadEarlier` scroll-triggered cascade was creating 500 delegates on startup. Bounded initial window + explicit scroll-back is correct. ## Phase 14: Multi-Agent Deepening — ID/Name Split & Empty Sessions (Aug 1) The multi-agent data structures introduced during the Great Flattening required further refinement to support true multi-agent sessions. ### ID/Name Split Sessions and agents now have **two identities**: - **`id`** (immutable UUIDv4, set at creation, never changes) - **`name`** (mutable human-readable string, defaults to first UUID segment) The sessions map is keyed by Name, not ID. Renaming a session or agent moves the directory in the 9P namespace — `mv` on the session directory calls `RenameFromRoot`, and `write` to the agent's `name` file calls `SetName`. Both entities expose `id` and `name` files in their 9P directories. ### Two-Step Session Creation Creation split into two phases: 1. `session/new` — write `name=` to create an empty session (no agent yet) 2. `session/{name}/agent/new` — write `cwd= backend= model= agent=` to create the agent This enables creating sessions in advance and attaching agents later. Empty sessions (`Core=nil`) are valid — they appear in `session/idx` with empty fields and are fully killable/renameable. ### Multi-Agent Data Structures - `session.Session` supports multiple agents (`Agents()` returns `[]*agent.Agent`) - `Session.AgentLogs` maps agent IDs to `*AgentLog` instances - Active agent is selectable, each agent has its own prompt/chat/state/log - `session/idx` emits one line per agent, not one line per session Key commits: 92 commits across Aug 1 touching session/, agent/, fs/, and kde/. ## Phase 15: 9P Declarative EDSL (Aug 2 — morning) The most significant structural change to the 9P server since its inception: **replaced the imperative filesystem with a declarative EDSL**. ### Before: Imperative Every 9P operation (stat, walk, read, write, open, create, remove) was hand-coded per node. The `fs/session/` sub-package contained ~3,100 lines of manual tree-building code split across 6 files (`files.go`, `root.go`, `perm.go`, `synth.go`, `create.go`, `persist.go`). Adding a new file meant updating stat, readdir, open, and permission code paths. ### After: Declarative EDSL The entire namespace is declared in a single `FsNodeDecl` tree in `fs/spec.go`: ```go var treeSpec = Dir("/", Leaf("backends", 0444, Read(readBackends), GID("agent")), Leaf("event", 0444, Read(readEvent), Stream(blockEvent), GID("agent")), Leaf("complete", 0666, Request(requestComplete), GID("agent")), // ... Dir("session", GID("agent"), Leaf("new", 0666, Request(requestSessionNew)), Leaf("idx", 0444, Read(readSessionIdx)), TemplateDir("{id}", listSessions), ), ) ``` `BuildTree()` in `fs/builder.go` walks the spec, validates invariants (directories can't have leaf handlers, at most one blocking variant, template nodes must have `List`, ownership inheritance), and produces a fully-wired `*fs.Tree`. ### Package consolidation - Entire `fs/session/` sub-package flattened into `fs/` — 6 files deleted - `cmd/olliesrv/bypass_tree.go` — bypass tree moved to `fs/bypassfiles.go` - `cmd/olliesrv/server.go` simplified from ~820+ lines to ~200 lines of thin 9P protocol handling; all filesystem logic is now in `fs/` - Handler files organized by scope: `rootfiles.go`, `sessionfiles.go`, `agentfiles.go`, `bypassfiles.go`, `procfiles.go` ### Net result - **−3,152 lines** deleted, **+2,086 lines** added across 24 files - Adding a file = adding one `Leaf()` call. No manual stat/readdir/write. - Validation at `BuildTree` time catches structural errors before serving. - Permission model inlined in the spec via `mode` and `GID()` — no separate `perm.go`. ## Phase 16: 9P Server Streamlining (Aug 2 — afternoon) Following the EDSL conversion, the 9P protocol server itself (`cmd/olliesrv/server.go`) received a focused cleanup over 13 commits: ### Fid management - **Server owns the fid map**: validates newfids (not already in use, not stale), garbage-collects orphaned fids. No more leaking fids on client disconnect. - **Store opened file on fid**: read/write use `f.entry` directly instead of re-resolving the path on every operation. Eliminates a class of TOCTOU bugs. ### Method extraction - `attach` and `open` extracted from `*Server` — pass groups and log explicitly - `GroupTable` extracted from `*Server` — permission checking uses bitmask OR instead of a boolean dance (`checkPermBits` → `hasPermBits`) - `checkPerm` extracted to package-level, permissions resolved in `open` - 9 `rootTree`-only methods extracted to package-level functions; switch statement replaced with handler map - Three thin wrapper methods inlined (`readFile`, `writeFile`, `FileTree` alias) - `handle` inlined into `Start`; renamed `Serve` → `Start`, `Shutdown` → `Kill` ### Result - `server.go` reduced from ~820 to ~200 lines - `main.go` simplified as bypass tree wiring moved to `fs/` - Server is now a thin 9P2000 protocol translator, not a filesystem ## Phase 17: Event Redesign & Chat Log Format (Aug 2) ### Global /event Replaced per-session event polling with a global event ring buffer and a single `/event` file. Events carry structured prefixes and delta descriptions: - `S new session/` — session created - `S kill session/` — session destroyed - `S rename session/ session/` — session renamed - `A kill session//agent/` — agent killed The event ring is a fixed-size circular buffer (100 slots) with a `sync.Cond` for blocking reads. Frontends block on `/event` and receive deltas since their last known offset. No polling, no timer, no D-Bus. Fixes along the way: - Dangling pointer in `eventRing` cond initialization (caused panics under load) - Double-unlock panic in `WaitEvent` - Spurious `[[[end]]]` markers on reasoning blocks (reasoning events were silently suppressed from chat but still advanced the role-transition state) ### Chat log: [[[type]]]/[[[end]]] block format The chat log switched from plain text to a structured block format: ``` [[[assistant:resp-abc123]]] Hello! How can I help? [[[end]]] [[[tool:file_read]]] File contents... [[[end]]] ``` This enables reliable parsing of multi-turn, multi-role conversations from the flat log. Each event type (`assistant`, `tool`, `call`, `retry`, `error`, `usage`, `info`, `exec`) opens a block and `[[[end]]]` closes it. `reasoning` and `reasoning_think` events are suppressed from the log entirely — they're included in the LLM context but invisible to the user. ### KDE submodule evolution The KDE frontend received extensive updates across both days: - **StreamFsm refactor** — chat block FSM with proper fence-post handling - **Responsive session tree** — selection/statewait stream management - **Explicit agent selection** — per-agent statewait/chat streams, only `switchAgent` starts streams; session click clears agent selection - **Emoji-labeled states** — visual state indicators (❌ for errors, ⚠️ for warnings) - **Double-click rename** — inline renaming of agent nodes - **Robust session loading** — loads sessions on startup, handles empty state - **Context menu** — kill session, per-session operations - **Event-driven updates** — replaced timer-based polling with blocking `/event` - **Fence fix** — fixed a bug where code fences could eat surrounding content - Removed D-Bus entirely; all communication via `9p` plan9port binary ### Updated Filesystem Layout (post-refactor) ``` / ← root (owned by system user) ├── backends backends list ├── help help text ├── models model list (cached) ├── agents agent config list ├── ctl root control (invalidate, kill) ├── event global event stream (blocking read) ├── complete request-response: code completion ├── generate request-response: one-shot LLM generation ├── route request-response: model routing ├── bypass/ │ ├── policy global bypass policy │ └── pending/{sname} pending bypass requests └── session/ ├── new write key=value to create session ├── idx session index (one line per agent) ├── {name}/ │ ├── env session environment variables │ ├── ctl session control (kill, save, invalidate) │ ├── plan session-scoped markdown checklist │ ├── id immutable session UUID │ ├── name mutable session name (write to rename) │ ├── bypass per-session bypass policy │ └── agent/ │ ├── new write config to create agent (rdwr) │ └── {aname}/ │ ├── prompt, prompt.prev, fifo.in, fifo.out │ ├── chat (streaming), log (64KB window) │ ├── state, statewait (blocking) │ ├── cfg, ctl, cwd │ ├── id (immutable), name (mutable) │ ├── offset, usage, cost, ctxsz, models │ ├── systemprompt, context, tail │ ├── tools, toolsrv.loaded, toolsrv.rev │ └── proc/{pid} ``` ## Updated Current Topology ``` ollie/ ← single Go module ├── cmd/ │ ├── olliesrv/ ← 9P server (sessions, agents, backends) │ │ └── internal/ agent/, backend/, bypass/, fs/, session/, prompts/, toolclient/ │ ├── toolsrv/ ← 9P tool execution server (separate process) │ │ └── internal/ fs/, exec/, registry/, sandbox/ │ ├── ollie-9p/ ← 9P client CLI │ └── ollie-remote/ ← remote execution binary ├── toolsrv/ ← 9P client library for toolsrv ├── fsedsl/ ← filesystem declaration EDSL (used by both servers) ├── env/ ← environment helpers ├── log/ ← structured logging ├── paths/ ← XDG path resolution ├── kde/ ← KDE Plasma integration (part of the main repository) ├── data/tools/ ← tool executables + .meta sidecar files ├── data/agents/ ← agent configs (JSON) ├── data/prompts/ ← system prompt fragments (markdown) ├── data/skills/ ← domain knowledge modules (markdown) ├── data/scripts/ ← ollie-remount, o CLI ├── data/services/ ← systemd, xdg-autostart ├── prompts/ ← embedded prompt templates └── doc/ ← documentation ``` ## Phase 18: Zero Built-in Tools (Aug 2–3) The most radical simplification yet: **zero tools compiled into the Go binary.** Every tool — including `shell` and `reasoning_think` — is now an external script loaded dynamically through the 9P filesystem. ### What changed **Before**: `tools/builtin/builtins.go` returned four Go-compiled handlers (`shell`, `reasoning_think`, `tool_list`, `tool_load`, `tool_active`). These were hard-linked into every agent's tool definition, consuming LLM context slots even when unused. The tool registry was a two-tier system: built-in handlers (Go code, always available) + script tools (loaded on demand via `tool_load`). **After**: `tools/builtin/` was removed entirely — `builtins.go` last returned `nil`. All tools are loaded through a single convergent path: writing the tool name to `session/{sname}/agent/{aname}/tools`. The `shell` and `reasoning_think` tools are now external scripts (`data/tools/shell`, `data/tools/reasoning_think`) with `.meta` sidecars — identical to `file_read`, `file_grep`, or any other tool. ### Unified tool loading Tool loading follows a single path shared by both 9P writes and autoLoad initialization: ``` 9P write to agent/{id}/tools │ ▼ session.loadTool(name, agent) │ ├── Check disallow list └── RPC "tool_load" → ollie-remote process ``` The `tool_load` tool itself is a bash script (`data/tools/tool_load`) that writes the tool name to the 9P filesystem — it's a meta-tool that uses the same interface as every other operation. ### Agent config: `autoLoad` Agent configuration files now declare which tools to load at startup via an `autoLoad` array: ```json { "autoLoad": [ "file_read", "file_edit", "file_glob", "file_grep", "file_write", "tool_load" ] } ``` Each agent profile (default, copilot, explorer, librarian, navigator, taskmanager, theo) has its own `autoLoad` list tailored to its domain. The `allowTools` field provides a secondary security gate. ### Tool output format classification Each tool's `.meta` can declare an `outputFormat` field specifying the source-fence language for its output: ```json { "description": "...", "outputFormat": "markdown" } ``` The agent loop propagates this to the chat log as the fence language (`[[[tool:file_read]]]\`\`\`markdown`), enabling syntax-highlighted output in the KDE GUI. Tools without `outputFormat` default to plaintext fences. ### Remote execution simplified `ollie-remote` no longer embeds tool scripts or sandbox config: - Removed `//go:embed all:tools` — tools come from `$OLLIE_TOOLS_PATH` in the environment - Removed embedded sandbox `default.yaml` — sandbox config is provisioned alongside tools - Removed `--tools` flag — uses `~/.config/ollie/tools` by default ### SSH bootstrap simplified The SSH bootstrap sequence was cut from a complex multi-stage protocol to a single tarball transfer: | Before | After | |--------|-------| | SHA256 hash verification | No hashing | | gzip + base64 binary encoding | Raw `tar czf` tarball | | `OLLIE_LOADER_START` / `OLLIE_LOADER_READY` handshake | Single `OLLIE_LISTEN_READY` signal | | Separate `bootstrap.sh` template (88 lines) | Inline 10-line script | | Per-binary transfer (ollie-remote + native sandbox) | Single `tar` of `~/.config/ollie/` | The simplified bootstrap: ```sh CACHE_DIR="${XDG_CACHE_HOME:-$HOME/.cache}/ollie" mkdir -p "$CACHE_DIR" tar xzf - -C "$CACHE_DIR" # receives tarball from local export OLLIE_TOOLS_PATH="$CACHE_DIR/tools" export PATH="$CACHE_DIR/bin:$PATH" exec "$CACHE_DIR/bin/ollie-remote" serve --cwd --listen ``` ### 9P namespace changes - `agent/{id}/tools` — **read** lists all currently loaded tools (via the session's `toolsConn`), **write** loads a tool by name (via `session.loadTool`) - Removed `agent/{id}/toolsrv.loaded` — subsumed by `tools` read - Removed `agent/{id}/toolsrv.rev` — no longer needed - Added root-level `/tools` — lists all discoverable tools on disk (global catalog) - Removed `HandlerCtx.ToolReg` — the in-memory Go-level registry is no longer passed through 9P handlers ### Files removed | File | Status | |------|--------| | `tools/builtin/` | Package removed entirely — all tool logic lives in external scripts | | `tools/builtin/shell.go` | Deleted (now `data/tools/shell` + `data/tools/shell.meta`) | | `tools/builtin/reasoning.go` | Deleted (now `data/tools/reasoning_think` + `data/tools/reasoning_think.meta`) | | `tools/builtin/tool.go` | Deleted (`tool_list`/`tool_load`/`tool_active` replaced by 9P filesystem mechanism) | | `toolsrv/bootstrap.sh` | Deleted (inline bootstrap replaces multi-stage protocol) | | `cmd/ollie-remote/tools/.gitkeep` | Deleted (no embedded tools) | ### Lines of code - **−530 lines** from the Go core (removed built-in handlers, embedded tools, bootstrap script) - **+82 lines** for external tool scripts + `.meta` sidecars (`shell`, `reasoning_think`, `tool_load`) - **Net**: tools system is smaller, simpler, and more uniform ### Why it matters The zero-tools transition is the culmination of the architectural trajectory that began with the tool registry in Phase 10: **every facility is a file; nothing is hard-coded.** 1. **Uniformity** — There is no distinction between "core" tools and "script" tools. `shell`, `reasoning_think`, `file_read`, `file_write` — all loaded the same way, all have `.meta` sidecars, all called via the same RPC mechanism. 2. **Minimal core** — The Go binary no longer contains any tool logic. The agent loop is pure coordination: stream LLM → parse tool calls → dispatch to remote process → loop. Tool definitions, schemas, and execution are entirely external. 3. **Agent-choosable tools** — Each agent profile declares exactly which tools it needs. No agent pays the context-window cost of `shell` and `reasoning_think` unless its config asks for them. 4. **9P as the sole control plane** — Tool loading is now a file write. The same mechanism (`ollie-9p write ...`) that submits prompts and reads state also manages tool loading. No special RPC, no separate tool-manager protocol. 5. **Simpler remote deployment** — No embedded files, no binary transfer protocol, no hash verification. Provisioning a remote host is `tar czf ~/.config/ollie | ssh host tar xzf -`. The result: the Go core is a 9P file server and an agent loop, nothing more. All behavior — every tool, skill, prompt, and configuration — lives in data files on disk. This is the Plan 9 ideal: a small, fixed kernel; all policy and capability in the filesystem. ### SLOC current (Aug 3) | Component | Lines (excl. tests) | |-----------|--------------------| | agent + backend + toolsrv + session | 11,828 | | fs + cmd/olliesrv | 4,838 | | kde gui | 6,491 | | **Total** | **23,157** | Zero dead exported functions remain (verified via LSP + grep across all repos). ## Phase 19: EDSL Extraction & Structural Cleanup (Aug 3–4) The declarative EDSL created in Phase 15 was extracted into a standalone, reusable library. This wasn't just moving code — the EDSL itself was **created from scratch on Aug 2** to replace ~3,100 lines of imperative filesystem code. Within 48 hours it went from concept to extracted library. The trajectory: 1. **Aug 2 morning**: Created the EDSL (Phase 15) — replaced imperative tree-building with declarative spec 2. **Aug 2 afternoon**: Streamlined the 9P server (Phase 16) — reduced to thin protocol handler 3. **Aug 3**: Extracted EDSL to `fsedsl/` as a reusable library 4. **Aug 4**: Structural cleanup — removed dead packages, improved `o` CLI This rapid iteration exemplifies the project's development style: build the right abstraction, validate it works, then extract it for reuse. ### fsedsl: Standalone Library The generic EDSL types and builder logic extracted from `fs/` into `fsedsl/`: ``` fsedsl/ ├── fsnode.go FsNodeDecl[C], NodeOption[C], Dir/Leaf/Each constructors ├── builder.go BuildTree — walks spec, validates, wires handlers ├── tree.go Tree, File, FileConfig, FileTree types ├── option.go Doc, Read, Write, Stream, BlockOnce, Request, etc. └── README.md Integration guide ``` The library is **protocol-agnostic** — it knows nothing about 9P. It produces a `*Tree` structure that any protocol (9P, FUSE, HTTP, gRPC) can walk. The `fs/` package becomes a thin adapter: type aliases + ollie-specific handlers. Key design: - Generic over context type `C` — ollie uses `HandlerCtx`, others can use anything - Single `BuildTree[C]()` call validates and wires the entire namespace - Validation at build time: no leaf handlers on directories, at most one blocking variant per file, template nodes require `List`, ownership inheritance ### fs/ Package Consolidation After EDSL extraction, the fs package was restructured: **Before** (scattered): ``` fs/ ├── spec.go namespace declaration ├── builder.go tree builder (moved to fsedsl) ├── adapter.go type aliases ├── session/ sub-package with 6 files │ ├── files.go │ ├── root.go │ ├── perm.go │ └── ... └── bypassfiles.go ``` **After** (flat, by scope): ``` fs/ ├── spec.go single source of truth — entire namespace ├── fsnode.go type aliases from fsedsl ├── builder.go BuildTree wrapper ├── rootfiles.go root-level handlers (backends, models, help, ctl) ├── sessionfiles.go session-level handlers (env, ctl, name) ├── agentfiles.go agent-level handlers (prompt, chat, state, tools) ├── bypassfiles.go bypass broker handlers └── procfiles.go detached process handlers ``` No sub-packages. Handler files are organized by 9P tree scope. ### Session Package Refactors Major restructuring of session lifecycle: - **Session/Agent split** — `session.Session` manages lifecycle, `agent.Agent` manages conversation. Clear ownership boundaries. - **Persistence to session/** — `fs/persist.go` removed, persistence logic moved to `session/` package - **Manager eliminated** — Package-level registry replaces `Manager` struct. CRUD via `session.Lookup()`, `session.All()`, `session.CreateFromRoot()`, etc. - **Remote promoted to session level** — Remote execution managed at session, not agent level ### Event Bus: Ring Buffer → Pubsub The event ring buffer added in Phase 17 lasted exactly one day. On Aug 3, it was replaced with `github.com/simonfxr/pubsub`: **Why the ring buffer failed**: - Off-by-one bugs in circular buffer indexing - Race conditions between `sync.Cond` broadcasts and context cancellation - Panics from double-unlock when multiple readers raced to the same event - The offset-based API leaked implementation details to clients **What pubsub provides**: - Topic-based routing (`session.{id}.new`, `agent.{id}.state`) - Wildcard subscriptions (`session.*`, `*`) - Channel-based delivery with automatic cleanup on context cancellation - A decade of production hardening by someone else The backwards-compatible `PostEvent()` wrapper translates the old format (`A agentId state thinking`) to the new topic format (`agent.agentId.state` with payload `thinking`). Existing code kept working. **Lines changed**: +103 / -72. Net increase, but the new code is correct. ### Dead Code Removal | Removed | Reason | |---------|--------| | `cmd/Ollie` (Acme frontend) | Use `o` CLI to compose acme front-end | | `mount/` package | FUSE mount replaced by direct 9P client (`ollie-9p`) | | `ollie-watchdog` script | No longer needed — 9P client doesn't have FUSE stale mount issues | | `doc/experiments/` | Historical experiments moved to git history | | `doc/help.md` | Now generated dynamically from `Doc()` strings in spec | ### o CLI Improvements The terminal CLI (`data/scripts/o`) was simplified: - **Removed auto-session/agent** — Context must be set explicitly via `export` or `eval $(o env ...)`. Eliminates slow `ollie-9p read session/idx` calls. - **Path classification** — Paths categorized as root/session/agent level. Clear error messages: "chat requires agent context". - **TUI percentage layout** — Fixed-size splits replaced with percentages (`-l 25%`). Adapts to terminal size. - **Block marker filtering** — Chat stream piped through `grep -v '\[\[\[.*\]\]\]'` to hide internal markers. ### Documentation - Removed outdated docs (`doc/experiments/`, stale `help.md`) - Renamed doc files to lowercase-kebab-case - Updated usage.md for new `o` CLI behavior - Removed mount/ and watchdog references from architecture docs ### Gantt Update The timeline now extends to Aug 4, 2026 — ~3.75 months of development. ### SLOC current (Aug 4) | Component | Lines (excl. tests) | |-----------|--------------------| | agent + toolsrv + session | 9,077 | | fs + cmd/olliesrv | 2,919 | | fsedsl (extracted library) | 1,220 | | kde gui | 4,162 | | **Total (core)** | **17,378** | Note: Aug 3 count included `backend/` (4,366 lines) and had different kde scope. Comparable core (fs + agent + toolsrv + session + fsedsl + kde gui) shrank from ~23k to ~17k. --- ## "Good Idea Fairy" Cleanup (Aug 5) A brutal simplification pass that deleted ~960 lines of speculative infrastructure. ### Hooks — Deleted Entirely The entire hooks system was removed: | Hook | What it did | Why deleted | |------|-------------|-------------| | `agentSpawn` | Run commands on session creation | Prompt composition handles initialization | | `preTurn` | Inject context before each turn | Removed earlier; prompt assembly does this | | `postTurn` | Run commands after each turn | Never used in practice | | `preTool` | Gate tool execution | Security model moved to sandbox | | `postTool` | Transform tool results | Never used | | `preCompact` | Run before context compaction | No use case | | `postCompact` | Run after compaction | No use case | | `turnError` | Handle backend errors | Replaced by retry logic in the loop | Hooks are a "standard feature" in agent frameworks — lifecycle callbacks that let external code react to events. In practice, they added complexity without solving real problems. The prompt system handles initialization. The sandbox handles security. Error handling belongs in the loop. Every hook was either unused or doing something that belonged elsewhere. **−237 lines** from `agent/hooks.go`. ### Other Deletions | What | Why | |------|-----| | `backend/noop.go` | Test backend, never used. Tests use real backends or mocks. | | `backend/oneshot.go` | Alternative generation mode, never used. `generate` file handles one-shot. | | `cmd/ollie-9p/mount/` | FUSE mount code, replaced by `ollie-9p` direct 9P client. | | `data/tools/route.meta` | Model routing tool, replaced by explicit `/model` commands. | | `toolsrv/schema.go` | Schema validation code, never enabled. | ### Turn State Machine The turn state machine in `agent/turn.go` was simplified from 158 lines to ~40. The `TurnCtx` struct was not yet eliminated (that came later in Aug 11), but the state transitions were flattened. ### Philosophy "Good idea fairy" is military slang for someone who shows up with clever suggestions that sound good but create work. The deleted code was all reasonable in isolation — hooks are a standard pattern, test backends are useful, FUSE mounts are convenient. But each added surface area that had to be maintained, tested, and reasoned about. None of it solved actual problems better than simpler alternatives. --- ## Aug 9, 2026 — Simplification & Namespace Cleanup Major simplification pass: ~1,450 lines removed from the core runtime. ### Structural - **Section-based preamble** — `Preamble` struct with `Set/Get/String` replaces string surgery (-270 lines) - **Session** — Export immutable fields, extract `buildAgent` helper (-115 lines) - **Prompt resolver** — Gutted to file-path-only resolution (-127 lines) - **TaskState subsystem** — Removed entirely (-246 lines) - **GenerationParams** — Embedded in AgentConfig, JSON tags added (-34 lines) - **sync.Cond → close-channel** — `WaitChange` uses idiomatic channel signal - **Dead code** — `session.New()`, env map, prevPrompt field, usage_log.go, contextDebug() ### Namespace The agent namespace went from 17 files to 10. Redundant files merged or moved to ctl: | Before | After | |--------|-------| | `fifo.in` + `fifo.out` | `fifo` (write=enqueue, read=dequeue) | | `cost` + `usage` + `ctxsz` | `stats` (key=value lines) | | `chat` | `chat` (filtered text) + `chat.raw` (full markup) | | `cwd`, `models`, `tools`, `systemprompt` | ctl commands | | `connection`, `state`, `context`, `tail`, `offset`, `prompt.prev` | removed | ### ctl Unification All ctl files (root, session, agent) now share: ```go type rdwrHandler func(ctx HandlerCtx, args []string) ([]byte, error) ``` Agent ctl is **rdwr** (request-response): write command, read result. Commands with no args return current state (e.g., `/model` prints the model). Agent ctl commands: `stop`, `compact`, `clear`, `inject`, `agent`, `model`, `models`, `tools`, `tool_load`, `cwd`, `name`, `backend`, `systemprompt`. ### Command Surface `/` prefix in prompts == `o ctl`. No special-cased commands in the agent. - `/model qwen3:8b` → switches model, returns new model name - `/tools` → lists loaded tools - `/inject look at this` → injects prompt mid-turn (overwrites pending) - `!q` in the REPL → exits the TUI (local-only) ### elevate → bypass The sandbox escape mechanism was renamed from `elevate` to `bypass` throughout (package, namespace, env vars, tool args, all docs). ### Streaming Fix The TUI streaming regression (line-by-line instead of char-by-char since Aug 4) was caused by piping through `grep -v` — grep is inherently line-buffered. Fixed by: - `chat` file now serves filtered text (markers + fences stripped server-side via `stripMarkers` state machine) - `chat.raw` serves full markup for GUIs that parse blocks - TUI no longer pipes through grep for streaming ### CLI (`o` script) - `readloop` → `read -l` - `chatstream` → removed (`o read chat` streams directly) - `stop`, `kill` → removed (use `o ctl stop`, `o ctl kill`) - `ctl` uses `ollie-9p rdwr` (gets response) - REPL: `/` → `o ctl`, `!q` → exit TUI - Background statewait loop trapped on EXIT (fixes zombie) ### SLOC (Aug 9) | Component | Lines (excl. tests, backends, fsedsl, generated) | |-----------|--------------------:| | agent | 3,167 | | toolsrv | 2,521 | | fs | 1,774 | | session | 1,386 | | cmd/olliesrv | 1,161 | | bypass | 733 | | lib9p | 690 | | cmd/ollie-9p | 356 | | cmd/ollie-remote | 369 | | sandbox | 293 | | log + env + paths + format + prompts | 428 | | **Total (core)** | **12,878** | --- ## Aug 9, 2026 (cont.) — Prompt Audit & Maintainability Pass Systematic audit of the prompt construction system and codebase maintainability. ### Prompt System - **System prompt**: ~200 → ~80 lines. Removed tool-first table, API docs section (moved to agent prompt), dead 9P entries, output protocol (moved per-agent). - **AllowTools**: removed entirely (field, RPC, config). Tools are loaded dynamically; static allowlists served no purpose. - **Phantom refs fixed**: `route` removed from driver autoLoad, `reasoning_think` + `shell` added to default autoLoad. - **Output protocol**: moved from system prompt to per-agent prompts. Theo's brevity rule applied to all agents except copilot. - **Tool docs**: merged listing + documentation into single `renderTools()`. Removed JSON schema dump (function-calling API provides it natively). Removed `BuildToolListing` dead code. - **PRIME_* env vars**: removed entirely. Replaced `SessionInfra.PromptEnv []string` with typed `Platform string` + `IsGitRepo bool` fields. ### Code Quality - **`OnToolsChanged`**: changed from `func(string)` to `func()` signal. Agent re-fetches tool list on signal instead of receiving a pre-formatted string it ignores. - **`Preamble` sections**: type-safe `Section` type replaces raw strings. Compile-time typo detection. - **`extractToolResult`**: documented the tool output protocol (JSON content-block envelope with raw-text fallback). - **Package godoc**: `session/` package comment documenting Init→Create→Register→Kill lifecycle. - **`PromptEnv()`**: deleted (was a no-op returning nil after PRIME_* removal). ### Documentation - Moved 7 reference docs from `doc/resources/` to `doc/` (architecture, 9p, writing-tools, tool-registry, core, remote-execution, edsl). Left whitepaper material (evolution, multi-agent, misc, no-mcp, ideas) in `resources/`. - Rewrote `data/agents/README.md` to match current config schema and agent roster. - Fixed stale cross-references in README.md and doc/tool-registry.md. ### SLOC (Aug 9, post-audit) | Component | Lines (excl. tests, generated) | |-----------|--------------------:| | agent | 3,159 | | toolsrv | 2,464 | | session | 1,377 | | fs | 1,774 | | cmd/olliesrv | 1,161 | | bypass | 733 | | lib9p | 690 | | cmd/ollie-9p | 356 | | cmd/ollie-remote | 358 | | sandbox | 293 | | log + env + paths + format + prompts | 428 | | **Total (core)** | **12,793** | Excludes: backends (4,229), fsedsl (1,030), tests, KDE, tools, generated code. ## Phase 20: toolsrv 9P Migration & Process Isolation (Aug 10–11) The tool server completed its evolution from an in-process library to a **fully independent 9P server**. Locally it runs as a child process of olliesrv; for remote execution it runs on a remote host, connected via SSH Unix socket forwarding. ### Before `toolsrv` was a Go package (`ollie/toolsrv`) with a `Server` struct that ran in-process within olliesrv. Tool calls were Go function calls — no process boundary, no separate namespace. The package mixed client code, server code, registry, sandbox wrappers, and execution logic in a flat directory. ### After Two distinct components: 1. **`cmd/toolsrv/`** — a standalone binary that serves its own 9P2000 filesystem over a Unix socket. Has its own `internal/` packages: - `internal/fs/` — filesystem spec (fsedsl), server state, process management - `internal/exec/` — sandboxed tool execution (native Landlock, bypass broker) - `internal/registry/` — session-scoped tool registry - `internal/sandbox/` — native Landlock configuration and enforcement 2. **`toolsrv/`** (root package) — a 9P client library. `Conn` dials toolsrv over a Unix socket, authenticates via Tauth, and exposes methods like `CallTool`, `LoadTool`, `ListTools`, `Ping`. ### toolsrv 9P Namespace ``` / ├── ctl write: load , unload , env K=V, cwd ├── tools read: list loaded tools (JSON), write: tool name to load ├── info read: platform, arch └── proc/ ├── new rdwr: write tool+args, blocks, read result ├── new.bg write: tool+args, returns pid immediately └── {pid}/ ├── out read: output ├── wait read: blocks until exit, returns exit code ├── stat read: running/exited, runtime, tool └── ctl write: signal , dismiss ``` Both `cmd/toolsrv` and `cmd/olliesrv` declare their namespaces using the same `fsedsl` library. Both implement their own 9P protocol handlers (using `9fans.net/go/plan9`) — they are intentionally separate servers that communicate over a socket. ### Process Lifecycle Three-layer defense against orphaned toolsrv processes: 1. **Pdeathsig** (local only) — `SysProcAttr{Pdeathsig: SIGTERM}` on local spawn. When olliesrv dies, the kernel terminates toolsrv immediately. For SSH-spawned toolsrv, Pdeathsig is set on the `ssh` process itself — when SSH dies, the remote shell receives SIGHUP which typically cascades to toolsrv. 2. **Idle timeout** (local + remote) — toolsrv tracks active 9P connections. After the first client connects and then all connections drop, a 30s timer starts. If no new connection arrives, toolsrv exits cleanly. 60s startup grace period for the initial connection. This is the primary cleanup mechanism for remote toolsrv where Pdeathsig doesn't apply directly. 3. **Kill-before-respawn** — `ProcessKeeper.Dial()` kills the old process before spawning a replacement. Prevents accumulation during reconnect cycles. ### Authentication toolsrv uses 9P Tauth for authentication. The first client sets the shared secret; subsequent clients must provide the same secret. This replaces the previous token-in-environment approach and is compatible with socket permission security. ### Structural Consistency Both servers now follow the same physical layout: ``` cmd/{server}/ ├── main.go entry point ├── server.go 9P protocol handler (top-level, like Plan 9 tradition) └── internal/ ├── fs/ filesystem spec + handlers + state ├── ... domain-specific packages ``` ### Parallel Tool Execution Replaced binary ReadOnly/not batching with resource-based conflict scheduling. Tool calls within a single turn are grouped into parallel batches based on a three-class scope system declared in `.meta`: - **scope "read"** — never conflicts (always parallel) - **scope "write"** — conflicts only on same file path - **scope "global"** (or unset, default) — full serialization barrier The model's read-before-write pattern (enforced by the turn-based protocol) guarantees that parallel writes are safe: each file_edit carries its own `old_string` context from a prior read, and edits on different paths are provably independent. Shell and tools with opaque effects (like `lsp_rename`) declare scope "global" and always run alone. Unknown/unset scope defaults to global — tools must opt in to parallelism. ### Background Process Interrupts Any tool call can include `"background": true` to execute asynchronously via `proc/new.bg`. The model receives a process ID immediately and continues working. Background processes have no timeout — they run until they exit naturally, are stopped via `proc/{id}/ctl`, or die with the session. Output is streamed in real-time into `proc/{id}/out`. Upon completion, the process exit notification is injected as a **user prompt** (written to the agent's `prompt` file) containing a `` block: ```xml --- FAIL: TestFoo (0.00s) foo_test.go:12: expected 3, got 2 FAIL ``` This uses automatic prompt queueing: if the agent is still working, the interrupt arrives on the next idle transition; if the agent is already idle, it arrives immediately. The model sees these as natural follow-up prompts — no polling, no in-loop injection. It can react to failures or stop processes via the proc ctl file (`term`, `kill`, `dismiss`). **Process lifecycle**: toolsrv owns all procs. Session death (toolsrv exit) kills all procs via `KillAll()` + `Pdeathsig`. Exited procs remain in the tree for 10 minutes after last read, then are garbage collected. ### Universal Dispatch Flags All tools receive four optional parameters injected into their schemas at runtime (not declared per-tool): - `bypass` — sandbox escape via bypass broker - `timeout` — execution timeout in seconds - `sandbox` — sandbox profile override - `background` — async execution with auto-injected output ### Why Not Concurrent-by-Default Foreground tool calls block the model until all results arrive — deliberately. The alternative (execute everything concurrently, deliver results as queued prompts) was considered and rejected: it turns one coherent reasoning step into N separate generation cycles, each with partial information. The model can't reason about results together, every interrupt costs a full inference round-trip, and the end result is sequential execution but slower and more expensive. Background execution remains an explicit opt-in for genuinely long-running processes where the model doesn't need the result immediately. ## Phase 22: Meta-Only Tool Definitions (Aug 11–12) A `.meta` file can now define a tool with no companion executable. The `cmd` field is treated as a shell command string — not a path to resolve, but a command to run directly. JSON arguments are passed on stdin. Before: ```json {"cmd": "rg"} // resolved via PATH, then executed with --flags ``` After: ```json {"cmd": "jq -r '.pattern' | xargs rg"} // shell command, executed as-is ``` This changes what a "tool" is. Previously, tools were executables with `.meta` sidecars. Now a tool can be purely declarative: a `.meta` file that shells out to existing CLIs. Wrap ripgrep, kubectl, or an MCP bridge without writing any code. The tools directory is prepended to `$PATH` during execution, so meta-only tools can reference other tools in the same directory. **Why this matters**: any CLI becomes a tool with 20 lines of JSON. The barrier to adding capabilities dropped from "write a script" to "write a schema." ## Phase 23: Bypass via 9P (Aug 11) The bypass broker — the mechanism for escaping the sandbox — was previously a Unix socket with a custom protocol. It now routes through the 9P filesystem: - toolsrv exposes `bypass/pending` (blocking read) and `bypass/resolve` (write) - olliesrv subscribes to pending requests and routes them to D-Bus for approval - Approved/denied responses flow back through 9P This unifies the control plane. Everything is files. The bypass socket is gone. ## Phase 24: fs/ Flattening (Aug 11) The olliesrv filesystem package had accumulated abstractions: - `AgentLog` — chat log buffer wrapper - `SessionNode` — session state container - `RootState` — root-level state These were eliminated. Chat logging moved to the agent package. Session state moved to the session package. The fs/ package now contains: - `spec.go` — the EDSL declaration with all handlers inlined - `support.go` — helper functions - `newroot.go` — tree construction ~600 lines deleted. The handlers live where the data lives. ## Phase 25: Sandbox Simplification (Aug 11) Named sandbox profiles (`default`, `restricted`, `remote`) were removed. There is now one config: `sandbox.yaml`. Per-project overrides via `.ollie-sandbox.yaml` remain, but the multi-profile system is gone. The `sandbox` parameter on tool calls now does nothing — kept for compatibility but ignored. ## Phase 26: Agent Loop Cleanup (Aug 11) The `TurnCtx` struct — which carried per-turn state through the loop — was eliminated. Loop functions became methods on `*Agent`, accessing state directly. The turn state machine in `turn.go` was simplified. ~100 lines deleted, control flow is clearer. ## Phase 27: Tree-sitter Code Intelligence (Aug 15) Code navigation and structural editing moved into native Go tools backed by Tree-sitter grammars. The tool set now includes: - `codebase_overview` for repository structure - `code_outline` for one-file declarations - `code_symbols` for workspace-wide symbol searches - `code_dependencies` for imports and includes - `code_query` for syntax-aware searches - `code_rewrite` for structural rewrites The tools support Go, JavaScript, TypeScript, TSX, Python, Rust, C, C++, JSON, YAML, PHP, and Markdown. Agent prompts now require structural tools for repository exploration and reserve text search for simple content searches. This makes targeted edits safer than broad textual replacement. ## Phase 28: Go File Tools and Tool-Surface Cleanup (Aug 16) The core file tools (`file_read`, `file_write`, `file_edit`, `file_grep`, and `file_glob`) were replaced with Go binaries and shared implementation code. Tests were added for the file-tool package and existing LSP and web-tool coverage was expanded. The change removes the old Python implementations from the runtime path while preserving the same tool contracts. Obsolete process-management tools and the `logseq` tool were removed. The installation recipe was corrected for the compiled file tools. Workspace-path placeholders were standardized across prompts and tool metadata so agents use the actual working directory instead of guessed home paths. ## Phase 29: OptMem Persistent Memory (Aug 16) Ollie integrated [OptMem](https://github.com/VictorTaelin/OptMem) as its persistent memory backend. The legacy `memory_recall` and `memory_remember` scripts were replaced by metadata-defined tools that invoke the bundled `third_party/optmem/memo` executable directly. OptMem owns the append-only log and bounded B-tree index at `$XDG_DATA_HOME/ollie/optmem` (default: `~/.local/share/ollie/optmem`). Ollie does not maintain a second memory format or parallel store. Reads and writes use the same backend and serialize through the toolsrv path-lock system. Both memory tools are auto-loaded by every agent profile. Shared prompt guidance tells agents to recall relevant prior context before acting and remember durable decisions, outcomes, preferences, and non-obvious findings. The OptMem executable is installed under `$XDG_CONFIG_HOME/ollie/optmem` and explicitly granted `rwx` access by the native Landlock sandbox. This integration keeps memory as an ordinary Ollie tool while giving it a durable, searchable backend. It requires no MCP server, daemon, or new control-plane protocol. ## Phase 30: Markdown Parsing and Bounded Tool Output (Aug 16) Tree-sitter Markdown support was added to the code-intelligence layer. `.md` and `.markdown` files are parsed as Markdown, and headings are available to structural queries as `atx_heading` and `setext_heading` nodes. This makes `evolution.md` and other documentation amenable to the same targeted tooling as source code. The maximum tool result included in model context was reduced from 128 KiB to 32 KiB. Large command or file results are therefore less likely to crowd out the conversation and instructions. --- Commit count: ~60 commits over 3 days. The recent work added native structural and file tooling, persistent memory, tighter context bounds, and embedding- guided discovery while removing obsolete tool implementations. --- ## Dead Ends and Reversals Everything that was built and then killed, in roughly chronological order. | What | Why it was removed | |------|-------------------| | MCP servers (denote-mcp, 9beads-mcp) | Too heavyweight; scripts are simpler and faster | | TUI frontend | Replaced by `o` — a tiny shell script (40 lines of tmux) | | `pl/` planning namespace | Over-engineered; plan file is sufficient | | Beads integration | External dependency for something a checklist file handles | | `plan_create` / `plan_complete` / `task_add` / `task_check` tools | Five planning iterations before settling on a single markdown file | | SSH agent proxy | Attack surface not worth the convenience | | Separate ollie-dbus daemon (ollied) | Consolidated into olliesrv, then killed entirely | | D-Bus adapter (`org.ollie.SessionManager`) | 9P streaming is superior; no polling, no offset tracking, no frozen GUIs | | D-Bus-driven KDE frontends | Rewritten on pure 9P via plan9port `9p` binary | | Plasmoid / tray KDE components | Not useful enough to maintain | | FUSE mounts | Replaced by standalone 9P client binary; FUSE can't do blocking reads | | Tier routing (FAST/POWER) | Replaced by `/route` endpoint with real model discovery | | call_tool / pipe | Replaced by native tool registry with `tool_load` | | execute_code (multi-language) | Simplified to shell-only; tool scripts handle language choice | | chatwait | Reverted; acme tails `chat` directly | | Header comment metadata (`ollie:prompt`, `ollie:tier`, `args_json:`) | Replaced by `.meta` sidecar JSON; decouples metadata from language | | `contrib/` directory | Renamed to `data/`; nothing was community-contributed | | Script namespaces (s/, u/, x/) | Replaced by 9P request-response files | | s/sh, s/bfg, s/bbg | Replaced by `generate`/`complete`/`route` 9P files | | `agent.Core` interface | Nobody else implemented it; just indirection | | `Dispatcher` indirection in toolsrv | `toolsrv.Server` called directly | | `execute/` package | Merged into `tools/`; 3 functions didn't need a package | | `mgr/` package (Manager struct) | Replaced by package functions; `*fs.Tree` IS the collection | | `fs/session/` sub-package (6 files, ~3,100 lines) | Flattened into `fs/` — no more sub-package indirection | | `cmd/olliesrv/bypass_tree.go` | Bypass tree handlers moved to `fs/bypassfiles.go` | | Per-session/agent timestamp IDs | Replaced by UUIDv4 — immutable `id` + mutable `name` | | Single-step session creation | Split into `session/new` + `session/{name}/agent/new` | | Imperative filesystem (stat/walk/read/write per node) | Replaced by declarative EDSL (`fsedsl/`) | | Event ring buffer (100-slot circular buffer + sync.Cond) | Lasted one day; replaced by `pubsub` library. Off-by-one bugs, race conditions, panics. | | Built-in tool handlers (shell, reasoning_think, tool_list, tool_load, tool_active) | Replaced by external scripts loaded via 9P `agent/{id}/tools` write | | Go-compiled tool registry in `tools/builtin/` | Zero built-in tools — all tools are external scripts with `.meta` sidecars | | Embedded tool scripts in `ollie-remote` | Remote uses `$OLLIE_TOOLS_PATH` from environment; tools deployed via tarball | | SSH binary transfer (gzip+base64+hash-verify) | Replaced by simple `tar | ssh` of `~/.config/ollie/` | | `toolsrv.loaded` / `toolsrv.rev` 9P files | Removed — `tools` file handles both list and load | | `cmd/Ollie` (Acme frontend binary) | Replaced by `o` CLI composing acme | | `mount/` package | FUSE mount replaced by `ollie-9p` direct 9P client | | `ollie-watchdog` script | No FUSE stale mount issues with 9P client | | `doc/experiments/` | Historical experiments; moved to git history | | `doc/help.md` | Generated dynamically from `Doc()` strings in spec | | Hooks system (agentSpawn, preTurn, postTurn, preTool, postTool, preCompact, postCompact, turnError) | Never solved real problems; prompt handles init, sandbox handles security, retry handles errors | | `backend/noop.go` (test backend) | Never used; tests use real backends or mocks | | `backend/oneshot.go` | Never used; `generate` file handles one-shot | | `toolsrv/schema.go` (schema validation) | Never enabled | | `data/tools/route.meta` (model routing tool) | Replaced by explicit `/model` commands | | TaskState subsystem | Removed entirely (−246 lines); no use case survived | | `fifo.in` + `fifo.out` | Merged to single `fifo` (write=enqueue, read=dequeue) | | `cost` + `usage` + `ctxsz` files | Merged to `stats` (key=value lines) | | `connection`, `context`, `tail`, `offset`, `prompt.prev` agent files | Removed or moved to ctl | | AllowTools field | Tools loaded dynamically; static allowlists served no purpose | | PRIME_* env vars | Replaced by typed `Platform` + `IsGitRepo` fields | | `elevate` naming | Renamed to `bypass` throughout | | `superpowerd` (separate privilege escalation daemon) | Integrated into olliesrv as the bypass broker | | In-process toolsrv (Go library) | Replaced by separate 9P server process with socket boundary | | Binary ReadOnly/not tool batching | Replaced by scope-based conflict scheduling (read/write/global) | | `AgentLog` / `SessionNode` / `RootState` abstractions in fs/ | Eliminated; state moved to where data lives (agent/, session/) | | Named sandbox profiles (default, restricted, remote) | One config: `sandbox.yaml`. Multi-profile system was unused complexity | | `TurnCtx` struct | Eliminated; loop functions became methods on `*Agent` | | Bypass via custom Unix socket protocol | Replaced by bypass via 9P namespace | | `.meta` as sidecar to an executable | `.meta` IS the tool definition; executable is optional (tool-definitions.md) | | `subagent_generate` (JIT agent config tool) | Obsolete; sub-agents use existing profiles directly | | Fire-and-forget `subagent_spawn` via D-Bus | Replaced by blocking `ollie-9p rdwr agent/new` — returns the reply | | Sub-agents as peer agents (no parent, no return) | Replaced by task-scoped sub-agents with context inheritance and automatic cleanup | | `cascade` orchestrator script | Replaced by parallel `subagent_spawn` calls (scope: read, natural parallelism) | | Agent-loop batching as correctness mechanism | Moved to toolsrv path-lock table; agent batching is now just an optimization | | `virtfs.Request` naming | Renamed to `Rdwr` — it's an atomic operation, not a variant of read or write | | One-tool bootstrap (Phase 36) | Models skip the load step and call tools directly; explicit `autoLoad` per profile replaced it | | Lazy tool loading | Capability boundaries must be explicit; model compliance is not a security boundary | | `client_9p` tool hints with load instructions | Removed — tool hints now show loaded tools only | | `skills.Index` shared for skills and tools | Replaced by generic `embedding.Index[T]` with separate skill_match.go and tool_match.go | ## Feed file + Observer agents + BlockOnce/Stream refactor (Aug 13) Three related changes in one session. Net **+278 SLOC** across 21 files. ### What was added **`feed` file** — a change-detecting blocking read file on every agent. Write data in, internal consumer submits it as a prompt. Dedup built into the `BlockOnce` handler: same data written twice never wakes the reader. Enables real-time pair programming — an observer agent watches a human or another agent code. **Observer agent** (`data/agents/observer.json`, `data/prompts/agent-observer.md`) — read-only agent profile. Only loads `file_read`, `file_grep`, `file_glob`, `reasoning_think`. Prompt explicitly forbids writes. Receives diffs via feed, makes terse observations. **`ConsumeFeed`** — plain function that dials the 9P server via `lib9p` and reads from the agent's own feed file in a loop. Each read blocks until feed changes. Same pattern as `o read -l`. ### What was refactored **`virtfs.BlockOnce(readFn, signalFn)`** — previously a raw handler that took `(ctx, base)` and had to implement its own blocking. Now the framework handles the block-until-changed loop: call `readFn()`, compare hash to base, wait on `signalFn()` channel, repeat. Handlers become two-line closures. **`virtfs.Stream(readFn, signalFn)`** — same refactor. `streamChat` (40 lines of condvar + mutex + offset tracking) replaced by `a.ChatRead` + `a.ChatSignal` passed to `Stream(...)`. **Server timeout fallback** — when `BlockOnce` times out (5s, no change), the server falls back to the plain `Read` handler. Frontends get the current value as a heartbeat. Files without `Read` (like feed) return empty. ### What was removed - `streamChat()` in support.go (replaced by `ChatRead` method on agent) - `mergeCtx()` in support.go (no longer needed — blocking logic moved into virtfs framework) - `ObserverFeed` script (replaced by the `feed` file) - `WaitChange` usage outside filesystem handlers (feed consumer uses lib9p instead) ### What was fixed - **XDG fallback** in prompt resolver — `$XDG_CONFIG_HOME` now defaults to `$HOME/.config` when unset - **Duplicate tool headers** — `renderTools` skipped the generated `## name` when the tool prompt already has one - **FIFO drain** — queued prompts no longer orphaned on interrupt/panic/toolsrv failure - **Session restore ordering** — moved after 9P listener starts so feed consumers can connect ### Dead ends (killed during the session) | Attempt | Why killed | |---|---| | Custom channel-based Feed with its own signal infrastructure | Duplicated the agent's existing signalCh mechanism | | `Stream` mode for feed | Feed is a discrete value, not a byte stream | | Internal WaitChange-based consumer | Leaked internal plumbing; should be a plain 9P client | | `BlockOnceRaw` as primary API | Forced handlers to implement blocking themselves | | Concurrent-by-default tool execution | Rejected: turns one reasoning step into N generation cycles with partial info — sequential but slower and more expensive | ## Sub-Agents via 9P rdwr (Aug 14) Sub-agents implemented in **42 net lines of code**. The entire feature is wiring — no new infrastructure, no new concepts, no new processes. ### The mechanism `session/{sname}/agent/new` is an `Rdwr` file. Without `prompt=`, it creates a persistent agent and returns its ID (existing behavior). With `prompt=`, it enters sub-agent mode: creates a transient agent, submits the prompt, blocks until the agent finishes, returns the reply, and destroys the agent. ``` printf 'cwd=%s\nprompt=fix the tests\n' "$PWD" \ | ollie-9p rdwr session/$OLLIE_SESSION_ID/agent/new ``` ### Why it works in 42 lines The infrastructure was already there: - **`Rdwr` file primitive** (née `Request`) — atomic write-then-read, per-open isolation - **`Agent.Submit()`** — already blocks through the full turn - **`Agent.Reply()`** — already captures the final response - **`Session.RemoveAgent()`** — already handles cleanup - **Event bus** — normal agent lifecycle events fire (GUI sees sub-agents appear/disappear) The handler is just: parse params → create agent → submit → read reply → destroy → return. ### Path-based lock table Prerequisite: moved conflict serialization from the agent loop into toolsrv itself. All foreground tool calls now acquire a path-based lock before execution: - `scope "read"` — no lock (never conflicts) - `scope "write"` — exclusive lock on the file path - `scope "global"` — exclusive global lock This ensures sub-agents writing to the same file serialize correctly regardless of which agent initiated the call. The agent loop's batching remains as a performance optimization; correctness is enforced at the execution layer. ### Parallel dispatch `subagent_spawn` is declared `scope: "read"` — multiple spawn calls in the same turn run in parallel automatically. The calling agent blocks until all results return. No shell backgrounding, no polling, no coordination code. ```mermaid sequenceDiagram participant P as Parent Agent participant S1 as Sub-Agent 1 participant S2 as Sub-Agent 2 P->>S1: subagent_spawn(prompt="task A") P->>S2: subagent_spawn(prompt="task B") Note over P: blocked (parallel tool calls) S1-->>P: reply A S2-->>P: reply B Note over P: resumes with both results ``` ### What was killed - **`subagent_generate`** — JIT agent profile generation tool. Obsolete: sub-agents use existing profiles directly. - **D-Bus spawn path** — `subagent_spawn` previously used `dbus-send` to create sessions. Replaced by a 3-line `ollie-9p rdwr` call. ### Framework rename: `Request` → `Rdwr` The three atomic 9P operations are now named for what they are: - `Read` — non-blocking read - `Write` — non-blocking write (fire-and-forget) - `Rdwr` — atomic write-then-read (blocking, produces result) `BlockOnce` and `Stream` are special cases of `Read`. `Rdwr` is its own primitive — not a variant of either. ### Architecture validation This feature is evidence that the "everything is a file" architecture works at scale. A major capability (parallel sub-agents with automatic serialization) landed as pure wiring because: 1. The 9P namespace already exposes `agent/new` as an rdwr file 2. The agent loop already blocks and produces results 3. The tool server already serializes conflicting writes by path 4. The batching algorithm already parallelizes non-conflicting calls No new abstractions. No new processes. No new protocols. Just connecting existing pieces with 42 lines of glue. ### Context inheritance and isolation A sub-agent gets its own context window, history object, runtime state, tool registry, cache, step budget, and lifecycle. When spawned from an existing agent, the parent conversation messages are copied into the child's initial history via `RestoreHistoryFromMessages`. This is a one-time snapshot, not a live shared context. The child cannot append to, rewrite, or otherwise mutate the parent's history. The parent does not see child messages while the child runs. The child reports only its final reply through the blocking `subagent_spawn` result; the parent decides whether and how to incorporate that reply into its own context. This separation is deliberate. Shared conversation mutation would create ordering races, prompt contamination, and unclear ownership of tool results. Shared workspace resources remain coordinated independently through toolsrv's scope and path locks. ### Future work - **Shared toolsrv per host** — currently each session spawns its own toolsrv. Cross-session path serialization requires sessions to share a single toolsrv instance per host. - **Budget/depth controls** — token limits, step limits, and recursion depth caps for sub-agents. ## Phase 31: The Current Integration Boundary (Aug 17) The architecture was consolidated around explicit documentation and composition boundaries. The 9P filesystem remains the public control plane, while the agent core, prompt assembly, tool authoring, toolsrv, remote execution, and virtfs are documented as separate responsibilities. Common agent-framework features are deliberately not built into Ollie: MCP clients, embedded tool frameworks, planner/executor workflow engines, task graphs, schedulers, public coordination buses, frontend control planes, distributed agent state, and competing memory stores. These are integration points. They can be provided by tools, sessions, ordinary processes, or external systems such as OptMem and Beads. The current model is: ```text agent → toolsrv → executable or metadata-only tool → external system/state ``` Ollie supplies the agent loop, sessions, prompts, and 9P observation/control surface. The integration owns specialized state and semantics. ### Recent simplifications The period after the previous evolution entry completed a broad flattening and architecture cleanup: - Removed the remaining obsolete editor integration and consolidated 9P clients around the shared client implementation. - Moved the toolsrv client into `olliesrv` and split toolsrv protocol and metadata concerns into focused packages. - Removed obsolete environment/path packages and merged path handling into `util`; backend configuration became the single source for backend settings. - Added agent-scoped plans and metrics queries to the 9P namespace. - Consolidated tool metadata and tool authoring into one architecture; both executable-plus-`.meta` tools and metadata-only `cmd` tools are valid. - Replaced the old remote-execution, prompt-audit, EDSL, and 9P documents with architecture-focused documents, then deduplicated their boundaries. - Removed the legacy backend/provider environment fallbacks and the obsolete environment sample. Backend credentials, endpoints, models, and compaction models now live in `backends.conf`. - Documented the internal session event bus as an implementation detail rather than an external coordination API. - Made the composition boundary explicit: plan-and-execute systems such as Beads integrate through toolsrv; they are not Ollie runtime concerns. - Linked [OptMem](https://github.com/VictorTaelin/OptMem) as the external persistent-memory implementation. ### Current architecture At this point the runtime has two services and one integration surface: ```text 9P clients │ ▼ olliesrv ── sessions, agents, prompts, history, backends │ └─ authenticated 9P ── toolsrv ── registry, sandbox, processes └─ executable or metadata-only tools ``` `olliesrv` owns the agent runtime. It constructs the `virtfs` tree, serves the Ollie namespace, manages sessions and agents, resolves prompts, calls model backends, maintains history, and runs the agent loop. Its internal session event bus is used for runtime observers; it is not an external API. `toolsrv` is a separate authenticated 9P service. It owns tool metadata and host-variant discovery, per-agent loading, command execution, process state, output limits, sandbox policy, and bypass approval. Tool execution is not an in-process agent capability. A session may use a local toolsrv or an SSH-forwarded remote toolsrv; the agent runtime remains local. The 9P namespace is the external integration surface. Clients create sessions and agents, write prompts, read chat, wait on state, feed observers, and issue control commands through files such as: ```text session/new session/{sname}/agent/new session/{sname}/agent/{aname}/prompt session/{sname}/agent/{aname}/chat session/{sname}/agent/{aname}/statewait session/{sname}/agent/{aname}/feed session/{sname}/agent/{aname}/ctl generate ``` A request follows this path: ```text client → 9P → olliesrv → agent loop → toolsrv → tool └──────────────→ backend ``` The client can be a shell command, editor integration, GUI, terminal, observer, or external workflow. The client supplies context and writes a prompt; Ollie supplies the runtime and 9P state surface. `virtfs` separates namespace declaration from 9P transport. Tool authoring, remote execution, prompting, and editor integration remain separate concerns documented by the focused architecture documents. ## Phase 32: Goals, Workflows, and Index Split (Aug 17) Session-level goals and the conductor workflow pattern became first-class features. The session and agent index formats were split for cleaner parsing and better sub-agent tree support in the GUI. ### Goals and Workflows Sessions gained goal-related files: - `session/{s}/goal` — write goal text to trigger a workflow - `session/{s}/goalstatus` — read/write status (running/complete/blocked) - `session/{s}/goalwait` — block until status changes Writing to `goal` triggers `runWorkflow()` if status is empty, complete, blocked, or error. The workflow script (default: `conductor`) creates a conductor agent that reads the goal, explores the codebase, breaks work into tasks, and delegates to sub-agents via `subagent_spawn`. The conductor agent profile includes code intelligence tools (`code_outline`, `code_symbols`, `code_query`, `codebase_overview`) and LSP tools (`lsp_definition`, `lsp_references`, etc.) for understanding the codebase before delegating. ### Index Format Split The single `session/idx` line that crammed session and agent data together was split into two files: **`session/idx`** — session list only: ``` session-id\tsession-name\tpaused\tconnected\tremote\tcwd ``` **`session/{s}/agent/idx`** — agent tree per session: ``` session-id\tagent-id\tagent-name\tparent-id\tdepth\tstate ``` This enabled: - Simpler GUI parsing with no embedded semicolon-separated agent lists - Proper sub-agent tree display with parent-id and depth - Expandable/collapsible agent hierarchy in the session tree - Auto-selection of top-level agents (depth 0) instead of first-in-list ### GUI Sub-Agent Tree The KDE GUI's `SessionModel` gained: - `hasChildren` role for agents with sub-agents - `m_agentExpanded` map for tracking agent expansion state - Recursive `appendAgentTree` that respects parent expansion - Expand/collapse arrows for agents with children ### Bug Fix: ollie-9p UNAME Fallback `ollie-9p` previously required `$OLLIE_UNAME` when `$OLLIE_SESSION_ID` was set, making it unusable from non-agent contexts. The fallback to `$USER` was added, allowing normal users to read session files like `goal` and `goalstatus`. ## Phase 33: Native Landlock and Remote Deployment Simplification (Current) Sandbox enforcement moved from an external sandbox executable into the `toolsrv` binary. The policy source remains the runtime `sandbox.yaml`, so permissions can still change without recompiling or redeploying the policy. Each restricted tool execution starts a short-lived `sandbox-exec` helper mode inside `toolsrv`. The helper loads the policy snapshot, sets `no_new_privs`, creates a Landlock ruleset, grants the configured filesystem permissions, and executes the tool. Restrictions are applied to the child rather than the long-lived toolsrv process because Landlock rules are inherited and cannot be removed. The external sandbox dependency was removed from local and remote execution. Remote bootstrap now transfers only the toolsrv binary and runtime configuration. It no longer locates, copies, or installs a separate sandbox executable. Remote toolsrv instances enforce the same native policy on the remote host, while `--yolo` remains the explicit way to disable enforcement. The project was also described more precisely as a distributed, integrating AI agent runtime: orchestration and model calls remain local while toolsrv, tools, and external systems can be composed locally or across remote hosts. ## Phase 34: Agent Peers and Consensus Workflow (Aug 19) Inter-agent communication gained a first-class mechanism: **peer links**. Previously, agents in the same session could only be addressed from external scripts or through sub-agent spawn (which is transient). The peer system adds persistent, topology-controlled messaging between agents within a session. ### Peer mechanism Each agent exposes a `peer/` directory in its 9P namespace. Entries are write-only files named after peer agents. Writing to `peer/{name}` delivers the message to the named agent's prompt handler — identical semantics to writing to that agent's `prompt` file, but scoped by the peer relationship. Peer links are **bidirectional**: `peeradd A` on agent B also adds B to A's peer set. This is enforced in the `peeradd`/`peerdel` ctl commands. The topology constrains who can talk to whom — if an agent has no peer link to another, it has no `peer/{name}` file and cannot message it. This is the access control surface. Implementation details: - Agent struct: `peers map[string]struct{}` + RWMutex, lazy-init - Methods: `AddPeer`, `RemovePeer`, `Peers` (sorted) - `peer/` declared as a `virtfs.Each` node — dynamic directory rebuilt from peer set - `peeradd`/`peerdel`/`peers` ctl commands with bidirectional enforcement - Peer cleanup on agent removal: `RemoveAgent` strips the dead agent from all remaining peer sets - Peers persisted in `PersistedAgent.Peers` field; restored after all agents are created - `peeradd`/`peerdel` trigger immediate `s.Save()` for durability Total Go additions: ~55 lines in agent.go, ~50 lines in spec.go, ~15 lines in persist.go and session.go. ### Consensus workflow The first workflow to use peers: `consensus`. Unlike the conductor (which decomposes a goal into sequential/parallel subtasks), consensus runs the **same task N times independently** and synthesizes agreement. The workflow creates: - 1 **foreman** agent (profile: foreman, temp 0.3) — synthesis only, no investigation - N **panelist** agents (profile: panelist, temp 0.7) — independent analysis Each panelist is linked as a peer of the foreman (bidirectional). Panelists cannot message each other — the topology enforces the protocol. Flow: 1. Workflow script creates agents, establishes peer links, primes all 2. Each panelist investigates the goal from a different angle (correctness, simplicity, edge cases) 3. Panelists write findings to `peer/foreman` when done 4. Each delivery triggers a turn on the foreman 5. After receiving all N reports, the foreman synthesizes consensus in its chat output 6. Foreman writes "complete" to goalstatus The consensus output lives in the foreman's chat — the goal file is never overwritten. This respects the separation: goal = what to do, chat = what was learned. ### Design insight Peers vs sub-agents is not a replacement — it's a complementary primitive: - **Sub-agents** are stateless, transient, fire-and-forget. Good for decomposition. - **Peers** are persistent, stateful, conversational. Good for deliberation. The conductor workflow uses sub-agents (decompose → delegate → collect). The consensus workflow uses peers (investigate independently → report → synthesize). Both are wiring over the same 9P primitives. ## Phase 35: Embedding-Guided Tool and Skill Discovery (Aug 20) Ollie added a local embedding subsystem for semantic discovery of tools and skills. It loads the `all-MiniLM-L6-v2` sentence-transformer through ONNX Runtime, tokenizes descriptions, produces 384-dimensional vectors, and ranks matches with cosine similarity. The skill index scans configured `SKILL.md` directories, parses their frontmatter, precomputes description embeddings, and matches each user request against the indexed skills. The agent injects up to three relevant skills when they exceed the configured similarity threshold. Tool metadata is indexed by the same mechanism, allowing the agent to inject up to five relevant tool hints without placing every tool in the model context. The embedding model and ONNX Runtime are installed under the XDG data model directory by `make install-models` / `make install-data`. Skill directories may be overridden through `embedding.conf`; the default is the installed skills directory. Matching is lazy and cached per process, so the common path pays the model-loading cost once. This is a significant architectural shift for small models: discovery becomes semantic rather than dependent on exact tool or skill names, while progressive disclosure keeps the prompt bounded. The embedding model is local and separate from the conversational backend, so provider choice does not affect discovery. --- ## Phase 36: One-Tool Bootstrap and Progressive Capability Loading (Aug 21) The default agent profile now starts with exactly one callable tool: `client_9p`. The previous static `autoLoad` list was removed. This makes the runtime capability surface explicit: the agent can begin with the namespace client and load everything else through its own agent `ctl` file. Semantic matching remains a hint, not an implicit capability grant. When a request matches a tool description, the agent receives the tool name and the 9P operation required to load it. The model calls `client_9p` with: ```text rdwr session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/ctl tool_load ``` The agent runtime refreshes its model-facing schemas after a load or unload. `toolsrv` exposes an agent-scoped `tools_rev` request/response file so the client can detect registry changes without rebuilding tool state on every tool round. A revision of zero is treated as unavailable and preserves the safe refresh behavior. This completes the progressive-disclosure path: ```text client_9p → agent ctl → toolsrv registry → loaded tool schema → tool call ``` The important boundary is unchanged. `olliesrv` still owns prompting and the agent loop; `toolsrv` remains authoritative for discovery, loading, execution, sandboxing, and process state. The change removes ambient startup capability without adding a new orchestration subsystem. The `client_9p` script also expands `$OLLIE_SESSION_ID` and `$OLLIE_UNAME` in virtual namespace paths. This is required because JSON argument parsing does not perform shell expansion, and keeps the documented namespace examples directly callable by the model. ## Phase 37: Explicit Tool Loading and Generic Embedding Index (Aug 21) Lazy tool loading — where the agent could call any tool by name and have it loaded automatically — was removed. Tools now require explicit `autoLoad` declarations in agent configs. This is a reversal of the Phase 36 progressive loading experiment. ### Why lazy loading failed The one-tool bootstrap approach (Phase 36) relied on semantic hints telling the model to load tools through `client_9p`. In practice: 1. **Models called tools directly instead of loading them first.** Even with explicit instructions, models would attempt to call `file_read` or `shell` without the intermediate `client_9p tool_load` step. 2. **The indirection added latency and context cost.** Every tool use required an extra round-trip: hint → model decides to load → load call → refresh → actual tool call. This doubled the turns for common operations. 3. **Capability boundaries became unclear.** An agent's effective capability was "whatever it decides to load," which is not the same as "what it's configured to do." A code-review agent shouldn't have `shell` access just because it asked for it. ### The fix: explicit autoLoad per profile Each agent profile now declares exactly which tools it starts with: ```json { "name": "default", "autoLoad": ["shell", "file_read", "file_edit", "file_grep", "file_glob", "file_write", "lsp_hover", "lsp_definition", "lsp_references", "lsp_symbols", "lsp_diagnostics", "lsp_completion", "lsp_rename", "code_outline", "code_symbols", "code_query", "codebase_overview", "code_dependencies", "code_rewrite", "client_9p", "subagent_spawn", "skill_list", "skill_load", "memory_wake", "memory_remember", "memory_recall", "memory_zoom", "web_fetch", "reasoning_think"] } ``` Role-specific profiles (conductor, reviewer, panelist, etc.) load only what they need. An observer loads read-only tools. A foreman loads coordination tools. This makes capabilities explicit and auditable. ### Generic embedding index The embedding package gained a generic `Index[T]` type that replaces the skills-specific `skills.Index`. The same index structure now handles both skill matching (entries are `skills.Skill`) and tool matching (entries are `toolsrv.Meta`). ```go type Index[T any] struct { model *Model entries []T vectors []Vector textFn func(T) string } ``` Tool matching was split from skill matching: - `skill_match.go` — builds skill index once per process, matches per turn - `tool_match.go` — builds tool index per turn from loaded tools only The tool index now reflects the agent's actual loaded tools, not all installed metadata. This aligns semantic discovery with the explicit loading model. ### Agent config alignment All 14 agent configs were updated to declare tools appropriate to their roles: | Profile | Purpose | Key tools | |---------|---------|-----------| | default | General coding | Full tool set | | conductor | Task decomposition | Code intel + subagent_spawn | | reviewer | Code review | Read-only + LSP | | foreman | Consensus synthesis | Coordination only | | panelist | Independent analysis | Read + reasoning | | observer | Watch and comment | Read-only | | driver | Remote execution | Shell + file tools | | copilot | IDE assistance | Code intel + LSP | | writer | Documentation | File tools + reasoning | | researcher | Investigation | Read + web + reasoning | | planner | Architecture | Read + reasoning | | debugger | Troubleshooting | Full tool set | | tester | Test writing | Code + shell | | refactorer | Code transformation | Code + LSP + rewrite | ### What was removed - `load-on-call` tool loading (the Phase 36 mechanism) - `client_9p` load hints in tool-hints injection - `skills.Index` (replaced by generic `embedding.Index[T]`) - Combined skill/tool matching in `skill_match.go` ### Source changes ```text embedding/index.go +89 new generic Index[T] skills/skills.go -70 removed Index, kept Skill type skill_match.go -40 removed tool matching tool_match.go +55 new per-turn tool index data/agents/*.json +14 autoLoad declarations ``` ## Phase 38: Streaming Rdwr and Event Filtering (Aug 22) A new 9P primitive for filtered event subscriptions, and cleanup of unused blocking mechanisms. ### The streaming rdwr pattern The `event` file gained write-then-stream semantics: write a filter pattern, then stream matching events. This is a new 9P interaction pattern — atomic write followed by indefinite streaming read on the same open fid. ```sh # Subscribe to agent state changes only echo "session.*.agent.*.state" | ollie-9p rdwrs event ``` Filter syntax uses `*` to match one segment and `>` to match all remaining segments: - `session.*.agent.*.state` — all agent state changes - `session.abc123.>` — all events for one session - `*` alone matches everything (default behavior) Implementation uses per-fid state in the server: - `eventFilter` stores the compiled pattern - `eventCh` receives matching events - `eventCancel` cleans up on fid clunk The `ollie-9p` client gained an `rdwrs` command for streaming rdwr operations. ### GUI event consolidation The KDE GUI switched from per-agent statewait streams to a single server-wide event stream. This reduced connection overhead and simplified the streaming architecture. The GUI now: - Opens one `event` stream per daemon connection - Parses event topics to dispatch state updates - Handles bypass requests through the same event path ### What was removed **`statewait` file** — replaced by the `event` stream with filtered subscriptions. The per-agent blocking read was superseded by the more flexible event filtering. **`bypasswait` file** — redundant since bypass requests flow through the event stream as `session.{sid}.bypass.request` events. **`feed` file and observer agents** — the feed mechanism (change-detecting BlockOnce input) was documented but never used by any frontend or script. The observer agent pattern was never adopted. Removed: `feed.go`, `FeedWrite`, `ConsumeFeed`, `WatchFeed`, and all related session wiring. ### Event topics The event stream now carries all real-time state: | Topic | Payload | |-------|---------| | `session.{sid}.new` | name | | `session.{sid}.kill` | — | | `session.{sid}.rename` | oldName newName | | `session.{sid}.pause` | — | | `session.{sid}.resume` | — | | `session.{sid}.agent.{aid}.new` | — | | `session.{sid}.agent.{aid}.kill` | — | | `session.{sid}.agent.{aid}.state` | idle\|calling\|thinking\|paused | | `session.{sid}.agent.{aid}.bypass.request` | id\tcmd\tcwd | | `session.{sid}.agent.{aid}.bypass.resolved` | id\tapproved/denied | ### Source changes ```text server.go +120 per-fid event filter state, write/read handlers session/event.go +45 SubscribeEventsFiltered, MatchTopic ollie-9p/main.go +25 rdwrs command o (script) +10 filtered event subscription for tui ollie9pclient.cpp -50 removed statewait streams agent/feed.go -46 deleted agent/agent.go -67 removed FeedWrite, ConsumeFeed agent/state.go -8 removed WatchFeed session/session.go -8 removed ConsumeFeed goroutines fs/spec.go -35 removed feed, statewait, bypasswait files ``` Net: **-222 lines** of unused infrastructure removed. ## Phase 39: Bypass Coordination and State Indicators (Oct 6) Cross-client bypass approval and real-time agent state visualization in the GUI. ### Bypass request/resolve coordination The bypass system now emits events for both requests and resolutions, enabling CLI and GUI to coordinate: **Request event** (existing): `session.{sid}.agent.{aid}.bypass.request` with payload `id\tcmd\tcwd` **Resolved event** (new): `session.{sid}.agent.{aid}.bypass.resolved` with payload `id\tapproved/denied` The GUI tracks pending bypasses per-agent in an in-memory collection (`m_pendingBypasses` QHash), enabling support for multiple concurrent pending bypasses (e.g., from background processes). When CLI resolves a bypass, the server emits the resolved event, and the GUI clears its state automatically. ### CLI bypass workflow Added three commands to the `o` CLI wrapper: ```bash o sess bypass # inspect pending: shows id, agent, cwd, cmd o sess approve [id] # approve (optional id for safety) o sess deny [id] # deny ``` The optional `id` argument prevents accidental approval of the wrong request when multiple bypass requests may have been issued. ### GUI indicators **Pending bypass indicator** — Yellow ⚠ to the left of agent name in session tree. Visible when `pendingBypassCount > 0` for that agent. Clears automatically on resolution (by GUI buttons or CLI). **Agent state indicator** — Colored dot to the right of agent name showing execution state: - 🟢 Green: idle - 🔵 Blue: thinking - 🟠 Orange: calling tool - ⚪ Gray: paused Both indicators use relative geometry (percentages of row height) for proper scaling across font sizes and DPI settings. ### Source changes ```text session/session.go +12 emit bypass.resolved event ollie9pclient.h +8 m_pendingBypasses, pendingBypassCount, bypassResolved signal ollie9pclient.cpp +45 handle bypass.resolved event, track pending counts ChatPane.qml +12 onBypassResolved handler SessionTree.qml +55 bypass indicator (⚠) and state indicator (dot) data/scripts/o +65 bypass, approve, deny commands ``` ## Phase 40: JSONL Chat Log Format (Oct 7) Replaced the custom `[[[role#id]]]...[[[end]]]` delimiter format with JSON Lines (JSONL) — one JSON object per line. Fixes streaming partial display and eliminates brittle regex parsing. ### Old format (deleted) ``` [[[user#abc12345]]] hello [[[end]]] [[[assistant#def67890]]] response text [[[end]]] ``` Problems: - `[[[end]]]` must appear on its own line, but streaming chunks don't respect line boundaries - Regex-based parsing is fragile and error-prone - No standard tooling for reading/searching ### New format (JSONL) ```json {"role":"user","id":"abc12345","content":"hello"} {"role":"assistant","id":"def67890","content":"response text"} {"role":"call","id":"aaa11111","name":"shell","content":"{\"cmd\":\"ls\"}"} {"role":"tool","id":"bbb22222","content":"file1.txt\nfile2.txt","format":"text"} ``` Each line is a complete JSON object with: - `role`: user, assistant, context, reasoning, call, tool, error, info, retry, stalled - `id`: 8-char hex block ID (deterministic from sha256) - `content`: block text - `name`: tool/function name (for call blocks) - `format`: output format hint (for tool blocks) - `partial`: true if streaming in progress ### Server changes **format/block.go** (new): `Block` struct with JSONL marshal/unmarshal, `RenderBlock` for filtered text output. **Deleted**: `format/event.go`, `format/format.go`, `format/format_test.go` — old delimiter format entirely removed. **agent/chat.go**: Dual logs — `rawLog` (JSONL for programmatic access) and `textLog` (rendered text for humans). `AppendBlock` writes to both. Streaming starts from beginning for replay. **agent/chatlog.go**: Event handler emits JSONL blocks via `flushPartial` (partial=true per chunk) and `closeBlock` (partial=false final). Skips internal events: `state`, `usage`, `limitretry`. **fs/spec.go**: New 9P files: - `log.raw` — JSONL snapshot (one-shot read, for GUI startup) - `log` — Rendered text snapshot (non-blocking, last 64KB) - `chat` — Rendered text stream (blocking, for TUI) - `block` — Rdwr lookup: write block ID, read JSON Deleted: `chat.raw`, `chat.search` ### GUI changes **chatblockmodel.cpp**: Complete rewrite. Uses `QJsonDocument` for parsing instead of regex state machine. Filters `context` role from display. Removed `m_inContext` tracking, `renderBlock` method, and regex patterns. **ollie9pclient.cpp**: `readLogForSession` reads from `/log.raw` (JSONL) instead of `/log` (plain text). `log.raw` is a one-shot snapshot read, not a stream; live updates arrive via the separate `event` stream. **chatblockmodel.h**: Added `Context` to `ChatBlock::Type` enum. Removed unused members. ### Context blocks Context injected by the server (user prompts, matched skills, tool hints) now emits as a separate `"context"` role block, followed by the actual user message. Context blocks are: - Sent to the LLM (wrapped in `` tags in history) - Filtered from GUI display (role-based, not pattern-based) - Filtered from rendered text log via `RenderBlock` ### Source changes ```text format/block.go +68 new Block struct, JSONL marshal/unmarshal, RenderBlock format/event.go -67 deleted format/format.go -107 deleted format/format_test.go -188 deleted agent/chat.go +95 dual logs, AppendBlock, BlockByID, streaming from start agent/chatlog.go +35 JSONL emission, skip internal events fs/spec.go +45 log.raw, log, block files; removed chat/chat.raw/chat.search chatblockmodel.cpp -400 JSONL parsing, removed regex state machine chatblockmodel.h +3 Context type, removed m_inContext ollie9pclient.cpp +5 read from log.raw ``` Net: **-680 lines** — simpler, more robust, standard format. ## Phase 41: Authoritative Live Chat Stream (Oct 10) Phase 40's JSONL `log.raw` doubled as both the persistent snapshot and the GUI's live stream. Because the JSONL buffer is append-only, `flushPartial` appended a *new* line for every streaming chunk — each carrying the full cumulative content — so a single assistant response left dozens of partial lines permanently in `log.raw`. The GUI, parsing the snapshot keyed by block ID, re-rendered the growing block once per partial line (O(N²) per response), and every reconnect replayed all historical partials. This produced the intermittent GUI rendering loops. It was fundamentally a data-input problem: partials were persisted, not just streamed. ### Fix: separate persistence from live delivery - **Partials are no longer persisted.** `AppendBlock` writes only finalized blocks to `rawLog`/`textLog`. A new `SetPartial` records the current in-flight block in a single `partialLine` field (bumping `partialVers`) and broadcasts it to stream readers — it never touches `rawLog`. `chatlog.go`'s `flushPartial` now calls `SetPartial`; `closeBlock` still calls `AppendBlock`, which clears the partial. - **`log.raw` is a finalized-only one-shot snapshot.** One line per logical block, bounded, no duplication. Used for explicit history loads (initial populate, bookmark reload), never polled. - **`chat.raw` is the authoritative live JSONL stream** (`Agent.RawLogStream`, `StreamRaw`). On open it replays `rawLog` (finals) from the start, then streams live deltas: newly finalized blocks (offset-safe, since `rawLog` is append-only and never rewritten) and the current partial whenever it changes (out-of-band, keyed on `partialVers`). The GUI collapses by block ID, so reconnects are O(history), not O(streaming chunks). Base encodes `:`. ### GUI `ollie9pclient.cpp`: `startActiveAgentStreams` streams `/chat.raw` instead of `/log.raw`. The one-shot `readLogForSession` still reads `/log.raw` for `loadChat` and bookmark reloads — `log.raw` is now purely a snapshot, never a live source. ### Source changes ```text agent/agent.go +4 partialLine/partialVers fields; cleared on Clear agent/chat.go +60 SetPartial, signalChat, RawLogStream; AppendBlock finals-only agent/chatlog.go ±1 flushPartial -> SetPartial fs/spec.go +7 chat.raw StreamRaw file ollie9pclient.cpp ±1 stream chat.raw, not log.raw ``` ## Phase 42: Tool and Bypass Visibility in Text Views (Oct 10) The rendered text views (`log`, `chat`) existed for clients without a JSONL parser or event stream — the TUI and the `o` script. But `RenderBlock` hid every `call` and `tool` block, so those views showed only user and assistant prose: no indication a tool ran, what it returned, or that a sandbox-bypass request was waiting for approval. Bypass requests were published solely as `event` topics, invisible to anyone reading `chat`. ### Changes - **`format/block.go`**: `RenderBlock` now surfaces `call` (`→ name args`), `tool` (output), `bypass` (`⚠ bypass requested (needs approval): ...`), and `bypass-resolved` (`bypass approved`/`denied`) blocks. `reasoning` and `context` stay hidden; partials still render empty. - **`session.go`**: `SetBypassPending` and `ResolveBypass` locate the requesting agent via `FindAgent(req.Env["OLLIE_UNAME"])` and append `bypass`/`bypass-resolved` chat blocks, so the pending-approval state and its outcome appear in `log`/`chat` alongside the existing `bypass.request`/`bypass.resolved` events. GUI clients, which read `chat.raw`, also receive these as plain-text blocks in addition to their dedicated approval dialog. ### Source changes ```text format/block.go +12 render call/tool/bypass/bypass-resolved format/block_test.go +72 new RenderBlock coverage session.go +20 emit bypass chat blocks on request/resolve agent/chat_test.go ±6 call content now expected in rendered log AGENTS.md ±1 log/chat filtering description ```