# Architectural Evolution

How the Ollie system-of-systems emerged and evolved.

This is a study log. It records what was built, what was killed, and why —
in the order it happened. The dead ends are as instructive as the survivors:
the rejected ideas at the end teach more about building agent systems than
the architecture that remains. The document evolves with the author's
understanding; when past decisions turn out to be wrong, they get recorded
here, not hidden.

## Timeline
```mermaid
gantt
    title Ollie Evolution
    dateFormat  YYYY-MM-DD
    axisFormat  %b %d
    section Foundation
    Monorepo + submodules          :done, 2026-04-11, 14d
    9P filesystem                  :done, 2026-04-14, 28d
    section Core Primitives
    Session lifecycle (session/)   :done, 2026-04-20, 30d
    Tool definitions                :done, 2026-05-01, 20d
    Skills system                  :done, 2026-05-10, 14d
    section Execution
    execute_code / shell           :done, 2026-05-08, 20d
    Batch jobs (b/ → session/bfg)  :done, 2026-05-15, 25d
    Sandbox (landrun)              :done, 2026-05-20, 40d
    section Multi-Agent
    Subagent spawn                 :done, 2026-05-25, 15d
    Cascade orchestrator           :done, 2026-06-05, 20d
    JIT agent generation           :done, 2026-07-10, 10d
    section Frontends
    TUI + Emacs (ellie)            :done, 2026-04-15, 45d
    Acme (Plan 9)                  :done, 2026-05-20, 30d
    KDE/Plasma                     :done, 2026-06-10, 40d
    section Control Planes
    D-Bus adapter (ollied)         :done, 2026-06-25, 20d
    D-Bus embedded in srv          :done, 2026-07-15, 10d
    D-Bus removed                  :done, 2026-07-31, 1d
    section Late Evolution
    Remote execution               :done, 2026-07-05, 15d
    Bypass broker               :done, 2026-07-10, 15d
    Tool registry (lazy)           :done, 2026-07-20, 10d
    FUSE → 9P client               :done, 2026-07-25, 5d
    The Great Flattening           :done, 2026-07-29, 2d
    9P Declarative EDSL            :done, 2026-08-02, 1d
    Zero built-in tools            :done, 2026-08-02, 2d
    EDSL extraction + cleanup      :done, 2026-08-03, 2d
    Good idea fairy cleanup        :done, 2026-08-05, 1d
    toolsrv 9P + process isolation :done, 2026-08-10, 2d
    Meta-only tool definitions     :done, 2026-08-11, 2d
    Bypass via 9P                  :done, 2026-08-11, 1d
    fs/ flattening                 :done, 2026-08-11, 1d
    Feed + observer agents         :done, 2026-08-13, 1d
    BlockOnce/Stream refactor      :done, 2026-08-13, 1d
    Sub-agents via 9P rdwr         :done, 2026-08-14, 1d
```
## Phase 1: Monorepo Bootstrap (Apr 11)
Started as independent git repos unified under a monorepo with submodules.
Initial components:
- **agent** — agent loop (Go): backend dispatch, tool execution, session state
- **fs**** — 9P file server exposing agent sessions as a synthetic filesystem
- **tui** — terminal UI (later removed)
- **el** — Emacs integration (ellie.el)
Build system was `mkfile` (Plan 9 make), later `Makefile`, finally `justfile`.
## Phase 2: The 9P Decision (Apr 14–28)
The defining architectural choice: **every agent primitive is a file**.
Sessions are directories. Sending a prompt is writing to `session/{sname}/prompt`.
Reading state is `cat session/{sname}/state`. This made the control plane
protocol-agnostic — any program that can read/write files can be a frontend.
Key files crystallized: `prompt`, `chat`, `state`, `ctl`, `cfg`, `statewait`.
The `session/new` file (write key=value pairs, read back session ID) became the
session factory.
## Phase 3: Tool System (Apr 28 – May 10)
Tools evolved through several incarnations:
1. **MCP servers** (denote-mcp, 9beads-mcp) — external processes, heavyweight
2. **Shell/Python scripts** in `~/.config/ollie/tools/` — lightweight, sandboxed
3. **execute_code** as the single built-in tool — all scripts invoked through it
4. **call_tool/pipe** — named tool dispatch and cross-tool pipelines
5. **Tool registry** (final form) — dynamic lazy-loading via `tool_list`/`tool_load`/`tool_active`; `execute_code` renamed to `shell`; tools hot-reloadable without server restart
The file tools (`file_read`, `file_write`, `file_edit`, `file_grep`, `file_glob`)
were extracted into standalone Python scripts. Memory (`memory_remember`,
`memory_recall`) and reasoning (`reasoning_think`) followed the same pattern.
MCP servers were removed early. Simple scripts won over complex daemons.
## Phase 4: Planning System Churn (May – Jul)
Planning went through the most iterations of any subsystem:
1. `pl/` directory namespace with flat file-per-task
2. `plan_create` / `plan_complete` tools
3. **Beads** integration (external issue tracker) — made optional, then dropped
4. `task_add` / `task_check` tools
5. Inline markdown plans forbidden
6. **Final form**: a single `session/{sname}/plan` file (markdown checklist), written by
   the agent, persisted across context compaction
The lesson: planning needed to be simple, agent-controlled, and not bureaucratic.
## Phase 5: Multi-Agent Coordination (May 25 – Jun 15)
Three patterns emerged:
- **subagent_spawn** — fire-and-forget session creation, parent never blocks
- **subagent_generate** — JIT agent config generation (role, constraints, tools)
- **cascade** — script-driven fan-out with `-max-workers`, `-retries`, optional
  synthesis step
```mermaid
flowchart TB
    UP["User prompt"]
    PARENT["Parent Session"]
    W1["Worker 1"]
    W2["Worker 2"]
    W3["Worker 3"]
    CASCADE["u/cascade\n(spawn + throttle)"]
    CW1["Cascade Worker"]
    CW2["Cascade Worker"]
    UP --> PARENT
    PARENT -- subagent_spawn --> W1
    PARENT -- subagent_spawn --> W2
    PARENT -- subagent_spawn --> W3
    W1 --> PARENT
    W2 --> PARENT
    W3 --> PARENT
    PARENT -- cascade --> CASCADE
    CASCADE --> CW1
    CASCADE --> CW2
```
Inter-agent communication uses the filesystem: agents write to each other's
`prompt` file.
## Phase 6: Frontend Proliferation
```mermaid
flowchart TB
    subgraph Core["agent.Agent"]
        AG["Agent Engine"]
    end
    subgraph Surfaces["Integration Surfaces"]
        P9["9P Filesystem\n(session/ namespace)"]
        DB["D-Bus\n(org.ollie.SessionManager)"]
    end
    subgraph Frontends
        SH["s/sh (terminal)"]
        ACME["acme (Plan 9)"]
        EL["ellie (Emacs)"]
        KG["KDE GUI"]
        KP["KDE Plasmoid"]
        KK["Kate Plugin"]
        KR["KRunner"]
        HTTP["curl / scripts"]
    end
    AG --> P9
    AG --> DB
    SH --> P9
    ACME --> P9
    EL --> P9
    HTTP --> P9
    KG --> DB
    KP --> DB
    KK --> DB
    KR --> DB
    WEB --> DB
```
- **s/sh** — shell script frontend (bash, briefly rc, back to bash)
- **acme** — Plan 9 editor integration with workspace-scoped navigator sessions
- **ellie.el** — Emacs with ghost-text completion
- **ollie-kde** — full Plasma integration: plasmoid, KRunner, Kate plugin, standalone GUI, GUI automation tools
- **TUI** — removed (Jul) in favor of s/sh and richer GUIs
## Phase 7: Security Model (May – Jul)
```mermaid
flowchart LR
    subgraph Permissions["9P Permissions"]
        OWNER["Owner → rw"]
        AGENT["Agent → restricted"]
        PEERS["Peers → read-only"]
    end
    subgraph Execution["Execution Sandbox"]
        AGT["Agent"]
        SANDBOX["landrun sandbox\n(restricted fs)"]
        TOOLS["Tool definitions"]
        BYPASS["x/bypass\n(socket broker)"]
        PRIV["Privileged Action"]
    end
    AGT --> SANDBOX --> TOOLS
    TOOLS -- needs escape --> ELEV --> PRIV
```
Evolution:
1. No sandboxing initially
2. YAML-based sandbox configs (landrun) for execute_code
3. Per-session 9P identity — agents can't read other sessions' tools
4. Bypass broker: socket-based, user-confirmed privilege escalation
5. SSH agent proxy added then removed (too much attack surface)
6. Final: integrated bypass broker replaces separate superpowerd adapter
## Phase 8: D-Bus as Second Control Plane (Jun 25 – Jul)
Originally everything was 9P-only. D-Bus was added for desktop integration:
1. Separate `ollie-dbus` daemon (ollied)
2. Embedded directly into `olliesrv` (9P server got `-no9p` flag)
3. KDE uses D-Bus exclusively; acme/sh/el use 9P
4. **9P and D-Bus do not share sessions** — independent stores, same core
## Phase 9: Remote Execution (Jul 5–19)
Split-brain architecture: orchestration stays local, code execution on a remote
machine via SSH.
- `ollie-remote` binary auto-deployed via SSH bootstrap
- Tools embedded in remote binary
- Streaming output notifications back to local session
- Session persistence across remote restarts
## Phase 10: Tool Registry (Jul 20–29)
The final major architectural shift — from "agent knows all tools upfront" to
lazy discovery:
1. Tools embed their own `ollie:prompt` metadata blocks
2. `tool_list` — discover available tools and descriptions
3. `tool_load` — promote a tool to a native callable
4. `tool_active` — introspect loaded tools
5. Skills get the same treatment: `skill_list`, `skill_load`, `skill_active`
This solved system prompt bloat — only load what's needed per task.
## Phase 11: The Great Flattening (Jul 29–30)
Largest single-day structural change: **−4,573 lines net** across the codebase.
Eliminated accumulated abstractions that no longer served a purpose.
### Core (−4,400 lines)
- **Killed `agent.Core` interface** — concrete `*agent.Agent` used directly.
  Nobody else implemented Core; the interface just added indirection.
- **Killed `Dispatcher` indirection** in toolsrv — `toolsrv.Server` called directly.
- **Merged `execute/` into `tools/`** — separate package for 3 functions was
  pointless overhead.
- **Renamed for clarity**: `agent.Session` → `agent.History`,
  `session.Core` → `session.Session`, `NewAgentCore` → `New`.
- **Extracted `agent/` package** from the monolithic session package.
- **Threaded context.Context** properly through daemon → session → agent
  (replaced ad-hoc interrupt mechanisms with cancellation).
- Dead code removal: `GlobalToolNames`, `ExtractReturnSchema`.
### 9P server (−214 lines)
- **Killed `mgr/` package entirely** — replaced `Manager` struct with package
  functions in `**fs**/`. The `*fs.Tree` IS the session collection; `rootState`
  (unexported) lives in `tree.Data`. CRUD via `NewRoot`, `Lookup`, `Create`,
  `Kill`, `Rename`, `Shutdown`.
- **Fixed infinite recursion** in `session/{sname}/agent/` directory — `pathType()`
  didn't recognize agent subdirectories, causing walk to succeed at any depth.
- **Fixed phantom session descent** — walk into non-existent sessions now fails
  immediately.
- **Fixed `/agents` empty read** — `makeStat` wasn't computing Length for root
  files, FUSE kernel saw 0 bytes and never issued a read.
- **Fixed `session.All()`** — was looking at `root.Children()` (empty) instead
  of `rootState.sessions`. Broke D-Bus ListSessions and GUI session restore.
### Frontends
- **acme**: paths updated `s/` → `session/`, agent files route through
  `session/{sname}/agent/{aname}/`.
- **ellie.el**: same path migration, added `ellie--agent-dir` for agent ID
  discovery.
- **KDE GUI**: no changes needed (D-Bus operates on Go objects, not paths).
### New filesystem layout
```
session/
├── new              write key=value to create session
├── idx              session index (tab-separated)
├── {sname}/
│   ├── plan         session-scoped markdown checklist
│   ├── env          session environment
│   └── agent/
│       └── {aname}/
│           ├── cfg          session configuration (key=value)
│           ├── ctl          control commands
│           ├── state        current state
│           ├── statewait    blocks until state changes
│           ├── chat         conversation log
│           ├── prompt       submit prompts
│           ├── prompt.prev  last submitted prompt
│           ├── offset       byte offset after last user prompt
│           ├── fifo.in      queue a prompt (FIFO input)
│           ├── fifo.out     pop queued prompt (FIFO output)
│           ├── context      rendered context window
│           ├── systemprompt rendered system prompt
│           ├── usage        token stats
│           ├── cost         cost estimate
│           ├── ctxsz        context size
│           ├── models       available models
│           ├── tools        tool management
│           ├── toolsrv.loaded  currently loaded tools
│           ├── toolsrv.rev     revision counter
│           ├── tail         exec helper
│           └── proc/        detached process output
```
### Architecture after (single Go module, no submodules for core/9p)
```
*fs.Tree (root)                     ← IS the session collection
  └── .Data = *rootState            ← sessions map, config, nextUID
       └── sessions[id] = *Session  ← owns Agent, context, cancel
Package functions (no Manager):
  session.NewRoot(cfg) → *fs.Tree
  session.Lookup(tree, id) → *Session
  session.All(tree) → []*Session
  session.CreateFromRoot(tree, args) → (id, error)
  session.KillFromRoot(tree, id)
  session.RenameFromRoot(tree, old, new)
  session.Shutdown(tree)
```
### SLOC after flattening
| Component | Lines |
|-----------|-------|
| agent + backend + toolsrv + session | 10,437 |
| **fs** + cmd/olliesrv | 7,925 |
| kde gui | 12,331 |
| **Total** | **32,538** |
Zero dead exported functions remain (verified via LSP + grep across all repos).
## Phase 12: Second Flattening & .meta Sidecar (Jul 30)

Another aggressive structural pass. Eliminated the entire script namespace
layer (s/, u/, x/), flattened `contrib/` into `data/`, and replaced
header-comment-based tool metadata with JSON sidecar files.

### Scripts removed (−1,100 lines)
- **s/{sh,b,bfg,bbg,ls,kill,cleanup}** — shell-based session frontends replaced
  by `ollie-9p rdwr generate/complete/route` and GUI frontends
- **u/{optimize,complete,cascade,escalate}** — utility scripts replaced by
  `generate`, `complete`, `route` files in the 9P namespace
- **x/{bd,prime,freeloader,task}** — internal plumbing scripts; PATH prepend
  (`prependOlliePath`) removed from toolsrv
- 9P session root trimmed to just `new` and `idx`; script-serving code removed
- `install-scripts` target now empty

### Tool metadata: .meta sidecar files
Header comment parsing (`ollie:prompt`, `ollie:tier`, `ollie:parallel read`,
`args_json:`) **eliminated entirely**. All tool metadata now lives in a JSON
sidecar file alongside the executable:

```
tools/
  file_edit       ← executable (any language)
  file_edit.meta  ← JSON metadata
```

```json
{
  "description": "Replace text in a file.",
  "prompt": "## file_edit\n\n...",
  "args": {"type":"object", ...},
  "tier": "cold",
  "readOnly": true
}
```

This decouples metadata from implementation language — compiled Go binaries,
Python scripts, and bash tools all use the same discovery mechanism.

### LSP tools ported to Go
The Python LSP bridge (`_lib/lsp/`) replaced with a pure Go implementation:

```
tools/
├── builtin/          ← in-process handlers (shell, reasoning, tool/skill registry)
└── lsp/              ← shared LSP client library
    └── cmd/          ← individual binaries (lsp_definition, lsp_hover, ...)
```

Architecture: each LSP tool binary embeds a bridge daemon (started on first
invocation via `--bridge` flag, persists via Unix socket, auto-exits after 5min
idle). No Python. No `_lib`. No external dependencies beyond the LSP servers
themselves (gopls, clangd, intelephense).

### Repo reorganization
- `contrib/{prompts,scripts,services,tools,skills,agents}` → `data/`
- `contrib/elisp` stays (Emacs frontend contribution)
- `tools/builtin/` — built-in tool handlers (moved from `tools/`, removed in Phase 18)
- `tools/lsp/` — Go LSP implementation (replaced `data/tools/_lib/lsp/`)

### Conditional variants for network transparency
Tools can declare multiple variants gated by host conditions (`binary`, `file`,
`os`, `arch`, `env`, `nenv`). First match determines the tool's schema and
executable. Enables one `.meta` to work across heterogeneous hosts — the
contract adapts to capabilities. Critical for `ollie-remote` deployments where
the remote host may have different tools, package managers, or init systems.

## Principles That Emerged
1. **Filesystem-as-API** — everything is read/write on synthetic files. No
   custom protocols needed.
   Monorepo coordinates versions.
3. **Scripts over servers** — MCP servers removed early. Simple scripts won.
4. **Progressive disclosure** — lazy tool/skill loading keeps system prompts
   small until complexity is needed.
5. **Plan simplicity** — after 5 iterations, planning settled on one markdown
   checklist file per session.
6. **Single control plane — 9P for everything.
   Streaming via blocking reads. No polling.
7. **Security by default** — sandboxed execution with explicit bypass,
   per-session identity.
## Phase 13: 9P Streaming & D-Bus Removal (Jul 31)

The D-Bus adapter (`org.ollie.SessionManager`) is deleted. All clients now
use 9P exclusively via blocking reads for natural streaming.

Key changes:
- **`chat` file**: blocking read that delivers tokens as the agent produces
  them. Per-fid offset. Never EOF (blocks between turns). EOF only on kill.
- **`log` file**: replaces old `chat`. Returns last 64KB (sliding window).
  Non-blocking, tail-able via Qid.Vers.
- **`kill`/`.` ctl command**: kills session → EOF on chat readers.
- **KDE GUI**: rewritten from scratch. Uses `9p` (plan9port) via QProcess.
  Streaming via `readyReadStandardOutput`. Plain text tail (8KB).
  No ChatBlockModel, no D-Bus, no ThemeManager. ~250 lines total.
- **Kate plugin**: all D-Bus calls replaced with `9p` subprocess calls.
  Streaming chat + statewait via persistent QProcess.
- **KRunner**: uses `9p` for session listing and one-shot generation.
- **Acme frontend**: uses 9fans.net/go plan9/client directly. Blocking
  read loop on `chat`. No FUSE mount.
- **Plasmoid, tray**: deleted (not useful enough to maintain).
- **dbus/ package**: deleted (-771 lines from server).

The only remaining godbus usage: `org.freedesktop.Notifications` for
bypass prompts (desktop integration, not ollie protocol).

Lessons:
- 9P's request-response model gives natural streaming for free.
  Server delays Rread until data arrives. No polling needed.
- FUSE mounts do NOT support blocking reads (return EOF immediately).
  Clients must use the 9P protocol directly.
- Never bind a growing string to a QTextArea with word-wrap.
  QTextDocument relayout is O(n) on the full content.
- The `loadEarlier` scroll-triggered cascade was creating 500 delegates
  on startup. Bounded initial window + explicit scroll-back is correct.

## Phase 14: Multi-Agent Deepening — ID/Name Split & Empty Sessions (Aug 1)

The multi-agent data structures introduced during the Great Flattening
required further refinement to support true multi-agent sessions.

### ID/Name Split
Sessions and agents now have **two identities**:
- **`id`** (immutable UUIDv4, set at creation, never changes)
- **`name`** (mutable human-readable string, defaults to first UUID segment)

The sessions map is keyed by Name, not ID. Renaming a session or agent moves
the directory in the 9P namespace — `mv` on the session directory calls
`RenameFromRoot`, and `write` to the agent's `name` file calls `SetName`.

Both entities expose `id` and `name` files in their 9P directories.

### Two-Step Session Creation
Creation split into two phases:
1. `session/new` — write `name=<name>` to create an empty session (no agent yet)
2. `session/{name}/agent/new` — write `cwd=<dir> backend=<backend> model=<model> agent=<agent>` to create the agent

This enables creating sessions in advance and attaching agents later. Empty
sessions (`Core=nil`) are valid — they appear in `session/idx` with empty
fields and are fully killable/renameable.

### Multi-Agent Data Structures
- `session.Session` supports multiple agents (`Agents()` returns `[]*agent.Agent`)
- `Session.AgentLogs` maps agent IDs to `*AgentLog` instances
- Active agent is selectable, each agent has its own prompt/chat/state/log
- `session/idx` emits one line per agent, not one line per session

Key commits: 92 commits across Aug 1 touching session/, agent/, fs/, and kde/.

## Phase 15: 9P Declarative EDSL (Aug 2 — morning)

The most significant structural change to the 9P server since its inception:
**replaced the imperative filesystem with a declarative EDSL**.

### Before: Imperative
Every 9P operation (stat, walk, read, write, open, create, remove) was
hand-coded per node. The `fs/session/` sub-package contained ~3,100 lines
of manual tree-building code split across 6 files (`files.go`, `root.go`,
`perm.go`, `synth.go`, `create.go`, `persist.go`). Adding a new file meant
updating stat, readdir, open, and permission code paths.

### After: Declarative EDSL
The entire namespace is declared in a single `FsNodeDecl` tree in `fs/spec.go`:
```go
var treeSpec = Dir("/",
    Leaf("backends", 0444, Read(readBackends), GID("agent")),
    Leaf("eventwait", 0444, Read(readEventwait), Stream(blockEventwait), GID("agent")),
    Leaf("complete", 0666, Request(requestComplete), GID("agent")),
    // ...
    Dir("session", GID("agent"),
        Leaf("new", 0666, Request(requestSessionNew)),
        Leaf("idx", 0444, Read(readSessionIdx)),
        TemplateDir("{id}", listSessions),
    ),
)
```

`BuildTree()` in `fs/builder.go` walks the spec, validates invariants
(directories can't have leaf handlers, at most one blocking variant,
template nodes must have `List`, ownership inheritance), and produces
a fully-wired `*fs.Tree`.

### Package consolidation
- Entire `fs/session/` sub-package flattened into `fs/` — 6 files deleted
- `cmd/olliesrv/bypass_tree.go` — bypass tree moved to `fs/bypassfiles.go`
- `cmd/olliesrv/server.go` simplified from ~820+ lines to ~200 lines of thin
  9P protocol handling; all filesystem logic is now in `fs/`
- Handler files organized by scope: `rootfiles.go`, `sessionfiles.go`,
  `agentfiles.go`, `bypassfiles.go`, `procfiles.go`

### Net result
- **−3,152 lines** deleted, **+2,086 lines** added across 24 files
- Adding a file = adding one `Leaf()` call. No manual stat/readdir/write.
- Validation at `BuildTree` time catches structural errors before serving.
- Permission model inlined in the spec via `mode` and `GID()` — no separate `perm.go`.

## Phase 16: 9P Server Streamlining (Aug 2 — afternoon)

Following the EDSL conversion, the 9P protocol server itself
(`cmd/olliesrv/server.go`) received a focused cleanup over 13 commits:

### Fid management
- **Server owns the fid map**: validates newfids (not already in use, not stale),
  garbage-collects orphaned fids. No more leaking fids on client disconnect.
- **Store opened file on fid**: read/write use `f.entry` directly instead of
  re-resolving the path on every operation. Eliminates a class of TOCTOU bugs.

### Method extraction
- `attach` and `open` extracted from `*Server` — pass groups and log explicitly
- `GroupTable` extracted from `*Server` — permission checking uses bitmask OR
  instead of a boolean dance (`checkPermBits` → `hasPermBits`)
- `checkPerm` extracted to package-level, permissions resolved in `open`
- 9 `rootTree`-only methods extracted to package-level functions; switch
  statement replaced with handler map
- Three thin wrapper methods inlined (`readFile`, `writeFile`, `FileTree` alias)
- `handle` inlined into `Start`; renamed `Serve` → `Start`, `Shutdown` → `Kill`

### Result
- `server.go` reduced from ~820 to ~200 lines
- `main.go` simplified as bypass tree wiring moved to `fs/`
- Server is now a thin 9P2000 protocol translator, not a filesystem

## Phase 17: Eventwait Redesign & Chat Log Format (Aug 2)

### Global /eventwait
Replaced per-session event polling with a global event ring buffer and a
single `/eventwait` file. Events carry structured prefixes and delta descriptions:
- `S new session/<name>` — session created
- `S kill session/<name>` — session destroyed
- `S rename session/<old> session/<new>` — session renamed
- `A kill session/<name>/agent/<aid>` — agent killed

The event ring is a fixed-size circular buffer (100 slots) with a `sync.Cond`
for blocking reads. Frontends block on `/eventwait` and receive deltas since
their last known offset. No polling, no timer, no D-Bus.

Fixes along the way:
- Dangling pointer in `eventRing` cond initialization (caused panics under load)
- Double-unlock panic in `WaitEvent`
- Spurious `[[[end]]]` markers on reasoning blocks (reasoning events were
  silently suppressed from chat but still advanced the role-transition state)

### Chat log: [[[type]]]/[[[end]]] block format
The chat log switched from plain text to a structured block format:
```
[[[assistant:resp-abc123]]]
Hello! How can I help?

[[[end]]]
[[[tool:file_read]]]
File contents...

[[[end]]]
```

This enables reliable parsing of multi-turn, multi-role conversations
from the flat log. Each event type (`assistant`, `tool`, `call`, `retry`,
`error`, `usage`, `info`, `exec`) opens a block and `[[[end]]]` closes it.
`reasoning` and `reasoning_think` events are suppressed from the log
entirely — they're included in the LLM context but invisible to the user.

### KDE submodule evolution
The KDE frontend received extensive updates across both days:
- **StreamFsm refactor** — chat block FSM with proper fence-post handling
- **Responsive session tree** — selection/statewait stream management
- **Explicit agent selection** — per-agent statewait/chat streams,
  only `switchAgent` starts streams; session click clears agent selection
- **Emoji-labeled states** — visual state indicators (❌ for errors, ⚠️ for warnings)
- **Double-click rename** — inline renaming of agent nodes
- **Robust session loading** — loads sessions on startup, handles empty state
- **Context menu** — kill session, per-session operations
- **Eventwait-driven updates** — replaced timer-based polling with blocking `/eventwait`
- **Fence fix** — fixed a bug where code fences could eat surrounding content
- Removed D-Bus entirely; all communication via `9p` plan9port binary

### Updated Filesystem Layout (post-refactor)
```
/                          ← root (owned by system user)
├── backends               backends list
├── help                   help text
├── models                 model list (cached)
├── agents                 agent config list
├── ctl                    root control (invalidate, kill)
├── eventwait              global event stream (blocking read)
├── complete               request-response: code completion
├── generate               request-response: one-shot LLM generation
├── route                  request-response: model routing
├── bypass/
│   ├── policy             global bypass policy
│   └── pending/{sname}       pending bypass requests
└── session/
    ├── new                write key=value to create session
    ├── idx                session index (one line per agent)
    ├── {name}/
    │   ├── env            session environment variables
    │   ├── ctl            session control (kill, save, invalidate)
    │   ├── plan           session-scoped markdown checklist
    │   ├── id             immutable session UUID
    │   ├── name           mutable session name (write to rename)
    │   ├── bypass        per-session bypass policy
    │   └── agent/
    │       ├── new        write config to create agent (rdwr)
    │       └── {aname}/
    │           ├── prompt, prompt.prev, fifo.in, fifo.out
    │           ├── chat (streaming), log (64KB window)
    │           ├── state, statewait (blocking)
    │           ├── cfg, ctl, cwd
    │           ├── id (immutable), name (mutable)
    │           ├── offset, usage, cost, ctxsz, models
    │           ├── systemprompt, context, tail
    │           ├── tools, toolsrv.loaded, toolsrv.rev
    │           └── proc/{pid}
```

## Updated Current Topology
```
ollie/                      ← single Go module
├── cmd/
│   ├── olliesrv/           ← 9P server (sessions, agents, backends)
│   │   └── internal/       agent/, backend/, bypass/, fs/, session/, prompts/, toolclient/
│   ├── toolsrv/            ← 9P tool execution server (separate process)
│   │   └── internal/       fs/, exec/, registry/, sandbox/
│   ├── ollie-9p/           ← 9P client CLI
│   └── ollie-remote/       ← remote execution binary
├── toolsrv/                ← 9P client library for toolsrv
├── fsedsl/                 ← filesystem declaration EDSL (used by both servers)
├── env/                    ← environment helpers
├── log/                    ← structured logging
├── paths/                  ← XDG path resolution
├── kde/                    ← KDE Plasma (submodule)
├── data/tools/             ← tool executables + .meta sidecar files
├── data/agents/            ← agent configs (JSON)
├── data/prompts/           ← system prompt fragments (markdown)
├── data/skills/            ← domain knowledge modules (markdown)
├── data/scripts/           ← ollie-remount, o CLI
├── data/services/          ← systemd, xdg-autostart
├── prompts/                ← embedded prompt templates
└── doc/                    ← documentation
```

## Phase 18: Zero Built-in Tools (Aug 2–3)

The most radical simplification yet: **zero tools compiled into the Go binary.** Every tool — including `shell` and `reasoning_think` — is now an external script loaded dynamically through the 9P filesystem.

### What changed

**Before**: `tools/builtin/builtins.go` returned four Go-compiled handlers (`shell`, `reasoning_think`, `tool_list`, `tool_load`, `tool_active`). These were hard-linked into every agent's tool definition, consuming LLM context slots even when unused. The tool registry was a two-tier system: built-in handlers (Go code, always available) + script tools (loaded on demand via `tool_load`).

**After**: `tools/builtin/` was removed entirely — `builtins.go` last returned `nil`. All tools are loaded through a single convergent path: writing the tool name to `session/{sname}/agent/{aname}/tools`. The `shell` and `reasoning_think` tools are now external scripts (`data/tools/shell`, `data/tools/reasoning_think`) with `.meta` sidecars — identical to `file_read`, `file_grep`, or any other tool.

### Unified tool loading

Tool loading follows a single path shared by both 9P writes and autoLoad initialization:

```
9P write to agent/{id}/tools
         │
         ▼
    session.loadTool(name, agent)
         │
         ├── Check disallow list
         └── RPC "tool_load" → ollie-remote process
```

The `tool_load` tool itself is a bash script (`data/tools/tool_load`) that writes the tool name to the 9P filesystem — it's a meta-tool that uses the same interface as every other operation.

### Agent config: `autoLoad`

Agent configuration files now declare which tools to load at startup via an `autoLoad` array:

```json
{
  "autoLoad": [
    "file_read",
    "file_edit",
    "file_glob",
    "file_grep",
    "file_write",
    "tool_load"
  ]
}
```

Each agent profile (default, copilot, explorer, librarian, navigator, taskmanager, theo) has its own `autoLoad` list tailored to its domain. The `allowTools` field provides a secondary security gate.

### Tool output format classification

Each tool's `.meta` can declare an `outputFormat` field specifying the source-fence language for its output:

```json
{
  "description": "...",
  "outputFormat": "markdown"
}
```

The agent loop propagates this to the chat log as the fence language (`[[[tool:file_read]]]\`\`\`markdown`), enabling syntax-highlighted output in the KDE GUI. Tools without `outputFormat` default to plaintext fences.

### Remote execution simplified

`ollie-remote` no longer embeds tool scripts or sandbox config:
- Removed `//go:embed all:tools` — tools come from `$OLLIE_TOOLS_PATH` in the environment
- Removed embedded sandbox `default.yaml` — sandbox config is provisioned alongside tools
- Removed `--tools` flag — uses `~/.config/ollie/tools` by default

### SSH bootstrap simplified

The SSH bootstrap sequence was cut from a complex multi-stage protocol to a single tarball transfer:

| Before | After |
|--------|-------|
| SHA256 hash verification | No hashing |
| gzip + base64 binary encoding | Raw `tar czf` tarball |
| `OLLIE_LOADER_START` / `OLLIE_LOADER_READY` handshake | Single `OLLIE_LISTEN_READY` signal |
| Separate `bootstrap.sh` template (88 lines) | Inline 10-line script |
| Per-binary transfer (ollie-remote + landrun) | Single `tar` of `~/.config/ollie/` |

The simplified bootstrap:

```sh
CACHE_DIR="${XDG_CACHE_HOME:-$HOME/.cache}/ollie"
mkdir -p "$CACHE_DIR"
tar xzf - -C "$CACHE_DIR"                    # receives tarball from local
export OLLIE_TOOLS_PATH="$CACHE_DIR/tools"
export PATH="$CACHE_DIR/bin:$PATH"
exec "$CACHE_DIR/bin/ollie-remote" serve --cwd <dir> --listen <sock>
```

### 9P namespace changes

- `agent/{id}/tools` — **read** lists all currently loaded tools (via the session's `toolsConn`), **write** loads a tool by name (via `session.loadTool`)
- Removed `agent/{id}/toolsrv.loaded` — subsumed by `tools` read
- Removed `agent/{id}/toolsrv.rev` — no longer needed
- Added root-level `/tools` — lists all discoverable tools on disk (global catalog)
- Removed `HandlerCtx.ToolReg` — the in-memory Go-level registry is no longer passed through 9P handlers

### Files removed

| File | Status |
|------|--------|
| `tools/builtin/` | Package removed entirely — all tool logic lives in external scripts |
| `tools/builtin/shell.go` | Deleted (now `data/tools/shell` + `data/tools/shell.meta`) |
| `tools/builtin/reasoning.go` | Deleted (now `data/tools/reasoning_think` + `data/tools/reasoning_think.meta`) |
| `tools/builtin/tool.go` | Deleted (`tool_list`/`tool_load`/`tool_active` replaced by 9P filesystem mechanism) |
| `toolsrv/bootstrap.sh` | Deleted (inline bootstrap replaces multi-stage protocol) |
| `cmd/ollie-remote/tools/.gitkeep` | Deleted (no embedded tools) |

### Lines of code

- **−530 lines** from the Go core (removed built-in handlers, embedded tools, bootstrap script)
- **+82 lines** for external tool scripts + `.meta` sidecars (`shell`, `reasoning_think`, `tool_load`)
- **Net**: tools system is smaller, simpler, and more uniform

### Why it matters

The zero-tools transition is the culmination of the architectural trajectory that began with the tool registry in Phase 10: **every facility is a file; nothing is hard-coded.**

1. **Uniformity** — There is no distinction between "core" tools and "script" tools. `shell`, `reasoning_think`, `file_read`, `file_write` — all loaded the same way, all have `.meta` sidecars, all called via the same RPC mechanism.

2. **Minimal core** — The Go binary no longer contains any tool logic. The agent loop is pure coordination: stream LLM → parse tool calls → dispatch to remote process → loop. Tool definitions, schemas, and execution are entirely external.

3. **Agent-choosable tools** — Each agent profile declares exactly which tools it needs. No agent pays the context-window cost of `shell` and `reasoning_think` unless its config asks for them.

4. **9P as the sole control plane** — Tool loading is now a file write. The same mechanism (`ollie-9p write ...`) that submits prompts and reads state also manages tool loading. No special RPC, no separate tool-manager protocol.

5. **Simpler remote deployment** — No embedded files, no binary transfer protocol, no hash verification. Provisioning a remote host is `tar czf ~/.config/ollie | ssh host tar xzf -`.

The result: the Go core is a 9P file server and an agent loop, nothing more. All behavior — every tool, skill, prompt, and configuration — lives in data files on disk. This is the Plan 9 ideal: a small, fixed kernel; all policy and capability in the filesystem.

### SLOC current (Aug 3)
| Component | Lines (excl. tests) |
|-----------|--------------------|
| agent + backend + toolsrv + session | 11,828 |
| fs + cmd/olliesrv | 4,838 |
| kde gui | 6,491 |
| **Total** | **23,157** |
Zero dead exported functions remain (verified via LSP + grep across all repos).

## Phase 19: EDSL Extraction & Structural Cleanup (Aug 3–4)

The declarative EDSL created in Phase 15 was extracted into a standalone,
reusable library. This wasn't just moving code — the EDSL itself was
**created from scratch on Aug 2** to replace ~3,100 lines of imperative
filesystem code. Within 48 hours it went from concept to extracted library.

The trajectory:
1. **Aug 2 morning**: Created the EDSL (Phase 15) — replaced imperative tree-building with declarative spec
2. **Aug 2 afternoon**: Streamlined the 9P server (Phase 16) — reduced to thin protocol handler
3. **Aug 3**: Extracted EDSL to `fsedsl/` as a reusable library
4. **Aug 4**: Structural cleanup — removed dead packages, improved `o` CLI

This rapid iteration exemplifies the project's development style: build the
right abstraction, validate it works, then extract it for reuse.

### fsedsl: Standalone Library

The generic EDSL types and builder logic extracted from `fs/` into `fsedsl/`:

```
fsedsl/
├── fsnode.go     FsNodeDecl[C], NodeOption[C], Dir/Leaf/Each constructors
├── builder.go    BuildTree — walks spec, validates, wires handlers
├── tree.go       Tree, File, FileConfig, FileTree types
├── option.go     Doc, Read, Write, Stream, BlockOnce, Request, etc.
└── README.md     Integration guide
```

The library is **protocol-agnostic** — it knows nothing about 9P. It produces
a `*Tree` structure that any protocol (9P, FUSE, HTTP, gRPC) can walk.
The `fs/` package becomes a thin adapter: type aliases + ollie-specific handlers.

Key design:
- Generic over context type `C` — ollie uses `HandlerCtx`, others can use anything
- Single `BuildTree[C]()` call validates and wires the entire namespace
- Validation at build time: no leaf handlers on directories, at most one blocking
  variant per file, template nodes require `List`, ownership inheritance

### fs/ Package Consolidation

After EDSL extraction, the fs package was restructured:

**Before** (scattered):
```
fs/
├── spec.go           namespace declaration
├── builder.go        tree builder (moved to fsedsl)
├── adapter.go        type aliases
├── session/          sub-package with 6 files
│   ├── files.go
│   ├── root.go
│   ├── perm.go
│   └── ...
└── bypassfiles.go
```

**After** (flat, by scope):
```
fs/
├── spec.go           single source of truth — entire namespace
├── fsnode.go         type aliases from fsedsl
├── builder.go        BuildTree wrapper
├── rootfiles.go      root-level handlers (backends, models, help, ctl)
├── sessionfiles.go   session-level handlers (env, ctl, name)
├── agentfiles.go     agent-level handlers (prompt, chat, state, tools)
├── bypassfiles.go   bypass broker handlers
└── procfiles.go      detached process handlers
```

No sub-packages. Handler files are organized by 9P tree scope.

### Session Package Refactors

Major restructuring of session lifecycle:

- **Session/Agent split** — `session.Session` manages lifecycle, `agent.Agent` manages conversation. Clear ownership boundaries.
- **Persistence to session/** — `fs/persist.go` removed, persistence logic moved to `session/` package
- **Manager eliminated** — Package-level registry replaces `Manager` struct. CRUD via `session.Lookup()`, `session.All()`, `session.CreateFromRoot()`, etc.
- **Remote promoted to session level** — Remote execution managed at session, not agent level

### Event Bus: Ring Buffer → Pubsub

The event ring buffer added in Phase 17 lasted exactly one day. On Aug 3, it
was replaced with `github.com/simonfxr/pubsub`:

**Why the ring buffer failed**:
- Off-by-one bugs in circular buffer indexing
- Race conditions between `sync.Cond` broadcasts and context cancellation
- Panics from double-unlock when multiple readers raced to the same event
- The offset-based API leaked implementation details to clients

**What pubsub provides**:
- Topic-based routing (`session.{id}.new`, `agent.{id}.state`)
- Wildcard subscriptions (`session.*`, `*`)
- Channel-based delivery with automatic cleanup on context cancellation
- A decade of production hardening by someone else

The backwards-compatible `PostEvent()` wrapper translates the old format
(`A agentId state thinking`) to the new topic format (`agent.agentId.state`
with payload `thinking`). Existing code kept working.

**Lines changed**: +103 / -72. Net increase, but the new code is correct.

### Dead Code Removal

| Removed | Reason |
|---------|--------|
| `cmd/Ollie` (Acme frontend) | Use `o` CLI to compose acme front-end |
| `mount/` package | FUSE mount replaced by direct 9P client (`ollie-9p`) |
| `ollie-watchdog` script | No longer needed — 9P client doesn't have FUSE stale mount issues |
| `doc/experiments/` | Historical experiments moved to git history |
| `doc/help.md` | Now generated dynamically from `Doc()` strings in spec |

### o CLI Improvements

The terminal CLI (`data/scripts/o`) was simplified:

- **Removed auto-session/agent** — Context must be set explicitly via `export` or `eval $(o env ...)`. Eliminates slow `ollie-9p read session/idx` calls.
- **Path classification** — Paths categorized as root/session/agent level. Clear error messages: "chat requires agent context".
- **TUI percentage layout** — Fixed-size splits replaced with percentages (`-l 25%`). Adapts to terminal size.
- **Block marker filtering** — Chat stream piped through `grep -v '\[\[\[.*\]\]\]'` to hide internal markers.

### Documentation

- Removed outdated docs (`doc/experiments/`, stale `help.md`)
- Renamed doc files to lowercase-kebab-case
- Updated usage.md for new `o` CLI behavior
- Removed mount/ and watchdog references from architecture docs

### Gantt Update

The timeline now extends to Aug 4, 2026 — ~3.75 months of development.

### SLOC current (Aug 4)
| Component | Lines (excl. tests) |
|-----------|--------------------|
| agent + toolsrv + session | 9,077 |
| fs + cmd/olliesrv | 2,919 |
| fsedsl (extracted library) | 1,220 |
| kde gui | 4,162 |
| **Total (core)** | **17,378** |

Note: Aug 3 count included `backend/` (4,366 lines) and had different kde scope.
Comparable core (fs + agent + toolsrv + session + fsedsl + kde gui) shrank from ~23k to ~17k.

---

## "Good Idea Fairy" Cleanup (Aug 5)

A brutal simplification pass that deleted ~960 lines of speculative infrastructure.

### Hooks — Deleted Entirely

The entire hooks system was removed:

| Hook | What it did | Why deleted |
|------|-------------|-------------|
| `agentSpawn` | Run commands on session creation | Prompt composition handles initialization |
| `preTurn` | Inject context before each turn | Removed earlier; prompt assembly does this |
| `postTurn` | Run commands after each turn | Never used in practice |
| `preTool` | Gate tool execution | Security model moved to sandbox |
| `postTool` | Transform tool results | Never used |
| `preCompact` | Run before context compaction | No use case |
| `postCompact` | Run after compaction | No use case |
| `turnError` | Handle backend errors | Replaced by retry logic in the loop |

Hooks are a "standard feature" in agent frameworks — lifecycle callbacks that
let external code react to events. In practice, they added complexity without
solving real problems. The prompt system handles initialization. The sandbox
handles security. Error handling belongs in the loop. Every hook was either
unused or doing something that belonged elsewhere.

**−237 lines** from `agent/hooks.go`.

### Other Deletions

| What | Why |
|------|-----|
| `backend/noop.go` | Test backend, never used. Tests use real backends or mocks. |
| `backend/oneshot.go` | Alternative generation mode, never used. `generate` file handles one-shot. |
| `cmd/ollie-9p/mount/` | FUSE mount code, replaced by `ollie-9p` direct 9P client. |
| `data/tools/route.meta` | Model routing tool, replaced by explicit `/model` commands. |
| `toolsrv/schema.go` | Schema validation code, never enabled. |

### Turn State Machine

The turn state machine in `agent/turn.go` was simplified from 158 lines to ~40.
The `TurnCtx` struct was not yet eliminated (that came later in Aug 11), but
the state transitions were flattened.

### Philosophy

"Good idea fairy" is military slang for someone who shows up with clever
suggestions that sound good but create work. The deleted code was all
reasonable in isolation — hooks are a standard pattern, test backends are
useful, FUSE mounts are convenient. But each added surface area that had to
be maintained, tested, and reasoned about. None of it solved actual problems
better than simpler alternatives.

---

## Aug 9, 2026 — Simplification & Namespace Cleanup

Major simplification pass: ~1,450 lines removed from the core runtime.

### Structural

- **Section-based preamble** — `Preamble` struct with `Set/Get/String` replaces string surgery (-270 lines)
- **Session** — Export immutable fields, extract `buildAgent` helper (-115 lines)
- **Prompt resolver** — Gutted to file-path-only resolution (-127 lines)
- **TaskState subsystem** — Removed entirely (-246 lines)
- **GenerationParams** — Embedded in AgentConfig, JSON tags added (-34 lines)
- **sync.Cond → close-channel** — `WaitChange` uses idiomatic channel signal
- **Dead code** — `session.New()`, env map, prevPrompt field, usage_log.go, contextDebug()

### Namespace

The agent namespace went from 17 files to 10. Redundant files merged or moved to ctl:

| Before | After |
|--------|-------|
| `fifo.in` + `fifo.out` | `fifo` (write=enqueue, read=dequeue) |
| `cost` + `usage` + `ctxsz` | `stats` (key=value lines) |
| `chat` | `chat` (filtered text) + `chat.raw` (full markup) |
| `cwd`, `models`, `tools`, `systemprompt` | ctl commands |
| `connection`, `state`, `context`, `tail`, `offset`, `prompt.prev` | removed |

### ctl Unification

All ctl files (root, session, agent) now share:
```go
type rdwrHandler func(ctx HandlerCtx, args []string) ([]byte, error)
```

Agent ctl is **rdwr** (request-response): write command, read result.
Commands with no args return current state (e.g., `/model` prints the model).

Agent ctl commands: `stop`, `compact`, `clear`, `inject`, `agent`, `model`, `models`, `tools`, `tool_load`, `cwd`, `name`, `backend`, `systemprompt`.

### Command Surface

`/` prefix in prompts == `o ctl`. No special-cased commands in the agent.
- `/model qwen3:8b` → switches model, returns new model name
- `/tools` → lists loaded tools
- `/inject look at this` → injects prompt mid-turn (overwrites pending)
- `!q` in the REPL → exits the TUI (local-only)

### elevate → bypass

The sandbox escape mechanism was renamed from `elevate` to `bypass` throughout (package, namespace, env vars, tool args, all docs).

### Streaming Fix

The TUI streaming regression (line-by-line instead of char-by-char since Aug 4) was caused by piping through `grep -v` — grep is inherently line-buffered. Fixed by:
- `chat` file now serves filtered text (markers + fences stripped server-side via `stripMarkers` state machine)
- `chat.raw` serves full markup for GUIs that parse blocks
- TUI no longer pipes through grep for streaming

### CLI (`o` script)

- `readloop` → `read -l`
- `chatstream` → removed (`o read chat` streams directly)
- `stop`, `kill` → removed (use `o ctl stop`, `o ctl kill`)
- `ctl` uses `ollie-9p rdwr` (gets response)
- REPL: `/` → `o ctl`, `!q` → exit TUI
- Background statewait loop trapped on EXIT (fixes zombie)

### SLOC (Aug 9)

| Component | Lines (excl. tests, backends, fsedsl, generated) |
|-----------|--------------------:|
| agent | 3,167 |
| toolsrv | 2,521 |
| fs | 1,774 |
| session | 1,386 |
| cmd/olliesrv | 1,161 |
| bypass | 733 |
| lib9p | 690 |
| cmd/ollie-9p | 356 |
| cmd/ollie-remote | 369 |
| sandbox | 293 |
| log + env + paths + format + prompts | 428 |
| **Total (core)** | **12,878** |

---

## Aug 9, 2026 (cont.) — Prompt Audit & Maintainability Pass

Systematic audit of the prompt construction system and codebase maintainability.

### Prompt System

- **System prompt**: ~200 → ~80 lines. Removed tool-first table, API docs section (moved to agent prompt), dead 9P entries, output protocol (moved per-agent).
- **AllowTools**: removed entirely (field, RPC, config). Tools are loaded dynamically; static allowlists served no purpose.
- **Phantom refs fixed**: `route` removed from driver autoLoad, `reasoning_think` + `shell` added to default autoLoad.
- **Output protocol**: moved from system prompt to per-agent prompts. Theo's brevity rule applied to all agents except copilot.
- **Tool docs**: merged listing + documentation into single `renderTools()`. Removed JSON schema dump (function-calling API provides it natively). Removed `BuildToolListing` dead code.
- **PRIME_* env vars**: removed entirely. Replaced `SessionInfra.PromptEnv []string` with typed `Platform string` + `IsGitRepo bool` fields.

### Code Quality

- **`OnToolsChanged`**: changed from `func(string)` to `func()` signal. Agent re-fetches tool list on signal instead of receiving a pre-formatted string it ignores.
- **`Preamble` sections**: type-safe `Section` type replaces raw strings. Compile-time typo detection.
- **`extractToolResult`**: documented the tool output protocol (JSON content-block envelope with raw-text fallback).
- **Package godoc**: `session/` package comment documenting Init→Create→Register→Kill lifecycle.
- **`PromptEnv()`**: deleted (was a no-op returning nil after PRIME_* removal).

### Documentation

- Moved 7 reference docs from `doc/resources/` to `doc/` (architecture, 9p, writing-tools, tool-registry, core, remote-execution, edsl). Left whitepaper material (evolution, multi-agent, misc, no-mcp, ideas) in `resources/`.
- Rewrote `data/agents/README.md` to match current config schema and agent roster.
- Fixed stale cross-references in README.md and doc/tool-registry.md.

### SLOC (Aug 9, post-audit)

| Component | Lines (excl. tests, generated) |
|-----------|--------------------:|
| agent | 3,159 |
| toolsrv | 2,464 |
| session | 1,377 |
| fs | 1,774 |
| cmd/olliesrv | 1,161 |
| bypass | 733 |
| lib9p | 690 |
| cmd/ollie-9p | 356 |
| cmd/ollie-remote | 358 |
| sandbox | 293 |
| log + env + paths + format + prompts | 428 |
| **Total (core)** | **12,793** |

Excludes: backends (4,229), fsedsl (1,030), tests, KDE, tools, generated code.

## Phase 20: toolsrv 9P Migration & Process Isolation (Aug 10–11)

The tool server completed its evolution from an in-process library to a
**fully independent 9P server**. Locally it runs as a child process of
olliesrv; for remote execution it runs on a remote host, connected via
SSH Unix socket forwarding.

### Before

`toolsrv` was a Go package (`ollie/toolsrv`) with a `Server` struct that ran
in-process within olliesrv. Tool calls were Go function calls — no process
boundary, no separate namespace. The package mixed client code, server code,
registry, sandbox wrappers, and execution logic in a flat directory.

### After

Two distinct components:

1. **`cmd/toolsrv/`** — a standalone binary that serves its own 9P2000
   filesystem over a Unix socket. Has its own `internal/` packages:
   - `internal/fs/` — filesystem spec (fsedsl), server state, process management
   - `internal/exec/` — sandboxed tool execution (landrun, bypass broker)
   - `internal/registry/` — session-scoped tool registry
   - `internal/sandbox/` — landrun configuration (unchanged)

2. **`toolsrv/`** (root package) — a 9P client library. `Conn` dials toolsrv
   over a Unix socket, authenticates via Tauth, and exposes methods like
   `CallTool`, `LoadTool`, `ListTools`, `Ping`.

### toolsrv 9P Namespace

```
/
├── ctl          write: load <tool>, unload <tool>, env K=V, cwd <path>
├── tools        read: list loaded tools (JSON), write: tool name to load
├── info         read: platform, arch
└── proc/
    ├── new      rdwr: write tool+args, blocks, read result
    ├── new.bg   write: tool+args, returns pid immediately
    └── {pid}/
        ├── out   read: output
        ├── wait  read: blocks until exit, returns exit code
        ├── stat  read: running/exited, runtime, tool
        └── ctl   write: signal <N>, dismiss
```

Both `cmd/toolsrv` and `cmd/olliesrv` declare their namespaces using the
same `fsedsl` library. Both implement their own 9P protocol handlers (using
`9fans.net/go/plan9`) — they are intentionally separate servers that
communicate over a socket.

### Process Lifecycle

Three-layer defense against orphaned toolsrv processes:

1. **Pdeathsig** (local only) — `SysProcAttr{Pdeathsig: SIGTERM}` on local
   spawn. When olliesrv dies, the kernel terminates toolsrv immediately.
   For SSH-spawned toolsrv, Pdeathsig is set on the `ssh` process itself —
   when SSH dies, the remote shell receives SIGHUP which typically cascades
   to toolsrv.

2. **Idle timeout** (local + remote) — toolsrv tracks active 9P connections.
   After the first client connects and then all connections drop, a 30s timer
   starts. If no new connection arrives, toolsrv exits cleanly. 60s startup
   grace period for the initial connection. This is the primary cleanup
   mechanism for remote toolsrv where Pdeathsig doesn't apply directly.

3. **Kill-before-respawn** — `ProcessKeeper.Dial()` kills the old process
   before spawning a replacement. Prevents accumulation during reconnect cycles.

### Authentication

toolsrv uses 9P Tauth for authentication. The first client sets the shared
secret; subsequent clients must provide the same secret. This replaces
the previous token-in-environment approach and is compatible with socket
permission security.

### Structural Consistency

Both servers now follow the same physical layout:

```
cmd/{server}/
├── main.go              entry point
├── server.go            9P protocol handler (top-level, like Plan 9 tradition)
└── internal/
    ├── fs/              filesystem spec + handlers + state
    ├── ...              domain-specific packages
```

### Parallel Tool Execution

Replaced binary ReadOnly/not batching with resource-based conflict scheduling.
Tool calls within a single turn are grouped into parallel batches based on
a three-class scope system declared in `.meta`:

- **scope "read"** — never conflicts (always parallel)
- **scope "write"** — conflicts only on same file path
- **scope "global"** (or unset, default) — full serialization barrier

The model's read-before-write pattern (enforced by the turn-based protocol)
guarantees that parallel writes are safe: each file_edit carries its own
`old_string` context from a prior read, and edits on different paths are
provably independent.

Shell and tools with opaque effects (like `lsp_rename`) declare scope "global"
and always run alone. Unknown/unset scope defaults to global — tools must
opt in to parallelism.

### Background Process Interrupts

Any tool call can include `"background": true` to execute asynchronously
via `proc/new.bg`. The model receives a process ID immediately and continues working.
Background processes have no timeout — they run until they exit naturally,
are stopped via `proc/{id}/ctl`, or die with the session.

Output is streamed in real-time into `proc/{id}/out`. Upon completion,
the process exit notification is injected as a **user prompt** (written to
the agent's `prompt` file) containing a `<system-proc-interrupt>` block:

```xml
<system-proc-interrupt id="42" cmd="go test ./..." status="exited" exit="1">
--- FAIL: TestFoo (0.00s)
    foo_test.go:12: expected 3, got 2
FAIL
</system-proc-interrupt>
```

This uses automatic prompt queueing: if the agent is still working,
the interrupt arrives on the next idle transition; if the agent is already
idle, it arrives immediately. The model sees these as natural follow-up
prompts — no polling, no in-loop injection. It can react to failures
or stop processes via the proc ctl file (`term`, `kill`, `dismiss`).

**Process lifecycle**: toolsrv owns all procs. Session death (toolsrv exit)
kills all procs via `KillAll()` + `Pdeathsig`. Exited procs remain in the
tree for 10 minutes after last read, then are garbage collected.

### Universal Dispatch Flags

All tools receive four optional parameters injected into their schemas at
runtime (not declared per-tool):
- `bypass` — sandbox escape via bypass broker
- `timeout` — execution timeout in seconds
- `sandbox` — sandbox profile override
- `background` — async execution with auto-injected output

### Why Not Concurrent-by-Default

Foreground tool calls block the model until all results arrive — deliberately.
The alternative (execute everything concurrently, deliver results as queued
prompts) was considered and rejected: it turns one coherent reasoning step
into N separate generation cycles, each with partial information. The model
can't reason about results together, every interrupt costs a full inference
round-trip, and the end result is sequential execution but slower and more
expensive. Background execution remains an explicit opt-in for genuinely
long-running processes where the model doesn't need the result immediately.

## Phase 22: Meta-Only Tool Definitions (Aug 11–12)

A `.meta` file can now define a tool with no companion executable. The `cmd`
field is treated as a shell command string — not a path to resolve, but a
command to run directly. JSON arguments are passed on stdin.

Before:
```json
{"cmd": "rg"}  // resolved via PATH, then executed with --flags
```

After:
```json
{"cmd": "jq -r '.pattern' | xargs rg"}  // shell command, executed as-is
```

This changes what a "tool" is. Previously, tools were executables with `.meta`
sidecars. Now a tool can be purely declarative: a `.meta` file that shells out
to existing CLIs. Wrap ripgrep, kubectl, or an MCP bridge without writing any
code.

The tools directory is prepended to `$PATH` during execution, so meta-only
tools can reference other tools in the same directory.

**Why this matters**: any CLI becomes a tool with 20 lines of JSON. The barrier
to adding capabilities dropped from "write a script" to "write a schema."

## Phase 23: Bypass via 9P (Aug 11)

The bypass broker — the mechanism for escaping the sandbox — was previously a
Unix socket with a custom protocol. It now routes through the 9P filesystem:

- toolsrv exposes `bypass/pending` (blocking read) and `bypass/resolve` (write)
- olliesrv subscribes to pending requests and routes them to D-Bus for approval
- Approved/denied responses flow back through 9P

This unifies the control plane. Everything is files. The bypass socket is gone.

## Phase 24: fs/ Flattening (Aug 11)

The olliesrv filesystem package had accumulated abstractions:

- `AgentLog` — chat log buffer wrapper
- `SessionNode` — session state container  
- `RootState` — root-level state

These were eliminated. Chat logging moved to the agent package. Session state
moved to the session package. The fs/ package now contains:

- `spec.go` — the EDSL declaration with all handlers inlined
- `support.go` — helper functions
- `newroot.go` — tree construction

~600 lines deleted. The handlers live where the data lives.

## Phase 25: Sandbox Simplification (Aug 11)

Named sandbox profiles (`default`, `restricted`, `remote`) were removed. There
is now one config: `sandbox.yaml`. Per-project overrides via `.ollie-sandbox.yaml`
remain, but the multi-profile system is gone.

The `sandbox` parameter on tool calls now does nothing — kept for compatibility
but ignored.

## Phase 26: Agent Loop Cleanup (Aug 11)

The `TurnCtx` struct — which carried per-turn state through the loop — was
eliminated. Loop functions became methods on `*Agent`, accessing state directly.
The turn state machine in `turn.go` was simplified.

~100 lines deleted, control flow is clearer.

---

Commit count: ~50 commits over 2 days. Net code change: approximately -800 lines.
The system does more with less.

---

## Dead Ends and Reversals

Everything that was built and then killed, in roughly chronological order.

| What | Why it was removed |
|------|-------------------|
| MCP servers (denote-mcp, 9beads-mcp) | Too heavyweight; scripts are simpler and faster |
| TUI frontend | Replaced by `o` — a tiny shell script (40 lines of tmux) |
| `pl/` planning namespace | Over-engineered; plan file is sufficient |
| Beads integration | External dependency for something a checklist file handles |
| `plan_create` / `plan_complete` / `task_add` / `task_check` tools | Five planning iterations before settling on a single markdown file |
| SSH agent proxy | Attack surface not worth the convenience |
| Separate ollie-dbus daemon (ollied) | Consolidated into olliesrv, then killed entirely |
| D-Bus adapter (`org.ollie.SessionManager`) | 9P streaming is superior; no polling, no offset tracking, no frozen GUIs |
| D-Bus-driven KDE frontends | Rewritten on pure 9P via plan9port `9p` binary |
| Plasmoid / tray KDE components | Not useful enough to maintain |
| FUSE mounts | Replaced by standalone 9P client binary; FUSE can't do blocking reads |
| Tier routing (FAST/POWER) | Replaced by `/route` endpoint with real model discovery |
| call_tool / pipe | Replaced by native tool registry with `tool_load` |
| execute_code (multi-language) | Simplified to shell-only; tool scripts handle language choice |
| chatwait | Reverted; acme tails `chat` directly |
| Header comment metadata (`ollie:prompt`, `ollie:tier`, `args_json:`) | Replaced by `.meta` sidecar JSON; decouples metadata from language |
| `contrib/` directory | Renamed to `data/`; nothing was community-contributed |
| Script namespaces (s/, u/, x/) | Replaced by 9P request-response files |
| s/sh, s/bfg, s/bbg | Replaced by `generate`/`complete`/`route` 9P files |
| `agent.Core` interface | Nobody else implemented it; just indirection |
| `Dispatcher` indirection in toolsrv | `toolsrv.Server` called directly |
| `execute/` package | Merged into `tools/`; 3 functions didn't need a package |
| `mgr/` package (Manager struct) | Replaced by package functions; `*fs.Tree` IS the collection |
| `fs/session/` sub-package (6 files, ~3,100 lines) | Flattened into `fs/` — no more sub-package indirection |
| `cmd/olliesrv/bypass_tree.go` | Bypass tree handlers moved to `fs/bypassfiles.go` |
| Per-session/agent timestamp IDs | Replaced by UUIDv4 — immutable `id` + mutable `name` |
| Single-step session creation | Split into `session/new` + `session/{name}/agent/new` |
| Imperative filesystem (stat/walk/read/write per node) | Replaced by declarative EDSL (`fsedsl/`) |
| Event ring buffer (100-slot circular buffer + sync.Cond) | Lasted one day; replaced by `pubsub` library. Off-by-one bugs, race conditions, panics. |
| Built-in tool handlers (shell, reasoning_think, tool_list, tool_load, tool_active) | Replaced by external scripts loaded via 9P `agent/{id}/tools` write |
| Go-compiled tool registry in `tools/builtin/` | Zero built-in tools — all tools are external scripts with `.meta` sidecars |
| Embedded tool scripts in `ollie-remote` | Remote uses `$OLLIE_TOOLS_PATH` from environment; tools deployed via tarball |
| SSH binary transfer (gzip+base64+hash-verify) | Replaced by simple `tar | ssh` of `~/.config/ollie/` |
| `toolsrv.loaded` / `toolsrv.rev` 9P files | Removed — `tools` file handles both list and load |
| `cmd/Ollie` (Acme frontend binary) | Replaced by `o` CLI composing acme |
| `mount/` package | FUSE mount replaced by `ollie-9p` direct 9P client |
| `ollie-watchdog` script | No FUSE stale mount issues with 9P client |
| `doc/experiments/` | Historical experiments; moved to git history |
| `doc/help.md` | Generated dynamically from `Doc()` strings in spec |
| Hooks system (agentSpawn, preTurn, postTurn, preTool, postTool, preCompact, postCompact, turnError) | Never solved real problems; prompt handles init, sandbox handles security, retry handles errors |
| `backend/noop.go` (test backend) | Never used; tests use real backends or mocks |
| `backend/oneshot.go` | Never used; `generate` file handles one-shot |
| `toolsrv/schema.go` (schema validation) | Never enabled |
| `data/tools/route.meta` (model routing tool) | Replaced by explicit `/model` commands |
| TaskState subsystem | Removed entirely (−246 lines); no use case survived |
| `fifo.in` + `fifo.out` | Merged to single `fifo` (write=enqueue, read=dequeue) |
| `cost` + `usage` + `ctxsz` files | Merged to `stats` (key=value lines) |
| `connection`, `context`, `tail`, `offset`, `prompt.prev` agent files | Removed or moved to ctl |
| AllowTools field | Tools loaded dynamically; static allowlists served no purpose |
| PRIME_* env vars | Replaced by typed `Platform` + `IsGitRepo` fields |
| `elevate` naming | Renamed to `bypass` throughout |
| In-process toolsrv (Go library) | Replaced by separate 9P server process with socket boundary |
| Binary ReadOnly/not tool batching | Replaced by scope-based conflict scheduling (read/write/global) |
| `AgentLog` / `SessionNode` / `RootState` abstractions in fs/ | Eliminated; state moved to where data lives (agent/, session/) |
| Named sandbox profiles (default, restricted, remote) | One config: `sandbox.yaml`. Multi-profile system was unused complexity |
| `TurnCtx` struct | Eliminated; loop functions became methods on `*Agent` |
| Bypass via custom Unix socket protocol | Replaced by bypass via 9P namespace |
| `subagent_generate` (JIT agent config tool) | Obsolete; sub-agents use existing profiles directly |
| Fire-and-forget `subagent_spawn` via D-Bus | Replaced by blocking `ollie-9p rdwr agent/new` — returns the reply |
| Sub-agents as peer agents (no parent, no return) | Replaced by task-scoped sub-agents with context inheritance and automatic cleanup |
| `cascade` orchestrator script | Replaced by parallel `subagent_spawn` calls (scope: read, natural parallelism) |
| Agent-loop batching as correctness mechanism | Moved to toolsrv path-lock table; agent batching is now just an optimization |
| `virtfs.Request` naming | Renamed to `Rdwr` — it's an atomic operation, not a variant of read or write |

## Feed file + Observer agents + BlockOnce/Stream refactor (Aug 13)

Three related changes in one session. Net **+278 SLOC** across 21 files.

### What was added

**`feed` file** — a change-detecting blocking read file on every agent. Write data in, internal consumer submits it as a prompt. Dedup built into the `BlockOnce` handler: same data written twice never wakes the reader. Enables real-time pair programming — an observer agent watches a human or another agent code.

**Observer agent** (`data/agents/observer.json`, `data/prompts/agent-observer.md`) — read-only agent profile. Only loads `file_read`, `file_grep`, `file_glob`, `reasoning_think`. Prompt explicitly forbids writes. Receives diffs via feed, makes terse observations.

**`ConsumeFeed`** — plain function that dials the 9P server via `lib9p` and reads from the agent's own feed file in a loop. Each read blocks until feed changes. Same pattern as `o read -l`.

### What was refactored

**`virtfs.BlockOnce(readFn, signalFn)`** — previously a raw handler that took `(ctx, base)` and had to implement its own blocking. Now the framework handles the block-until-changed loop: call `readFn()`, compare hash to base, wait on `signalFn()` channel, repeat. Handlers become two-line closures.

**`virtfs.Stream(readFn, signalFn)`** — same refactor. `streamChat` (40 lines of condvar + mutex + offset tracking) replaced by `a.ChatRead` + `a.ChatSignal` passed to `Stream(...)`.

**Server timeout fallback** — when `BlockOnce` times out (5s, no change), the server falls back to the plain `Read` handler. Frontends get the current value as a heartbeat. Files without `Read` (like feed) return empty.

### What was removed

- `streamChat()` in support.go (replaced by `ChatRead` method on agent)
- `mergeCtx()` in support.go (no longer needed — blocking logic moved into virtfs framework)
- `ObserverFeed` script (replaced by the `feed` file)
- `WaitChange` usage outside filesystem handlers (feed consumer uses lib9p instead)

### What was fixed

- **XDG fallback** in prompt resolver — `$XDG_CONFIG_HOME` now defaults to `$HOME/.config` when unset
- **Duplicate tool headers** — `renderTools` skipped the generated `## name` when the tool prompt already has one
- **FIFO drain** — queued prompts no longer orphaned on interrupt/panic/toolsrv failure
- **Session restore ordering** — moved after 9P listener starts so feed consumers can connect

### Dead ends (killed during the session)

| Attempt | Why killed |
|---|---|
| Custom channel-based Feed with its own signal infrastructure | Duplicated the agent's existing signalCh mechanism |
| `Stream` mode for feed | Feed is a discrete value, not a byte stream |
| Internal WaitChange-based consumer | Leaked internal plumbing; should be a plain 9P client |
| `BlockOnceRaw` as primary API | Forced handlers to implement blocking themselves |
| Concurrent-by-default tool execution | Rejected: turns one reasoning step into N generation cycles with partial info — sequential but slower and more expensive |

## Sub-Agents via 9P rdwr (Aug 14)

Sub-agents implemented in **42 net lines of code**. The entire feature is
wiring — no new infrastructure, no new concepts, no new processes.

### The mechanism

`session/{sname}/agent/new` is an `Rdwr` file. Without `prompt=`, it creates
a persistent agent and returns its ID (existing behavior). With `prompt=`, it
enters sub-agent mode: creates a transient agent, submits the prompt, blocks
until the agent finishes, returns the reply, and destroys the agent.

```
printf 'cwd=%s\nprompt=fix the tests\n' "$PWD" \
  | ollie-9p rdwr session/$OLLIE_SESSION_ID/agent/new
```

### Why it works in 42 lines

The infrastructure was already there:
- **`Rdwr` file primitive** (née `Request`) — atomic write-then-read, per-open isolation
- **`Agent.Submit()`** — already blocks through the full turn
- **`Agent.Reply()`** — already captures the final response
- **`Session.RemoveAgent()`** — already handles cleanup
- **Event bus** — normal agent lifecycle events fire (GUI sees sub-agents appear/disappear)

The handler is just: parse params → create agent → submit → read reply → destroy → return.

### Path-based lock table

Prerequisite: moved conflict serialization from the agent loop into toolsrv
itself. All foreground tool calls now acquire a path-based lock before execution:

- `scope "read"` — no lock (never conflicts)
- `scope "write"` — exclusive lock on the file path
- `scope "global"` — exclusive global lock

This ensures sub-agents writing to the same file serialize correctly regardless
of which agent initiated the call. The agent loop's batching remains as a
performance optimization; correctness is enforced at the execution layer.

### Parallel dispatch

`subagent_spawn` is declared `scope: "read"` — multiple spawn calls in the same
turn run in parallel automatically. The calling agent blocks until all results
return. No shell backgrounding, no polling, no coordination code.

```mermaid
sequenceDiagram
    participant P as Parent Agent
    participant S1 as Sub-Agent 1
    participant S2 as Sub-Agent 2

    P->>S1: subagent_spawn(prompt="task A")
    P->>S2: subagent_spawn(prompt="task B")
    Note over P: blocked (parallel tool calls)
    S1-->>P: reply A
    S2-->>P: reply B
    Note over P: resumes with both results
```

### What was killed

- **`subagent_generate`** — JIT agent profile generation tool. Obsolete: sub-agents
  use existing profiles directly.
- **D-Bus spawn path** — `subagent_spawn` previously used `dbus-send` to create
  sessions. Replaced by a 3-line `ollie-9p rdwr` call.

### Framework rename: `Request` → `Rdwr`

The three atomic 9P operations are now named for what they are:
- `Read` — non-blocking read
- `Write` — non-blocking write (fire-and-forget)
- `Rdwr` — atomic write-then-read (blocking, produces result)

`BlockOnce` and `Stream` are special cases of `Read`.
`Rdwr` is its own primitive — not a variant of either.

### Architecture validation

This feature is evidence that the "everything is a file" architecture works at
scale. A major capability (parallel sub-agents with automatic serialization)
landed as pure wiring because:

1. The 9P namespace already exposes `agent/new` as an rdwr file
2. The agent loop already blocks and produces results
3. The tool server already serializes conflicting writes by path
4. The batching algorithm already parallelizes non-conflicting calls

No new abstractions. No new processes. No new protocols. Just connecting
existing pieces with 42 lines of glue.

### Context inheritance

Sub-agents inherit the parent's conversation history. When `parent=` is
specified in the payload, the handler clones the parent agent's messages
into the sub-agent's initial context via `RestoreHistoryFromMessages`.
This is mechanically identical to session restore — same function, same
data path. The sub-agent knows everything the parent knows without
rediscovering it.

### Future work

- **Shared toolsrv per host** — currently each session spawns its own toolsrv.
  Cross-session path serialization requires sessions to share a single toolsrv
  instance per host.
- **Budget/depth controls** — token limits, step limits, and recursion depth
  caps for sub-agents.