77 KiB

Raw Blame History

Architectural Evolution

How the Ollie system-of-systems emerged and evolved.

This is a study log. It records what was built, what was killed, and why — in the order it happened. The dead ends are as instructive as the survivors: the rejected ideas at the end teach more about building agent systems than the architecture that remains. The document evolves with the author's understanding; when past decisions turn out to be wrong, they get recorded here, not hidden.

Timeline

gantt
    title Ollie Evolution
    dateFormat  YYYY-MM-DD
    axisFormat  %b %d
    section Foundation
    Monorepo + submodules          :done, 2026-04-11, 14d
    9P filesystem                  :done, 2026-04-14, 28d
    section Core Primitives
    Session lifecycle (session/)   :done, 2026-04-20, 30d
    Tool definitions                :done, 2026-05-01, 20d
    Skills system                  :done, 2026-05-10, 14d
    section Execution
    execute_code / shell           :done, 2026-05-08, 20d
    Batch jobs (b/ → session/bfg)  :done, 2026-05-15, 25d
    Sandbox (landrun)              :done, 2026-05-20, 40d
    section Multi-Agent
    Subagent spawn                 :done, 2026-05-25, 15d
    Cascade orchestrator           :done, 2026-06-05, 20d
    JIT agent generation           :done, 2026-07-10, 10d
    section Frontends
    TUI + Emacs (ellie)            :done, 2026-04-15, 45d
    Acme (Plan 9)                  :done, 2026-05-20, 30d
    KDE/Plasma                     :done, 2026-06-10, 40d
    section Control Planes
    D-Bus adapter (ollied)         :done, 2026-06-25, 20d
    D-Bus embedded in srv          :done, 2026-07-15, 10d
    D-Bus removed                  :done, 2026-07-31, 1d
    section Late Evolution
    Remote execution               :done, 2026-07-05, 15d
    Bypass broker               :done, 2026-07-10, 15d
    Tool registry (lazy)           :done, 2026-07-20, 10d
    FUSE → 9P client               :done, 2026-07-25, 5d
    The Great Flattening           :done, 2026-07-29, 2d
    9P Declarative EDSL            :done, 2026-08-02, 1d
    Zero built-in tools            :done, 2026-08-02, 2d
    EDSL extraction + cleanup      :done, 2026-08-03, 2d
    Good idea fairy cleanup        :done, 2026-08-05, 1d
    toolsrv 9P + process isolation :done, 2026-08-10, 2d
    Meta-only tool definitions     :done, 2026-08-11, 2d
    Bypass via 9P                  :done, 2026-08-11, 1d
    fs/ flattening                 :done, 2026-08-11, 1d
    Feed + observer agents         :done, 2026-08-13, 1d
    BlockOnce/Stream refactor      :done, 2026-08-13, 1d
    Sub-agents via 9P rdwr         :done, 2026-08-14, 1d
    Code intelligence tools        :done, 2026-08-15, 1d
    Go file tools + cleanup        :done, 2026-08-16, 1d
    OptMem persistent memory       :done, 2026-08-16, 1d
    Markdown parsing + output cap  :done, 2026-08-16, 1d

Phase 1: Monorepo Bootstrap (Apr 11)

Started as independent git repos unified under a monorepo with submodules. Initial components:

agent — agent loop (Go): backend dispatch, tool execution, session state
fs** — 9P file server exposing agent sessions as a synthetic filesystem
tui — terminal UI (later removed)
el — Emacs integration (ellie.el) Build system was mkfile (Plan 9 make), later Makefile, finally justfile.

Phase 2: The 9P Decision (Apr 14–28)

The defining architectural choice: every agent primitive is a file. Sessions are directories. Sending a prompt is writing to session/{sname}/prompt. Reading state is cat session/{sname}/state. This made the control plane protocol-agnostic — any program that can read/write files can be a frontend. Key files crystallized: prompt, chat, state, ctl, cfg, statewait. The session/new file (write key=value pairs, read back session ID) became the session factory.

Phase 3: Tool System (Apr 28 – May 10)

Tools evolved through several incarnations:

MCP servers (denote-mcp, 9beads-mcp) — external processes, heavyweight
Shell/Python scripts in ~/.config/ollie/tools/ — lightweight, sandboxed
execute_code as the single built-in tool — all scripts invoked through it
call_tool/pipe — named tool dispatch and cross-tool pipelines
Tool registry (final form) — dynamic lazy-loading via tool_list/tool_load/tool_active; execute_code renamed to shell; tools hot-reloadable without server restart The file tools (file_read, file_write, file_edit, file_grep, file_glob) were extracted into standalone Python scripts. Memory (memory_remember, memory_recall) and reasoning (reasoning_think) followed the same pattern. The memory tools (memory_remember, memory_recall) now call OptMem directly. OptMem owns the persistent B-tree at $XDG_DATA_HOME/ollie/optmem (default: ~/.local/share/ollie/optmem); Ollie does not maintain a parallel memory directory or memory-file format. This keeps reads and writes on the same store and avoids competing writers. MCP servers were removed early. Simple scripts won over complex daemons; the OptMem-backed tools retain the lightweight tool interface without duplicating storage.

Phase 4: Planning System Churn (May – Jul)

Planning went through the most iterations of any subsystem:

pl/ directory namespace with flat file-per-task
plan_create / plan_complete tools
Beads integration (external issue tracker) — made optional, then dropped
task_add / task_check tools
Inline markdown plans forbidden
Final form: a single session/{sname}/plan file (markdown checklist), written by the agent, persisted across context compaction The lesson: planning needed to be simple, agent-controlled, and not bureaucratic.

Phase 5: Multi-Agent Coordination (May 25 – Jun 15)

Three patterns emerged:

subagent_spawn — fire-and-forget session creation, parent never blocks
subagent_generate — JIT agent config generation (role, constraints, tools)
cascade — script-driven fan-out with -max-workers, -retries, optional synthesis step

flowchart TB
    UP["User prompt"]
    PARENT["Parent Session"]
    W1["Worker 1"]
    W2["Worker 2"]
    W3["Worker 3"]
    CASCADE["u/cascade\n(spawn + throttle)"]
    CW1["Cascade Worker"]
    CW2["Cascade Worker"]
    UP --> PARENT
    PARENT -- subagent_spawn --> W1
    PARENT -- subagent_spawn --> W2
    PARENT -- subagent_spawn --> W3
    W1 --> PARENT
    W2 --> PARENT
    W3 --> PARENT
    PARENT -- cascade --> CASCADE
    CASCADE --> CW1
    CASCADE --> CW2

Inter-agent communication uses the filesystem: agents write to each other's prompt file.

Phase 6: Frontend Proliferation

flowchart TB
    subgraph Core["agent.Agent"]
        AG["Agent Engine"]
    end
    subgraph Surfaces["Integration Surfaces"]
        P9["9P Filesystem\n(session/ namespace)"]
        DB["D-Bus\n(org.ollie.SessionManager)"]
    end
    subgraph Frontends
        SH["s/sh (terminal)"]
        ACME["acme (Plan 9)"]
        EL["ellie (Emacs)"]
        KG["KDE GUI"]
        KP["KDE Plasmoid"]
        KK["Kate Plugin"]
        KR["KRunner"]
        HTTP["curl / scripts"]
    end
    AG --> P9
    AG --> DB
    SH --> P9
    ACME --> P9
    EL --> P9
    HTTP --> P9
    KG --> DB
    KP --> DB
    KK --> DB
    KR --> DB
    WEB --> DB

s/sh — shell script frontend (bash, briefly rc, back to bash)
acme — Plan 9 editor integration with workspace-scoped navigator sessions
ellie.el — Emacs with ghost-text completion
ollie-kde — full Plasma integration: plasmoid, KRunner, Kate plugin, standalone GUI, GUI automation tools
TUI — removed (Jul) in favor of s/sh and richer GUIs

Phase 7: Security Model (May – Jul)

flowchart LR
    subgraph Permissions["9P Permissions"]
        OWNER["Owner → rw"]
        AGENT["Agent → restricted"]
        PEERS["Peers → read-only"]
    end
    subgraph Execution["Execution Sandbox"]
        AGT["Agent"]
        SANDBOX["landrun sandbox\n(restricted fs)"]
        TOOLS["Tool definitions"]
        BYPASS["x/bypass\n(socket broker)"]
        PRIV["Privileged Action"]
    end
    AGT --> SANDBOX --> TOOLS
    TOOLS -- needs escape --> ELEV --> PRIV

Evolution:

No sandboxing initially
YAML-based sandbox configs (landrun) for execute_code
Per-session 9P identity — agents can't read other sessions' tools
Bypass broker: socket-based, user-confirmed privilege escalation
SSH agent proxy added then removed (too much attack surface)
Final: integrated bypass broker replaces separate superpowerd adapter

Phase 8: D-Bus as Second Control Plane (Jun 25 – Jul)

Originally everything was 9P-only. D-Bus was added for desktop integration:

Separate ollie-dbus daemon (ollied)
Embedded directly into olliesrv (9P server got -no9p flag)
KDE uses D-Bus exclusively; acme/sh/el use 9P
9P and D-Bus do not share sessions — independent stores, same core

Phase 9: Remote Execution (Jul 5–19)

Split-brain architecture: orchestration stays local, code execution on a remote machine via SSH.

ollie-remote binary auto-deployed via SSH bootstrap
Tools embedded in remote binary
Streaming output notifications back to local session
Session persistence across remote restarts

Phase 10: Tool Registry (Jul 20–29)

The final major architectural shift — from "agent knows all tools upfront" to lazy discovery:

Tools embed their own ollie:prompt metadata blocks
tool_list — discover available tools and descriptions
tool_load — promote a tool to a native callable
tool_active — introspect loaded tools
Skills get the same treatment: skill_list, skill_load, skill_active This solved system prompt bloat — only load what's needed per task.

Phase 11: The Great Flattening (Jul 29–30)

Largest single-day structural change: −4,573 lines net across the codebase. Eliminated accumulated abstractions that no longer served a purpose.

Core (−4,400 lines)

Killed agent.Core interface — concrete *agent.Agent used directly. Nobody else implemented Core; the interface just added indirection.
Killed Dispatcher indirection in toolsrv — toolsrv.Server called directly.
Merged execute/ into tools/ — separate package for 3 functions was pointless overhead.
Renamed for clarity: agent.Session → agent.History, session.Core → session.Session, NewAgentCore → New.
Extracted agent/ package from the monolithic session package.
Threaded context.Context properly through daemon → session → agent (replaced ad-hoc interrupt mechanisms with cancellation).
Dead code removal: GlobalToolNames, ExtractReturnSchema.

9P server (−214 lines)

Killed mgr/ package entirely — replaced Manager struct with package functions in **fs**/. The *fs.Tree IS the session collection; rootState (unexported) lives in tree.Data. CRUD via NewRoot, Lookup, Create, Kill, Rename, Shutdown.
Fixed infinite recursion in session/{sname}/agent/ directory — pathType() didn't recognize agent subdirectories, causing walk to succeed at any depth.
Fixed phantom session descent — walk into non-existent sessions now fails immediately.
Fixed /agents empty read — makeStat wasn't computing Length for root files, FUSE kernel saw 0 bytes and never issued a read.
Fixed session.All() — was looking at root.Children() (empty) instead of rootState.sessions. Broke D-Bus ListSessions and GUI session restore.

Frontends

acme: paths updated s/ → session/, agent files route through session/{sname}/agent/{aname}/.
ellie.el: same path migration, added ellie--agent-dir for agent ID discovery.
KDE GUI: no changes needed (D-Bus operates on Go objects, not paths).

New filesystem layout

session/
├── new              write key=value to create session
├── idx              session index (tab-separated)
├── {sname}/
│   ├── plan         session-scoped markdown checklist
│   ├── env          session environment
│   └── agent/
│       └── {aname}/
│           ├── cfg          session configuration (key=value)
│           ├── ctl          control commands
│           ├── state        current state
│           ├── statewait    blocks until state changes
│           ├── chat         conversation log
│           ├── prompt       submit prompts
│           ├── prompt.prev  last submitted prompt
│           ├── offset       byte offset after last user prompt
│           ├── fifo.in      queue a prompt (FIFO input)
│           ├── fifo.out     pop queued prompt (FIFO output)
│           ├── context      rendered context window
│           ├── systemprompt rendered system prompt
│           ├── usage        token stats
│           ├── cost         cost estimate
│           ├── ctxsz        context size
│           ├── models       available models
│           ├── tools        tool management
│           ├── toolsrv.loaded  currently loaded tools
│           ├── toolsrv.rev     revision counter
│           ├── tail         exec helper
│           └── proc/        detached process output

Architecture after (single Go module, no submodules for core/9p)

*fs.Tree (root)                     ← IS the session collection
  └── .Data = *rootState            ← sessions map, config, nextUID
       └── sessions[id] = *Session  ← owns Agent, context, cancel
Package functions (no Manager):
  session.NewRoot(cfg) → *fs.Tree
  session.Lookup(tree, id) → *Session
  session.All(tree) → []*Session
  session.CreateFromRoot(tree, args) → (id, error)
  session.KillFromRoot(tree, id)
  session.RenameFromRoot(tree, old, new)
  session.Shutdown(tree)

SLOC after flattening

Component	Lines
agent + backend + toolsrv + session	10,437
fs + cmd/olliesrv	7,925
kde gui	12,331
Total	32,538
Zero dead exported functions remain (verified via LSP + grep across all repos).

Phase 12: Second Flattening & .meta Sidecar (Jul 30)

Another aggressive structural pass. Eliminated the entire script namespace layer (s/, u/, x/), flattened contrib/ into data/, and replaced header-comment-based tool metadata with JSON sidecar files.

Scripts removed (−1,100 lines)

s/{sh,b,bfg,bbg,ls,kill,cleanup} — shell-based session frontends replaced by ollie-9p rdwr generate/complete/route and GUI frontends
u/{optimize,complete,cascade,escalate} — utility scripts replaced by generate, complete, route files in the 9P namespace
x/{bd,prime,freeloader,task} — internal plumbing scripts; PATH prepend (prependOlliePath) removed from toolsrv
9P session root trimmed to just new and idx; script-serving code removed
install-scripts target now empty

Tool metadata: .meta sidecar files

Header comment parsing (ollie:prompt, ollie:tier, ollie:parallel read, args_json:) eliminated entirely. All tool metadata now lives in a JSON sidecar file alongside the executable:

tools/
  file_edit       ← executable (any language)
  file_edit.meta  ← JSON metadata

{
  "description": "Replace text in a file.",
  "prompt": "## file_edit\n\n...",
  "args": {"type":"object", ...},
  "tier": "cold",
  "readOnly": true
}

This decouples metadata from implementation language — compiled Go binaries, Python scripts, and bash tools all use the same discovery mechanism.

LSP tools ported to Go

The Python LSP bridge (_lib/lsp/) replaced with a pure Go implementation:

tools/
├── builtin/          ← in-process handlers (shell, reasoning, tool/skill registry)
└── lsp/              ← shared LSP client library
    └── cmd/          ← individual binaries (lsp_definition, lsp_hover, ...)

Architecture: each LSP tool binary embeds a bridge daemon (started on first invocation via --bridge flag, persists via Unix socket, auto-exits after 5min idle). No Python. No _lib. No external dependencies beyond the LSP servers themselves (gopls, clangd, intelephense).

Repo reorganization

contrib/{prompts,scripts,services,tools,skills,agents} → data/
contrib/elisp stays (Emacs frontend contribution)
tools/builtin/ — built-in tool handlers (moved from tools/, removed in Phase 18)
tools/lsp/ — Go LSP implementation (replaced data/tools/_lib/lsp/)

Conditional variants for network transparency

Tools can declare multiple variants gated by host conditions (binary, file, os, arch, env, nenv). First match determines the tool's schema and executable. Enables one .meta to work across heterogeneous hosts — the contract adapts to capabilities. Critical for ollie-remote deployments where the remote host may have different tools, package managers, or init systems.

Principles That Emerged

Filesystem-as-API — everything is read/write on synthetic files. No custom protocols needed. Monorepo coordinates versions.
Scripts over servers — MCP servers removed early. Simple scripts won.
Progressive disclosure — lazy tool/skill loading keeps system prompts small until complexity is needed.
Plan simplicity — after 5 iterations, planning settled on one markdown checklist file per session.
**Single control plane — 9P for everything. Streaming via blocking reads. No polling.
Security by default — sandboxed execution with explicit bypass, per-session identity.

Phase 13: 9P Streaming & D-Bus Removal (Jul 31)

The D-Bus adapter (org.ollie.SessionManager) is deleted. All clients now use 9P exclusively via blocking reads for natural streaming.

Key changes:

chat file: blocking read that delivers tokens as the agent produces them. Per-fid offset. Never EOF (blocks between turns). EOF only on kill.
log file: replaces old chat. Returns last 64KB (sliding window). Non-blocking, tail-able via Qid.Vers.
kill/. ctl command: kills session → EOF on chat readers.
KDE GUI: rewritten from scratch. Uses 9p (plan9port) via QProcess. Streaming via readyReadStandardOutput. Plain text tail (8KB). No ChatBlockModel, no D-Bus, no ThemeManager. ~250 lines total.
Kate plugin: all D-Bus calls replaced with 9p subprocess calls. Streaming chat + statewait via persistent QProcess.
KRunner: uses 9p for session listing and one-shot generation.
Acme frontend: uses 9fans.net/go plan9/client directly. Blocking read loop on chat. No FUSE mount.
Plasmoid, tray: deleted (not useful enough to maintain).
dbus/ package: deleted (-771 lines from server).

The only remaining godbus usage: org.freedesktop.Notifications for bypass prompts (desktop integration, not ollie protocol).

Lessons:

9P's request-response model gives natural streaming for free. Server delays Rread until data arrives. No polling needed.
FUSE mounts do NOT support blocking reads (return EOF immediately). Clients must use the 9P protocol directly.
Never bind a growing string to a QTextArea with word-wrap. QTextDocument relayout is O(n) on the full content.
The loadEarlier scroll-triggered cascade was creating 500 delegates on startup. Bounded initial window + explicit scroll-back is correct.

Phase 14: Multi-Agent Deepening — ID/Name Split & Empty Sessions (Aug 1)

The multi-agent data structures introduced during the Great Flattening required further refinement to support true multi-agent sessions.

ID/Name Split

Sessions and agents now have two identities:

id (immutable UUIDv4, set at creation, never changes)
name (mutable human-readable string, defaults to first UUID segment)

The sessions map is keyed by Name, not ID. Renaming a session or agent moves the directory in the 9P namespace — mv on the session directory calls RenameFromRoot, and write to the agent's name file calls SetName.

Both entities expose id and name files in their 9P directories.

Two-Step Session Creation

Creation split into two phases:

session/new — write name=<name> to create an empty session (no agent yet)
session/{name}/agent/new — write cwd=<dir> backend=<backend> model=<model> agent=<agent> to create the agent

This enables creating sessions in advance and attaching agents later. Empty sessions (Core=nil) are valid — they appear in session/idx with empty fields and are fully killable/renameable.

Multi-Agent Data Structures

session.Session supports multiple agents (Agents() returns []*agent.Agent)
Session.AgentLogs maps agent IDs to *AgentLog instances
Active agent is selectable, each agent has its own prompt/chat/state/log
session/idx emits one line per agent, not one line per session

Key commits: 92 commits across Aug 1 touching session/, agent/, fs/, and kde/.

Phase 15: 9P Declarative EDSL (Aug 2 — morning)

The most significant structural change to the 9P server since its inception: replaced the imperative filesystem with a declarative EDSL.

Before: Imperative

Every 9P operation (stat, walk, read, write, open, create, remove) was hand-coded per node. The fs/session/ sub-package contained ~3,100 lines of manual tree-building code split across 6 files (files.go, root.go, perm.go, synth.go, create.go, persist.go). Adding a new file meant updating stat, readdir, open, and permission code paths.

After: Declarative EDSL

The entire namespace is declared in a single FsNodeDecl tree in fs/spec.go:

var treeSpec = Dir("/",
    Leaf("backends", 0444, Read(readBackends), GID("agent")),
    Leaf("eventwait", 0444, Read(readEventwait), Stream(blockEventwait), GID("agent")),
    Leaf("complete", 0666, Request(requestComplete), GID("agent")),
    // ...
    Dir("session", GID("agent"),
        Leaf("new", 0666, Request(requestSessionNew)),
        Leaf("idx", 0444, Read(readSessionIdx)),
        TemplateDir("{id}", listSessions),
    ),
)

BuildTree() in fs/builder.go walks the spec, validates invariants (directories can't have leaf handlers, at most one blocking variant, template nodes must have List, ownership inheritance), and produces a fully-wired *fs.Tree.

Package consolidation

Entire fs/session/ sub-package flattened into fs/ — 6 files deleted
cmd/olliesrv/bypass_tree.go — bypass tree moved to fs/bypassfiles.go
cmd/olliesrv/server.go simplified from ~820+ lines to ~200 lines of thin 9P protocol handling; all filesystem logic is now in fs/
Handler files organized by scope: rootfiles.go, sessionfiles.go, agentfiles.go, bypassfiles.go, procfiles.go

Net result

−3,152 lines deleted, +2,086 lines added across 24 files
Adding a file = adding one Leaf() call. No manual stat/readdir/write.
Validation at BuildTree time catches structural errors before serving.
Permission model inlined in the spec via mode and GID() — no separate perm.go.

Phase 16: 9P Server Streamlining (Aug 2 — afternoon)

Following the EDSL conversion, the 9P protocol server itself (cmd/olliesrv/server.go) received a focused cleanup over 13 commits:

Fid management

Server owns the fid map: validates newfids (not already in use, not stale), garbage-collects orphaned fids. No more leaking fids on client disconnect.
Store opened file on fid: read/write use f.entry directly instead of re-resolving the path on every operation. Eliminates a class of TOCTOU bugs.

Method extraction

attach and open extracted from *Server — pass groups and log explicitly
GroupTable extracted from *Server — permission checking uses bitmask OR instead of a boolean dance (checkPermBits → hasPermBits)
checkPerm extracted to package-level, permissions resolved in open
9 rootTree-only methods extracted to package-level functions; switch statement replaced with handler map
Three thin wrapper methods inlined (readFile, writeFile, FileTree alias)
handle inlined into Start; renamed Serve → Start, Shutdown → Kill

Result

server.go reduced from ~820 to ~200 lines
main.go simplified as bypass tree wiring moved to fs/
Server is now a thin 9P2000 protocol translator, not a filesystem

Phase 17: Eventwait Redesign & Chat Log Format (Aug 2)

Global /eventwait

Replaced per-session event polling with a global event ring buffer and a single /eventwait file. Events carry structured prefixes and delta descriptions:

S new session/<name> — session created
S kill session/<name> — session destroyed
S rename session/<old> session/<new> — session renamed
A kill session/<name>/agent/<aid> — agent killed

The event ring is a fixed-size circular buffer (100 slots) with a sync.Cond for blocking reads. Frontends block on /eventwait and receive deltas since their last known offset. No polling, no timer, no D-Bus.

Fixes along the way:

Dangling pointer in eventRing cond initialization (caused panics under load)
Double-unlock panic in WaitEvent
Spurious [[[end]]] markers on reasoning blocks (reasoning events were silently suppressed from chat but still advanced the role-transition state)

Chat log: [type]/[end] block format

The chat log switched from plain text to a structured block format:

[[[assistant:resp-abc123]]]
Hello! How can I help?

[[[end]]]
[[[tool:file_read]]]
File contents...

[[[end]]]

This enables reliable parsing of multi-turn, multi-role conversations from the flat log. Each event type (assistant, tool, call, retry, error, usage, info, exec) opens a block and [[[end]]] closes it. reasoning and reasoning_think events are suppressed from the log entirely — they're included in the LLM context but invisible to the user.

KDE submodule evolution

The KDE frontend received extensive updates across both days:

StreamFsm refactor — chat block FSM with proper fence-post handling
Responsive session tree — selection/statewait stream management
Explicit agent selection — per-agent statewait/chat streams, only switchAgent starts streams; session click clears agent selection
Emoji-labeled states — visual state indicators (❌ for errors, ⚠️ for warnings)
Double-click rename — inline renaming of agent nodes
Robust session loading — loads sessions on startup, handles empty state
Context menu — kill session, per-session operations
Eventwait-driven updates — replaced timer-based polling with blocking /eventwait
Fence fix — fixed a bug where code fences could eat surrounding content
Removed D-Bus entirely; all communication via 9p plan9port binary

Updated Filesystem Layout (post-refactor)

/                          ← root (owned by system user)
├── backends               backends list
├── help                   help text
├── models                 model list (cached)
├── agents                 agent config list
├── ctl                    root control (invalidate, kill)
├── eventwait              global event stream (blocking read)
├── complete               request-response: code completion
├── generate               request-response: one-shot LLM generation
├── route                  request-response: model routing
├── bypass/
│   ├── policy             global bypass policy
│   └── pending/{sname}       pending bypass requests
└── session/
    ├── new                write key=value to create session
    ├── idx                session index (one line per agent)
    ├── {name}/
    │   ├── env            session environment variables
    │   ├── ctl            session control (kill, save, invalidate)
    │   ├── plan           session-scoped markdown checklist
    │   ├── id             immutable session UUID
    │   ├── name           mutable session name (write to rename)
    │   ├── bypass        per-session bypass policy
    │   └── agent/
    │       ├── new        write config to create agent (rdwr)
    │       └── {aname}/
    │           ├── prompt, prompt.prev, fifo.in, fifo.out
    │           ├── chat (streaming), log (64KB window)
    │           ├── state, statewait (blocking)
    │           ├── cfg, ctl, cwd
    │           ├── id (immutable), name (mutable)
    │           ├── offset, usage, cost, ctxsz, models
    │           ├── systemprompt, context, tail
    │           ├── tools, toolsrv.loaded, toolsrv.rev
    │           └── proc/{pid}

Updated Current Topology

ollie/                      ← single Go module
├── cmd/
│   ├── olliesrv/           ← 9P server (sessions, agents, backends)
│   │   └── internal/       agent/, backend/, bypass/, fs/, session/, prompts/, toolclient/
│   ├── toolsrv/            ← 9P tool execution server (separate process)
│   │   └── internal/       fs/, exec/, registry/, sandbox/
│   ├── ollie-9p/           ← 9P client CLI
│   └── ollie-remote/       ← remote execution binary
├── toolsrv/                ← 9P client library for toolsrv
├── fsedsl/                 ← filesystem declaration EDSL (used by both servers)
├── env/                    ← environment helpers
├── log/                    ← structured logging
├── paths/                  ← XDG path resolution
├── kde/                    ← KDE Plasma (submodule)
├── data/tools/             ← tool executables + .meta sidecar files
├── data/agents/            ← agent configs (JSON)
├── data/prompts/           ← system prompt fragments (markdown)
├── data/skills/            ← domain knowledge modules (markdown)
├── data/scripts/           ← ollie-remount, o CLI
├── data/services/          ← systemd, xdg-autostart
├── prompts/                ← embedded prompt templates
└── doc/                    ← documentation

Phase 18: Zero Built-in Tools (Aug 2–3)

The most radical simplification yet: zero tools compiled into the Go binary. Every tool — including shell and reasoning_think — is now an external script loaded dynamically through the 9P filesystem.

What changed

Before: tools/builtin/builtins.go returned four Go-compiled handlers (shell, reasoning_think, tool_list, tool_load, tool_active). These were hard-linked into every agent's tool definition, consuming LLM context slots even when unused. The tool registry was a two-tier system: built-in handlers (Go code, always available) + script tools (loaded on demand via tool_load).

After: tools/builtin/ was removed entirely — builtins.go last returned nil. All tools are loaded through a single convergent path: writing the tool name to session/{sname}/agent/{aname}/tools. The shell and reasoning_think tools are now external scripts (data/tools/shell, data/tools/reasoning_think) with .meta sidecars — identical to file_read, file_grep, or any other tool.

Unified tool loading

Tool loading follows a single path shared by both 9P writes and autoLoad initialization:

9P write to agent/{id}/tools
         │
         ▼
    session.loadTool(name, agent)
         │
         ├── Check disallow list
         └── RPC "tool_load" → ollie-remote process

The tool_load tool itself is a bash script (data/tools/tool_load) that writes the tool name to the 9P filesystem — it's a meta-tool that uses the same interface as every other operation.

Agent config: `autoLoad`

Agent configuration files now declare which tools to load at startup via an autoLoad array:

{
  "autoLoad": [
    "file_read",
    "file_edit",
    "file_glob",
    "file_grep",
    "file_write",
    "tool_load"
  ]
}

Each agent profile (default, copilot, explorer, librarian, navigator, taskmanager, theo) has its own autoLoad list tailored to its domain. The allowTools field provides a secondary security gate.

Tool output format classification

Each tool's .meta can declare an outputFormat field specifying the source-fence language for its output:

{
  "description": "...",
  "outputFormat": "markdown"
}

The agent loop propagates this to the chat log as the fence language ([[[tool:file_read]]]\``markdown), enabling syntax-highlighted output in the KDE GUI. Tools without outputFormat` default to plaintext fences.

Remote execution simplified

ollie-remote no longer embeds tool scripts or sandbox config:

Removed //go:embed all:tools — tools come from $OLLIE_TOOLS_PATH in the environment
Removed embedded sandbox default.yaml — sandbox config is provisioned alongside tools
Removed --tools flag — uses ~/.config/ollie/tools by default

SSH bootstrap simplified

The SSH bootstrap sequence was cut from a complex multi-stage protocol to a single tarball transfer:

Before	After
SHA256 hash verification	No hashing
gzip + base64 binary encoding	Raw `tar czf` tarball
`OLLIE_LOADER_START` / `OLLIE_LOADER_READY` handshake	Single `OLLIE_LISTEN_READY` signal
Separate `bootstrap.sh` template (88 lines)	Inline 10-line script
Per-binary transfer (ollie-remote + landrun)	Single `tar` of `~/.config/ollie/`

The simplified bootstrap:

CACHE_DIR="${XDG_CACHE_HOME:-$HOME/.cache}/ollie"
mkdir -p "$CACHE_DIR"
tar xzf - -C "$CACHE_DIR"                    # receives tarball from local
export OLLIE_TOOLS_PATH="$CACHE_DIR/tools"
export PATH="$CACHE_DIR/bin:$PATH"
exec "$CACHE_DIR/bin/ollie-remote" serve --cwd <dir> --listen <sock>

9P namespace changes

agent/{id}/tools — read lists all currently loaded tools (via the session's toolsConn), write loads a tool by name (via session.loadTool)
Removed agent/{id}/toolsrv.loaded — subsumed by tools read
Removed agent/{id}/toolsrv.rev — no longer needed
Added root-level /tools — lists all discoverable tools on disk (global catalog)
Removed HandlerCtx.ToolReg — the in-memory Go-level registry is no longer passed through 9P handlers

Files removed

File	Status
`tools/builtin/`	Package removed entirely — all tool logic lives in external scripts
`tools/builtin/shell.go`	Deleted (now `data/tools/shell` + `data/tools/shell.meta`)
`tools/builtin/reasoning.go`	Deleted (now `data/tools/reasoning_think` + `data/tools/reasoning_think.meta`)
`tools/builtin/tool.go`	Deleted (`tool_list`/`tool_load`/`tool_active` replaced by 9P filesystem mechanism)
`toolsrv/bootstrap.sh`	Deleted (inline bootstrap replaces multi-stage protocol)
`cmd/ollie-remote/tools/.gitkeep`	Deleted (no embedded tools)

Lines of code

−530 lines from the Go core (removed built-in handlers, embedded tools, bootstrap script)
+82 lines for external tool scripts + .meta sidecars (shell, reasoning_think, tool_load)
Net: tools system is smaller, simpler, and more uniform

Why it matters

The zero-tools transition is the culmination of the architectural trajectory that began with the tool registry in Phase 10: every facility is a file; nothing is hard-coded.

Uniformity — There is no distinction between "core" tools and "script" tools. shell, reasoning_think, file_read, file_write — all loaded the same way, all have .meta sidecars, all called via the same RPC mechanism.
Minimal core — The Go binary no longer contains any tool logic. The agent loop is pure coordination: stream LLM → parse tool calls → dispatch to remote process → loop. Tool definitions, schemas, and execution are entirely external.
Agent-choosable tools — Each agent profile declares exactly which tools it needs. No agent pays the context-window cost of shell and reasoning_think unless its config asks for them.
9P as the sole control plane — Tool loading is now a file write. The same mechanism (ollie-9p write ...) that submits prompts and reads state also manages tool loading. No special RPC, no separate tool-manager protocol.
Simpler remote deployment — No embedded files, no binary transfer protocol, no hash verification. Provisioning a remote host is tar czf ~/.config/ollie | ssh host tar xzf -.

The result: the Go core is a 9P file server and an agent loop, nothing more. All behavior — every tool, skill, prompt, and configuration — lives in data files on disk. This is the Plan 9 ideal: a small, fixed kernel; all policy and capability in the filesystem.

SLOC current (Aug 3)

Component	Lines (excl. tests)
agent + backend + toolsrv + session	11,828
fs + cmd/olliesrv	4,838
kde gui	6,491
Total	23,157
Zero dead exported functions remain (verified via LSP + grep across all repos).

Phase 19: EDSL Extraction & Structural Cleanup (Aug 3–4)

The declarative EDSL created in Phase 15 was extracted into a standalone, reusable library. This wasn't just moving code — the EDSL itself was created from scratch on Aug 2 to replace ~3,100 lines of imperative filesystem code. Within 48 hours it went from concept to extracted library.

The trajectory:

Aug 2 morning: Created the EDSL (Phase 15) — replaced imperative tree-building with declarative spec
Aug 2 afternoon: Streamlined the 9P server (Phase 16) — reduced to thin protocol handler
Aug 3: Extracted EDSL to fsedsl/ as a reusable library
Aug 4: Structural cleanup — removed dead packages, improved o CLI

This rapid iteration exemplifies the project's development style: build the right abstraction, validate it works, then extract it for reuse.

fsedsl: Standalone Library

The generic EDSL types and builder logic extracted from fs/ into fsedsl/:

fsedsl/
├── fsnode.go     FsNodeDecl[C], NodeOption[C], Dir/Leaf/Each constructors
├── builder.go    BuildTree — walks spec, validates, wires handlers
├── tree.go       Tree, File, FileConfig, FileTree types
├── option.go     Doc, Read, Write, Stream, BlockOnce, Request, etc.
└── README.md     Integration guide

The library is protocol-agnostic — it knows nothing about 9P. It produces a *Tree structure that any protocol (9P, FUSE, HTTP, gRPC) can walk. The fs/ package becomes a thin adapter: type aliases + ollie-specific handlers.

Key design:

Generic over context type C — ollie uses HandlerCtx, others can use anything
Single BuildTree[C]() call validates and wires the entire namespace
Validation at build time: no leaf handlers on directories, at most one blocking variant per file, template nodes require List, ownership inheritance

fs/ Package Consolidation

After EDSL extraction, the fs package was restructured:

Before (scattered):

fs/
├── spec.go           namespace declaration
├── builder.go        tree builder (moved to fsedsl)
├── adapter.go        type aliases
├── session/          sub-package with 6 files
│   ├── files.go
│   ├── root.go
│   ├── perm.go
│   └── ...
└── bypassfiles.go

After (flat, by scope):

fs/
├── spec.go           single source of truth — entire namespace
├── fsnode.go         type aliases from fsedsl
├── builder.go        BuildTree wrapper
├── rootfiles.go      root-level handlers (backends, models, help, ctl)
├── sessionfiles.go   session-level handlers (env, ctl, name)
├── agentfiles.go     agent-level handlers (prompt, chat, state, tools)
├── bypassfiles.go   bypass broker handlers
└── procfiles.go      detached process handlers

No sub-packages. Handler files are organized by 9P tree scope.

Session Package Refactors

Major restructuring of session lifecycle:

Session/Agent split — session.Session manages lifecycle, agent.Agent manages conversation. Clear ownership boundaries.
Persistence to session/ — fs/persist.go removed, persistence logic moved to session/ package
Manager eliminated — Package-level registry replaces Manager struct. CRUD via session.Lookup(), session.All(), session.CreateFromRoot(), etc.
Remote promoted to session level — Remote execution managed at session, not agent level

Event Bus: Ring Buffer → Pubsub

The event ring buffer added in Phase 17 lasted exactly one day. On Aug 3, it was replaced with github.com/simonfxr/pubsub:

Why the ring buffer failed:

Off-by-one bugs in circular buffer indexing
Race conditions between sync.Cond broadcasts and context cancellation
Panics from double-unlock when multiple readers raced to the same event
The offset-based API leaked implementation details to clients

What pubsub provides:

Topic-based routing (session.{id}.new, agent.{id}.state)
Wildcard subscriptions (session.*, *)
Channel-based delivery with automatic cleanup on context cancellation
A decade of production hardening by someone else

The backwards-compatible PostEvent() wrapper translates the old format (A agentId state thinking) to the new topic format (agent.agentId.state with payload thinking). Existing code kept working.

Lines changed: +103 / -72. Net increase, but the new code is correct.

Dead Code Removal

Removed	Reason
`cmd/Ollie` (Acme frontend)	Use `o` CLI to compose acme front-end
`mount/` package	FUSE mount replaced by direct 9P client (`ollie-9p`)
`ollie-watchdog` script	No longer needed — 9P client doesn't have FUSE stale mount issues
`doc/experiments/`	Historical experiments moved to git history
`doc/help.md`	Now generated dynamically from `Doc()` strings in spec

o CLI Improvements

The terminal CLI (data/scripts/o) was simplified:

Removed auto-session/agent — Context must be set explicitly via export or eval $(o env ...). Eliminates slow ollie-9p read session/idx calls.
Path classification — Paths categorized as root/session/agent level. Clear error messages: "chat requires agent context".
TUI percentage layout — Fixed-size splits replaced with percentages (-l 25%). Adapts to terminal size.
Block marker filtering — Chat stream piped through grep -v '\[\[\[.*\]\]\]' to hide internal markers.

Documentation

Removed outdated docs (doc/experiments/, stale help.md)
Renamed doc files to lowercase-kebab-case
Updated usage.md for new o CLI behavior
Removed mount/ and watchdog references from architecture docs

Gantt Update

The timeline now extends to Aug 4, 2026 — ~3.75 months of development.

SLOC current (Aug 4)

Component	Lines (excl. tests)
agent + toolsrv + session	9,077
fs + cmd/olliesrv	2,919
fsedsl (extracted library)	1,220
kde gui	4,162
Total (core)	17,378

Note: Aug 3 count included backend/ (4,366 lines) and had different kde scope. Comparable core (fs + agent + toolsrv + session + fsedsl + kde gui) shrank from ~23k to ~17k.

"Good Idea Fairy" Cleanup (Aug 5)

A brutal simplification pass that deleted ~960 lines of speculative infrastructure.

Hooks — Deleted Entirely

The entire hooks system was removed:

Hook	What it did	Why deleted
`agentSpawn`	Run commands on session creation	Prompt composition handles initialization
`preTurn`	Inject context before each turn	Removed earlier; prompt assembly does this
`postTurn`	Run commands after each turn	Never used in practice
`preTool`	Gate tool execution	Security model moved to sandbox
`postTool`	Transform tool results	Never used
`preCompact`	Run before context compaction	No use case
`postCompact`	Run after compaction	No use case
`turnError`	Handle backend errors	Replaced by retry logic in the loop

Hooks are a "standard feature" in agent frameworks — lifecycle callbacks that let external code react to events. In practice, they added complexity without solving real problems. The prompt system handles initialization. The sandbox handles security. Error handling belongs in the loop. Every hook was either unused or doing something that belonged elsewhere.

−237 lines from agent/hooks.go.

Other Deletions

What	Why
`backend/noop.go`	Test backend, never used. Tests use real backends or mocks.
`backend/oneshot.go`	Alternative generation mode, never used. `generate` file handles one-shot.
`cmd/ollie-9p/mount/`	FUSE mount code, replaced by `ollie-9p` direct 9P client.
`data/tools/route.meta`	Model routing tool, replaced by explicit `/model` commands.
`toolsrv/schema.go`	Schema validation code, never enabled.

Turn State Machine

The turn state machine in agent/turn.go was simplified from 158 lines to ~40. The TurnCtx struct was not yet eliminated (that came later in Aug 11), but the state transitions were flattened.

Philosophy

"Good idea fairy" is military slang for someone who shows up with clever suggestions that sound good but create work. The deleted code was all reasonable in isolation — hooks are a standard pattern, test backends are useful, FUSE mounts are convenient. But each added surface area that had to be maintained, tested, and reasoned about. None of it solved actual problems better than simpler alternatives.

Aug 9, 2026 — Simplification & Namespace Cleanup

Major simplification pass: ~1,450 lines removed from the core runtime.

Structural

Section-based preamble — Preamble struct with Set/Get/String replaces string surgery (-270 lines)
Session — Export immutable fields, extract buildAgent helper (-115 lines)
Prompt resolver — Gutted to file-path-only resolution (-127 lines)
TaskState subsystem — Removed entirely (-246 lines)
GenerationParams — Embedded in AgentConfig, JSON tags added (-34 lines)
sync.Cond → close-channel — WaitChange uses idiomatic channel signal
Dead code — session.New(), env map, prevPrompt field, usage_log.go, contextDebug()

Namespace

The agent namespace went from 17 files to 10. Redundant files merged or moved to ctl:

Before	After
`fifo.in` + `fifo.out`	`fifo` (write=enqueue, read=dequeue)
`cost` + `usage` + `ctxsz`	`stats` (key=value lines)
`chat`	`chat` (filtered text) + `chat.raw` (full markup)
`cwd`, `models`, `tools`, `systemprompt`	ctl commands
`connection`, `state`, `context`, `tail`, `offset`, `prompt.prev`	removed

ctl Unification

All ctl files (root, session, agent) now share:

type rdwrHandler func(ctx HandlerCtx, args []string) ([]byte, error)

Agent ctl is rdwr (request-response): write command, read result. Commands with no args return current state (e.g., /model prints the model).

Agent ctl commands: stop, compact, clear, inject, agent, model, models, tools, tool_load, cwd, name, backend, systemprompt.

Command Surface

/ prefix in prompts == o ctl. No special-cased commands in the agent.

/model qwen3:8b → switches model, returns new model name
/tools → lists loaded tools
/inject look at this → injects prompt mid-turn (overwrites pending)
!q in the REPL → exits the TUI (local-only)

elevate → bypass

The sandbox escape mechanism was renamed from elevate to bypass throughout (package, namespace, env vars, tool args, all docs).

Streaming Fix

The TUI streaming regression (line-by-line instead of char-by-char since Aug 4) was caused by piping through grep -v — grep is inherently line-buffered. Fixed by:

chat file now serves filtered text (markers + fences stripped server-side via stripMarkers state machine)
chat.raw serves full markup for GUIs that parse blocks
TUI no longer pipes through grep for streaming

CLI (`o` script)

readloop → read -l
chatstream → removed (o read chat streams directly)
stop, kill → removed (use o ctl stop, o ctl kill)
ctl uses ollie-9p rdwr (gets response)
REPL: / → o ctl, !q → exit TUI
Background statewait loop trapped on EXIT (fixes zombie)

SLOC (Aug 9)

Component	Lines (excl. tests, backends, fsedsl, generated)
agent	3,167
toolsrv	2,521
fs	1,774
session	1,386
cmd/olliesrv	1,161
bypass	733
lib9p	690
cmd/ollie-9p	356
cmd/ollie-remote	369
sandbox	293
log + env + paths + format + prompts	428
Total (core)	12,878

Aug 9, 2026 (cont.) — Prompt Audit & Maintainability Pass

Systematic audit of the prompt construction system and codebase maintainability.

Prompt System

System prompt: ~200 → ~80 lines. Removed tool-first table, API docs section (moved to agent prompt), dead 9P entries, output protocol (moved per-agent).
AllowTools: removed entirely (field, RPC, config). Tools are loaded dynamically; static allowlists served no purpose.
Phantom refs fixed: route removed from driver autoLoad, reasoning_think + shell added to default autoLoad.
Output protocol: moved from system prompt to per-agent prompts. Theo's brevity rule applied to all agents except copilot.
Tool docs: merged listing + documentation into single renderTools(). Removed JSON schema dump (function-calling API provides it natively). Removed BuildToolListing dead code.
PRIME_ env vars*: removed entirely. Replaced SessionInfra.PromptEnv []string with typed Platform string + IsGitRepo bool fields.

Code Quality

OnToolsChanged: changed from func(string) to func() signal. Agent re-fetches tool list on signal instead of receiving a pre-formatted string it ignores.
Preamble sections: type-safe Section type replaces raw strings. Compile-time typo detection.
extractToolResult: documented the tool output protocol (JSON content-block envelope with raw-text fallback).
Package godoc: session/ package comment documenting Init→Create→Register→Kill lifecycle.
PromptEnv(): deleted (was a no-op returning nil after PRIME_* removal).

Documentation

Moved 7 reference docs from doc/resources/ to doc/ (architecture, 9p, writing-tools, tool-registry, core, remote-execution, edsl). Left whitepaper material (evolution, multi-agent, misc, no-mcp, ideas) in resources/.
Rewrote data/agents/README.md to match current config schema and agent roster.
Fixed stale cross-references in README.md and doc/tool-registry.md.

SLOC (Aug 9, post-audit)

Component	Lines (excl. tests, generated)
agent	3,159
toolsrv	2,464
session	1,377
fs	1,774
cmd/olliesrv	1,161
bypass	733
lib9p	690
cmd/ollie-9p	356
cmd/ollie-remote	358
sandbox	293
log + env + paths + format + prompts	428
Total (core)	12,793

Excludes: backends (4,229), fsedsl (1,030), tests, KDE, tools, generated code.

Phase 20: toolsrv 9P Migration & Process Isolation (Aug 10–11)

The tool server completed its evolution from an in-process library to a fully independent 9P server. Locally it runs as a child process of olliesrv; for remote execution it runs on a remote host, connected via SSH Unix socket forwarding.

Before

toolsrv was a Go package (ollie/toolsrv) with a Server struct that ran in-process within olliesrv. Tool calls were Go function calls — no process boundary, no separate namespace. The package mixed client code, server code, registry, sandbox wrappers, and execution logic in a flat directory.

After

Two distinct components:

cmd/toolsrv/ — a standalone binary that serves its own 9P2000 filesystem over a Unix socket. Has its own internal/ packages:
- internal/fs/ — filesystem spec (fsedsl), server state, process management
- internal/exec/ — sandboxed tool execution (landrun, bypass broker)
- internal/registry/ — session-scoped tool registry
- internal/sandbox/ — landrun configuration (unchanged)
toolsrv/ (root package) — a 9P client library. Conn dials toolsrv over a Unix socket, authenticates via Tauth, and exposes methods like CallTool, LoadTool, ListTools, Ping.

toolsrv 9P Namespace

/
├── ctl          write: load <tool>, unload <tool>, env K=V, cwd <path>
├── tools        read: list loaded tools (JSON), write: tool name to load
├── info         read: platform, arch
└── proc/
    ├── new      rdwr: write tool+args, blocks, read result
    ├── new.bg   write: tool+args, returns pid immediately
    └── {pid}/
        ├── out   read: output
        ├── wait  read: blocks until exit, returns exit code
        ├── stat  read: running/exited, runtime, tool
        └── ctl   write: signal <N>, dismiss

Both cmd/toolsrv and cmd/olliesrv declare their namespaces using the same fsedsl library. Both implement their own 9P protocol handlers (using 9fans.net/go/plan9) — they are intentionally separate servers that communicate over a socket.

Process Lifecycle

Three-layer defense against orphaned toolsrv processes:

Pdeathsig (local only) — SysProcAttr{Pdeathsig: SIGTERM} on local spawn. When olliesrv dies, the kernel terminates toolsrv immediately. For SSH-spawned toolsrv, Pdeathsig is set on the ssh process itself — when SSH dies, the remote shell receives SIGHUP which typically cascades to toolsrv.
Idle timeout (local + remote) — toolsrv tracks active 9P connections. After the first client connects and then all connections drop, a 30s timer starts. If no new connection arrives, toolsrv exits cleanly. 60s startup grace period for the initial connection. This is the primary cleanup mechanism for remote toolsrv where Pdeathsig doesn't apply directly.
Kill-before-respawn — ProcessKeeper.Dial() kills the old process before spawning a replacement. Prevents accumulation during reconnect cycles.

Authentication

toolsrv uses 9P Tauth for authentication. The first client sets the shared secret; subsequent clients must provide the same secret. This replaces the previous token-in-environment approach and is compatible with socket permission security.

Structural Consistency

Both servers now follow the same physical layout:

cmd/{server}/
├── main.go              entry point
├── server.go            9P protocol handler (top-level, like Plan 9 tradition)
└── internal/
    ├── fs/              filesystem spec + handlers + state
    ├── ...              domain-specific packages

Parallel Tool Execution

Replaced binary ReadOnly/not batching with resource-based conflict scheduling. Tool calls within a single turn are grouped into parallel batches based on a three-class scope system declared in .meta:

scope "read" — never conflicts (always parallel)
scope "write" — conflicts only on same file path
scope "global" (or unset, default) — full serialization barrier

The model's read-before-write pattern (enforced by the turn-based protocol) guarantees that parallel writes are safe: each file_edit carries its own old_string context from a prior read, and edits on different paths are provably independent.

Shell and tools with opaque effects (like lsp_rename) declare scope "global" and always run alone. Unknown/unset scope defaults to global — tools must opt in to parallelism.

Background Process Interrupts

Any tool call can include "background": true to execute asynchronously via proc/new.bg. The model receives a process ID immediately and continues working. Background processes have no timeout — they run until they exit naturally, are stopped via proc/{id}/ctl, or die with the session.

Output is streamed in real-time into proc/{id}/out. Upon completion, the process exit notification is injected as a user prompt (written to the agent's prompt file) containing a <system-proc-interrupt> block:

<system-proc-interrupt id="42" cmd="go test ./..." status="exited" exit="1">
--- FAIL: TestFoo (0.00s)
    foo_test.go:12: expected 3, got 2
FAIL
</system-proc-interrupt>

This uses automatic prompt queueing: if the agent is still working, the interrupt arrives on the next idle transition; if the agent is already idle, it arrives immediately. The model sees these as natural follow-up prompts — no polling, no in-loop injection. It can react to failures or stop processes via the proc ctl file (term, kill, dismiss).

Process lifecycle: toolsrv owns all procs. Session death (toolsrv exit) kills all procs via KillAll() + Pdeathsig. Exited procs remain in the tree for 10 minutes after last read, then are garbage collected.

Universal Dispatch Flags

All tools receive four optional parameters injected into their schemas at runtime (not declared per-tool):

bypass — sandbox escape via bypass broker
timeout — execution timeout in seconds
sandbox — sandbox profile override
background — async execution with auto-injected output

Why Not Concurrent-by-Default

Foreground tool calls block the model until all results arrive — deliberately. The alternative (execute everything concurrently, deliver results as queued prompts) was considered and rejected: it turns one coherent reasoning step into N separate generation cycles, each with partial information. The model can't reason about results together, every interrupt costs a full inference round-trip, and the end result is sequential execution but slower and more expensive. Background execution remains an explicit opt-in for genuinely long-running processes where the model doesn't need the result immediately.

Phase 22: Meta-Only Tool Definitions (Aug 11–12)

A .meta file can now define a tool with no companion executable. The cmd field is treated as a shell command string — not a path to resolve, but a command to run directly. JSON arguments are passed on stdin.

Before:

{"cmd": "rg"}  // resolved via PATH, then executed with --flags

After:

{"cmd": "jq -r '.pattern' | xargs rg"}  // shell command, executed as-is

This changes what a "tool" is. Previously, tools were executables with .meta sidecars. Now a tool can be purely declarative: a .meta file that shells out to existing CLIs. Wrap ripgrep, kubectl, or an MCP bridge without writing any code.

The tools directory is prepended to $PATH during execution, so meta-only tools can reference other tools in the same directory.

Why this matters: any CLI becomes a tool with 20 lines of JSON. The barrier to adding capabilities dropped from "write a script" to "write a schema."

Phase 23: Bypass via 9P (Aug 11)

The bypass broker — the mechanism for escaping the sandbox — was previously a Unix socket with a custom protocol. It now routes through the 9P filesystem:

toolsrv exposes bypass/pending (blocking read) and bypass/resolve (write)
olliesrv subscribes to pending requests and routes them to D-Bus for approval
Approved/denied responses flow back through 9P

This unifies the control plane. Everything is files. The bypass socket is gone.

Phase 24: fs/ Flattening (Aug 11)

The olliesrv filesystem package had accumulated abstractions:

AgentLog — chat log buffer wrapper
SessionNode — session state container
RootState — root-level state

These were eliminated. Chat logging moved to the agent package. Session state moved to the session package. The fs/ package now contains:

spec.go — the EDSL declaration with all handlers inlined
support.go — helper functions
newroot.go — tree construction

~600 lines deleted. The handlers live where the data lives.

Phase 25: Sandbox Simplification (Aug 11)

Named sandbox profiles (default, restricted, remote) were removed. There is now one config: sandbox.yaml. Per-project overrides via .ollie-sandbox.yaml remain, but the multi-profile system is gone.

The sandbox parameter on tool calls now does nothing — kept for compatibility but ignored.

Phase 26: Agent Loop Cleanup (Aug 11)

The TurnCtx struct — which carried per-turn state through the loop — was eliminated. Loop functions became methods on *Agent, accessing state directly. The turn state machine in turn.go was simplified.

~100 lines deleted, control flow is clearer.

Phase 27: Tree-sitter Code Intelligence (Aug 15)

Code navigation and structural editing moved into native Go tools backed by Tree-sitter grammars. The tool set now includes:

codebase_overview for repository structure
code_outline for one-file declarations
code_symbols for workspace-wide symbol searches
code_dependencies for imports and includes
code_query for syntax-aware searches
code_rewrite for structural rewrites

The tools support Go, JavaScript, TypeScript, TSX, Python, Rust, C, C++, JSON, YAML, PHP, and Markdown. Agent prompts now require structural tools for repository exploration and reserve text search for simple content searches. This makes targeted edits safer than broad textual replacement.

Phase 28: Go File Tools and Tool-Surface Cleanup (Aug 16)

The core file tools (file_read, file_write, file_edit, file_grep, and file_glob) were replaced with Go binaries and shared implementation code. Tests were added for the file-tool package and existing LSP and web-tool coverage was expanded. The change removes the old Python implementations from the runtime path while preserving the same tool contracts.

Obsolete process-management tools and the logseq tool were removed. The installation recipe was corrected for the compiled file tools. Workspace-path placeholders were standardized across prompts and tool metadata so agents use the actual working directory instead of guessed home paths.

Phase 29: OptMem Persistent Memory (Aug 16)

Ollie integrated OptMem as its persistent memory backend. The legacy memory_recall and memory_remember scripts were replaced by metadata-defined tools that invoke the bundled third_party/optmem/memo executable directly.

OptMem owns the append-only log and bounded B-tree index at $XDG_DATA_HOME/ollie/optmem (default: ~/.local/share/ollie/optmem). Ollie does not maintain a second memory format or parallel store. Reads and writes use the same backend and serialize through the toolsrv path-lock system.

Both memory tools are auto-loaded by every agent profile. Shared prompt guidance tells agents to recall relevant prior context before acting and remember durable decisions, outcomes, preferences, and non-obvious findings. The OptMem executable is installed under $XDG_CONFIG_HOME/ollie/optmem and explicitly granted rwx access by the landrun sandbox.

This integration keeps memory as an ordinary Ollie tool while giving it a durable, searchable backend. It requires no MCP server, daemon, or new control-plane protocol.

Phase 30: Markdown Parsing and Bounded Tool Output (Aug 16)

Tree-sitter Markdown support was added to the code-intelligence layer. .md and .markdown files are parsed as Markdown, and headings are available to structural queries as atx_heading and setext_heading nodes. This makes evolution.md and other documentation amenable to the same targeted tooling as source code.

The maximum tool result included in model context was reduced from 128 KiB to 32 KiB. Large command or file results are therefore less likely to crowd out the conversation and instructions.

Commit count: ~60 commits over 3 days. The recent work added native structural and file tooling, persistent memory, and tighter context bounds while removing obsolete tool implementations.

Dead Ends and Reversals

Everything that was built and then killed, in roughly chronological order.

What	Why it was removed
MCP servers (denote-mcp, 9beads-mcp)	Too heavyweight; scripts are simpler and faster
TUI frontend	Replaced by `o` — a tiny shell script (40 lines of tmux)
`pl/` planning namespace	Over-engineered; plan file is sufficient
Beads integration	External dependency for something a checklist file handles
`plan_create` / `plan_complete` / `task_add` / `task_check` tools	Five planning iterations before settling on a single markdown file
SSH agent proxy	Attack surface not worth the convenience
Separate ollie-dbus daemon (ollied)	Consolidated into olliesrv, then killed entirely
D-Bus adapter (`org.ollie.SessionManager`)	9P streaming is superior; no polling, no offset tracking, no frozen GUIs
D-Bus-driven KDE frontends	Rewritten on pure 9P via plan9port `9p` binary
Plasmoid / tray KDE components	Not useful enough to maintain
FUSE mounts	Replaced by standalone 9P client binary; FUSE can't do blocking reads
Tier routing (FAST/POWER)	Replaced by `/route` endpoint with real model discovery
call_tool / pipe	Replaced by native tool registry with `tool_load`
execute_code (multi-language)	Simplified to shell-only; tool scripts handle language choice
chatwait	Reverted; acme tails `chat` directly
Header comment metadata (`ollie:prompt`, `ollie:tier`, `args_json:`)	Replaced by `.meta` sidecar JSON; decouples metadata from language
`contrib/` directory	Renamed to `data/`; nothing was community-contributed
Script namespaces (s/, u/, x/)	Replaced by 9P request-response files
s/sh, s/bfg, s/bbg	Replaced by `generate`/`complete`/`route` 9P files
`agent.Core` interface	Nobody else implemented it; just indirection
`Dispatcher` indirection in toolsrv	`toolsrv.Server` called directly
`execute/` package	Merged into `tools/`; 3 functions didn't need a package
`mgr/` package (Manager struct)	Replaced by package functions; `*fs.Tree` IS the collection
`fs/session/` sub-package (6 files, ~3,100 lines)	Flattened into `fs/` — no more sub-package indirection
`cmd/olliesrv/bypass_tree.go`	Bypass tree handlers moved to `fs/bypassfiles.go`
Per-session/agent timestamp IDs	Replaced by UUIDv4 — immutable `id` + mutable `name`
Single-step session creation	Split into `session/new` + `session/{name}/agent/new`
Imperative filesystem (stat/walk/read/write per node)	Replaced by declarative EDSL (`fsedsl/`)
Event ring buffer (100-slot circular buffer + sync.Cond)	Lasted one day; replaced by `pubsub` library. Off-by-one bugs, race conditions, panics.
Built-in tool handlers (shell, reasoning_think, tool_list, tool_load, tool_active)	Replaced by external scripts loaded via 9P `agent/{id}/tools` write
Go-compiled tool registry in `tools/builtin/`	Zero built-in tools — all tools are external scripts with `.meta` sidecars
Embedded tool scripts in `ollie-remote`	Remote uses `$OLLIE_TOOLS_PATH` from environment; tools deployed via tarball
SSH binary transfer (gzip+base64+hash-verify)	Replaced by simple `tar
`toolsrv.loaded` / `toolsrv.rev` 9P files	Removed — `tools` file handles both list and load
`cmd/Ollie` (Acme frontend binary)	Replaced by `o` CLI composing acme
`mount/` package	FUSE mount replaced by `ollie-9p` direct 9P client
`ollie-watchdog` script	No FUSE stale mount issues with 9P client
`doc/experiments/`	Historical experiments; moved to git history
`doc/help.md`	Generated dynamically from `Doc()` strings in spec
Hooks system (agentSpawn, preTurn, postTurn, preTool, postTool, preCompact, postCompact, turnError)	Never solved real problems; prompt handles init, sandbox handles security, retry handles errors
`backend/noop.go` (test backend)	Never used; tests use real backends or mocks
`backend/oneshot.go`	Never used; `generate` file handles one-shot
`toolsrv/schema.go` (schema validation)	Never enabled
`data/tools/route.meta` (model routing tool)	Replaced by explicit `/model` commands
TaskState subsystem	Removed entirely (−246 lines); no use case survived
`fifo.in` + `fifo.out`	Merged to single `fifo` (write=enqueue, read=dequeue)
`cost` + `usage` + `ctxsz` files	Merged to `stats` (key=value lines)
`connection`, `context`, `tail`, `offset`, `prompt.prev` agent files	Removed or moved to ctl
AllowTools field	Tools loaded dynamically; static allowlists served no purpose
PRIME_* env vars	Replaced by typed `Platform` + `IsGitRepo` fields
`elevate` naming	Renamed to `bypass` throughout
`superpowerd` (separate privilege escalation daemon)	Integrated into olliesrv as the bypass broker
In-process toolsrv (Go library)	Replaced by separate 9P server process with socket boundary
Binary ReadOnly/not tool batching	Replaced by scope-based conflict scheduling (read/write/global)
`AgentLog` / `SessionNode` / `RootState` abstractions in fs/	Eliminated; state moved to where data lives (agent/, session/)
Named sandbox profiles (default, restricted, remote)	One config: `sandbox.yaml`. Multi-profile system was unused complexity
`TurnCtx` struct	Eliminated; loop functions became methods on `*Agent`
Bypass via custom Unix socket protocol	Replaced by bypass via 9P namespace
`.meta` as sidecar to an executable	`.meta` IS the tool definition; executable is optional (tool-definitions.md)
`subagent_generate` (JIT agent config tool)	Obsolete; sub-agents use existing profiles directly
Fire-and-forget `subagent_spawn` via D-Bus	Replaced by blocking `ollie-9p rdwr agent/new` — returns the reply
Sub-agents as peer agents (no parent, no return)	Replaced by task-scoped sub-agents with context inheritance and automatic cleanup
`cascade` orchestrator script	Replaced by parallel `subagent_spawn` calls (scope: read, natural parallelism)
Agent-loop batching as correctness mechanism	Moved to toolsrv path-lock table; agent batching is now just an optimization
`virtfs.Request` naming	Renamed to `Rdwr` — it's an atomic operation, not a variant of read or write

Feed file + Observer agents + BlockOnce/Stream refactor (Aug 13)

Three related changes in one session. Net +278 SLOC across 21 files.

What was added

feed file — a change-detecting blocking read file on every agent. Write data in, internal consumer submits it as a prompt. Dedup built into the BlockOnce handler: same data written twice never wakes the reader. Enables real-time pair programming — an observer agent watches a human or another agent code.

Observer agent (data/agents/observer.json, data/prompts/agent-observer.md) — read-only agent profile. Only loads file_read, file_grep, file_glob, reasoning_think. Prompt explicitly forbids writes. Receives diffs via feed, makes terse observations.

ConsumeFeed — plain function that dials the 9P server via lib9p and reads from the agent's own feed file in a loop. Each read blocks until feed changes. Same pattern as o read -l.

What was refactored

virtfs.BlockOnce(readFn, signalFn) — previously a raw handler that took (ctx, base) and had to implement its own blocking. Now the framework handles the block-until-changed loop: call readFn(), compare hash to base, wait on signalFn() channel, repeat. Handlers become two-line closures.

virtfs.Stream(readFn, signalFn) — same refactor. streamChat (40 lines of condvar + mutex + offset tracking) replaced by a.ChatRead + a.ChatSignal passed to Stream(...).

Server timeout fallback — when BlockOnce times out (5s, no change), the server falls back to the plain Read handler. Frontends get the current value as a heartbeat. Files without Read (like feed) return empty.

What was removed

streamChat() in support.go (replaced by ChatRead method on agent)
mergeCtx() in support.go (no longer needed — blocking logic moved into virtfs framework)
ObserverFeed script (replaced by the feed file)
WaitChange usage outside filesystem handlers (feed consumer uses lib9p instead)

What was fixed

XDG fallback in prompt resolver — $XDG_CONFIG_HOME now defaults to $HOME/.config when unset
Duplicate tool headers — renderTools skipped the generated ## name when the tool prompt already has one
FIFO drain — queued prompts no longer orphaned on interrupt/panic/toolsrv failure
Session restore ordering — moved after 9P listener starts so feed consumers can connect

Dead ends (killed during the session)

Attempt	Why killed
Custom channel-based Feed with its own signal infrastructure	Duplicated the agent's existing signalCh mechanism
`Stream` mode for feed	Feed is a discrete value, not a byte stream
Internal WaitChange-based consumer	Leaked internal plumbing; should be a plain 9P client
`BlockOnceRaw` as primary API	Forced handlers to implement blocking themselves
Concurrent-by-default tool execution	Rejected: turns one reasoning step into N generation cycles with partial info — sequential but slower and more expensive

Sub-Agents via 9P rdwr (Aug 14)

Sub-agents implemented in 42 net lines of code. The entire feature is wiring — no new infrastructure, no new concepts, no new processes.

The mechanism

session/{sname}/agent/new is an Rdwr file. Without prompt=, it creates a persistent agent and returns its ID (existing behavior). With prompt=, it enters sub-agent mode: creates a transient agent, submits the prompt, blocks until the agent finishes, returns the reply, and destroys the agent.

printf 'cwd=%s\nprompt=fix the tests\n' "$PWD" \
  | ollie-9p rdwr session/$OLLIE_SESSION_ID/agent/new

Why it works in 42 lines

The infrastructure was already there:

Rdwr file primitive (née Request) — atomic write-then-read, per-open isolation
Agent.Submit() — already blocks through the full turn
Agent.Reply() — already captures the final response
Session.RemoveAgent() — already handles cleanup
Event bus — normal agent lifecycle events fire (GUI sees sub-agents appear/disappear)

The handler is just: parse params → create agent → submit → read reply → destroy → return.

Path-based lock table

Prerequisite: moved conflict serialization from the agent loop into toolsrv itself. All foreground tool calls now acquire a path-based lock before execution:

scope "read" — no lock (never conflicts)
scope "write" — exclusive lock on the file path
scope "global" — exclusive global lock

This ensures sub-agents writing to the same file serialize correctly regardless of which agent initiated the call. The agent loop's batching remains as a performance optimization; correctness is enforced at the execution layer.

Parallel dispatch

subagent_spawn is declared scope: "read" — multiple spawn calls in the same turn run in parallel automatically. The calling agent blocks until all results return. No shell backgrounding, no polling, no coordination code.

sequenceDiagram
    participant P as Parent Agent
    participant S1 as Sub-Agent 1
    participant S2 as Sub-Agent 2

    P->>S1: subagent_spawn(prompt="task A")
    P->>S2: subagent_spawn(prompt="task B")
    Note over P: blocked (parallel tool calls)
    S1-->>P: reply A
    S2-->>P: reply B
    Note over P: resumes with both results

What was killed

subagent_generate — JIT agent profile generation tool. Obsolete: sub-agents use existing profiles directly.
D-Bus spawn path — subagent_spawn previously used dbus-send to create sessions. Replaced by a 3-line ollie-9p rdwr call.

Framework rename: `Request` → `Rdwr`

The three atomic 9P operations are now named for what they are:

Read — non-blocking read
Write — non-blocking write (fire-and-forget)
Rdwr — atomic write-then-read (blocking, produces result)

BlockOnce and Stream are special cases of Read. Rdwr is its own primitive — not a variant of either.

Architecture validation

This feature is evidence that the "everything is a file" architecture works at scale. A major capability (parallel sub-agents with automatic serialization) landed as pure wiring because:

The 9P namespace already exposes agent/new as an rdwr file
The agent loop already blocks and produces results
The tool server already serializes conflicting writes by path
The batching algorithm already parallelizes non-conflicting calls

No new abstractions. No new processes. No new protocols. Just connecting existing pieces with 42 lines of glue.

Context inheritance and isolation

A sub-agent gets its own context window, history object, runtime state, tool registry, cache, step budget, and lifecycle. When spawned from an existing agent, the parent conversation messages are copied into the child's initial history via RestoreHistoryFromMessages.

This is a one-time snapshot, not a live shared context. The child cannot append to, rewrite, or otherwise mutate the parent's history. The parent does not see child messages while the child runs. The child reports only its final reply through the blocking subagent_spawn result; the parent decides whether and how to incorporate that reply into its own context.

This separation is deliberate. Shared conversation mutation would create ordering races, prompt contamination, and unclear ownership of tool results. Shared workspace resources remain coordinated independently through toolsrv's scope and path locks.

Future work

Shared toolsrv per host — currently each session spawns its own toolsrv. Cross-session path serialization requires sessions to share a single toolsrv instance per host.
Budget/depth controls — token limits, step limits, and recursion depth caps for sub-agents.

77 KiB Raw Blame History Unescape Escape

Architectural Evolution

Timeline

Phase 1: Monorepo Bootstrap (Apr 11)

Phase 2: The 9P Decision (Apr 14–28)

Phase 3: Tool System (Apr 28 – May 10)

Phase 4: Planning System Churn (May – Jul)

Phase 5: Multi-Agent Coordination (May 25 – Jun 15)

Phase 6: Frontend Proliferation

Phase 7: Security Model (May – Jul)

Phase 8: D-Bus as Second Control Plane (Jun 25 – Jul)

Phase 9: Remote Execution (Jul 5–19)

Phase 10: Tool Registry (Jul 20–29)

Phase 11: The Great Flattening (Jul 29–30)

Core (−4,400 lines)

9P server (−214 lines)

Frontends

New filesystem layout

Architecture after (single Go module, no submodules for core/9p)

SLOC after flattening

Phase 12: Second Flattening & .meta Sidecar (Jul 30)

Scripts removed (−1,100 lines)

Tool metadata: .meta sidecar files

LSP tools ported to Go

Repo reorganization

Conditional variants for network transparency

Principles That Emerged

Phase 13: 9P Streaming & D-Bus Removal (Jul 31)

Phase 14: Multi-Agent Deepening — ID/Name Split & Empty Sessions (Aug 1)

ID/Name Split

Two-Step Session Creation

Multi-Agent Data Structures

Phase 15: 9P Declarative EDSL (Aug 2 — morning)

Before: Imperative

After: Declarative EDSL

Package consolidation

Net result

Phase 16: 9P Server Streamlining (Aug 2 — afternoon)

Fid management

Method extraction

Result

Phase 17: Eventwait Redesign & Chat Log Format (Aug 2)

Global /eventwait

Chat log: [type]/[end] block format

KDE submodule evolution

Updated Filesystem Layout (post-refactor)

Updated Current Topology

Phase 18: Zero Built-in Tools (Aug 2–3)

What changed

Unified tool loading

Agent config: autoLoad

Tool output format classification

Remote execution simplified

SSH bootstrap simplified

9P namespace changes

Files removed

Lines of code

Why it matters

SLOC current (Aug 3)

Phase 19: EDSL Extraction & Structural Cleanup (Aug 3–4)

fsedsl: Standalone Library

fs/ Package Consolidation

Session Package Refactors

Event Bus: Ring Buffer → Pubsub

Dead Code Removal

o CLI Improvements

Documentation

Gantt Update

SLOC current (Aug 4)

"Good Idea Fairy" Cleanup (Aug 5)

Hooks — Deleted Entirely

Other Deletions

Turn State Machine

Philosophy

Aug 9, 2026 — Simplification & Namespace Cleanup

Structural

Namespace

ctl Unification

Command Surface

elevate → bypass

77 KiB

Raw Blame History

Agent config: `autoLoad`

CLI (`o` script)

Framework rename: `Request` → `Rdwr`