17 KiB

Raw Blame History

Architectural Evolution

How the Ollie system-of-systems emerged and evolved. ~1300 commits over ~3.5 months (Apr 11 – Jul 30, 2026).

Timeline

                         Apr          May          Jun          Jul
                         |----|--------|--------|--------|--------|
Foundation
  Monorepo + submodules  ===
  9P filesystem          . ============

Core Primitives
  Session lifecycle (s/)    ==================
  Tool scripts                  ============
  Skills system                   ========

Execution
  execute_code                       ==========
  Batch jobs (b/→s/bfg)               ==================
  Sandbox (landrun)                      ====================================

Multi-Agent
  Subagent spawn                           ==========
  Cascade orchestrator                        ==========
  JIT agent generation                              ====

Frontends
  TUI + Emacs (ellie)    =====================================
  Acme (Plan 9)                               ============================
  Web UI + httpgw                          ====================
  KDE/Plasma                                      ========================

Control Planes
  D-Bus adapter (ollied)                             ===============
  D-Bus embedded in srv                                    =====

Late Evolution
  Remote execution                                      ==========
  Elevation broker                                         ==========
  Tool registry (lazy)                                          ======
  Reactions / feedback                                       =====
  FUSE → 9P client                                          ====

Phase 1: Monorepo Bootstrap (Apr 11)

Started as independent git repos unified under a monorepo with submodules. Initial components:

core — agent loop (Go): backend dispatch, tool execution, session state
9p — 9P file server exposing agent sessions as a synthetic filesystem
tui — terminal UI (later removed)
el — Emacs integration (ellie.el)

Build system was mkfile (Plan 9 make), later Makefile, finally justfile.

Phase 2: The 9P Decision (Apr 14–28)

The defining architectural choice: every agent primitive is a file.

Sessions are directories. Sending a prompt is writing to s/{id}/prompt. Reading state is cat s/{id}/state. This made the control plane protocol-agnostic — any program that can read/write files can be a frontend.

Key files crystallized: prompt, chat, state, ctl, cfg, statewait.

The s/new file (write key=value pairs, read back session ID) became the session factory.

Phase 3: Tool System (Apr 28 – May 10)

Tools evolved through several incarnations:

MCP servers (denote-mcp, 9beads-mcp) — external processes, heavyweight
Shell/Python scripts in ~/.config/ollie/tools/ — lightweight, sandboxed
execute_code as the single built-in tool — all scripts invoked through it
call_tool/pipe — named tool dispatch and cross-tool pipelines
Tool registry (final form) — lazy-loading via tool_list/tool_load/tool_active; execute_code renamed to shell

The file tools (file_read, file_write, file_edit, file_grep, file_glob) were extracted into standalone Python scripts. Memory (memory_remember, memory_recall) and reasoning (reasoning_think) followed the same pattern.

MCP servers were removed early. Simple scripts won over complex daemons.

Phase 4: Planning System Churn (May – Jul)

Planning went through the most iterations of any subsystem:

pl/ directory namespace with flat file-per-task
plan_create / plan_complete tools
Beads integration (external issue tracker) — made optional, then dropped
task_add / task_check tools
Inline markdown plans forbidden
Final form: a single s/{id}/plan file (markdown checklist), written by the agent, persisted across context compaction

The lesson: planning needed to be simple, agent-controlled, and not bureaucratic.

Phase 5: Multi-Agent Coordination (May 25 – Jun 15)

Three patterns emerged:

subagent_spawn — fire-and-forget session creation, parent never blocks
subagent_generate — JIT agent config generation (role, constraints, tools)
cascade — script-driven fan-out with -max-workers, -retries, optional synthesis step

                 ┌──────────────────┐
                 │   User prompt    │
                 └────────┬─────────┘
                          │
                 ┌────────▼─────────┐
                 │  Parent Session  │
                 └──┬──────┬──────┬─┘
        subagent_   │      │      │   subagent_
          spawn     │      │      │     spawn
           ┌────────▼┐  ┌──▼───┐  ┌▼────────┐
           │Worker 1 │  │Wkr 2 │  │Worker 3 │
           └────┬────┘  └──┬───┘  └────┬────┘
                │          │           │
                └──────────┼───────────┘
                   write to parent's prompt

                 ┌────────▼─────────┐
                 │  Parent Session  │──── cascade ───┐
                 └──────────────────┘                │
                                          ┌─────────▼──────────┐
                                          │   u/cascade        │
                                          │ (spawn + throttle) │
                                          └───┬──────────┬─────┘
                                              │          │
                                         ┌────▼───┐ ┌───▼────┐
                                         │Cascade │ │Cascade │
                                         │Worker  │ │Worker  │
                                         └────────┘ └────────┘

Inter-agent communication uses the filesystem: agents write to each other's prompt file or use bidirectional peer/ links.

Phase 6: Frontend Proliferation

                        ┌─────────────────┐
                        │   agent.Core    │
                        └───┬─────────┬───┘
                            │         │
                     ┌──────▼───┐ ┌───▼──────┐
                     │ 9P Server│ │D-Bus Svc │
                     └──┬──┬──┬┘ └──┬─────┬──┘
                        │  │  │     │     │
              ┌─────────┘  │  └──┐  │     └──────────┐
              │            │     │  │                 │
        ┌─────▼───┐  ┌────▼──┐  │  ├────────┐  ┌────▼────┐
        │  s/sh   │  │ acme  │  │  │  KDE   │  │ Web UI  │
        └─────────┘  └───────┘  │  │Plasmoid│  │(httpgw) │
                           ┌────▼──┤KRunner │  └─────────┘
                           │ellie  │Kate    │
                           └───────┤GUI     │
                                   └────────┘

s/sh — shell script frontend (bash, briefly rc, back to bash)
acme — Plan 9 editor integration with Oi (quick query), Kmpl (completion)
ellie.el — Emacs with ghost-text completion
httpgw — HTTP→D-Bus gateway
webui — browser frontend via httpgw
ollie-kde — full Plasma integration: plasmoid, KRunner, Kate plugin, standalone GUI, GUI automation tools
TUI — removed (Jul) in favor of s/sh and richer GUIs

Phase 7: Security Model (May – Jul)

  9P Permissions            Execution Sandbox
  ──────────────            ─────────────────
  Owner  → rw               Agent
  Agent  → restricted         │
  Peers  → read-only          ▼
                            landrun sandbox (restricted fs)
                              │
                              ▼
                            Tool scripts
                              │ needs escape?
                              ▼
                            x/elevate (socket broker)
                              │
                              ▼
                            Integrated Broker
                              │ user confirms
                              ▼
                            Privileged Action

Evolution:

No sandboxing initially
YAML-based sandbox configs (landrun) for execute_code
Per-session 9P identity — agents can't read other sessions' tools
Elevation broker: socket-based, user-confirmed privilege escalation
SSH agent proxy added then removed (too much attack surface)
Final: integrated elevation broker replaces separate superpowerd adapter

Phase 8: D-Bus as Second Control Plane (Jun 25 – Jul)

Originally everything was 9P-only. D-Bus was added for desktop integration:

Separate ollie-dbus daemon (ollied)
Embedded directly into olliesrv (9P server got -nodbus flag)
KDE uses D-Bus exclusively; acme/sh/el use 9P
9P and D-Bus do not share sessions — independent stores, same core

Phase 9: Remote Execution (Jul 5–19)

Split-brain architecture: orchestration stays local, code execution on a remote machine via SSH.

ollie-remote binary auto-deployed via SSH bootstrap
Tools embedded in remote binary
Streaming output notifications back to local session
Session persistence across remote restarts

Phase 10: Tool Registry (Jul 20–29)

The final major architectural shift — from "agent knows all tools upfront" to lazy discovery:

Tools embed their own ollie:prompt metadata blocks
tool_list — discover available tools and descriptions
tool_load — promote a tool to a native callable
tool_active — introspect loaded tools
Skills get the same treatment: skill_list, skill_load, skill_active

This solved system prompt bloat — only load what's needed per task.

Phase 11: The Great Flattening (Jul 29–30)

Largest single-day structural change: −4,573 lines net across the codebase. Eliminated accumulated abstractions that no longer served a purpose.

Core repo (−4,400 lines)

Killed agent.Core interface — concrete *agent.Agent used directly. Nobody else implemented Core; the interface just added indirection.
Killed session.Core interface — same pattern, same outcome.
Killed Dispatcher indirection in toolsrv — tools.Server called directly.
Merged execute/ into tools/ — separate package for 3 functions was pointless overhead.
Renamed packages for clarity: agent.Session → agent.History, session.Core → session.Session, NewAgentCore → New.
Extracted agent/ package from the monolithic session package.
Threaded context.Context properly through daemon → session → agent (replaced ad-hoc interrupt mechanisms with cancellation).
Dead code removal: GlobalToolNames, ExtractReturnSchema.

9P server repo (−214 lines)

Killed mgr/ package entirely — replaced Manager struct with package functions in session/. The fs.Tree IS the session collection; rootState (unexported) lives in tree.Data. CRUD via NewRoot, Lookup, Create, Kill, Rename, Shutdown.
Fixed infinite recursion in session/{sid}/agent/ directory — pathType() didn't recognize agent subdirectories, causing walk to succeed at any depth.
Fixed phantom session descent — walk into non-existent sessions now fails immediately.
Fixed /agents empty read — makeStat wasn't computing Length for root files, FUSE kernel saw 0 bytes and never issued a read.
Fixed session.All() — was looking at root.Children() (empty) instead of rootState.sessions. Broke D-Bus ListSessions and GUI session restore.

Frontends

acme: paths updated s/ → session/, agent files route through session/{id}/agent/{aid}/.
ellie.el: same path migration, added ellie--agent-dir for agent ID discovery.
KDE GUI: no changes needed (D-Bus operates on Go objects, not paths).

New filesystem layout

session/
├── new              write key=value to create session
├── idx              session index (tab-separated)
├── {sid}/
│   ├── plan         session-scoped markdown checklist
│   ├── env          session environment
│   └── agent/
│       └── {aid}/
│           ├── chat        conversation log
│           ├── prompt      submit prompts
│           ├── cfg         agent configuration
│           ├── ctl         control commands
│           ├── state       current state
│           ├── statewait   blocks until state changes
│           ├── usage       token stats
│           ├── cost        cost estimate
│           ├── ctxsz       context size
│           ├── systemprompt rendered system prompt
│           ├── tools       tool management
│           ├── tools.loaded currently loaded tools
│           └── proc/
│               └── {pid}   detached process output

Architecture after

*fs.Tree (root)                     ← IS the session collection
  └── .Data = *rootState            ← sessions map, config, nextUID
       └── sessions[id] = *Session  ← owns Agent, context, cancel

Package functions (no Manager):
  session.NewRoot(cfg) → *fs.Tree
  session.Lookup(tree, id) → *Session
  session.All(tree) → []*Session
  session.CreateFromRoot(tree, args) → (id, error)
  session.KillFromRoot(tree, id)
  session.RenameFromRoot(tree, old, new)
  session.Shutdown(tree)

SLOC after flattening

Component	Lines
core (minus backend)	10,437
9p server	7,925
kde gui	12,331
el + acme + httpgw	1,845
Total	32,538

Zero dead exported functions remain (verified via LSP + grep across all repos).

Current Topology

ollie/                      ← monorepo root
├── core/                   ← agent loop, backends, tool dispatch (Go)
├── 9p/                     ← 9P server + embedded D-Bus adapter (Go)
├── acme/                   ← Plan 9 acme frontend (Go)
├── el/                     ← Emacs integration (elisp)
├── kde/                    ← KDE Plasma: plasmoid, GUI, Kate, KRunner (C++/QML)
├── httpgw/                 ← HTTP→D-Bus gateway (Go)
├── webui/                  ← browser frontend (JS)
├── remote/                 ← split-brain execution server (Go)
├── tools/                  ← 37 tool scripts (Python)
├── scripts/{s,u,x}/       ← filesystem-hosted scripts
├── agents/                 ← agent configs (JSON)
├── prompts/                ← system prompt fragments (markdown)
├── skills/                 ← domain knowledge modules (markdown)
├── sandbox/                ← landrun sandbox profiles (YAML)
├── services/               ← init/service files
└── experiments/            ← multi-agent trial docs

Principles That Emerged

Filesystem-as-API — everything is read/write on synthetic files. No custom protocols needed.
Submodule independence — each component has its own repo, tests, build. Monorepo coordinates versions.
Scripts over servers — MCP servers removed early. Simple scripts won.
Progressive disclosure — lazy tool/skill loading keeps system prompts small until complexity is needed.
Plan simplicity — after 5 iterations, planning settled on one markdown checklist file per session.
Dual control plane — 9P for power users/editors, D-Bus for desktop. Same core, different surfaces.
Security by default — sandboxed execution with explicit elevation, per-session identity.

Dead Ends and Reversals

What	Why it was removed
MCP servers	Too heavyweight; scripts are simpler and faster
TUI frontend	Replaced by s/sh (lighter) and KDE GUI (richer)
`pl/` planning namespace	Over-engineered; plan file is sufficient
Beads integration	External dependency for something a checklist file handles
SSH agent proxy	Attack surface not worth the convenience
Separate ollie-dbus daemon	Consolidated into olliesrv
FUSE mounts	Replaced by standalone 9P client binary
Tier routing (FAST/POWER)	Replaced by /route endpoint with real model discovery
call_tool / pipe	Replaced by native tool registry with tool_load
execute_code (multi-lang)	Simplified to shell-only; tool scripts handle language choice
chatwait	Reverted; acme tails chat directly

17 KiB Raw Blame History Unescape Escape