2383 lines
112 KiB
Markdown
2383 lines
112 KiB
Markdown
# Architectural Evolution
|
||
|
||
## AnviLLM: Research That Led to Ollie
|
||
|
||
Before Ollie, there was [`anvillm`](https://git.lneely.de/lkn/anvillm): the working laboratory in which the
|
||
filesystem-as-agent idea first became concrete. Beginning on February 9, 2026,
|
||
AnviLLM orchestrated Claude Code, Kiro, and Ollama sessions in `tmux`, exposing
|
||
session state, control, output, and inter-agent communication through a 9P
|
||
namespace. It answered the first question: **can independent agent CLIs be
|
||
made to cooperate through ordinary file operations?** Yes—with caveats.
|
||
|
||
AnviLLM was deliberately exploratory. Its history records the path from a
|
||
monolithic client to the `anvilsrv` daemon and Assist client; from polling and
|
||
process inspection to hooks, events, and streaming reads; from ad-hoc prompts
|
||
to fire-and-forget workflows, roles, a supervisor, and a conductor; and from
|
||
simple messages to mailboxes, beads, and cross-backend coordination. It also
|
||
tried several frontends—Acme, Emacs, a web interface, and a curses TUI—and
|
||
proved that a 9P surface lets each of them remain replaceable.
|
||
|
||
That experiment left durable lessons. A filesystem is a powerful integration
|
||
surface: `read`, `write`, pipes, and blocking streams compose without a
|
||
special-purpose orchestration protocol. But wrapping external CLIs makes the
|
||
runtime depend on backend-specific hooks and fragile process heuristics; even
|
||
“running” versus “idle” cannot be inferred uniformly. A 9P socket is not an
|
||
authentication system: Unix ownership and permissions must carry the boundary.
|
||
State, recovery, cancellation, and message delivery must be explicit rather
|
||
than inferred from terminal behavior. And orchestration belongs outside the
|
||
agent core, where scripts and clients can evolve independently.
|
||
|
||
Ollie is the continuation of that research. It keeps AnviLLM's strongest
|
||
result—the 9P namespace as the API—while moving the agent loop, backends,
|
||
tools, sandbox, persistence, and context management into a runtime designed
|
||
for direct control. AnviLLM established what the filesystem surface could
|
||
unlock; its limitations clarified what the runtime must own. This log begins
|
||
with that lineage.
|
||
|
||
## How the Ollie system-of-systems emerged and evolved.
|
||
|
||
This is a study log. It records what was built, what was killed, and why —
|
||
in the order it happened. The dead ends are as instructive as the survivors:
|
||
the rejected ideas at the end teach more about building agent systems than
|
||
the architecture that remains. The document evolves with the author's
|
||
understanding; when past decisions turn out to be wrong, they get recorded
|
||
here, not hidden.
|
||
|
||
## Timeline
|
||
```mermaid
|
||
gantt
|
||
title Ollie Evolution
|
||
dateFormat YYYY-MM-DD
|
||
axisFormat %b %d
|
||
section Foundation
|
||
Monorepo + submodules :done, 2026-04-11, 14d
|
||
9P filesystem :done, 2026-04-14, 28d
|
||
section Core Primitives
|
||
Session lifecycle (session/) :done, 2026-04-20, 30d
|
||
Tool definitions :done, 2026-05-01, 20d
|
||
Skills system :done, 2026-05-10, 14d
|
||
section Execution
|
||
execute_code / shell :done, 2026-05-08, 20d
|
||
Batch jobs (b/ → session/bfg) :done, 2026-05-15, 25d
|
||
Sandbox (native Landlock) :done, 2026-05-20, 40d
|
||
section Multi-Agent
|
||
Subagent spawn :done, 2026-05-25, 15d
|
||
Cascade orchestrator :done, 2026-06-05, 20d
|
||
JIT agent generation :done, 2026-07-10, 10d
|
||
section Frontends
|
||
TUI + Emacs (ellie) :done, 2026-04-15, 45d
|
||
Acme (Plan 9) :done, 2026-05-20, 30d
|
||
KDE/Plasma :done, 2026-06-10, 40d
|
||
section Control Planes
|
||
D-Bus adapter (ollied) :done, 2026-06-25, 20d
|
||
D-Bus embedded in srv :done, 2026-07-15, 10d
|
||
D-Bus removed :done, 2026-07-31, 1d
|
||
section Late Evolution
|
||
Remote execution :done, 2026-07-05, 15d
|
||
Bypass broker :done, 2026-07-10, 15d
|
||
Tool registry (lazy) :done, 2026-07-20, 10d
|
||
FUSE → 9P client :done, 2026-07-25, 5d
|
||
The Great Flattening :done, 2026-07-29, 2d
|
||
9P Declarative EDSL :done, 2026-08-02, 1d
|
||
Zero built-in tools :done, 2026-08-02, 2d
|
||
EDSL extraction + cleanup :done, 2026-08-03, 2d
|
||
Good idea fairy cleanup :done, 2026-08-05, 1d
|
||
toolsrv 9P + process isolation :done, 2026-08-10, 2d
|
||
Meta-only tool definitions :done, 2026-08-11, 2d
|
||
Bypass via 9P :done, 2026-08-11, 1d
|
||
fs/ flattening :done, 2026-08-11, 1d
|
||
Feed + observer agents :done, 2026-08-13, 1d
|
||
BlockOnce/Stream refactor :done, 2026-08-13, 1d
|
||
Sub-agents via 9P rdwr :done, 2026-08-14, 1d
|
||
Code intelligence tools :done, 2026-08-15, 1d
|
||
Go file tools + cleanup :done, 2026-08-16, 1d
|
||
OptMem persistent memory :done, 2026-08-16, 1d
|
||
Markdown parsing + output cap :done, 2026-08-16, 1d
|
||
Goals + conductor workflow :done, 2026-08-17, 1d
|
||
Split session/agent indexes :done, 2026-08-17, 1d
|
||
Agent peers + consensus :done, 2026-08-19, 1d
|
||
Embedding-guided discovery :done, 2026-08-20, 1d
|
||
One-tool bootstrap discovery :done, 2026-08-21, 1d
|
||
Explicit tool loading :done, 2026-08-21, 1d
|
||
Streaming rdwr + event filters :done, 2026-08-22, 1d
|
||
Bypass coordination + state UI :done, 2026-10-06, 1d
|
||
JSONL chat log format :done, 2026-10-07, 1d
|
||
Live chat stream (chat.raw) :done, 2026-10-10, 1d
|
||
Tool/bypass text visibility :done, 2026-10-10, 1d
|
||
```
|
||
|
||
## Current Size (Oct 6)
|
||
|
||
| Component | Lines | Notes |
|
||
|-----------|------:|-------|
|
||
| Go core (excluding backends) | ~22,000 | Agent, session, tools, 9P |
|
||
| Compiled tools (tools/) | ~3,000 | Code intel, file, LSP, web |
|
||
| KDE integration (kde/) | ~15,700 | GUI, Kate, KRunner |
|
||
| Script tools (data/tools/) | ~2,600 | Python/Bash tools |
|
||
| CLI scripts (data/scripts/) | ~960 | `o` wrapper and helpers |
|
||
| **Core runtime** | **~25,000** | Go core + compiled tools |
|
||
| **With KDE** | **~40,700** | Core + KDE integration |
|
||
|
||
Agent profiles: 14. The interesting logic is ~25K of core runtime plus ~15.7K of KDE integration. Backend adapters (~8.3K), prompts, skills, and docs are excluded from this count.
|
||
|
||
## Phase 1: Monorepo Bootstrap (Apr 11)
|
||
Started as independent git repos unified under a monorepo with submodules.
|
||
Initial components:
|
||
- **agent** — agent loop (Go): backend dispatch, tool execution, session state
|
||
- **fs**** — 9P file server exposing agent sessions as a synthetic filesystem
|
||
- **tui** — terminal UI (later removed)
|
||
- **el** — Emacs integration (ellie.el)
|
||
Build system was `mkfile` (Plan 9 make), later `Makefile`, finally `justfile`.
|
||
## Phase 2: The 9P Decision (Apr 14–28)
|
||
The defining architectural choice: **every agent primitive is a file**.
|
||
Sessions are directories. Sending a prompt is writing to `session/{sname}/prompt`.
|
||
Reading state is `cat session/{sname}/state`. This made the control plane
|
||
protocol-agnostic — any program that can read/write files can be a frontend.
|
||
Key files crystallized: `prompt`, `chat`, `state`, `ctl`, `cfg`, `statewait`.
|
||
The `session/new` file (write key=value pairs, read back session ID) became the
|
||
session factory.
|
||
## Phase 3: Tool System (Apr 28 – May 10)
|
||
Tools evolved through several incarnations:
|
||
1. **MCP servers** (denote-mcp, 9beads-mcp) — external processes, heavyweight
|
||
2. **Shell/Python scripts** in `~/.config/ollie/tools/` — lightweight, sandboxed
|
||
3. **execute_code** as the single built-in tool — all scripts invoked through it
|
||
4. **call_tool/pipe** — named tool dispatch and cross-tool pipelines
|
||
5. **Tool registry** (final form) — dynamic lazy-loading via `tool_list`/`tool_load`/`tool_active`; `execute_code` renamed to `shell`; tools hot-reloadable without server restart
|
||
The file tools (`file_read`, `file_write`, `file_edit`, `file_grep`, `file_glob`)
|
||
were extracted into standalone Python scripts. Memory (`memory_remember`,
|
||
`memory_recall`) and reasoning (`reasoning_think`) followed the same pattern.
|
||
The memory tools (`memory_remember`, `memory_recall`) now call OptMem directly. OptMem owns the persistent B-tree at `$XDG_DATA_HOME/ollie/optmem` (default: `~/.local/share/ollie/optmem`); Ollie does not maintain a parallel memory directory or memory-file format. This keeps reads and writes on the same store and avoids competing writers. MCP servers were removed early. Simple scripts won over complex daemons; the OptMem-backed tools retain the lightweight tool interface without duplicating storage.
|
||
## Phase 4: Planning System Churn (May – Jul)
|
||
Planning went through the most iterations of any subsystem:
|
||
1. `pl/` directory namespace with flat file-per-task
|
||
2. `plan_create` / `plan_complete` tools
|
||
3. **Beads** integration (external issue tracker) — made optional, then dropped
|
||
4. `task_add` / `task_check` tools
|
||
5. Inline markdown plans forbidden
|
||
6. **Final form**: a single `session/{sname}/plan` file (markdown checklist), written by
|
||
the agent, persisted across context compaction
|
||
The lesson: planning needed to be simple, agent-controlled, and not bureaucratic.
|
||
## Phase 5: Multi-Agent Coordination (May 25 – Jun 15)
|
||
Three patterns emerged:
|
||
- **subagent_spawn** — fire-and-forget session creation, parent never blocks
|
||
- **subagent_generate** — JIT agent config generation (role, constraints, tools)
|
||
- **cascade** — script-driven fan-out with `-max-workers`, `-retries`, optional
|
||
synthesis step
|
||
```mermaid
|
||
flowchart TB
|
||
UP["User prompt"]
|
||
PARENT["Parent Session"]
|
||
W1["Worker 1"]
|
||
W2["Worker 2"]
|
||
W3["Worker 3"]
|
||
CASCADE["u/cascade\n(spawn + throttle)"]
|
||
CW1["Cascade Worker"]
|
||
CW2["Cascade Worker"]
|
||
UP --> PARENT
|
||
PARENT -- subagent_spawn --> W1
|
||
PARENT -- subagent_spawn --> W2
|
||
PARENT -- subagent_spawn --> W3
|
||
W1 --> PARENT
|
||
W2 --> PARENT
|
||
W3 --> PARENT
|
||
PARENT -- cascade --> CASCADE
|
||
CASCADE --> CW1
|
||
CASCADE --> CW2
|
||
```
|
||
Inter-agent communication uses the filesystem: agents write to each other's
|
||
`prompt` file.
|
||
## Phase 6: Frontend Proliferation
|
||
```mermaid
|
||
flowchart TB
|
||
subgraph Core["agent.Agent"]
|
||
AG["Agent Engine"]
|
||
end
|
||
subgraph Surfaces["Integration Surfaces"]
|
||
P9["9P Filesystem\n(session/ namespace)"]
|
||
DB["D-Bus\n(org.ollie.SessionManager)"]
|
||
end
|
||
subgraph Frontends
|
||
SH["s/sh (terminal)"]
|
||
ACME["acme (Plan 9)"]
|
||
EL["ellie (Emacs)"]
|
||
KG["KDE GUI"]
|
||
KP["KDE Plasmoid"]
|
||
KK["Kate Plugin"]
|
||
KR["KRunner"]
|
||
HTTP["curl / scripts"]
|
||
end
|
||
AG --> P9
|
||
AG --> DB
|
||
SH --> P9
|
||
ACME --> P9
|
||
EL --> P9
|
||
HTTP --> P9
|
||
KG --> DB
|
||
KP --> DB
|
||
KK --> DB
|
||
KR --> DB
|
||
WEB --> DB
|
||
```
|
||
- **s/sh** — shell script frontend (bash, briefly rc, back to bash)
|
||
- **acme** — Plan 9 editor integration with workspace-scoped navigator sessions
|
||
- **ellie.el** — Emacs with ghost-text completion
|
||
- **ollie-kde** — full Plasma integration: plasmoid, KRunner, Kate plugin, standalone GUI, GUI automation tools
|
||
- **TUI** — removed (Jul) in favor of s/sh and richer GUIs
|
||
## Phase 7: Security Model (May – Jul)
|
||
```mermaid
|
||
flowchart LR
|
||
subgraph Permissions["9P Permissions"]
|
||
OWNER["Owner → rw"]
|
||
AGENT["Agent → restricted"]
|
||
PEERS["Peers → read-only"]
|
||
end
|
||
subgraph Execution["Execution Sandbox"]
|
||
AGT["Agent"]
|
||
SANDBOX["native Landlock sandbox\n(restricted fs)"]
|
||
TOOLS["Tool definitions"]
|
||
BYPASS["x/bypass\n(socket broker)"]
|
||
PRIV["Privileged Action"]
|
||
end
|
||
AGT --> SANDBOX --> TOOLS
|
||
TOOLS -- needs escape --> ELEV --> PRIV
|
||
```
|
||
Evolution:
|
||
1. No sandboxing initially
|
||
2. YAML-based native Landlock sandbox configs for execute_code
|
||
3. Per-session 9P identity — agents can't read other sessions' tools
|
||
4. Bypass broker: socket-based, user-confirmed privilege escalation
|
||
5. SSH agent proxy added then removed (too much attack surface)
|
||
6. Final: integrated bypass broker replaces separate superpowerd adapter
|
||
## Phase 8: D-Bus as Second Control Plane (Jun 25 – Jul)
|
||
Originally everything was 9P-only. D-Bus was added for desktop integration:
|
||
1. Separate `ollie-dbus` daemon (ollied)
|
||
2. Embedded directly into `olliesrv` (9P server got `-no9p` flag)
|
||
3. KDE uses D-Bus exclusively; acme/sh/el use 9P
|
||
4. **9P and D-Bus do not share sessions** — independent stores, same core
|
||
## Phase 9: Remote Execution (Jul 5–19)
|
||
Split-brain architecture: orchestration stays local, code execution on a remote
|
||
machine via SSH.
|
||
- `ollie-remote` binary auto-deployed via SSH bootstrap
|
||
- Tools embedded in remote binary
|
||
- Streaming output notifications back to local session
|
||
- Session persistence across remote restarts
|
||
## Phase 10: Tool Registry (Jul 20–29)
|
||
The final major architectural shift — from "agent knows all tools upfront" to
|
||
lazy discovery:
|
||
1. Tools embed their own `ollie:prompt` metadata blocks
|
||
2. `tool_list` — discover available tools and descriptions
|
||
3. `tool_load` — promote a tool to a native callable
|
||
4. `tool_active` — introspect loaded tools
|
||
5. Skills get the same treatment: `skill_list`, `skill_load`, `skill_active`
|
||
This solved system prompt bloat — only load what's needed per task.
|
||
## Phase 11: The Great Flattening (Jul 29–30)
|
||
Largest single-day structural change: **−4,573 lines net** across the codebase.
|
||
Eliminated accumulated abstractions that no longer served a purpose.
|
||
### Core (−4,400 lines)
|
||
- **Killed `agent.Core` interface** — concrete `*agent.Agent` used directly.
|
||
Nobody else implemented Core; the interface just added indirection.
|
||
- **Killed `Dispatcher` indirection** in toolsrv — `toolsrv.Server` called directly.
|
||
- **Merged `execute/` into `tools/`** — separate package for 3 functions was
|
||
pointless overhead.
|
||
- **Renamed for clarity**: `agent.Session` → `agent.History`,
|
||
`session.Core` → `session.Session`, `NewAgentCore` → `New`.
|
||
- **Extracted `agent/` package** from the monolithic session package.
|
||
- **Threaded context.Context** properly through daemon → session → agent
|
||
(replaced ad-hoc interrupt mechanisms with cancellation).
|
||
- Dead code removal: `GlobalToolNames`, `ExtractReturnSchema`.
|
||
### 9P server (−214 lines)
|
||
- **Killed `mgr/` package entirely** — replaced `Manager` struct with package
|
||
functions in `**fs**/`. The `*fs.Tree` IS the session collection; `rootState`
|
||
(unexported) lives in `tree.Data`. CRUD via `NewRoot`, `Lookup`, `Create`,
|
||
`Kill`, `Rename`, `Shutdown`.
|
||
- **Fixed infinite recursion** in `session/{sname}/agent/` directory — `pathType()`
|
||
didn't recognize agent subdirectories, causing walk to succeed at any depth.
|
||
- **Fixed phantom session descent** — walk into non-existent sessions now fails
|
||
immediately.
|
||
- **Fixed `/agents` empty read** — `makeStat` wasn't computing Length for root
|
||
files, FUSE kernel saw 0 bytes and never issued a read.
|
||
- **Fixed `session.All()`** — was looking at `root.Children()` (empty) instead
|
||
of `rootState.sessions`. Broke D-Bus ListSessions and GUI session restore.
|
||
### Frontends
|
||
- **acme**: paths updated `s/` → `session/`, agent files route through
|
||
`session/{sname}/agent/{aname}/`.
|
||
- **ellie.el**: same path migration, added `ellie--agent-dir` for agent ID
|
||
discovery.
|
||
- **KDE GUI**: no changes needed (D-Bus operates on Go objects, not paths).
|
||
### New filesystem layout
|
||
```
|
||
session/
|
||
├── new write key=value to create session
|
||
├── idx session index (tab-separated)
|
||
├── {sname}/
|
||
│ ├── plan session-scoped markdown checklist
|
||
│ ├── env session environment
|
||
│ └── agent/
|
||
│ └── {aname}/
|
||
│ ├── cfg session configuration (key=value)
|
||
│ ├── ctl control commands
|
||
│ ├── state current state
|
||
│ ├── statewait blocks until state changes
|
||
│ ├── chat conversation log
|
||
│ ├── prompt submit prompts
|
||
│ ├── prompt.prev last submitted prompt
|
||
│ ├── offset byte offset after last user prompt
|
||
│ ├── fifo.in queue a prompt (FIFO input)
|
||
│ ├── fifo.out pop queued prompt (FIFO output)
|
||
│ ├── context rendered context window
|
||
│ ├── systemprompt rendered system prompt
|
||
│ ├── usage token stats
|
||
│ ├── cost cost estimate
|
||
│ ├── ctxsz context size
|
||
│ ├── models available models
|
||
│ ├── tools tool management
|
||
│ ├── toolsrv.loaded currently loaded tools
|
||
│ ├── toolsrv.rev revision counter
|
||
│ ├── tail exec helper
|
||
│ └── proc/ detached process output
|
||
```
|
||
### Architecture after (single Go module, no submodules for core/9p)
|
||
```
|
||
*fs.Tree (root) ← IS the session collection
|
||
└── .Data = *rootState ← sessions map, config, nextUID
|
||
└── sessions[id] = *Session ← owns Agent, context, cancel
|
||
Package functions (no Manager):
|
||
session.NewRoot(cfg) → *fs.Tree
|
||
session.Lookup(tree, id) → *Session
|
||
session.All(tree) → []*Session
|
||
session.CreateFromRoot(tree, args) → (id, error)
|
||
session.KillFromRoot(tree, id)
|
||
session.RenameFromRoot(tree, old, new)
|
||
session.Shutdown(tree)
|
||
```
|
||
### SLOC after flattening
|
||
| Component | Lines |
|
||
|-----------|-------|
|
||
| agent + backend + toolsrv + session | 10,437 |
|
||
| **fs** + cmd/olliesrv | 7,925 |
|
||
| kde gui | 12,331 |
|
||
| **Total** | **32,538** |
|
||
Zero dead exported functions remain (verified via LSP + grep across all repos).
|
||
## Phase 12: Second Flattening & .meta Sidecar (Jul 30)
|
||
|
||
Another aggressive structural pass. Eliminated the entire script namespace
|
||
layer (s/, u/, x/), flattened `contrib/` into `data/`, and replaced
|
||
header-comment-based tool metadata with JSON sidecar files.
|
||
|
||
### Scripts removed (−1,100 lines)
|
||
- **s/{sh,b,bfg,bbg,ls,kill,cleanup}** — shell-based session frontends replaced
|
||
by `ollie-9p rdwr generate/complete/route` and GUI frontends
|
||
- **u/{optimize,complete,cascade,escalate}** — utility scripts replaced by
|
||
`generate`, `complete`, `route` files in the 9P namespace
|
||
- **x/{bd,prime,freeloader,task}** — internal plumbing scripts; PATH prepend
|
||
(`prependOlliePath`) removed from toolsrv
|
||
- 9P session root trimmed to just `new` and `idx`; script-serving code removed
|
||
- `install-scripts` target now empty
|
||
|
||
### Tool metadata: .meta sidecar files
|
||
Header comment parsing (`ollie:prompt`, `ollie:tier`, `ollie:parallel read`,
|
||
`args_json:`) **eliminated entirely**. All tool metadata now lives in a JSON
|
||
sidecar file alongside the executable:
|
||
|
||
```
|
||
tools/
|
||
file_edit ← executable (any language)
|
||
file_edit.meta ← JSON metadata
|
||
```
|
||
|
||
```json
|
||
{
|
||
"description": "Replace text in a file.",
|
||
"prompt": "## file_edit\n\n...",
|
||
"args": {"type":"object", ...},
|
||
"tier": "cold",
|
||
"readOnly": true
|
||
}
|
||
```
|
||
|
||
This decouples metadata from implementation language — compiled Go binaries,
|
||
Python scripts, and bash tools all use the same discovery mechanism.
|
||
|
||
### LSP tools ported to Go
|
||
The Python LSP bridge (`_lib/lsp/`) replaced with a pure Go implementation:
|
||
|
||
```
|
||
tools/
|
||
├── builtin/ ← in-process handlers (shell, reasoning, tool/skill registry)
|
||
└── lsp/ ← shared LSP client library
|
||
└── cmd/ ← individual binaries (lsp_definition, lsp_hover, ...)
|
||
```
|
||
|
||
Architecture: each LSP tool binary embeds a bridge daemon (started on first
|
||
invocation via `--bridge` flag, persists via Unix socket, auto-exits after 5min
|
||
idle). No Python. No `_lib`. No external dependencies beyond the LSP servers
|
||
themselves (gopls, clangd, intelephense).
|
||
|
||
### Repo reorganization
|
||
- `contrib/{prompts,scripts,services,tools,skills,agents}` → `data/`
|
||
- `contrib/elisp` stays (Emacs frontend contribution)
|
||
- `tools/builtin/` — built-in tool handlers (moved from `tools/`, removed in Phase 18)
|
||
- `tools/lsp/` — Go LSP implementation (replaced `data/tools/_lib/lsp/`)
|
||
|
||
### Conditional variants for network transparency
|
||
Tools can declare multiple variants gated by host conditions (`binary`, `file`,
|
||
`os`, `arch`, `env`, `nenv`). First match determines the tool's schema and
|
||
executable. Enables one `.meta` to work across heterogeneous hosts — the
|
||
contract adapts to capabilities. Critical for `ollie-remote` deployments where
|
||
the remote host may have different tools, package managers, or init systems.
|
||
|
||
## Principles That Emerged
|
||
1. **Filesystem-as-API** — everything is read/write on synthetic files. No
|
||
custom protocols needed.
|
||
Monorepo coordinates versions.
|
||
3. **Scripts over servers** — MCP servers removed early. Simple scripts won.
|
||
4. **Progressive disclosure** — lazy tool/skill loading keeps system prompts
|
||
small until complexity is needed.
|
||
5. **Plan simplicity** — after 5 iterations, planning settled on one markdown
|
||
checklist file per session.
|
||
6. **Single control plane — 9P for everything.
|
||
Streaming via blocking reads. No polling.
|
||
7. **Security by default** — sandboxed execution with explicit bypass,
|
||
per-session identity.
|
||
## Phase 13: 9P Streaming & D-Bus Removal (Jul 31)
|
||
|
||
The D-Bus adapter (`org.ollie.SessionManager`) is deleted. All clients now
|
||
use 9P exclusively via blocking reads for natural streaming.
|
||
|
||
Key changes:
|
||
- **`chat` file**: blocking read that delivers tokens as the agent produces
|
||
them. Per-fid offset. Never EOF (blocks between turns). EOF only on kill.
|
||
- **`log` file**: replaces old `chat`. Returns last 64KB (sliding window).
|
||
Non-blocking, tail-able via Qid.Vers.
|
||
- **`kill`/`.` ctl command**: kills session → EOF on chat readers.
|
||
- **KDE GUI**: rewritten from scratch. Uses `9p` (plan9port) via QProcess.
|
||
Streaming via `readyReadStandardOutput`. Plain text tail (8KB).
|
||
No ChatBlockModel, no D-Bus, no ThemeManager. ~250 lines total.
|
||
- **Kate plugin**: all D-Bus calls replaced with `9p` subprocess calls.
|
||
Streaming chat + statewait via persistent QProcess.
|
||
- **KRunner**: uses `9p` for session listing and one-shot generation.
|
||
- **Acme frontend**: uses 9fans.net/go plan9/client directly. Blocking
|
||
read loop on `chat`. No FUSE mount.
|
||
- **Plasmoid, tray**: deleted (not useful enough to maintain).
|
||
- **dbus/ package**: deleted (-771 lines from server).
|
||
|
||
The only remaining godbus usage: `org.freedesktop.Notifications` for
|
||
bypass prompts (desktop integration, not ollie protocol).
|
||
|
||
Lessons:
|
||
- 9P's request-response model gives natural streaming for free.
|
||
Server delays Rread until data arrives. No polling needed.
|
||
- FUSE mounts do NOT support blocking reads (return EOF immediately).
|
||
Clients must use the 9P protocol directly.
|
||
- Never bind a growing string to a QTextArea with word-wrap.
|
||
QTextDocument relayout is O(n) on the full content.
|
||
- The `loadEarlier` scroll-triggered cascade was creating 500 delegates
|
||
on startup. Bounded initial window + explicit scroll-back is correct.
|
||
|
||
## Phase 14: Multi-Agent Deepening — ID/Name Split & Empty Sessions (Aug 1)
|
||
|
||
The multi-agent data structures introduced during the Great Flattening
|
||
required further refinement to support true multi-agent sessions.
|
||
|
||
### ID/Name Split
|
||
Sessions and agents now have **two identities**:
|
||
- **`id`** (immutable UUIDv4, set at creation, never changes)
|
||
- **`name`** (mutable human-readable string, defaults to first UUID segment)
|
||
|
||
The sessions map is keyed by Name, not ID. Renaming a session or agent moves
|
||
the directory in the 9P namespace — `mv` on the session directory calls
|
||
`RenameFromRoot`, and `write` to the agent's `name` file calls `SetName`.
|
||
|
||
Both entities expose `id` and `name` files in their 9P directories.
|
||
|
||
### Two-Step Session Creation
|
||
Creation split into two phases:
|
||
1. `session/new` — write `name=<name>` to create an empty session (no agent yet)
|
||
2. `session/{name}/agent/new` — write `cwd=<dir> backend=<backend> model=<model> agent=<agent>` to create the agent
|
||
|
||
This enables creating sessions in advance and attaching agents later. Empty
|
||
sessions (`Core=nil`) are valid — they appear in `session/idx` with empty
|
||
fields and are fully killable/renameable.
|
||
|
||
### Multi-Agent Data Structures
|
||
- `session.Session` supports multiple agents (`Agents()` returns `[]*agent.Agent`)
|
||
- `Session.AgentLogs` maps agent IDs to `*AgentLog` instances
|
||
- Active agent is selectable, each agent has its own prompt/chat/state/log
|
||
- `session/idx` emits one line per agent, not one line per session
|
||
|
||
Key commits: 92 commits across Aug 1 touching session/, agent/, fs/, and kde/.
|
||
|
||
## Phase 15: 9P Declarative EDSL (Aug 2 — morning)
|
||
|
||
The most significant structural change to the 9P server since its inception:
|
||
**replaced the imperative filesystem with a declarative EDSL**.
|
||
|
||
### Before: Imperative
|
||
Every 9P operation (stat, walk, read, write, open, create, remove) was
|
||
hand-coded per node. The `fs/session/` sub-package contained ~3,100 lines
|
||
of manual tree-building code split across 6 files (`files.go`, `root.go`,
|
||
`perm.go`, `synth.go`, `create.go`, `persist.go`). Adding a new file meant
|
||
updating stat, readdir, open, and permission code paths.
|
||
|
||
### After: Declarative EDSL
|
||
The entire namespace is declared in a single `FsNodeDecl` tree in `fs/spec.go`:
|
||
```go
|
||
var treeSpec = Dir("/",
|
||
Leaf("backends", 0444, Read(readBackends), GID("agent")),
|
||
Leaf("event", 0444, Read(readEvent), Stream(blockEvent), GID("agent")),
|
||
Leaf("complete", 0666, Request(requestComplete), GID("agent")),
|
||
// ...
|
||
Dir("session", GID("agent"),
|
||
Leaf("new", 0666, Request(requestSessionNew)),
|
||
Leaf("idx", 0444, Read(readSessionIdx)),
|
||
TemplateDir("{id}", listSessions),
|
||
),
|
||
)
|
||
```
|
||
|
||
`BuildTree()` in `fs/builder.go` walks the spec, validates invariants
|
||
(directories can't have leaf handlers, at most one blocking variant,
|
||
template nodes must have `List`, ownership inheritance), and produces
|
||
a fully-wired `*fs.Tree`.
|
||
|
||
### Package consolidation
|
||
- Entire `fs/session/` sub-package flattened into `fs/` — 6 files deleted
|
||
- `cmd/olliesrv/bypass_tree.go` — bypass tree moved to `fs/bypassfiles.go`
|
||
- `cmd/olliesrv/server.go` simplified from ~820+ lines to ~200 lines of thin
|
||
9P protocol handling; all filesystem logic is now in `fs/`
|
||
- Handler files organized by scope: `rootfiles.go`, `sessionfiles.go`,
|
||
`agentfiles.go`, `bypassfiles.go`, `procfiles.go`
|
||
|
||
### Net result
|
||
- **−3,152 lines** deleted, **+2,086 lines** added across 24 files
|
||
- Adding a file = adding one `Leaf()` call. No manual stat/readdir/write.
|
||
- Validation at `BuildTree` time catches structural errors before serving.
|
||
- Permission model inlined in the spec via `mode` and `GID()` — no separate `perm.go`.
|
||
|
||
## Phase 16: 9P Server Streamlining (Aug 2 — afternoon)
|
||
|
||
Following the EDSL conversion, the 9P protocol server itself
|
||
(`cmd/olliesrv/server.go`) received a focused cleanup over 13 commits:
|
||
|
||
### Fid management
|
||
- **Server owns the fid map**: validates newfids (not already in use, not stale),
|
||
garbage-collects orphaned fids. No more leaking fids on client disconnect.
|
||
- **Store opened file on fid**: read/write use `f.entry` directly instead of
|
||
re-resolving the path on every operation. Eliminates a class of TOCTOU bugs.
|
||
|
||
### Method extraction
|
||
- `attach` and `open` extracted from `*Server` — pass groups and log explicitly
|
||
- `GroupTable` extracted from `*Server` — permission checking uses bitmask OR
|
||
instead of a boolean dance (`checkPermBits` → `hasPermBits`)
|
||
- `checkPerm` extracted to package-level, permissions resolved in `open`
|
||
- 9 `rootTree`-only methods extracted to package-level functions; switch
|
||
statement replaced with handler map
|
||
- Three thin wrapper methods inlined (`readFile`, `writeFile`, `FileTree` alias)
|
||
- `handle` inlined into `Start`; renamed `Serve` → `Start`, `Shutdown` → `Kill`
|
||
|
||
### Result
|
||
- `server.go` reduced from ~820 to ~200 lines
|
||
- `main.go` simplified as bypass tree wiring moved to `fs/`
|
||
- Server is now a thin 9P2000 protocol translator, not a filesystem
|
||
|
||
## Phase 17: Event Redesign & Chat Log Format (Aug 2)
|
||
|
||
### Global /event
|
||
Replaced per-session event polling with a global event ring buffer and a
|
||
single `/event` file. Events carry structured prefixes and delta descriptions:
|
||
- `S new session/<name>` — session created
|
||
- `S kill session/<name>` — session destroyed
|
||
- `S rename session/<old> session/<new>` — session renamed
|
||
- `A kill session/<name>/agent/<aid>` — agent killed
|
||
|
||
The event ring is a fixed-size circular buffer (100 slots) with a `sync.Cond`
|
||
for blocking reads. Frontends block on `/event` and receive deltas since
|
||
their last known offset. No polling, no timer, no D-Bus.
|
||
|
||
Fixes along the way:
|
||
- Dangling pointer in `eventRing` cond initialization (caused panics under load)
|
||
- Double-unlock panic in `WaitEvent`
|
||
- Spurious `[[[end]]]` markers on reasoning blocks (reasoning events were
|
||
silently suppressed from chat but still advanced the role-transition state)
|
||
|
||
### Chat log: [[[type]]]/[[[end]]] block format
|
||
The chat log switched from plain text to a structured block format:
|
||
```
|
||
[[[assistant:resp-abc123]]]
|
||
Hello! How can I help?
|
||
|
||
[[[end]]]
|
||
[[[tool:file_read]]]
|
||
File contents...
|
||
|
||
[[[end]]]
|
||
```
|
||
|
||
This enables reliable parsing of multi-turn, multi-role conversations
|
||
from the flat log. Each event type (`assistant`, `tool`, `call`, `retry`,
|
||
`error`, `usage`, `info`, `exec`) opens a block and `[[[end]]]` closes it.
|
||
`reasoning` and `reasoning_think` events are suppressed from the log
|
||
entirely — they're included in the LLM context but invisible to the user.
|
||
|
||
### KDE submodule evolution
|
||
The KDE frontend received extensive updates across both days:
|
||
- **StreamFsm refactor** — chat block FSM with proper fence-post handling
|
||
- **Responsive session tree** — selection/statewait stream management
|
||
- **Explicit agent selection** — per-agent statewait/chat streams,
|
||
only `switchAgent` starts streams; session click clears agent selection
|
||
- **Emoji-labeled states** — visual state indicators (❌ for errors, ⚠️ for warnings)
|
||
- **Double-click rename** — inline renaming of agent nodes
|
||
- **Robust session loading** — loads sessions on startup, handles empty state
|
||
- **Context menu** — kill session, per-session operations
|
||
- **Event-driven updates** — replaced timer-based polling with blocking `/event`
|
||
- **Fence fix** — fixed a bug where code fences could eat surrounding content
|
||
- Removed D-Bus entirely; all communication via `9p` plan9port binary
|
||
|
||
### Updated Filesystem Layout (post-refactor)
|
||
```
|
||
/ ← root (owned by system user)
|
||
├── backends backends list
|
||
├── help help text
|
||
├── models model list (cached)
|
||
├── agents agent config list
|
||
├── ctl root control (invalidate, kill)
|
||
├── event global event stream (blocking read)
|
||
├── complete request-response: code completion
|
||
├── generate request-response: one-shot LLM generation
|
||
├── route request-response: model routing
|
||
├── bypass/
|
||
│ ├── policy global bypass policy
|
||
│ └── pending/{sname} pending bypass requests
|
||
└── session/
|
||
├── new write key=value to create session
|
||
├── idx session index (one line per agent)
|
||
├── {name}/
|
||
│ ├── env session environment variables
|
||
│ ├── ctl session control (kill, save, invalidate)
|
||
│ ├── plan session-scoped markdown checklist
|
||
│ ├── id immutable session UUID
|
||
│ ├── name mutable session name (write to rename)
|
||
│ ├── bypass per-session bypass policy
|
||
│ └── agent/
|
||
│ ├── new write config to create agent (rdwr)
|
||
│ └── {aname}/
|
||
│ ├── prompt, prompt.prev, fifo.in, fifo.out
|
||
│ ├── chat (streaming), log (64KB window)
|
||
│ ├── state, statewait (blocking)
|
||
│ ├── cfg, ctl, cwd
|
||
│ ├── id (immutable), name (mutable)
|
||
│ ├── offset, usage, cost, ctxsz, models
|
||
│ ├── systemprompt, context, tail
|
||
│ ├── tools, toolsrv.loaded, toolsrv.rev
|
||
│ └── proc/{pid}
|
||
```
|
||
|
||
## Updated Current Topology
|
||
```
|
||
ollie/ ← single Go module
|
||
├── cmd/
|
||
│ ├── olliesrv/ ← 9P server (sessions, agents, backends)
|
||
│ │ └── internal/ agent/, backend/, bypass/, fs/, session/, prompts/, toolclient/
|
||
│ ├── toolsrv/ ← 9P tool execution server (separate process)
|
||
│ │ └── internal/ fs/, exec/, registry/, sandbox/
|
||
│ ├── ollie-9p/ ← 9P client CLI
|
||
│ └── ollie-remote/ ← remote execution binary
|
||
├── toolsrv/ ← 9P client library for toolsrv
|
||
├── fsedsl/ ← filesystem declaration EDSL (used by both servers)
|
||
├── env/ ← environment helpers
|
||
├── log/ ← structured logging
|
||
├── paths/ ← XDG path resolution
|
||
├── kde/ ← KDE Plasma integration (part of the main repository)
|
||
├── data/tools/ ← tool executables + .meta sidecar files
|
||
├── data/agents/ ← agent configs (JSON)
|
||
├── data/prompts/ ← system prompt fragments (markdown)
|
||
├── data/skills/ ← domain knowledge modules (markdown)
|
||
├── data/scripts/ ← ollie-remount, o CLI
|
||
├── data/services/ ← systemd, xdg-autostart
|
||
├── prompts/ ← embedded prompt templates
|
||
└── doc/ ← documentation
|
||
```
|
||
|
||
## Phase 18: Zero Built-in Tools (Aug 2–3)
|
||
|
||
The most radical simplification yet: **zero tools compiled into the Go binary.** Every tool — including `shell` and `reasoning_think` — is now an external script loaded dynamically through the 9P filesystem.
|
||
|
||
### What changed
|
||
|
||
**Before**: `tools/builtin/builtins.go` returned four Go-compiled handlers (`shell`, `reasoning_think`, `tool_list`, `tool_load`, `tool_active`). These were hard-linked into every agent's tool definition, consuming LLM context slots even when unused. The tool registry was a two-tier system: built-in handlers (Go code, always available) + script tools (loaded on demand via `tool_load`).
|
||
|
||
**After**: `tools/builtin/` was removed entirely — `builtins.go` last returned `nil`. All tools are loaded through a single convergent path: writing the tool name to `session/{sname}/agent/{aname}/tools`. The `shell` and `reasoning_think` tools are now external scripts (`data/tools/shell`, `data/tools/reasoning_think`) with `.meta` sidecars — identical to `file_read`, `file_grep`, or any other tool.
|
||
|
||
### Unified tool loading
|
||
|
||
Tool loading follows a single path shared by both 9P writes and autoLoad initialization:
|
||
|
||
```
|
||
9P write to agent/{id}/tools
|
||
│
|
||
▼
|
||
session.loadTool(name, agent)
|
||
│
|
||
├── Check disallow list
|
||
└── RPC "tool_load" → ollie-remote process
|
||
```
|
||
|
||
The `tool_load` tool itself is a bash script (`data/tools/tool_load`) that writes the tool name to the 9P filesystem — it's a meta-tool that uses the same interface as every other operation.
|
||
|
||
### Agent config: `autoLoad`
|
||
|
||
Agent configuration files now declare which tools to load at startup via an `autoLoad` array:
|
||
|
||
```json
|
||
{
|
||
"autoLoad": [
|
||
"file_read",
|
||
"file_edit",
|
||
"file_glob",
|
||
"file_grep",
|
||
"file_write",
|
||
"tool_load"
|
||
]
|
||
}
|
||
```
|
||
|
||
Each agent profile (default, copilot, explorer, librarian, navigator, taskmanager, theo) has its own `autoLoad` list tailored to its domain. The `allowTools` field provides a secondary security gate.
|
||
|
||
### Tool output format classification
|
||
|
||
Each tool's `.meta` can declare an `outputFormat` field specifying the source-fence language for its output:
|
||
|
||
```json
|
||
{
|
||
"description": "...",
|
||
"outputFormat": "markdown"
|
||
}
|
||
```
|
||
|
||
The agent loop propagates this to the chat log as the fence language (`[[[tool:file_read]]]\`\`\`markdown`), enabling syntax-highlighted output in the KDE GUI. Tools without `outputFormat` default to plaintext fences.
|
||
|
||
### Remote execution simplified
|
||
|
||
`ollie-remote` no longer embeds tool scripts or sandbox config:
|
||
- Removed `//go:embed all:tools` — tools come from `$OLLIE_TOOLS_PATH` in the environment
|
||
- Removed embedded sandbox `default.yaml` — sandbox config is provisioned alongside tools
|
||
- Removed `--tools` flag — uses `~/.config/ollie/tools` by default
|
||
|
||
### SSH bootstrap simplified
|
||
|
||
The SSH bootstrap sequence was cut from a complex multi-stage protocol to a single tarball transfer:
|
||
|
||
| Before | After |
|
||
|--------|-------|
|
||
| SHA256 hash verification | No hashing |
|
||
| gzip + base64 binary encoding | Raw `tar czf` tarball |
|
||
| `OLLIE_LOADER_START` / `OLLIE_LOADER_READY` handshake | Single `OLLIE_LISTEN_READY` signal |
|
||
| Separate `bootstrap.sh` template (88 lines) | Inline 10-line script |
|
||
| Per-binary transfer (ollie-remote + native sandbox) | Single `tar` of `~/.config/ollie/` |
|
||
|
||
The simplified bootstrap:
|
||
|
||
```sh
|
||
CACHE_DIR="${XDG_CACHE_HOME:-$HOME/.cache}/ollie"
|
||
mkdir -p "$CACHE_DIR"
|
||
tar xzf - -C "$CACHE_DIR" # receives tarball from local
|
||
export OLLIE_TOOLS_PATH="$CACHE_DIR/tools"
|
||
export PATH="$CACHE_DIR/bin:$PATH"
|
||
exec "$CACHE_DIR/bin/ollie-remote" serve --cwd <dir> --listen <sock>
|
||
```
|
||
|
||
### 9P namespace changes
|
||
|
||
- `agent/{id}/tools` — **read** lists all currently loaded tools (via the session's `toolsConn`), **write** loads a tool by name (via `session.loadTool`)
|
||
- Removed `agent/{id}/toolsrv.loaded` — subsumed by `tools` read
|
||
- Removed `agent/{id}/toolsrv.rev` — no longer needed
|
||
- Added root-level `/tools` — lists all discoverable tools on disk (global catalog)
|
||
- Removed `HandlerCtx.ToolReg` — the in-memory Go-level registry is no longer passed through 9P handlers
|
||
|
||
### Files removed
|
||
|
||
| File | Status |
|
||
|------|--------|
|
||
| `tools/builtin/` | Package removed entirely — all tool logic lives in external scripts |
|
||
| `tools/builtin/shell.go` | Deleted (now `data/tools/shell` + `data/tools/shell.meta`) |
|
||
| `tools/builtin/reasoning.go` | Deleted (now `data/tools/reasoning_think` + `data/tools/reasoning_think.meta`) |
|
||
| `tools/builtin/tool.go` | Deleted (`tool_list`/`tool_load`/`tool_active` replaced by 9P filesystem mechanism) |
|
||
| `toolsrv/bootstrap.sh` | Deleted (inline bootstrap replaces multi-stage protocol) |
|
||
| `cmd/ollie-remote/tools/.gitkeep` | Deleted (no embedded tools) |
|
||
|
||
### Lines of code
|
||
|
||
- **−530 lines** from the Go core (removed built-in handlers, embedded tools, bootstrap script)
|
||
- **+82 lines** for external tool scripts + `.meta` sidecars (`shell`, `reasoning_think`, `tool_load`)
|
||
- **Net**: tools system is smaller, simpler, and more uniform
|
||
|
||
### Why it matters
|
||
|
||
The zero-tools transition is the culmination of the architectural trajectory that began with the tool registry in Phase 10: **every facility is a file; nothing is hard-coded.**
|
||
|
||
1. **Uniformity** — There is no distinction between "core" tools and "script" tools. `shell`, `reasoning_think`, `file_read`, `file_write` — all loaded the same way, all have `.meta` sidecars, all called via the same RPC mechanism.
|
||
|
||
2. **Minimal core** — The Go binary no longer contains any tool logic. The agent loop is pure coordination: stream LLM → parse tool calls → dispatch to remote process → loop. Tool definitions, schemas, and execution are entirely external.
|
||
|
||
3. **Agent-choosable tools** — Each agent profile declares exactly which tools it needs. No agent pays the context-window cost of `shell` and `reasoning_think` unless its config asks for them.
|
||
|
||
4. **9P as the sole control plane** — Tool loading is now a file write. The same mechanism (`ollie-9p write ...`) that submits prompts and reads state also manages tool loading. No special RPC, no separate tool-manager protocol.
|
||
|
||
5. **Simpler remote deployment** — No embedded files, no binary transfer protocol, no hash verification. Provisioning a remote host is `tar czf ~/.config/ollie | ssh host tar xzf -`.
|
||
|
||
The result: the Go core is a 9P file server and an agent loop, nothing more. All behavior — every tool, skill, prompt, and configuration — lives in data files on disk. This is the Plan 9 ideal: a small, fixed kernel; all policy and capability in the filesystem.
|
||
|
||
### SLOC current (Aug 3)
|
||
| Component | Lines (excl. tests) |
|
||
|-----------|--------------------|
|
||
| agent + backend + toolsrv + session | 11,828 |
|
||
| fs + cmd/olliesrv | 4,838 |
|
||
| kde gui | 6,491 |
|
||
| **Total** | **23,157** |
|
||
Zero dead exported functions remain (verified via LSP + grep across all repos).
|
||
|
||
## Phase 19: EDSL Extraction & Structural Cleanup (Aug 3–4)
|
||
|
||
The declarative EDSL created in Phase 15 was extracted into a standalone,
|
||
reusable library. This wasn't just moving code — the EDSL itself was
|
||
**created from scratch on Aug 2** to replace ~3,100 lines of imperative
|
||
filesystem code. Within 48 hours it went from concept to extracted library.
|
||
|
||
The trajectory:
|
||
1. **Aug 2 morning**: Created the EDSL (Phase 15) — replaced imperative tree-building with declarative spec
|
||
2. **Aug 2 afternoon**: Streamlined the 9P server (Phase 16) — reduced to thin protocol handler
|
||
3. **Aug 3**: Extracted EDSL to `fsedsl/` as a reusable library
|
||
4. **Aug 4**: Structural cleanup — removed dead packages, improved `o` CLI
|
||
|
||
This rapid iteration exemplifies the project's development style: build the
|
||
right abstraction, validate it works, then extract it for reuse.
|
||
|
||
### fsedsl: Standalone Library
|
||
|
||
The generic EDSL types and builder logic extracted from `fs/` into `fsedsl/`:
|
||
|
||
```
|
||
fsedsl/
|
||
├── fsnode.go FsNodeDecl[C], NodeOption[C], Dir/Leaf/Each constructors
|
||
├── builder.go BuildTree — walks spec, validates, wires handlers
|
||
├── tree.go Tree, File, FileConfig, FileTree types
|
||
├── option.go Doc, Read, Write, Stream, BlockOnce, Request, etc.
|
||
└── README.md Integration guide
|
||
```
|
||
|
||
The library is **protocol-agnostic** — it knows nothing about 9P. It produces
|
||
a `*Tree` structure that any protocol (9P, FUSE, HTTP, gRPC) can walk.
|
||
The `fs/` package becomes a thin adapter: type aliases + ollie-specific handlers.
|
||
|
||
Key design:
|
||
- Generic over context type `C` — ollie uses `HandlerCtx`, others can use anything
|
||
- Single `BuildTree[C]()` call validates and wires the entire namespace
|
||
- Validation at build time: no leaf handlers on directories, at most one blocking
|
||
variant per file, template nodes require `List`, ownership inheritance
|
||
|
||
### fs/ Package Consolidation
|
||
|
||
After EDSL extraction, the fs package was restructured:
|
||
|
||
**Before** (scattered):
|
||
```
|
||
fs/
|
||
├── spec.go namespace declaration
|
||
├── builder.go tree builder (moved to fsedsl)
|
||
├── adapter.go type aliases
|
||
├── session/ sub-package with 6 files
|
||
│ ├── files.go
|
||
│ ├── root.go
|
||
│ ├── perm.go
|
||
│ └── ...
|
||
└── bypassfiles.go
|
||
```
|
||
|
||
**After** (flat, by scope):
|
||
```
|
||
fs/
|
||
├── spec.go single source of truth — entire namespace
|
||
├── fsnode.go type aliases from fsedsl
|
||
├── builder.go BuildTree wrapper
|
||
├── rootfiles.go root-level handlers (backends, models, help, ctl)
|
||
├── sessionfiles.go session-level handlers (env, ctl, name)
|
||
├── agentfiles.go agent-level handlers (prompt, chat, state, tools)
|
||
├── bypassfiles.go bypass broker handlers
|
||
└── procfiles.go detached process handlers
|
||
```
|
||
|
||
No sub-packages. Handler files are organized by 9P tree scope.
|
||
|
||
### Session Package Refactors
|
||
|
||
Major restructuring of session lifecycle:
|
||
|
||
- **Session/Agent split** — `session.Session` manages lifecycle, `agent.Agent` manages conversation. Clear ownership boundaries.
|
||
- **Persistence to session/** — `fs/persist.go` removed, persistence logic moved to `session/` package
|
||
- **Manager eliminated** — Package-level registry replaces `Manager` struct. CRUD via `session.Lookup()`, `session.All()`, `session.CreateFromRoot()`, etc.
|
||
- **Remote promoted to session level** — Remote execution managed at session, not agent level
|
||
|
||
### Event Bus: Ring Buffer → Pubsub
|
||
|
||
The event ring buffer added in Phase 17 lasted exactly one day. On Aug 3, it
|
||
was replaced with `github.com/simonfxr/pubsub`:
|
||
|
||
**Why the ring buffer failed**:
|
||
- Off-by-one bugs in circular buffer indexing
|
||
- Race conditions between `sync.Cond` broadcasts and context cancellation
|
||
- Panics from double-unlock when multiple readers raced to the same event
|
||
- The offset-based API leaked implementation details to clients
|
||
|
||
**What pubsub provides**:
|
||
- Topic-based routing (`session.{id}.new`, `agent.{id}.state`)
|
||
- Wildcard subscriptions (`session.*`, `*`)
|
||
- Channel-based delivery with automatic cleanup on context cancellation
|
||
- A decade of production hardening by someone else
|
||
|
||
The backwards-compatible `PostEvent()` wrapper translates the old format
|
||
(`A agentId state thinking`) to the new topic format (`agent.agentId.state`
|
||
with payload `thinking`). Existing code kept working.
|
||
|
||
**Lines changed**: +103 / -72. Net increase, but the new code is correct.
|
||
|
||
### Dead Code Removal
|
||
|
||
| Removed | Reason |
|
||
|---------|--------|
|
||
| `cmd/Ollie` (Acme frontend) | Use `o` CLI to compose acme front-end |
|
||
| `mount/` package | FUSE mount replaced by direct 9P client (`ollie-9p`) |
|
||
| `ollie-watchdog` script | No longer needed — 9P client doesn't have FUSE stale mount issues |
|
||
| `doc/experiments/` | Historical experiments moved to git history |
|
||
| `doc/help.md` | Now generated dynamically from `Doc()` strings in spec |
|
||
|
||
### o CLI Improvements
|
||
|
||
The terminal CLI (`data/scripts/o`) was simplified:
|
||
|
||
- **Removed auto-session/agent** — Context must be set explicitly via `export` or `eval $(o env ...)`. Eliminates slow `ollie-9p read session/idx` calls.
|
||
- **Path classification** — Paths categorized as root/session/agent level. Clear error messages: "chat requires agent context".
|
||
- **TUI percentage layout** — Fixed-size splits replaced with percentages (`-l 25%`). Adapts to terminal size.
|
||
- **Block marker filtering** — Chat stream piped through `grep -v '\[\[\[.*\]\]\]'` to hide internal markers.
|
||
|
||
### Documentation
|
||
|
||
- Removed outdated docs (`doc/experiments/`, stale `help.md`)
|
||
- Renamed doc files to lowercase-kebab-case
|
||
- Updated usage.md for new `o` CLI behavior
|
||
- Removed mount/ and watchdog references from architecture docs
|
||
|
||
### Gantt Update
|
||
|
||
The timeline now extends to Aug 4, 2026 — ~3.75 months of development.
|
||
|
||
### SLOC current (Aug 4)
|
||
| Component | Lines (excl. tests) |
|
||
|-----------|--------------------|
|
||
| agent + toolsrv + session | 9,077 |
|
||
| fs + cmd/olliesrv | 2,919 |
|
||
| fsedsl (extracted library) | 1,220 |
|
||
| kde gui | 4,162 |
|
||
| **Total (core)** | **17,378** |
|
||
|
||
Note: Aug 3 count included `backend/` (4,366 lines) and had different kde scope.
|
||
Comparable core (fs + agent + toolsrv + session + fsedsl + kde gui) shrank from ~23k to ~17k.
|
||
|
||
---
|
||
|
||
## "Good Idea Fairy" Cleanup (Aug 5)
|
||
|
||
A brutal simplification pass that deleted ~960 lines of speculative infrastructure.
|
||
|
||
### Hooks — Deleted Entirely
|
||
|
||
The entire hooks system was removed:
|
||
|
||
| Hook | What it did | Why deleted |
|
||
|------|-------------|-------------|
|
||
| `agentSpawn` | Run commands on session creation | Prompt composition handles initialization |
|
||
| `preTurn` | Inject context before each turn | Removed earlier; prompt assembly does this |
|
||
| `postTurn` | Run commands after each turn | Never used in practice |
|
||
| `preTool` | Gate tool execution | Security model moved to sandbox |
|
||
| `postTool` | Transform tool results | Never used |
|
||
| `preCompact` | Run before context compaction | No use case |
|
||
| `postCompact` | Run after compaction | No use case |
|
||
| `turnError` | Handle backend errors | Replaced by retry logic in the loop |
|
||
|
||
Hooks are a "standard feature" in agent frameworks — lifecycle callbacks that
|
||
let external code react to events. In practice, they added complexity without
|
||
solving real problems. The prompt system handles initialization. The sandbox
|
||
handles security. Error handling belongs in the loop. Every hook was either
|
||
unused or doing something that belonged elsewhere.
|
||
|
||
**−237 lines** from `agent/hooks.go`.
|
||
|
||
### Other Deletions
|
||
|
||
| What | Why |
|
||
|------|-----|
|
||
| `backend/noop.go` | Test backend, never used. Tests use real backends or mocks. |
|
||
| `backend/oneshot.go` | Alternative generation mode, never used. `generate` file handles one-shot. |
|
||
| `cmd/ollie-9p/mount/` | FUSE mount code, replaced by `ollie-9p` direct 9P client. |
|
||
| `data/tools/route.meta` | Model routing tool, replaced by explicit `/model` commands. |
|
||
| `toolsrv/schema.go` | Schema validation code, never enabled. |
|
||
|
||
### Turn State Machine
|
||
|
||
The turn state machine in `agent/turn.go` was simplified from 158 lines to ~40.
|
||
The `TurnCtx` struct was not yet eliminated (that came later in Aug 11), but
|
||
the state transitions were flattened.
|
||
|
||
### Philosophy
|
||
|
||
"Good idea fairy" is military slang for someone who shows up with clever
|
||
suggestions that sound good but create work. The deleted code was all
|
||
reasonable in isolation — hooks are a standard pattern, test backends are
|
||
useful, FUSE mounts are convenient. But each added surface area that had to
|
||
be maintained, tested, and reasoned about. None of it solved actual problems
|
||
better than simpler alternatives.
|
||
|
||
---
|
||
|
||
## Aug 9, 2026 — Simplification & Namespace Cleanup
|
||
|
||
Major simplification pass: ~1,450 lines removed from the core runtime.
|
||
|
||
### Structural
|
||
|
||
- **Section-based preamble** — `Preamble` struct with `Set/Get/String` replaces string surgery (-270 lines)
|
||
- **Session** — Export immutable fields, extract `buildAgent` helper (-115 lines)
|
||
- **Prompt resolver** — Gutted to file-path-only resolution (-127 lines)
|
||
- **TaskState subsystem** — Removed entirely (-246 lines)
|
||
- **GenerationParams** — Embedded in AgentConfig, JSON tags added (-34 lines)
|
||
- **sync.Cond → close-channel** — `WaitChange` uses idiomatic channel signal
|
||
- **Dead code** — `session.New()`, env map, prevPrompt field, usage_log.go, contextDebug()
|
||
|
||
### Namespace
|
||
|
||
The agent namespace went from 17 files to 10. Redundant files merged or moved to ctl:
|
||
|
||
| Before | After |
|
||
|--------|-------|
|
||
| `fifo.in` + `fifo.out` | `fifo` (write=enqueue, read=dequeue) |
|
||
| `cost` + `usage` + `ctxsz` | `stats` (key=value lines) |
|
||
| `chat` | `chat` (filtered text) + `chat.raw` (full markup) |
|
||
| `cwd`, `models`, `tools`, `systemprompt` | ctl commands |
|
||
| `connection`, `state`, `context`, `tail`, `offset`, `prompt.prev` | removed |
|
||
|
||
### ctl Unification
|
||
|
||
All ctl files (root, session, agent) now share:
|
||
```go
|
||
type rdwrHandler func(ctx HandlerCtx, args []string) ([]byte, error)
|
||
```
|
||
|
||
Agent ctl is **rdwr** (request-response): write command, read result.
|
||
Commands with no args return current state (e.g., `/model` prints the model).
|
||
|
||
Agent ctl commands: `stop`, `compact`, `clear`, `inject`, `agent`, `model`, `models`, `tools`, `tool_load`, `cwd`, `name`, `backend`, `systemprompt`.
|
||
|
||
### Command Surface
|
||
|
||
`/` prefix in prompts == `o ctl`. No special-cased commands in the agent.
|
||
- `/model qwen3:8b` → switches model, returns new model name
|
||
- `/tools` → lists loaded tools
|
||
- `/inject look at this` → injects prompt mid-turn (overwrites pending)
|
||
- `!q` in the REPL → exits the TUI (local-only)
|
||
|
||
### elevate → bypass
|
||
|
||
The sandbox escape mechanism was renamed from `elevate` to `bypass` throughout (package, namespace, env vars, tool args, all docs).
|
||
|
||
### Streaming Fix
|
||
|
||
The TUI streaming regression (line-by-line instead of char-by-char since Aug 4) was caused by piping through `grep -v` — grep is inherently line-buffered. Fixed by:
|
||
- `chat` file now serves filtered text (markers + fences stripped server-side via `stripMarkers` state machine)
|
||
- `chat.raw` serves full markup for GUIs that parse blocks
|
||
- TUI no longer pipes through grep for streaming
|
||
|
||
### CLI (`o` script)
|
||
|
||
- `readloop` → `read -l`
|
||
- `chatstream` → removed (`o read chat` streams directly)
|
||
- `stop`, `kill` → removed (use `o ctl stop`, `o ctl kill`)
|
||
- `ctl` uses `ollie-9p rdwr` (gets response)
|
||
- REPL: `/` → `o ctl`, `!q` → exit TUI
|
||
- Background statewait loop trapped on EXIT (fixes zombie)
|
||
|
||
### SLOC (Aug 9)
|
||
|
||
| Component | Lines (excl. tests, backends, fsedsl, generated) |
|
||
|-----------|--------------------:|
|
||
| agent | 3,167 |
|
||
| toolsrv | 2,521 |
|
||
| fs | 1,774 |
|
||
| session | 1,386 |
|
||
| cmd/olliesrv | 1,161 |
|
||
| bypass | 733 |
|
||
| lib9p | 690 |
|
||
| cmd/ollie-9p | 356 |
|
||
| cmd/ollie-remote | 369 |
|
||
| sandbox | 293 |
|
||
| log + env + paths + format + prompts | 428 |
|
||
| **Total (core)** | **12,878** |
|
||
|
||
---
|
||
|
||
## Aug 9, 2026 (cont.) — Prompt Audit & Maintainability Pass
|
||
|
||
Systematic audit of the prompt construction system and codebase maintainability.
|
||
|
||
### Prompt System
|
||
|
||
- **System prompt**: ~200 → ~80 lines. Removed tool-first table, API docs section (moved to agent prompt), dead 9P entries, output protocol (moved per-agent).
|
||
- **AllowTools**: removed entirely (field, RPC, config). Tools are loaded dynamically; static allowlists served no purpose.
|
||
- **Phantom refs fixed**: `route` removed from driver autoLoad, `reasoning_think` + `shell` added to default autoLoad.
|
||
- **Output protocol**: moved from system prompt to per-agent prompts. Theo's brevity rule applied to all agents except copilot.
|
||
- **Tool docs**: merged listing + documentation into single `renderTools()`. Removed JSON schema dump (function-calling API provides it natively). Removed `BuildToolListing` dead code.
|
||
- **PRIME_* env vars**: removed entirely. Replaced `SessionInfra.PromptEnv []string` with typed `Platform string` + `IsGitRepo bool` fields.
|
||
|
||
### Code Quality
|
||
|
||
- **`OnToolsChanged`**: changed from `func(string)` to `func()` signal. Agent re-fetches tool list on signal instead of receiving a pre-formatted string it ignores.
|
||
- **`Preamble` sections**: type-safe `Section` type replaces raw strings. Compile-time typo detection.
|
||
- **`extractToolResult`**: documented the tool output protocol (JSON content-block envelope with raw-text fallback).
|
||
- **Package godoc**: `session/` package comment documenting Init→Create→Register→Kill lifecycle.
|
||
- **`PromptEnv()`**: deleted (was a no-op returning nil after PRIME_* removal).
|
||
|
||
### Documentation
|
||
|
||
- Moved 7 reference docs from `doc/resources/` to `doc/` (architecture, 9p, writing-tools, tool-registry, core, remote-execution, edsl). Left whitepaper material (evolution, multi-agent, misc, no-mcp, ideas) in `resources/`.
|
||
- Rewrote `data/agents/README.md` to match current config schema and agent roster.
|
||
- Fixed stale cross-references in README.md and doc/tool-registry.md.
|
||
|
||
### SLOC (Aug 9, post-audit)
|
||
|
||
| Component | Lines (excl. tests, generated) |
|
||
|-----------|--------------------:|
|
||
| agent | 3,159 |
|
||
| toolsrv | 2,464 |
|
||
| session | 1,377 |
|
||
| fs | 1,774 |
|
||
| cmd/olliesrv | 1,161 |
|
||
| bypass | 733 |
|
||
| lib9p | 690 |
|
||
| cmd/ollie-9p | 356 |
|
||
| cmd/ollie-remote | 358 |
|
||
| sandbox | 293 |
|
||
| log + env + paths + format + prompts | 428 |
|
||
| **Total (core)** | **12,793** |
|
||
|
||
Excludes: backends (4,229), fsedsl (1,030), tests, KDE, tools, generated code.
|
||
|
||
## Phase 20: toolsrv 9P Migration & Process Isolation (Aug 10–11)
|
||
|
||
The tool server completed its evolution from an in-process library to a
|
||
**fully independent 9P server**. Locally it runs as a child process of
|
||
olliesrv; for remote execution it runs on a remote host, connected via
|
||
SSH Unix socket forwarding.
|
||
|
||
### Before
|
||
|
||
`toolsrv` was a Go package (`ollie/toolsrv`) with a `Server` struct that ran
|
||
in-process within olliesrv. Tool calls were Go function calls — no process
|
||
boundary, no separate namespace. The package mixed client code, server code,
|
||
registry, sandbox wrappers, and execution logic in a flat directory.
|
||
|
||
### After
|
||
|
||
Two distinct components:
|
||
|
||
1. **`cmd/toolsrv/`** — a standalone binary that serves its own 9P2000
|
||
filesystem over a Unix socket. Has its own `internal/` packages:
|
||
- `internal/fs/` — filesystem spec (fsedsl), server state, process management
|
||
- `internal/exec/` — sandboxed tool execution (native Landlock, bypass broker)
|
||
- `internal/registry/` — session-scoped tool registry
|
||
- `internal/sandbox/` — native Landlock configuration and enforcement
|
||
|
||
2. **`toolsrv/`** (root package) — a 9P client library. `Conn` dials toolsrv
|
||
over a Unix socket, authenticates via Tauth, and exposes methods like
|
||
`CallTool`, `LoadTool`, `ListTools`, `Ping`.
|
||
|
||
### toolsrv 9P Namespace
|
||
|
||
```
|
||
/
|
||
├── ctl write: load <tool>, unload <tool>, env K=V, cwd <path>
|
||
├── tools read: list loaded tools (JSON), write: tool name to load
|
||
├── info read: platform, arch
|
||
└── proc/
|
||
├── new rdwr: write tool+args, blocks, read result
|
||
├── new.bg write: tool+args, returns pid immediately
|
||
└── {pid}/
|
||
├── out read: output
|
||
├── wait read: blocks until exit, returns exit code
|
||
├── stat read: running/exited, runtime, tool
|
||
└── ctl write: signal <N>, dismiss
|
||
```
|
||
|
||
Both `cmd/toolsrv` and `cmd/olliesrv` declare their namespaces using the
|
||
same `fsedsl` library. Both implement their own 9P protocol handlers (using
|
||
`9fans.net/go/plan9`) — they are intentionally separate servers that
|
||
communicate over a socket.
|
||
|
||
### Process Lifecycle
|
||
|
||
Three-layer defense against orphaned toolsrv processes:
|
||
|
||
1. **Pdeathsig** (local only) — `SysProcAttr{Pdeathsig: SIGTERM}` on local
|
||
spawn. When olliesrv dies, the kernel terminates toolsrv immediately.
|
||
For SSH-spawned toolsrv, Pdeathsig is set on the `ssh` process itself —
|
||
when SSH dies, the remote shell receives SIGHUP which typically cascades
|
||
to toolsrv.
|
||
|
||
2. **Idle timeout** (local + remote) — toolsrv tracks active 9P connections.
|
||
After the first client connects and then all connections drop, a 30s timer
|
||
starts. If no new connection arrives, toolsrv exits cleanly. 60s startup
|
||
grace period for the initial connection. This is the primary cleanup
|
||
mechanism for remote toolsrv where Pdeathsig doesn't apply directly.
|
||
|
||
3. **Kill-before-respawn** — `ProcessKeeper.Dial()` kills the old process
|
||
before spawning a replacement. Prevents accumulation during reconnect cycles.
|
||
|
||
### Authentication
|
||
|
||
toolsrv uses 9P Tauth for authentication. The first client sets the shared
|
||
secret; subsequent clients must provide the same secret. This replaces
|
||
the previous token-in-environment approach and is compatible with socket
|
||
permission security.
|
||
|
||
### Structural Consistency
|
||
|
||
Both servers now follow the same physical layout:
|
||
|
||
```
|
||
cmd/{server}/
|
||
├── main.go entry point
|
||
├── server.go 9P protocol handler (top-level, like Plan 9 tradition)
|
||
└── internal/
|
||
├── fs/ filesystem spec + handlers + state
|
||
├── ... domain-specific packages
|
||
```
|
||
|
||
### Parallel Tool Execution
|
||
|
||
Replaced binary ReadOnly/not batching with resource-based conflict scheduling.
|
||
Tool calls within a single turn are grouped into parallel batches based on
|
||
a three-class scope system declared in `.meta`:
|
||
|
||
- **scope "read"** — never conflicts (always parallel)
|
||
- **scope "write"** — conflicts only on same file path
|
||
- **scope "global"** (or unset, default) — full serialization barrier
|
||
|
||
The model's read-before-write pattern (enforced by the turn-based protocol)
|
||
guarantees that parallel writes are safe: each file_edit carries its own
|
||
`old_string` context from a prior read, and edits on different paths are
|
||
provably independent.
|
||
|
||
Shell and tools with opaque effects (like `lsp_rename`) declare scope "global"
|
||
and always run alone. Unknown/unset scope defaults to global — tools must
|
||
opt in to parallelism.
|
||
|
||
### Background Process Interrupts
|
||
|
||
Any tool call can include `"background": true` to execute asynchronously
|
||
via `proc/new.bg`. The model receives a process ID immediately and continues working.
|
||
Background processes have no timeout — they run until they exit naturally,
|
||
are stopped via `proc/{id}/ctl`, or die with the session.
|
||
|
||
Output is streamed in real-time into `proc/{id}/out`. Upon completion,
|
||
the process exit notification is injected as a **user prompt** (written to
|
||
the agent's `prompt` file) containing a `<system-proc-interrupt>` block:
|
||
|
||
```xml
|
||
<system-proc-interrupt id="42" cmd="go test ./..." status="exited" exit="1">
|
||
--- FAIL: TestFoo (0.00s)
|
||
foo_test.go:12: expected 3, got 2
|
||
FAIL
|
||
</system-proc-interrupt>
|
||
```
|
||
|
||
This uses automatic prompt queueing: if the agent is still working,
|
||
the interrupt arrives on the next idle transition; if the agent is already
|
||
idle, it arrives immediately. The model sees these as natural follow-up
|
||
prompts — no polling, no in-loop injection. It can react to failures
|
||
or stop processes via the proc ctl file (`term`, `kill`, `dismiss`).
|
||
|
||
**Process lifecycle**: toolsrv owns all procs. Session death (toolsrv exit)
|
||
kills all procs via `KillAll()` + `Pdeathsig`. Exited procs remain in the
|
||
tree for 10 minutes after last read, then are garbage collected.
|
||
|
||
### Universal Dispatch Flags
|
||
|
||
All tools receive four optional parameters injected into their schemas at
|
||
runtime (not declared per-tool):
|
||
- `bypass` — sandbox escape via bypass broker
|
||
- `timeout` — execution timeout in seconds
|
||
- `sandbox` — sandbox profile override
|
||
- `background` — async execution with auto-injected output
|
||
|
||
### Why Not Concurrent-by-Default
|
||
|
||
Foreground tool calls block the model until all results arrive — deliberately.
|
||
The alternative (execute everything concurrently, deliver results as queued
|
||
prompts) was considered and rejected: it turns one coherent reasoning step
|
||
into N separate generation cycles, each with partial information. The model
|
||
can't reason about results together, every interrupt costs a full inference
|
||
round-trip, and the end result is sequential execution but slower and more
|
||
expensive. Background execution remains an explicit opt-in for genuinely
|
||
long-running processes where the model doesn't need the result immediately.
|
||
|
||
## Phase 22: Meta-Only Tool Definitions (Aug 11–12)
|
||
|
||
A `.meta` file can now define a tool with no companion executable. The `cmd`
|
||
field is treated as a shell command string — not a path to resolve, but a
|
||
command to run directly. JSON arguments are passed on stdin.
|
||
|
||
Before:
|
||
```json
|
||
{"cmd": "rg"} // resolved via PATH, then executed with --flags
|
||
```
|
||
|
||
After:
|
||
```json
|
||
{"cmd": "jq -r '.pattern' | xargs rg"} // shell command, executed as-is
|
||
```
|
||
|
||
This changes what a "tool" is. Previously, tools were executables with `.meta`
|
||
sidecars. Now a tool can be purely declarative: a `.meta` file that shells out
|
||
to existing CLIs. Wrap ripgrep, kubectl, or an MCP bridge without writing any
|
||
code.
|
||
|
||
The tools directory is prepended to `$PATH` during execution, so meta-only
|
||
tools can reference other tools in the same directory.
|
||
|
||
**Why this matters**: any CLI becomes a tool with 20 lines of JSON. The barrier
|
||
to adding capabilities dropped from "write a script" to "write a schema."
|
||
|
||
## Phase 23: Bypass via 9P (Aug 11)
|
||
|
||
The bypass broker — the mechanism for escaping the sandbox — was previously a
|
||
Unix socket with a custom protocol. It now routes through the 9P filesystem:
|
||
|
||
- toolsrv exposes `bypass/pending` (blocking read) and `bypass/resolve` (write)
|
||
- olliesrv subscribes to pending requests and routes them to D-Bus for approval
|
||
- Approved/denied responses flow back through 9P
|
||
|
||
This unifies the control plane. Everything is files. The bypass socket is gone.
|
||
|
||
## Phase 24: fs/ Flattening (Aug 11)
|
||
|
||
The olliesrv filesystem package had accumulated abstractions:
|
||
|
||
- `AgentLog` — chat log buffer wrapper
|
||
- `SessionNode` — session state container
|
||
- `RootState` — root-level state
|
||
|
||
These were eliminated. Chat logging moved to the agent package. Session state
|
||
moved to the session package. The fs/ package now contains:
|
||
|
||
- `spec.go` — the EDSL declaration with all handlers inlined
|
||
- `support.go` — helper functions
|
||
- `newroot.go` — tree construction
|
||
|
||
~600 lines deleted. The handlers live where the data lives.
|
||
|
||
## Phase 25: Sandbox Simplification (Aug 11)
|
||
|
||
Named sandbox profiles (`default`, `restricted`, `remote`) were removed. There
|
||
is now one config: `sandbox.yaml`. Per-project overrides via `.ollie-sandbox.yaml`
|
||
remain, but the multi-profile system is gone.
|
||
|
||
The `sandbox` parameter on tool calls now does nothing — kept for compatibility
|
||
but ignored.
|
||
|
||
## Phase 26: Agent Loop Cleanup (Aug 11)
|
||
|
||
The `TurnCtx` struct — which carried per-turn state through the loop — was
|
||
eliminated. Loop functions became methods on `*Agent`, accessing state directly.
|
||
The turn state machine in `turn.go` was simplified.
|
||
|
||
~100 lines deleted, control flow is clearer.
|
||
|
||
## Phase 27: Tree-sitter Code Intelligence (Aug 15)
|
||
|
||
Code navigation and structural editing moved into native Go tools backed by
|
||
Tree-sitter grammars. The tool set now includes:
|
||
|
||
- `codebase_overview` for repository structure
|
||
- `code_outline` for one-file declarations
|
||
- `code_symbols` for workspace-wide symbol searches
|
||
- `code_dependencies` for imports and includes
|
||
- `code_query` for syntax-aware searches
|
||
- `code_rewrite` for structural rewrites
|
||
|
||
The tools support Go, JavaScript, TypeScript, TSX, Python, Rust, C, C++, JSON,
|
||
YAML, PHP, and Markdown. Agent prompts now require structural tools for
|
||
repository exploration and reserve text search for simple content searches.
|
||
This makes targeted edits safer than broad textual replacement.
|
||
|
||
## Phase 28: Go File Tools and Tool-Surface Cleanup (Aug 16)
|
||
|
||
The core file tools (`file_read`, `file_write`, `file_edit`, `file_grep`, and
|
||
`file_glob`) were replaced with Go binaries and shared implementation code.
|
||
Tests were added for the file-tool package and existing LSP and web-tool
|
||
coverage was expanded. The change removes the old Python implementations from
|
||
the runtime path while preserving the same tool contracts.
|
||
|
||
Obsolete process-management tools and the `logseq` tool were removed. The
|
||
installation recipe was corrected for the compiled file tools. Workspace-path
|
||
placeholders were standardized across prompts and tool metadata so agents use
|
||
the actual working directory instead of guessed home paths.
|
||
|
||
## Phase 29: OptMem Persistent Memory (Aug 16)
|
||
|
||
Ollie integrated [OptMem](https://github.com/VictorTaelin/OptMem) as its
|
||
persistent memory backend. The legacy `memory_recall` and `memory_remember`
|
||
scripts were replaced by metadata-defined tools that invoke the bundled
|
||
`third_party/optmem/memo` executable directly.
|
||
|
||
OptMem owns the append-only log and bounded B-tree index at
|
||
`$XDG_DATA_HOME/ollie/optmem` (default: `~/.local/share/ollie/optmem`). Ollie
|
||
does not maintain a second memory format or parallel store. Reads and writes
|
||
use the same backend and serialize through the toolsrv path-lock system.
|
||
|
||
Both memory tools are auto-loaded by every agent profile. Shared prompt
|
||
guidance tells agents to recall relevant prior context before acting and
|
||
remember durable decisions, outcomes, preferences, and non-obvious findings.
|
||
The OptMem executable is installed under `$XDG_CONFIG_HOME/ollie/optmem` and
|
||
explicitly granted `rwx` access by the native Landlock sandbox.
|
||
|
||
This integration keeps memory as an ordinary Ollie tool while giving it a
|
||
durable, searchable backend. It requires no MCP server, daemon, or new
|
||
control-plane protocol.
|
||
|
||
## Phase 30: Markdown Parsing and Bounded Tool Output (Aug 16)
|
||
|
||
Tree-sitter Markdown support was added to the code-intelligence layer. `.md`
|
||
and `.markdown` files are parsed as Markdown, and headings are available to
|
||
structural queries as `atx_heading` and `setext_heading` nodes. This makes
|
||
`evolution.md` and other documentation amenable to the same targeted tooling
|
||
as source code.
|
||
|
||
The maximum tool result included in model context was reduced from 128 KiB to
|
||
32 KiB. Large command or file results are therefore less likely to crowd out
|
||
the conversation and instructions.
|
||
|
||
---
|
||
|
||
Commit count: ~60 commits over 3 days. The recent work added native structural
|
||
and file tooling, persistent memory, tighter context bounds, and embedding-
|
||
guided discovery while removing obsolete tool implementations.
|
||
|
||
---
|
||
|
||
## Dead Ends and Reversals
|
||
|
||
Everything that was built and then killed, in roughly chronological order.
|
||
|
||
| What | Why it was removed |
|
||
|------|-------------------|
|
||
| MCP servers (denote-mcp, 9beads-mcp) | Too heavyweight; scripts are simpler and faster |
|
||
| TUI frontend | Replaced by `o` — a tiny shell script (40 lines of tmux) |
|
||
| `pl/` planning namespace | Over-engineered; plan file is sufficient |
|
||
| Beads integration | External dependency for something a checklist file handles |
|
||
| `plan_create` / `plan_complete` / `task_add` / `task_check` tools | Five planning iterations before settling on a single markdown file |
|
||
| SSH agent proxy | Attack surface not worth the convenience |
|
||
| Separate ollie-dbus daemon (ollied) | Consolidated into olliesrv, then killed entirely |
|
||
| D-Bus adapter (`org.ollie.SessionManager`) | 9P streaming is superior; no polling, no offset tracking, no frozen GUIs |
|
||
| D-Bus-driven KDE frontends | Rewritten on pure 9P via plan9port `9p` binary |
|
||
| Plasmoid / tray KDE components | Not useful enough to maintain |
|
||
| FUSE mounts | Replaced by standalone 9P client binary; FUSE can't do blocking reads |
|
||
| Tier routing (FAST/POWER) | Replaced by `/route` endpoint with real model discovery |
|
||
| call_tool / pipe | Replaced by native tool registry with `tool_load` |
|
||
| execute_code (multi-language) | Simplified to shell-only; tool scripts handle language choice |
|
||
| chatwait | Reverted; acme tails `chat` directly |
|
||
| Header comment metadata (`ollie:prompt`, `ollie:tier`, `args_json:`) | Replaced by `.meta` sidecar JSON; decouples metadata from language |
|
||
| `contrib/` directory | Renamed to `data/`; nothing was community-contributed |
|
||
| Script namespaces (s/, u/, x/) | Replaced by 9P request-response files |
|
||
| s/sh, s/bfg, s/bbg | Replaced by `generate`/`complete`/`route` 9P files |
|
||
| `agent.Core` interface | Nobody else implemented it; just indirection |
|
||
| `Dispatcher` indirection in toolsrv | `toolsrv.Server` called directly |
|
||
| `execute/` package | Merged into `tools/`; 3 functions didn't need a package |
|
||
| `mgr/` package (Manager struct) | Replaced by package functions; `*fs.Tree` IS the collection |
|
||
| `fs/session/` sub-package (6 files, ~3,100 lines) | Flattened into `fs/` — no more sub-package indirection |
|
||
| `cmd/olliesrv/bypass_tree.go` | Bypass tree handlers moved to `fs/bypassfiles.go` |
|
||
| Per-session/agent timestamp IDs | Replaced by UUIDv4 — immutable `id` + mutable `name` |
|
||
| Single-step session creation | Split into `session/new` + `session/{name}/agent/new` |
|
||
| Imperative filesystem (stat/walk/read/write per node) | Replaced by declarative EDSL (`fsedsl/`) |
|
||
| Event ring buffer (100-slot circular buffer + sync.Cond) | Lasted one day; replaced by `pubsub` library. Off-by-one bugs, race conditions, panics. |
|
||
| Built-in tool handlers (shell, reasoning_think, tool_list, tool_load, tool_active) | Replaced by external scripts loaded via 9P `agent/{id}/tools` write |
|
||
| Go-compiled tool registry in `tools/builtin/` | Zero built-in tools — all tools are external scripts with `.meta` sidecars |
|
||
| Embedded tool scripts in `ollie-remote` | Remote uses `$OLLIE_TOOLS_PATH` from environment; tools deployed via tarball |
|
||
| SSH binary transfer (gzip+base64+hash-verify) | Replaced by simple `tar | ssh` of `~/.config/ollie/` |
|
||
| `toolsrv.loaded` / `toolsrv.rev` 9P files | Removed — `tools` file handles both list and load |
|
||
| `cmd/Ollie` (Acme frontend binary) | Replaced by `o` CLI composing acme |
|
||
| `mount/` package | FUSE mount replaced by `ollie-9p` direct 9P client |
|
||
| `ollie-watchdog` script | No FUSE stale mount issues with 9P client |
|
||
| `doc/experiments/` | Historical experiments; moved to git history |
|
||
| `doc/help.md` | Generated dynamically from `Doc()` strings in spec |
|
||
| Hooks system (agentSpawn, preTurn, postTurn, preTool, postTool, preCompact, postCompact, turnError) | Never solved real problems; prompt handles init, sandbox handles security, retry handles errors |
|
||
| `backend/noop.go` (test backend) | Never used; tests use real backends or mocks |
|
||
| `backend/oneshot.go` | Never used; `generate` file handles one-shot |
|
||
| `toolsrv/schema.go` (schema validation) | Never enabled |
|
||
| `data/tools/route.meta` (model routing tool) | Replaced by explicit `/model` commands |
|
||
| TaskState subsystem | Removed entirely (−246 lines); no use case survived |
|
||
| `fifo.in` + `fifo.out` | Merged to single `fifo` (write=enqueue, read=dequeue) |
|
||
| `cost` + `usage` + `ctxsz` files | Merged to `stats` (key=value lines) |
|
||
| `connection`, `context`, `tail`, `offset`, `prompt.prev` agent files | Removed or moved to ctl |
|
||
| AllowTools field | Tools loaded dynamically; static allowlists served no purpose |
|
||
| PRIME_* env vars | Replaced by typed `Platform` + `IsGitRepo` fields |
|
||
| `elevate` naming | Renamed to `bypass` throughout |
|
||
| `superpowerd` (separate privilege escalation daemon) | Integrated into olliesrv as the bypass broker |
|
||
| In-process toolsrv (Go library) | Replaced by separate 9P server process with socket boundary |
|
||
| Binary ReadOnly/not tool batching | Replaced by scope-based conflict scheduling (read/write/global) |
|
||
| `AgentLog` / `SessionNode` / `RootState` abstractions in fs/ | Eliminated; state moved to where data lives (agent/, session/) |
|
||
| Named sandbox profiles (default, restricted, remote) | One config: `sandbox.yaml`. Multi-profile system was unused complexity |
|
||
| `TurnCtx` struct | Eliminated; loop functions became methods on `*Agent` |
|
||
| Bypass via custom Unix socket protocol | Replaced by bypass via 9P namespace |
|
||
| `.meta` as sidecar to an executable | `.meta` IS the tool definition; executable is optional (tool-definitions.md) |
|
||
| `subagent_generate` (JIT agent config tool) | Obsolete; sub-agents use existing profiles directly |
|
||
| Fire-and-forget `subagent_spawn` via D-Bus | Replaced by blocking `ollie-9p rdwr agent/new` — returns the reply |
|
||
| Sub-agents as peer agents (no parent, no return) | Replaced by task-scoped sub-agents with context inheritance and automatic cleanup |
|
||
| `cascade` orchestrator script | Replaced by parallel `subagent_spawn` calls (scope: read, natural parallelism) |
|
||
| Agent-loop batching as correctness mechanism | Moved to toolsrv path-lock table; agent batching is now just an optimization |
|
||
| `virtfs.Request` naming | Renamed to `Rdwr` — it's an atomic operation, not a variant of read or write |
|
||
| One-tool bootstrap (Phase 36) | Models skip the load step and call tools directly; explicit `autoLoad` per profile replaced it |
|
||
| Lazy tool loading | Capability boundaries must be explicit; model compliance is not a security boundary |
|
||
| `client_9p` tool hints with load instructions | Removed — tool hints now show loaded tools only |
|
||
| `skills.Index` shared for skills and tools | Replaced by generic `embedding.Index[T]` with separate skill_match.go and tool_match.go |
|
||
|
||
## Feed file + Observer agents + BlockOnce/Stream refactor (Aug 13)
|
||
|
||
Three related changes in one session. Net **+278 SLOC** across 21 files.
|
||
|
||
### What was added
|
||
|
||
**`feed` file** — a change-detecting blocking read file on every agent. Write data in, internal consumer submits it as a prompt. Dedup built into the `BlockOnce` handler: same data written twice never wakes the reader. Enables real-time pair programming — an observer agent watches a human or another agent code.
|
||
|
||
**Observer agent** (`data/agents/observer.json`, `data/prompts/agent-observer.md`) — read-only agent profile. Only loads `file_read`, `file_grep`, `file_glob`, `reasoning_think`. Prompt explicitly forbids writes. Receives diffs via feed, makes terse observations.
|
||
|
||
**`ConsumeFeed`** — plain function that dials the 9P server via `lib9p` and reads from the agent's own feed file in a loop. Each read blocks until feed changes. Same pattern as `o read -l`.
|
||
|
||
### What was refactored
|
||
|
||
**`virtfs.BlockOnce(readFn, signalFn)`** — previously a raw handler that took `(ctx, base)` and had to implement its own blocking. Now the framework handles the block-until-changed loop: call `readFn()`, compare hash to base, wait on `signalFn()` channel, repeat. Handlers become two-line closures.
|
||
|
||
**`virtfs.Stream(readFn, signalFn)`** — same refactor. `streamChat` (40 lines of condvar + mutex + offset tracking) replaced by `a.ChatRead` + `a.ChatSignal` passed to `Stream(...)`.
|
||
|
||
**Server timeout fallback** — when `BlockOnce` times out (5s, no change), the server falls back to the plain `Read` handler. Frontends get the current value as a heartbeat. Files without `Read` (like feed) return empty.
|
||
|
||
### What was removed
|
||
|
||
- `streamChat()` in support.go (replaced by `ChatRead` method on agent)
|
||
- `mergeCtx()` in support.go (no longer needed — blocking logic moved into virtfs framework)
|
||
- `ObserverFeed` script (replaced by the `feed` file)
|
||
- `WaitChange` usage outside filesystem handlers (feed consumer uses lib9p instead)
|
||
|
||
### What was fixed
|
||
|
||
- **XDG fallback** in prompt resolver — `$XDG_CONFIG_HOME` now defaults to `$HOME/.config` when unset
|
||
- **Duplicate tool headers** — `renderTools` skipped the generated `## name` when the tool prompt already has one
|
||
- **FIFO drain** — queued prompts no longer orphaned on interrupt/panic/toolsrv failure
|
||
- **Session restore ordering** — moved after 9P listener starts so feed consumers can connect
|
||
|
||
### Dead ends (killed during the session)
|
||
|
||
| Attempt | Why killed |
|
||
|---|---|
|
||
| Custom channel-based Feed with its own signal infrastructure | Duplicated the agent's existing signalCh mechanism |
|
||
| `Stream` mode for feed | Feed is a discrete value, not a byte stream |
|
||
| Internal WaitChange-based consumer | Leaked internal plumbing; should be a plain 9P client |
|
||
| `BlockOnceRaw` as primary API | Forced handlers to implement blocking themselves |
|
||
| Concurrent-by-default tool execution | Rejected: turns one reasoning step into N generation cycles with partial info — sequential but slower and more expensive |
|
||
|
||
## Sub-Agents via 9P rdwr (Aug 14)
|
||
|
||
Sub-agents implemented in **42 net lines of code**. The entire feature is
|
||
wiring — no new infrastructure, no new concepts, no new processes.
|
||
|
||
### The mechanism
|
||
|
||
`session/{sname}/agent/new` is an `Rdwr` file. Without `prompt=`, it creates
|
||
a persistent agent and returns its ID (existing behavior). With `prompt=`, it
|
||
enters sub-agent mode: creates a transient agent, submits the prompt, blocks
|
||
until the agent finishes, returns the reply, and destroys the agent.
|
||
|
||
```
|
||
printf 'cwd=%s\nprompt=fix the tests\n' "$PWD" \
|
||
| ollie-9p rdwr session/$OLLIE_SESSION_ID/agent/new
|
||
```
|
||
|
||
### Why it works in 42 lines
|
||
|
||
The infrastructure was already there:
|
||
- **`Rdwr` file primitive** (née `Request`) — atomic write-then-read, per-open isolation
|
||
- **`Agent.Submit()`** — already blocks through the full turn
|
||
- **`Agent.Reply()`** — already captures the final response
|
||
- **`Session.RemoveAgent()`** — already handles cleanup
|
||
- **Event bus** — normal agent lifecycle events fire (GUI sees sub-agents appear/disappear)
|
||
|
||
The handler is just: parse params → create agent → submit → read reply → destroy → return.
|
||
|
||
### Path-based lock table
|
||
|
||
Prerequisite: moved conflict serialization from the agent loop into toolsrv
|
||
itself. All foreground tool calls now acquire a path-based lock before execution:
|
||
|
||
- `scope "read"` — no lock (never conflicts)
|
||
- `scope "write"` — exclusive lock on the file path
|
||
- `scope "global"` — exclusive global lock
|
||
|
||
This ensures sub-agents writing to the same file serialize correctly regardless
|
||
of which agent initiated the call. The agent loop's batching remains as a
|
||
performance optimization; correctness is enforced at the execution layer.
|
||
|
||
### Parallel dispatch
|
||
|
||
`subagent_spawn` is declared `scope: "read"` — multiple spawn calls in the same
|
||
turn run in parallel automatically. The calling agent blocks until all results
|
||
return. No shell backgrounding, no polling, no coordination code.
|
||
|
||
```mermaid
|
||
sequenceDiagram
|
||
participant P as Parent Agent
|
||
participant S1 as Sub-Agent 1
|
||
participant S2 as Sub-Agent 2
|
||
|
||
P->>S1: subagent_spawn(prompt="task A")
|
||
P->>S2: subagent_spawn(prompt="task B")
|
||
Note over P: blocked (parallel tool calls)
|
||
S1-->>P: reply A
|
||
S2-->>P: reply B
|
||
Note over P: resumes with both results
|
||
```
|
||
|
||
### What was killed
|
||
|
||
- **`subagent_generate`** — JIT agent profile generation tool. Obsolete: sub-agents
|
||
use existing profiles directly.
|
||
- **D-Bus spawn path** — `subagent_spawn` previously used `dbus-send` to create
|
||
sessions. Replaced by a 3-line `ollie-9p rdwr` call.
|
||
|
||
### Framework rename: `Request` → `Rdwr`
|
||
|
||
The three atomic 9P operations are now named for what they are:
|
||
- `Read` — non-blocking read
|
||
- `Write` — non-blocking write (fire-and-forget)
|
||
- `Rdwr` — atomic write-then-read (blocking, produces result)
|
||
|
||
`BlockOnce` and `Stream` are special cases of `Read`.
|
||
`Rdwr` is its own primitive — not a variant of either.
|
||
|
||
### Architecture validation
|
||
|
||
This feature is evidence that the "everything is a file" architecture works at
|
||
scale. A major capability (parallel sub-agents with automatic serialization)
|
||
landed as pure wiring because:
|
||
|
||
1. The 9P namespace already exposes `agent/new` as an rdwr file
|
||
2. The agent loop already blocks and produces results
|
||
3. The tool server already serializes conflicting writes by path
|
||
4. The batching algorithm already parallelizes non-conflicting calls
|
||
|
||
No new abstractions. No new processes. No new protocols. Just connecting
|
||
existing pieces with 42 lines of glue.
|
||
|
||
### Context inheritance and isolation
|
||
|
||
A sub-agent gets its own context window, history object, runtime state, tool
|
||
registry, cache, step budget, and lifecycle. When spawned from an existing
|
||
agent, the parent conversation messages are copied into the child's initial
|
||
history via `RestoreHistoryFromMessages`.
|
||
|
||
This is a one-time snapshot, not a live shared context. The child cannot append
|
||
to, rewrite, or otherwise mutate the parent's history. The parent does not see
|
||
child messages while the child runs. The child reports only its final reply
|
||
through the blocking `subagent_spawn` result; the parent decides whether and
|
||
how to incorporate that reply into its own context.
|
||
|
||
This separation is deliberate. Shared conversation mutation would create
|
||
ordering races, prompt contamination, and unclear ownership of tool results.
|
||
Shared workspace resources remain coordinated independently through toolsrv's
|
||
scope and path locks.
|
||
|
||
### Future work
|
||
|
||
- **Shared toolsrv per host** — currently each session spawns its own toolsrv.
|
||
Cross-session path serialization requires sessions to share a single toolsrv
|
||
instance per host.
|
||
- **Budget/depth controls** — token limits, step limits, and recursion depth
|
||
caps for sub-agents.
|
||
|
||
## Phase 31: The Current Integration Boundary (Aug 17)
|
||
|
||
The architecture was consolidated around explicit documentation and composition
|
||
boundaries. The 9P filesystem remains the public control plane, while the
|
||
agent core, prompt assembly, tool authoring, toolsrv, remote execution, and
|
||
virtfs are documented as separate responsibilities.
|
||
|
||
Common agent-framework features are deliberately not built into Ollie:
|
||
MCP clients, embedded tool frameworks, planner/executor workflow engines,
|
||
task graphs, schedulers, public coordination buses, frontend control planes,
|
||
distributed agent state, and competing memory stores. These are integration
|
||
points. They can be provided by tools, sessions, ordinary processes, or
|
||
external systems such as OptMem and Beads.
|
||
|
||
The current model is:
|
||
|
||
```text
|
||
agent → toolsrv → executable or metadata-only tool → external system/state
|
||
```
|
||
|
||
Ollie supplies the agent loop, sessions, prompts, and 9P observation/control
|
||
surface. The integration owns specialized state and semantics.
|
||
|
||
### Recent simplifications
|
||
|
||
The period after the previous evolution entry completed a broad flattening and
|
||
architecture cleanup:
|
||
|
||
- Removed the remaining obsolete editor integration and consolidated 9P
|
||
clients around the shared client implementation.
|
||
- Moved the toolsrv client into `olliesrv` and split toolsrv protocol and
|
||
metadata concerns into focused packages.
|
||
- Removed obsolete environment/path packages and merged path handling into
|
||
`util`; backend configuration became the single source for backend settings.
|
||
- Added agent-scoped plans and metrics queries to the 9P namespace.
|
||
- Consolidated tool metadata and tool authoring into one architecture; both
|
||
executable-plus-`.meta` tools and metadata-only `cmd` tools are valid.
|
||
- Replaced the old remote-execution, prompt-audit, EDSL, and 9P documents with
|
||
architecture-focused documents, then deduplicated their boundaries.
|
||
- Removed the legacy backend/provider environment fallbacks and the obsolete
|
||
environment sample. Backend credentials, endpoints, models, and compaction
|
||
models now live in `backends.conf`.
|
||
- Documented the internal session event bus as an implementation detail rather
|
||
than an external coordination API.
|
||
- Made the composition boundary explicit: plan-and-execute systems such as
|
||
Beads integrate through toolsrv; they are not Ollie runtime concerns.
|
||
- Linked [OptMem](https://github.com/VictorTaelin/OptMem) as the external
|
||
persistent-memory implementation.
|
||
|
||
### Current architecture
|
||
|
||
At this point the runtime has two services and one integration surface:
|
||
|
||
```text
|
||
9P clients
|
||
│
|
||
▼
|
||
olliesrv ── sessions, agents, prompts, history, backends
|
||
│
|
||
└─ authenticated 9P ── toolsrv ── registry, sandbox, processes
|
||
└─ executable or metadata-only tools
|
||
```
|
||
|
||
`olliesrv` owns the agent runtime. It constructs the `virtfs` tree, serves the Ollie namespace, manages
|
||
sessions and agents, resolves prompts, calls model backends, maintains history,
|
||
and runs the agent loop. Its internal session event bus is used for runtime observers; it is not an external API.
|
||
|
||
`toolsrv` is a separate authenticated 9P service. It owns tool metadata and
|
||
host-variant discovery, per-agent loading, command execution, process state,
|
||
output limits, sandbox policy, and bypass approval. Tool execution is not an
|
||
in-process agent capability. A session may use a local toolsrv or an SSH-forwarded
|
||
remote toolsrv; the agent runtime remains local.
|
||
|
||
The 9P namespace is the external integration surface. Clients create sessions
|
||
and agents, write prompts, read chat, wait on state, feed observers, and issue
|
||
control commands through files such as:
|
||
|
||
```text
|
||
session/new
|
||
session/{sname}/agent/new
|
||
session/{sname}/agent/{aname}/prompt
|
||
session/{sname}/agent/{aname}/chat
|
||
session/{sname}/agent/{aname}/statewait
|
||
session/{sname}/agent/{aname}/feed
|
||
session/{sname}/agent/{aname}/ctl
|
||
generate
|
||
```
|
||
|
||
A request follows this path:
|
||
|
||
```text
|
||
client → 9P → olliesrv → agent loop → toolsrv → tool
|
||
└──────────────→ backend
|
||
```
|
||
|
||
The client can be a shell command, editor integration, GUI, terminal, observer,
|
||
or external workflow. The client supplies context and writes a prompt; Ollie
|
||
supplies the runtime and 9P state surface. `virtfs` separates namespace
|
||
declaration from 9P transport. Tool authoring, remote execution, prompting, and
|
||
editor integration remain separate concerns documented by the focused
|
||
architecture documents.
|
||
|
||
## Phase 32: Goals, Workflows, and Index Split (Aug 17)
|
||
|
||
Session-level goals and the conductor workflow pattern became first-class features.
|
||
The session and agent index formats were split for cleaner parsing and better
|
||
sub-agent tree support in the GUI.
|
||
|
||
### Goals and Workflows
|
||
|
||
Sessions gained goal-related files:
|
||
- `session/{s}/goal` — write goal text to trigger a workflow
|
||
- `session/{s}/goalstatus` — read/write status (running/complete/blocked)
|
||
- `session/{s}/goalwait` — block until status changes
|
||
|
||
Writing to `goal` triggers `runWorkflow()` if status is empty, complete, blocked,
|
||
or error. The workflow script (default: `conductor`) creates a conductor agent
|
||
that reads the goal, explores the codebase, breaks work into tasks, and delegates
|
||
to sub-agents via `subagent_spawn`.
|
||
|
||
The conductor agent profile includes code intelligence tools (`code_outline`,
|
||
`code_symbols`, `code_query`, `codebase_overview`) and LSP tools (`lsp_definition`,
|
||
`lsp_references`, etc.) for understanding the codebase before delegating.
|
||
|
||
### Index Format Split
|
||
|
||
The single `session/idx` line that crammed session and agent data together was
|
||
split into two files:
|
||
|
||
**`session/idx`** — session list only:
|
||
```
|
||
session-id\tsession-name\tpaused\tconnected\tremote\tcwd
|
||
```
|
||
|
||
**`session/{s}/agent/idx`** — agent tree per session:
|
||
```
|
||
session-id\tagent-id\tagent-name\tparent-id\tdepth\tstate
|
||
```
|
||
|
||
This enabled:
|
||
- Simpler GUI parsing with no embedded semicolon-separated agent lists
|
||
- Proper sub-agent tree display with parent-id and depth
|
||
- Expandable/collapsible agent hierarchy in the session tree
|
||
- Auto-selection of top-level agents (depth 0) instead of first-in-list
|
||
|
||
### GUI Sub-Agent Tree
|
||
|
||
The KDE GUI's `SessionModel` gained:
|
||
- `hasChildren` role for agents with sub-agents
|
||
- `m_agentExpanded` map for tracking agent expansion state
|
||
- Recursive `appendAgentTree` that respects parent expansion
|
||
- Expand/collapse arrows for agents with children
|
||
|
||
### Bug Fix: ollie-9p UNAME Fallback
|
||
`ollie-9p` previously required `$OLLIE_UNAME` when `$OLLIE_SESSION_ID` was set,
|
||
making it unusable from non-agent contexts. The fallback to `$USER` was added,
|
||
allowing normal users to read session files like `goal` and `goalstatus`.
|
||
|
||
## Phase 33: Native Landlock and Remote Deployment Simplification (Current)
|
||
|
||
Sandbox enforcement moved from an external sandbox executable into the
|
||
`toolsrv` binary. The policy source remains the runtime `sandbox.yaml`, so
|
||
permissions can still change without recompiling or redeploying the policy.
|
||
|
||
Each restricted tool execution starts a short-lived `sandbox-exec` helper mode
|
||
inside `toolsrv`. The helper loads the policy snapshot, sets `no_new_privs`,
|
||
creates a Landlock ruleset, grants the configured filesystem permissions, and
|
||
executes the tool. Restrictions are applied to the child rather than the
|
||
long-lived toolsrv process because Landlock rules are inherited and cannot be
|
||
removed.
|
||
|
||
The external sandbox dependency was removed from local and remote execution.
|
||
Remote bootstrap now transfers only the toolsrv binary and runtime
|
||
configuration. It no longer locates, copies, or installs a separate sandbox
|
||
executable. Remote toolsrv instances enforce the same native policy on the
|
||
remote host, while `--yolo` remains the explicit way to disable enforcement.
|
||
|
||
The project was also described more precisely as a distributed, integrating AI
|
||
agent runtime: orchestration and model calls remain local while toolsrv,
|
||
tools, and external systems can be composed locally or across remote hosts.
|
||
|
||
## Phase 34: Agent Peers and Consensus Workflow (Aug 19)
|
||
|
||
Inter-agent communication gained a first-class mechanism: **peer links**.
|
||
Previously, agents in the same session could only be addressed from external
|
||
scripts or through sub-agent spawn (which is transient). The peer system adds
|
||
persistent, topology-controlled messaging between agents within a session.
|
||
|
||
### Peer mechanism
|
||
|
||
Each agent exposes a `peer/` directory in its 9P namespace. Entries are
|
||
write-only files named after peer agents. Writing to `peer/{name}` delivers
|
||
the message to the named agent's prompt handler — identical semantics to
|
||
writing to that agent's `prompt` file, but scoped by the peer relationship.
|
||
|
||
Peer links are **bidirectional**: `peeradd A` on agent B also adds B to A's
|
||
peer set. This is enforced in the `peeradd`/`peerdel` ctl commands. The
|
||
topology constrains who can talk to whom — if an agent has no peer link to
|
||
another, it has no `peer/{name}` file and cannot message it. This is the
|
||
access control surface.
|
||
|
||
Implementation details:
|
||
- Agent struct: `peers map[string]struct{}` + RWMutex, lazy-init
|
||
- Methods: `AddPeer`, `RemovePeer`, `Peers` (sorted)
|
||
- `peer/` declared as a `virtfs.Each` node — dynamic directory rebuilt from peer set
|
||
- `peeradd`/`peerdel`/`peers` ctl commands with bidirectional enforcement
|
||
- Peer cleanup on agent removal: `RemoveAgent` strips the dead agent from all remaining peer sets
|
||
- Peers persisted in `PersistedAgent.Peers` field; restored after all agents are created
|
||
- `peeradd`/`peerdel` trigger immediate `s.Save()` for durability
|
||
|
||
Total Go additions: ~55 lines in agent.go, ~50 lines in spec.go, ~15 lines in persist.go and session.go.
|
||
|
||
### Consensus workflow
|
||
|
||
The first workflow to use peers: `consensus`. Unlike the conductor (which
|
||
decomposes a goal into sequential/parallel subtasks), consensus runs the
|
||
**same task N times independently** and synthesizes agreement.
|
||
|
||
The workflow creates:
|
||
- 1 **foreman** agent (profile: foreman, temp 0.3) — synthesis only, no investigation
|
||
- N **panelist** agents (profile: panelist, temp 0.7) — independent analysis
|
||
|
||
Each panelist is linked as a peer of the foreman (bidirectional). Panelists
|
||
cannot message each other — the topology enforces the protocol.
|
||
|
||
Flow:
|
||
1. Workflow script creates agents, establishes peer links, primes all
|
||
2. Each panelist investigates the goal from a different angle (correctness, simplicity, edge cases)
|
||
3. Panelists write findings to `peer/foreman` when done
|
||
4. Each delivery triggers a turn on the foreman
|
||
5. After receiving all N reports, the foreman synthesizes consensus in its chat output
|
||
6. Foreman writes "complete" to goalstatus
|
||
|
||
The consensus output lives in the foreman's chat — the goal file is never
|
||
overwritten. This respects the separation: goal = what to do, chat = what
|
||
was learned.
|
||
|
||
### Design insight
|
||
|
||
Peers vs sub-agents is not a replacement — it's a complementary primitive:
|
||
- **Sub-agents** are stateless, transient, fire-and-forget. Good for decomposition.
|
||
- **Peers** are persistent, stateful, conversational. Good for deliberation.
|
||
|
||
The conductor workflow uses sub-agents (decompose → delegate → collect).
|
||
The consensus workflow uses peers (investigate independently → report → synthesize).
|
||
Both are wiring over the same 9P primitives.
|
||
|
||
## Phase 35: Embedding-Guided Tool and Skill Discovery (Aug 20)
|
||
|
||
Ollie added a local embedding subsystem for semantic discovery of tools and
|
||
skills. It loads the `all-MiniLM-L6-v2` sentence-transformer through ONNX
|
||
Runtime, tokenizes descriptions, produces 384-dimensional vectors, and ranks
|
||
matches with cosine similarity.
|
||
|
||
The skill index scans configured `SKILL.md` directories, parses their
|
||
frontmatter, precomputes description embeddings, and matches each user request
|
||
against the indexed skills. The agent injects up to three relevant skills when
|
||
they exceed the configured similarity threshold. Tool metadata is indexed by
|
||
the same mechanism, allowing the agent to inject up to five relevant tool hints
|
||
without placing every tool in the model context.
|
||
|
||
The embedding model and ONNX Runtime are installed under the XDG data model
|
||
directory by `make install-models` / `make install-data`. Skill directories may
|
||
be overridden through `embedding.conf`; the default is the installed skills
|
||
directory. Matching is lazy and cached per process, so the common path pays the
|
||
model-loading cost once.
|
||
|
||
This is a significant architectural shift for small models: discovery becomes
|
||
semantic rather than dependent on exact tool or skill names, while progressive
|
||
disclosure keeps the prompt bounded. The embedding model is local and separate
|
||
from the conversational backend, so provider choice does not affect discovery.
|
||
|
||
---
|
||
|
||
## Phase 36: One-Tool Bootstrap and Progressive Capability Loading (Aug 21)
|
||
|
||
The default agent profile now starts with exactly one callable tool: `client_9p`.
|
||
The previous static `autoLoad` list was removed. This makes the runtime
|
||
capability surface explicit: the agent can begin with the namespace client and
|
||
load everything else through its own agent `ctl` file.
|
||
|
||
Semantic matching remains a hint, not an implicit capability grant. When a
|
||
request matches a tool description, the agent receives the tool name and the
|
||
9P operation required to load it. The model calls `client_9p` with:
|
||
|
||
```text
|
||
rdwr session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/ctl
|
||
tool_load <name>
|
||
```
|
||
|
||
The agent runtime refreshes its model-facing schemas after a load or unload.
|
||
`toolsrv` exposes an agent-scoped `tools_rev` request/response file so the
|
||
client can detect registry changes without rebuilding tool state on every tool
|
||
round. A revision of zero is treated as unavailable and preserves the safe
|
||
refresh behavior.
|
||
|
||
This completes the progressive-disclosure path:
|
||
|
||
```text
|
||
client_9p → agent ctl → toolsrv registry → loaded tool schema → tool call
|
||
```
|
||
|
||
The important boundary is unchanged. `olliesrv` still owns prompting and the
|
||
agent loop; `toolsrv` remains authoritative for discovery, loading, execution,
|
||
sandboxing, and process state. The change removes ambient startup capability
|
||
without adding a new orchestration subsystem.
|
||
|
||
The `client_9p` script also expands `$OLLIE_SESSION_ID` and `$OLLIE_UNAME` in
|
||
virtual namespace paths. This is required because JSON argument parsing does not
|
||
perform shell expansion, and keeps the documented namespace examples directly
|
||
callable by the model.
|
||
|
||
## Phase 37: Explicit Tool Loading and Generic Embedding Index (Aug 21)
|
||
|
||
Lazy tool loading — where the agent could call any tool by name and have it
|
||
loaded automatically — was removed. Tools now require explicit `autoLoad`
|
||
declarations in agent configs. This is a reversal of the Phase 36 progressive
|
||
loading experiment.
|
||
|
||
### Why lazy loading failed
|
||
|
||
The one-tool bootstrap approach (Phase 36) relied on semantic hints telling the
|
||
model to load tools through `client_9p`. In practice:
|
||
|
||
1. **Models called tools directly instead of loading them first.** Even with
|
||
explicit instructions, models would attempt to call `file_read` or `shell`
|
||
without the intermediate `client_9p tool_load` step.
|
||
|
||
2. **The indirection added latency and context cost.** Every tool use required
|
||
an extra round-trip: hint → model decides to load → load call → refresh →
|
||
actual tool call. This doubled the turns for common operations.
|
||
|
||
3. **Capability boundaries became unclear.** An agent's effective capability
|
||
was "whatever it decides to load," which is not the same as "what it's
|
||
configured to do." A code-review agent shouldn't have `shell` access just
|
||
because it asked for it.
|
||
|
||
### The fix: explicit autoLoad per profile
|
||
|
||
Each agent profile now declares exactly which tools it starts with:
|
||
|
||
```json
|
||
{
|
||
"name": "default",
|
||
"autoLoad": ["shell", "file_read", "file_edit", "file_grep", "file_glob",
|
||
"file_write", "lsp_hover", "lsp_definition", "lsp_references",
|
||
"lsp_symbols", "lsp_diagnostics", "lsp_completion", "lsp_rename",
|
||
"code_outline", "code_symbols", "code_query", "codebase_overview",
|
||
"code_dependencies", "code_rewrite", "client_9p", "subagent_spawn",
|
||
"skill_list", "skill_load", "memory_wake", "memory_remember",
|
||
"memory_recall", "memory_zoom", "web_fetch", "reasoning_think"]
|
||
}
|
||
```
|
||
|
||
Role-specific profiles (conductor, reviewer, panelist, etc.) load only what
|
||
they need. An observer loads read-only tools. A foreman loads coordination
|
||
tools. This makes capabilities explicit and auditable.
|
||
|
||
### Generic embedding index
|
||
|
||
The embedding package gained a generic `Index[T]` type that replaces the
|
||
skills-specific `skills.Index`. The same index structure now handles both
|
||
skill matching (entries are `skills.Skill`) and tool matching (entries are
|
||
`toolsrv.Meta`).
|
||
|
||
```go
|
||
type Index[T any] struct {
|
||
model *Model
|
||
entries []T
|
||
vectors []Vector
|
||
textFn func(T) string
|
||
}
|
||
```
|
||
|
||
Tool matching was split from skill matching:
|
||
- `skill_match.go` — builds skill index once per process, matches per turn
|
||
- `tool_match.go` — builds tool index per turn from loaded tools only
|
||
|
||
The tool index now reflects the agent's actual loaded tools, not all installed
|
||
metadata. This aligns semantic discovery with the explicit loading model.
|
||
|
||
### Agent config alignment
|
||
|
||
All 14 agent configs were updated to declare tools appropriate to their roles:
|
||
|
||
| Profile | Purpose | Key tools |
|
||
|---------|---------|-----------|
|
||
| default | General coding | Full tool set |
|
||
| conductor | Task decomposition | Code intel + subagent_spawn |
|
||
| reviewer | Code review | Read-only + LSP |
|
||
| foreman | Consensus synthesis | Coordination only |
|
||
| panelist | Independent analysis | Read + reasoning |
|
||
| observer | Watch and comment | Read-only |
|
||
| driver | Remote execution | Shell + file tools |
|
||
| copilot | IDE assistance | Code intel + LSP |
|
||
| writer | Documentation | File tools + reasoning |
|
||
| researcher | Investigation | Read + web + reasoning |
|
||
| planner | Architecture | Read + reasoning |
|
||
| debugger | Troubleshooting | Full tool set |
|
||
| tester | Test writing | Code + shell |
|
||
| refactorer | Code transformation | Code + LSP + rewrite |
|
||
|
||
### What was removed
|
||
|
||
- `load-on-call` tool loading (the Phase 36 mechanism)
|
||
- `client_9p` load hints in tool-hints injection
|
||
- `skills.Index` (replaced by generic `embedding.Index[T]`)
|
||
- Combined skill/tool matching in `skill_match.go`
|
||
|
||
### Source changes
|
||
|
||
```text
|
||
embedding/index.go +89 new generic Index[T]
|
||
skills/skills.go -70 removed Index, kept Skill type
|
||
skill_match.go -40 removed tool matching
|
||
tool_match.go +55 new per-turn tool index
|
||
data/agents/*.json +14 autoLoad declarations
|
||
```
|
||
|
||
## Phase 38: Streaming Rdwr and Event Filtering (Aug 22)
|
||
|
||
A new 9P primitive for filtered event subscriptions, and cleanup of unused blocking mechanisms.
|
||
|
||
### The streaming rdwr pattern
|
||
|
||
The `event` file gained write-then-stream semantics: write a filter pattern, then stream matching events. This is a new 9P interaction pattern — atomic write followed by indefinite streaming read on the same open fid.
|
||
|
||
```sh
|
||
# Subscribe to agent state changes only
|
||
echo "session.*.agent.*.state" | ollie-9p rdwrs event
|
||
```
|
||
|
||
Filter syntax uses `*` to match one segment and `>` to match all remaining segments:
|
||
- `session.*.agent.*.state` — all agent state changes
|
||
- `session.abc123.>` — all events for one session
|
||
- `*` alone matches everything (default behavior)
|
||
|
||
Implementation uses per-fid state in the server:
|
||
- `eventFilter` stores the compiled pattern
|
||
- `eventCh` receives matching events
|
||
- `eventCancel` cleans up on fid clunk
|
||
|
||
The `ollie-9p` client gained an `rdwrs` command for streaming rdwr operations.
|
||
|
||
### GUI event consolidation
|
||
|
||
The KDE GUI switched from per-agent statewait streams to a single server-wide event stream. This reduced connection overhead and simplified the streaming architecture. The GUI now:
|
||
- Opens one `event` stream per daemon connection
|
||
- Parses event topics to dispatch state updates
|
||
- Handles bypass requests through the same event path
|
||
|
||
### What was removed
|
||
|
||
**`statewait` file** — replaced by the `event` stream with filtered subscriptions. The per-agent blocking read was superseded by the more flexible event filtering.
|
||
|
||
**`bypasswait` file** — redundant since bypass requests flow through the event stream as `session.{sid}.bypass.request` events.
|
||
|
||
**`feed` file and observer agents** — the feed mechanism (change-detecting BlockOnce input) was documented but never used by any frontend or script. The observer agent pattern was never adopted. Removed: `feed.go`, `FeedWrite`, `ConsumeFeed`, `WatchFeed`, and all related session wiring.
|
||
|
||
### Event topics
|
||
|
||
The event stream now carries all real-time state:
|
||
|
||
| Topic | Payload |
|
||
|-------|---------|
|
||
| `session.{sid}.new` | name |
|
||
| `session.{sid}.kill` | — |
|
||
| `session.{sid}.rename` | oldName newName |
|
||
| `session.{sid}.pause` | — |
|
||
| `session.{sid}.resume` | — |
|
||
| `session.{sid}.agent.{aid}.new` | — |
|
||
| `session.{sid}.agent.{aid}.kill` | — |
|
||
| `session.{sid}.agent.{aid}.state` | idle\|calling\|thinking\|paused |
|
||
| `session.{sid}.agent.{aid}.bypass.request` | id\tcmd\tcwd |
|
||
| `session.{sid}.agent.{aid}.bypass.resolved` | id\tapproved/denied |
|
||
|
||
### Source changes
|
||
|
||
```text
|
||
server.go +120 per-fid event filter state, write/read handlers
|
||
session/event.go +45 SubscribeEventsFiltered, MatchTopic
|
||
ollie-9p/main.go +25 rdwrs command
|
||
o (script) +10 filtered event subscription for tui
|
||
ollie9pclient.cpp -50 removed statewait streams
|
||
agent/feed.go -46 deleted
|
||
agent/agent.go -67 removed FeedWrite, ConsumeFeed
|
||
agent/state.go -8 removed WatchFeed
|
||
session/session.go -8 removed ConsumeFeed goroutines
|
||
fs/spec.go -35 removed feed, statewait, bypasswait files
|
||
```
|
||
|
||
Net: **-222 lines** of unused infrastructure removed.
|
||
|
||
## Phase 39: Bypass Coordination and State Indicators (Oct 6)
|
||
|
||
Cross-client bypass approval and real-time agent state visualization in the GUI.
|
||
|
||
### Bypass request/resolve coordination
|
||
|
||
The bypass system now emits events for both requests and resolutions, enabling CLI and GUI to coordinate:
|
||
|
||
**Request event** (existing): `session.{sid}.agent.{aid}.bypass.request` with payload `id\tcmd\tcwd`
|
||
|
||
**Resolved event** (new): `session.{sid}.agent.{aid}.bypass.resolved` with payload `id\tapproved/denied`
|
||
|
||
The GUI tracks pending bypasses per-agent in an in-memory collection (`m_pendingBypasses` QHash), enabling support for multiple concurrent pending bypasses (e.g., from background processes). When CLI resolves a bypass, the server emits the resolved event, and the GUI clears its state automatically.
|
||
|
||
### CLI bypass workflow
|
||
|
||
Added three commands to the `o` CLI wrapper:
|
||
|
||
```bash
|
||
o sess bypass # inspect pending: shows id, agent, cwd, cmd
|
||
o sess approve [id] # approve (optional id for safety)
|
||
o sess deny [id] # deny
|
||
```
|
||
|
||
The optional `id` argument prevents accidental approval of the wrong request when multiple bypass requests may have been issued.
|
||
|
||
### GUI indicators
|
||
|
||
**Pending bypass indicator** — Yellow ⚠ to the left of agent name in session tree. Visible when `pendingBypassCount > 0` for that agent. Clears automatically on resolution (by GUI buttons or CLI).
|
||
|
||
**Agent state indicator** — Colored dot to the right of agent name showing execution state:
|
||
- 🟢 Green: idle
|
||
- 🔵 Blue: thinking
|
||
- 🟠 Orange: calling tool
|
||
- ⚪ Gray: paused
|
||
|
||
Both indicators use relative geometry (percentages of row height) for proper scaling across font sizes and DPI settings.
|
||
|
||
### Source changes
|
||
|
||
```text
|
||
session/session.go +12 emit bypass.resolved event
|
||
ollie9pclient.h +8 m_pendingBypasses, pendingBypassCount, bypassResolved signal
|
||
ollie9pclient.cpp +45 handle bypass.resolved event, track pending counts
|
||
ChatPane.qml +12 onBypassResolved handler
|
||
SessionTree.qml +55 bypass indicator (⚠) and state indicator (dot)
|
||
data/scripts/o +65 bypass, approve, deny commands
|
||
```
|
||
|
||
## Phase 40: JSONL Chat Log Format (Oct 7)
|
||
|
||
Replaced the custom `[[[role#id]]]...[[[end]]]` delimiter format with JSON Lines (JSONL) — one JSON object per line. Fixes streaming partial display and eliminates brittle regex parsing.
|
||
|
||
### Old format (deleted)
|
||
|
||
```
|
||
[[[user#abc12345]]]
|
||
hello
|
||
[[[end]]]
|
||
[[[assistant#def67890]]]
|
||
response text
|
||
[[[end]]]
|
||
```
|
||
|
||
Problems:
|
||
- `[[[end]]]` must appear on its own line, but streaming chunks don't respect line boundaries
|
||
- Regex-based parsing is fragile and error-prone
|
||
- No standard tooling for reading/searching
|
||
|
||
### New format (JSONL)
|
||
|
||
```json
|
||
{"role":"user","id":"abc12345","content":"hello"}
|
||
{"role":"assistant","id":"def67890","content":"response text"}
|
||
{"role":"call","id":"aaa11111","name":"shell","content":"{\"cmd\":\"ls\"}"}
|
||
{"role":"tool","id":"bbb22222","content":"file1.txt\nfile2.txt","format":"text"}
|
||
```
|
||
|
||
Each line is a complete JSON object with:
|
||
- `role`: user, assistant, context, reasoning, call, tool, error, info, retry, stalled
|
||
- `id`: 8-char hex block ID (deterministic from sha256)
|
||
- `content`: block text
|
||
- `name`: tool/function name (for call blocks)
|
||
- `format`: output format hint (for tool blocks)
|
||
- `partial`: true if streaming in progress
|
||
|
||
### Server changes
|
||
|
||
**format/block.go** (new): `Block` struct with JSONL marshal/unmarshal, `RenderBlock` for filtered text output.
|
||
|
||
**Deleted**: `format/event.go`, `format/format.go`, `format/format_test.go` — old delimiter format entirely removed.
|
||
|
||
**agent/chat.go**: Dual logs — `rawLog` (JSONL for programmatic access) and `textLog` (rendered text for humans). `AppendBlock` writes to both. Streaming starts from beginning for replay.
|
||
|
||
**agent/chatlog.go**: Event handler emits JSONL blocks via `flushPartial` (partial=true per chunk) and `closeBlock` (partial=false final). Skips internal events: `state`, `usage`, `limitretry`.
|
||
|
||
**fs/spec.go**: New 9P files:
|
||
- `log.raw` — JSONL snapshot (one-shot read, for GUI startup)
|
||
- `log` — Rendered text snapshot (non-blocking, last 64KB)
|
||
- `chat` — Rendered text stream (blocking, for TUI)
|
||
- `block` — Rdwr lookup: write block ID, read JSON
|
||
|
||
Deleted: `chat.raw`, `chat.search`
|
||
|
||
### GUI changes
|
||
|
||
**chatblockmodel.cpp**: Complete rewrite. Uses `QJsonDocument` for parsing instead of regex state machine. Filters `context` role from display. Removed `m_inContext` tracking, `renderBlock` method, and regex patterns.
|
||
|
||
**ollie9pclient.cpp**: `readLogForSession` reads from `/log.raw` (JSONL) instead of `/log` (plain text). `log.raw` is a one-shot snapshot read, not a stream; live updates arrive via the separate `event` stream.
|
||
|
||
**chatblockmodel.h**: Added `Context` to `ChatBlock::Type` enum. Removed unused members.
|
||
|
||
### Context blocks
|
||
|
||
Context injected by the server (user prompts, matched skills, tool hints) now emits as a separate `"context"` role block, followed by the actual user message. Context blocks are:
|
||
- Sent to the LLM (wrapped in `<context>` tags in history)
|
||
- Filtered from GUI display (role-based, not pattern-based)
|
||
- Filtered from rendered text log via `RenderBlock`
|
||
|
||
### Source changes
|
||
|
||
```text
|
||
format/block.go +68 new Block struct, JSONL marshal/unmarshal, RenderBlock
|
||
format/event.go -67 deleted
|
||
format/format.go -107 deleted
|
||
format/format_test.go -188 deleted
|
||
agent/chat.go +95 dual logs, AppendBlock, BlockByID, streaming from start
|
||
agent/chatlog.go +35 JSONL emission, skip internal events
|
||
fs/spec.go +45 log.raw, log, block files; removed chat/chat.raw/chat.search
|
||
chatblockmodel.cpp -400 JSONL parsing, removed regex state machine
|
||
chatblockmodel.h +3 Context type, removed m_inContext
|
||
ollie9pclient.cpp +5 read from log.raw
|
||
```
|
||
|
||
Net: **-680 lines** — simpler, more robust, standard format.
|
||
|
||
## Phase 41: Authoritative Live Chat Stream (Oct 10)
|
||
|
||
Phase 40's JSONL `log.raw` doubled as both the persistent snapshot and the GUI's live stream. Because the JSONL buffer is append-only, `flushPartial` appended a *new* line for every streaming chunk — each carrying the full cumulative content — so a single assistant response left dozens of partial lines permanently in `log.raw`. The GUI, parsing the snapshot keyed by block ID, re-rendered the growing block once per partial line (O(N²) per response), and every reconnect replayed all historical partials. This produced the intermittent GUI rendering loops. It was fundamentally a data-input problem: partials were persisted, not just streamed.
|
||
|
||
### Fix: separate persistence from live delivery
|
||
|
||
- **Partials are no longer persisted.** `AppendBlock` writes only finalized blocks to `rawLog`/`textLog`. A new `SetPartial` records the current in-flight block in a single `partialLine` field (bumping `partialVers`) and broadcasts it to stream readers — it never touches `rawLog`. `chatlog.go`'s `flushPartial` now calls `SetPartial`; `closeBlock` still calls `AppendBlock`, which clears the partial.
|
||
- **`log.raw` is a finalized-only one-shot snapshot.** One line per logical block, bounded, no duplication. Used for explicit history loads (initial populate, bookmark reload), never polled.
|
||
- **`chat.raw` is the authoritative live JSONL stream** (`Agent.RawLogStream`, `StreamRaw`). On open it replays `rawLog` (finals) from the start, then streams live deltas: newly finalized blocks (offset-safe, since `rawLog` is append-only and never rewritten) and the current partial whenever it changes (out-of-band, keyed on `partialVers`). The GUI collapses by block ID, so reconnects are O(history), not O(streaming chunks). Base encodes `<rawOffset>:<partialVers>`.
|
||
|
||
### GUI
|
||
|
||
`ollie9pclient.cpp`: `startActiveAgentStreams` streams `/chat.raw` instead of `/log.raw`. The one-shot `readLogForSession` still reads `/log.raw` for `loadChat` and bookmark reloads — `log.raw` is now purely a snapshot, never a live source.
|
||
|
||
### Source changes
|
||
|
||
```text
|
||
agent/agent.go +4 partialLine/partialVers fields; cleared on Clear
|
||
agent/chat.go +60 SetPartial, signalChat, RawLogStream; AppendBlock finals-only
|
||
agent/chatlog.go ±1 flushPartial -> SetPartial
|
||
fs/spec.go +7 chat.raw StreamRaw file
|
||
ollie9pclient.cpp ±1 stream chat.raw, not log.raw
|
||
```
|
||
|
||
## Phase 42: Tool and Bypass Visibility in Text Views (Oct 10)
|
||
|
||
The rendered text views (`log`, `chat`) existed for clients without a JSONL parser or event stream — the TUI and the `o` script. But `RenderBlock` hid every `call` and `tool` block, so those views showed only user and assistant prose: no indication a tool ran, what it returned, or that a sandbox-bypass request was waiting for approval. Bypass requests were published solely as `event` topics, invisible to anyone reading `chat`.
|
||
|
||
### Changes
|
||
|
||
- **`format/block.go`**: `RenderBlock` now surfaces `call` (`→ name args`), `tool` (output), `bypass` (`⚠ bypass requested (needs approval): ...`), and `bypass-resolved` (`bypass approved`/`denied`) blocks. `reasoning` and `context` stay hidden; partials still render empty.
|
||
- **`session.go`**: `SetBypassPending` and `ResolveBypass` locate the requesting agent via `FindAgent(req.Env["OLLIE_UNAME"])` and append `bypass`/`bypass-resolved` chat blocks, so the pending-approval state and its outcome appear in `log`/`chat` alongside the existing `bypass.request`/`bypass.resolved` events. GUI clients, which read `chat.raw`, also receive these as plain-text blocks in addition to their dedicated approval dialog.
|
||
|
||
### Source changes
|
||
|
||
```text
|
||
format/block.go +12 render call/tool/bypass/bypass-resolved
|
||
format/block_test.go +72 new RenderBlock coverage
|
||
session.go +20 emit bypass chat blocks on request/resolve
|
||
agent/chat_test.go ±6 call content now expected in rendered log
|
||
AGENTS.md ±1 log/chat filtering description
|
||
```
|