docs: update architecture docs for Aug 1-2 refactors
Covers the major changes from the past two days:
- 9P Declarative EDSL (fs/spec.go + fs/builder.go, fs/session/ → fs/ flattening)
- 9P Server streamlining (fid map, GroupTable, method extraction)
- Multi-Agent Deepening (ID/Name split, UUIDv4, two-step creation, empty sessions)
- Eventwait redesign (global /eventwait ring buffer, structured delta events)
- Chat log [[[type]]]/[[[end]]] block format
- KDE submodule evolution (StreamFsm, responsive tree, emoji states, eventwait)
Updates: AGENTS.md, ARCHITECTURE.md, EVOLUTION.md, MULTI_AGENT.md,
CORE.md, edsl.md
This commit is contained in:
Ollie Agent
parent
bb152f401d
commit
6c6c133d76
|
|
@ -9,7 +9,10 @@ ollie/ ← you are here
|
|||
├── agent/ (Go) Agent loop, history, hooks, commands
|
||||
├── toolsrv/ (Go) Tool server, registry, sandboxed execution
|
||||
├── session/ (Go) Session lifecycle, config, persistence
|
||||
├── fs/session/ (Go) 9P filesystem tree
|
||||
├── fs/ (Go) 9P filesystem tree (EDSL-declared, flat package)
|
||||
│ ├── spec.go Single source of truth — entire namespace declared here
|
||||
│ ├── fsnode.go Core FsNodeDecl type + Dir/Leaf/TemplateDir constructors
|
||||
│ └── builder.go BuildTree — walks spec, validates, wires handlers
|
||||
├── cmd/ (Go) Binaries (olliesrv, ollie-9p, ollie-remote)
|
||||
├── tools/ (Go) Tool implementations:
|
||||
│ ├── builtin/ Built-in handlers (shell, reasoning, tool/skill registry)
|
||||
|
|
@ -87,6 +90,7 @@ just test-remote
|
|||
4. **Sandbox** (`sandbox/`): Landlock-based. Config in `sandbox/*.yaml` defines filesystem access per profile.
|
||||
5. **Backends** (`backend/`): Ollama, OpenAI-compatible, Anthropic, Copilot, Kiro, Gemini, CodeWhisperer. Selectable per-session.
|
||||
6. **Prompts assembled at runtime**: Agent JSON `prompt` array specifies which prompt files to concatenate. Static prompt files can be included directly; the base system prompt is embedded in the binary and always prepended.
|
||||
7. **9P namespace declared via EDSL**: The entire filesystem is a single recursive `FsNodeDecl` tree in `fs/spec.go`, built by `BuildTree()` in `fs/builder.go`. The `fs/` package is flat — no sub-package — handler files (`rootfiles.go`, `sessionfiles.go`, `agentfiles.go`) are organized by scope. See `doc/edsl.md` for the full reference.
|
||||
## Key Files
|
||||
| What | Where |
|
||||
|------|-------|
|
||||
|
|
@ -94,7 +98,8 @@ just test-remote
|
|||
| Tool server | `toolsrv/server.go` |
|
||||
| Remote execution | `toolsrv/remote.go`, `cmd/ollie-remote/` |
|
||||
| Sandbox enforcement | `sandbox/` |
|
||||
| 9P filesystem | `fs/` |
|
||||
| 9P filesystem (EDSL spec) | `fs/spec.go` |
|
||||
| 9P filesystem (builder) | `fs/builder.go` |
|
||||
| Session management | `session/session.go` |
|
||||
| System prompt template | Embedded in binary |
|
||||
| Agent configs | `agents/*.json` |
|
||||
|
|
|
|||
|
|
@ -2,21 +2,37 @@
|
|||
## Philosophy
|
||||
ollie's design philosophy is Emacs: a small, extensible core. The Go runtime (`agent.Agent`) defines what an agent *is* — an event loop, a backend connection, a message history, and three built-in primitives (`shell`, tool registry, skill registry). Everything else — orchestration, scheduling, workflows, UIs — is pushed out to the surrounding environment.
|
||||
The primary integration philosophy is Plan 9's "everything is a file." `olliesrv` exposes agent state and behaviors as files in a 9P namespace. Any program that can read and write files can drive an agent: shell scripts, editors, web apps, cron, containers.
|
||||
All clients communicate via 9P. Desktop notifications use D-Bus directly (org.freedesktop.Notifications) for elevation prompts only.
|
||||
All clients communicate via 9P exclusively. Desktop notifications use D-Bus directly (org.freedesktop.Notifications) for elevation prompts only — this is the last remaining D-Bus dependency.
|
||||
Design principles:
|
||||
- **Small extensible core.** The agent runtime is minimal; capabilities come from composing external scripts.
|
||||
- **Three built-in primitives.** `shell`, tool registry (`tool_load`/`tool_list`/`tool_active`), and skill registry (`skill_load`/`skill_list`/`skill_active`) are the only tools compiled into the core. Everything else is a script.
|
||||
- **One integration path: 9P filesystem. Streaming via blocking reads. No polling.
|
||||
- **9P namespace declared via EDSL.** The entire filesystem is a single recursive `FsNodeDecl` tree in `fs/spec.go`, validated and built by `BuildTree()` in `fs/builder.go`. Adding a file means adding a `Leaf()` call — no manual stat/readdir/write wiring.
|
||||
- **No framework lock-in.** Frontends are decoupled via whichever interface they prefer; the core doesn't know or care.
|
||||
## Repository Structure
|
||||
Single Go module (`ollie`) with two Git submodules for decoupled frontends:
|
||||
Single Go module (`ollie`) with one Git submodule (`kde/`) for the KDE frontend:
|
||||
```
|
||||
ollie/
|
||||
├── agent/ Agent loop, history, hooks, prompt resolution, commands
|
||||
├── backend/ LLM providers (Anthropic, OpenAI, Ollama, Gemini, Copilot, CodeWhisperer)
|
||||
├── toolsrv/ Tool server: sandboxed execution, tool registry, skill management
|
||||
├── session/ Session lifecycle (config, creation, persistence)
|
||||
├── fs/session/ 9P filesystem tree for sessions
|
||||
├── fs/ 9P filesystem: EDSL spec + handlers (flat package)
|
||||
│ ├── spec.go Namespace declaration (single source of truth)
|
||||
│ ├── fsnode.go FsNodeDecl type + Dir/Leaf/TemplateDir constructors
|
||||
│ ├── builder.go BuildTree — spec -> *Tree wiring
|
||||
│ ├── rootfiles.go Root-level handlers (backends, models, eventwait, complete, generate, route)
|
||||
│ ├── sessionfiles.go Session-level handlers (env, ctl, plan, agent/, ...)
|
||||
│ ├── agentfiles.go Agent-level handlers (prompt, chat, state, cfg, ...)
|
||||
│ ├── elevatefiles.go Elevation handlers (policy, pending)
|
||||
│ ├── procfiles.go Process handlers
|
||||
│ ├── lifecycle.go Session create/kill/rename/shutdown + event ring
|
||||
│ ├── newroot.go NewRoot — tree construction + persistence restore
|
||||
│ ├── persist.go Session persistence to disk
|
||||
│ ├── types.go Session/AgentLog types
|
||||
│ ├── tree.go 9P *Tree (from fs package)
|
||||
│ ├── fs.go 9P File/FileConfig implementations
|
||||
│ └── format.go Event formatting helpers
|
||||
├── detach/ Background process management (ring buffer, signal)
|
||||
├── elevate/ Elevation broker (privilege escalation daemon)
|
||||
├── sandbox/ Landlock sandbox config YAML
|
||||
|
|
@ -28,13 +44,14 @@ ollie/
|
|||
│ ├── olliesrv/ 9P server
|
||||
│ ├── ollie-9p/ 9P client
|
||||
│ └── ollie-remote/ Remote execution server
|
||||
├── kde/ KDE integration (submodule) — plasmoid, GUI, Kate plugin, KRunner, tray
|
||||
├── agents/ Agent config JSONs (default, coding, orchestrator, worker, ...)
|
||||
├── prompts/ Prompt templates (markdown)
|
||||
├── tools/ Tool scripts (file_read, lsp_*, memory_*, subagent_*, ...)
|
||||
├── skills/ Domain knowledge files (markdown)
|
||||
├── scripts/ User-facing scripts organized by namespace (s/, u/, x/)
|
||||
├── sandbox/ Sandbox config YAML files
|
||||
├── kde/ KDE integration (submodule) — standalone GUI, Kate plugin, KRunner
|
||||
├── data/agents/ Agent config JSONs (default, coding, orchestrator, worker, ...)
|
||||
├── data/prompts/ Prompt templates (markdown)
|
||||
├── data/tools/ Tool scripts + .meta sidecar files
|
||||
├── data/skills/ Domain knowledge modules (markdown)
|
||||
├── data/scripts/ Helper scripts (ollie-remount, ollie-watchdog)
|
||||
├── data/services/ Systemd/xdg-autostart service files
|
||||
├── prompts/ Embedded prompt templates (compiled into binary)
|
||||
└── doc/ Documentation
|
||||
```
|
||||
## System Overview
|
||||
|
|
@ -43,15 +60,17 @@ flowchart TB
|
|||
subgraph Frontends["Frontends"]
|
||||
ACME["acme (Plan 9)"]
|
||||
ELLIE["ellie (Emacs)"]
|
||||
KDE["KDE GUI / Plasmoid"]
|
||||
KDE["KDE GUI / Kate / KRunner"]
|
||||
SH["s/sh (terminal)"]
|
||||
HTTP["curl / scripts"]
|
||||
end
|
||||
subgraph Integration["Integration Layer"]
|
||||
direction LR
|
||||
P9["9P Filesystem"]
|
||||
P9["9P Filesystem\n(ollie-9p client)"]
|
||||
end
|
||||
subgraph Server["olliesrv"]
|
||||
direction TB
|
||||
NS["9P Namespace\nsession/ · backends · models"]
|
||||
NS["9P Namespace\nEDSL-declared in fs/spec.go"]
|
||||
end
|
||||
subgraph Core["Agent Engine (per session)"]
|
||||
LOOP["Agent Loop\n(agent/loop.go)"]
|
||||
|
|
@ -65,15 +84,13 @@ flowchart TB
|
|||
GEMINI["Gemini"]
|
||||
CW["CodeWhisperer"]
|
||||
end
|
||||
TUI --> P9
|
||||
SH --> P9
|
||||
ACME --> P9
|
||||
ELLIE --> P9
|
||||
KDE --> DBUS
|
||||
WEB --> DBUS
|
||||
KDE --> P9
|
||||
HTTP --> P9
|
||||
P9 --> NS
|
||||
DBUS --> DBA
|
||||
NS --> LOOP
|
||||
DBA --> LOOP
|
||||
LOOP <--> SESS
|
||||
LOOP --> TRV
|
||||
LOOP <--> Backends
|
||||
|
|
@ -172,61 +189,84 @@ Agent configs declare prompts as a JSON array of shell commands. Each command is
|
|||
`olliesrv` is the central daemon. It implements a 9P2000 file server that exposes the entire agent namespace:
|
||||
### Namespace Layout
|
||||
```
|
||||
session/
|
||||
├── new write key=value to create session
|
||||
├── idx session index (tab-separated)
|
||||
├── ls exec to list sessions
|
||||
├── kill exec to kill a session
|
||||
├── sh interactive shell
|
||||
├── b batch one-shot (create, submit, wait, print, kill)
|
||||
├── bfg batch foreground (submit to existing, wait, print)
|
||||
├── bbg batch background (submit, return path)
|
||||
├── cleanup kill idle sessions
|
||||
├── {sid}/
|
||||
│ ├── plan session-scoped markdown checklist
|
||||
│ ├── env session environment
|
||||
│ └── agent/
|
||||
│ └── {aid}/
|
||||
│ ├── cfg session config (key=value)
|
||||
│ ├── ctl control commands (stop, kill, compact, model, ...)
|
||||
│ ├── state current state (idle/thinking/calling)
|
||||
│ ├── statewait blocking read until state changes
|
||||
│ ├── chat full chat history (streamable)
|
||||
│ ├── offset byte offset after last user prompt
|
||||
│ ├── prompt write to submit a prompt
|
||||
│ ├── prompt.prev last submitted prompt
|
||||
│ ├── fifo.in queue a prompt (FIFO input)
|
||||
│ ├── fifo.out pop queued prompt (FIFO output)
|
||||
│ ├── context rendered context window
|
||||
│ ├── systemprompt rendered system prompt
|
||||
│ ├── usage token usage stats
|
||||
│ ├── cost estimated cost
|
||||
│ ├── ctxsz context size
|
||||
│ ├── models available models
|
||||
│ ├── tools tool management (read all, write name to load)
|
||||
│ ├── toolsrv.loaded currently loaded tools
|
||||
│ ├── toolsrv.rev revision counter
|
||||
│ ├── tail exec helper
|
||||
│ └── proc/ detached background processes
|
||||
/ ← root (owned by system user)
|
||||
├── backends backends list
|
||||
├── help help text
|
||||
├── models model list (cached)
|
||||
├── agents agent config list
|
||||
├── ctl root control (invalidate, kill)
|
||||
├── eventwait global event stream (blocking read)
|
||||
├── complete request-response: code completion
|
||||
├── generate request-response: one-shot LLM generation
|
||||
├── route request-response: model routing
|
||||
├── elevate/
|
||||
│ ├── policy global elevation policy
|
||||
│ └── pending/{id} pending elevation requests (approve/deny/persist)
|
||||
└── session/
|
||||
├── new write key=value to create session
|
||||
├── idx session index (one line per agent)
|
||||
├── {name}/
|
||||
│ ├── env session environment variables
|
||||
│ ├── ctl session control (kill, save, invalidate)
|
||||
│ ├── plan session-scoped markdown checklist
|
||||
│ ├── id immutable session UUID
|
||||
│ ├── name mutable session name (write to rename)
|
||||
│ ├── elevate per-session elevation policy
|
||||
│ └── agent/
|
||||
│ ├── new write config to create agent (rdwr)
|
||||
│ └── {aid}/
|
||||
│ ├── prompt submit a prompt
|
||||
│ ├── prompt.prev last submitted prompt
|
||||
│ ├── fifo.in queue a prompt
|
||||
│ ├── fifo.out pop queued prompt
|
||||
│ ├── chat streaming chat log (blocking read)
|
||||
│ ├── log last 64KB of chat (non-blocking)
|
||||
│ ├── state current state (idle/thinking/calling)
|
||||
│ ├── statewait blocking read until state changes
|
||||
│ ├── cfg agent config (key=value)
|
||||
│ ├── ctl agent control (stop, compact, ...)
|
||||
│ ├── cwd working directory
|
||||
│ ├── id immutable agent UUID
|
||||
│ ├── name mutable agent name
|
||||
│ ├── offset byte offset after last user prompt
|
||||
│ ├── usage token usage stats
|
||||
│ ├── cost estimated cost
|
||||
│ ├── ctxsz context size
|
||||
│ ├── models available models
|
||||
│ ├── systemprompt rendered system prompt
|
||||
│ ├── context rendered context window
|
||||
│ ├── tail exec helper for tailing chat
|
||||
│ ├── tools tool management (read all, write name to load)
|
||||
│ ├── toolsrv.loaded currently loaded tools
|
||||
│ ├── toolsrv.rev revision counter
|
||||
│ └── proc/{pid} detached process output
|
||||
```
|
||||
### 9P Interaction Model
|
||||
All agent interaction uses the `ollie-9p` client, which auto-discovers the server via `$NAMESPACE` and identifies the caller via `$OLLIE_UNAME`. The 9P filesystem exposes session state, control, and inter-agent communication as files:
|
||||
- Sessions are directories under `session/`
|
||||
- Session creation is a two-step process: `session/new` (name only) then `session/{name}/agent/new` (full config)
|
||||
- Sessions and agents have **immutable UUIDs** (`id` file) and **mutable human-readable names** (`name` file); writing to `name` renames the entry
|
||||
- Prompts, state, chat history, and config are readable/writable files
|
||||
- Streaming via blocking reads: `chat`, `statewait`, `eventwait` block the 9P read until data arrives
|
||||
- Permission enforcement uses 9P user principals
|
||||
### Permission Model (`fs/perm.go`)
|
||||
All permissions are declared in a single registry (`Perms`). Key design:
|
||||
- The entire namespace is declared as a single EDSL tree in `fs/spec.go` — see `doc/edsl.md`
|
||||
### Permission Model (`fs/spec.go`)
|
||||
Permissions are declared inline in the EDSL spec via `mode` and `GID()` options. There is no separate permission registry — the spec is the source of truth:
|
||||
- `prompt` is mode `0666` — group+other can write, owner (the agent) cannot
|
||||
- `chat` is mode `0444` — read-only for everyone
|
||||
- `ctl` is mode `0666` — anyone can send control commands
|
||||
- `plan` is mode `0666` — readable by peer agents
|
||||
### Session Management (`fs/session/`)
|
||||
- `eventwait` is `0444` — blocking read for global event stream
|
||||
- `session/new` is `0666` with `Request` handler — anyone can create sessions
|
||||
- Ownership inherits: empty `UID`/`GID` values inherit from parent node, all the way up to root
|
||||
### Session Management (`fs/`)
|
||||
The `*fs.Tree` IS the session collection. Package functions manage lifecycle:
|
||||
- `NewRoot(cfg)` — create the root tree
|
||||
- `NewRoot(cfg)` — create the root tree from the EDSL spec (`fs/spec.go` → `BuildTree` → wired `*Tree`)
|
||||
- `Lookup(tree, id)` — find a session by ID
|
||||
- `All(tree)` — list all sessions
|
||||
- `CreateFromRoot(tree, args)` — create a new session
|
||||
- `CreateFromRoot(tree, args)` — create a new session (two-step: session name, then agent creation)
|
||||
- `KillFromRoot(tree, id)` — kill a session
|
||||
- `RenameFromRoot(tree, old, new)` — rename a session
|
||||
- `Shutdown(tree)` — clean shutdown
|
||||
The adapter is best-effort: if no session bus is available (e.g., headless/container), `olliesrv` continues functioning as a pure 9P server.
|
||||
- `Shutdown(tree)` — clean shutdown (interrupt all, wait for idle, persist, close)
|
||||
|
||||
Event streaming uses a global ring buffer (`eventRing`) with a global `/eventwait` file. Events carry prefixes (`S` for session, `A` for agent) with structured descriptions (`new`, `kill`, `rename`). Frontends block on `/eventwait` and receive delta events without polling.
|
||||
|
|
|
|||
23
doc/CORE.md
23
doc/CORE.md
|
|
@ -57,17 +57,22 @@ flowchart TB
|
|||
SESS["session.go\nSession, Config"]
|
||||
end
|
||||
|
||||
subgraph FsSession["fs/session/ package"]
|
||||
TREE["tree.go\n*fs.Tree"]
|
||||
ROOT["root.go\nNewRoot()"]
|
||||
CREATE["create.go\nCreateFromRoot()"]
|
||||
FILES["files.go\nfile operations"]
|
||||
PERM["perm.go\npermission registry"]
|
||||
subgraph Fs["fs/ package"]
|
||||
TREE["tree.go\n*Tree"]
|
||||
NEWROOT["newroot.go\nNewRoot()"]
|
||||
LIFECYCLE["lifecycle.go\nCreate/Kill/Rename"]
|
||||
SESSFILES["sessionfiles.go\nsession file handlers"]
|
||||
AGENTFILES["agentfiles.go\nagent file handlers"]
|
||||
ROOTFILES["rootfiles.go\nroot file handlers"]
|
||||
ELEVATEFILES["elevatefiles.go\nelevation handlers"]
|
||||
PROCFILES["procfiles.go\nprocess handlers"]
|
||||
PERSIST["persist.go\nsession persistence"]
|
||||
FSGO["fs.go\n9P filesystem"]
|
||||
FS_GO["fs.go\n9P File/FileConfig"]
|
||||
FORMAT["format.go\nevent formatting"]
|
||||
SYNTH["synth.go\nsynthetic files"]
|
||||
TYPES["types.go\nSession struct"]
|
||||
SPEC["spec.go\nEDSL namespace declaration"]
|
||||
BUILDER["builder.go\nBuildTree"]
|
||||
FSNODE["fsnode.go\nFsNodeDecl"]
|
||||
end
|
||||
|
||||
subgraph Support["Supporting Packages"]
|
||||
|
|
@ -82,7 +87,7 @@ flowchart TB
|
|||
Agent --> Backend
|
||||
Agent --> Toolsrv
|
||||
Agent --> Session
|
||||
FsSession --> Session
|
||||
Fs --> Session
|
||||
Toolsrv --> SANDBOX
|
||||
Toolsrv --> DETACH
|
||||
Toolsrv --> ELEVATE
|
||||
|
|
|
|||
295
doc/EVOLUTION.md
295
doc/EVOLUTION.md
|
|
@ -350,34 +350,6 @@ Credentials are requested via `kdialog`/`zenity` on the local desktop, or forwar
|
|||
over SSH for remote execution. The tool itself has no knowledge of sudo — the
|
||||
privilege wrapping is entirely in the dispatch layer.
|
||||
|
||||
## Current Topology
|
||||
```
|
||||
ollie/ ← single Go module
|
||||
├── agent/ ← agent loop, history, hooks, commands
|
||||
├── backend/ ← LLM providers (6 backends)
|
||||
├── toolsrv/ ← tool server, registry, remote execution
|
||||
├── session/ ← session lifecycle, config
|
||||
├── **fs**/ ← 9P filesystem tree
|
||||
├── dbus/ ← D-Bus adapter
|
||||
├── detach/ ← background process management
|
||||
├── elevate/ ← elevation broker
|
||||
├── sandbox/ ← landrun sandbox config
|
||||
├── env/ ← environment helpers
|
||||
├── log/ ← structured logging
|
||||
├── paths/ ← XDG path resolution
|
||||
├── mount/ ← 9P FUSE mount client (network transparency)
|
||||
├── cmd/ ← binaries (olliesrv, ollie-9p, ollie-remote)
|
||||
├── kde/ ← KDE Plasma (submodule)
|
||||
├── data/tools/ ← tool executables + .meta sidecar files
|
||||
├── data/agents/ ← agent configs (JSON)
|
||||
├── data/prompts/ ← system prompt fragments (markdown)
|
||||
├── data/skills/ ← domain knowledge modules (markdown)
|
||||
├── data/scripts/ ← ollie-remount, ollie-watchdog
|
||||
├── data/services/ ← systemd, xdg-autostart
|
||||
├── prompts/ ← embedded prompt templates
|
||||
├── sandbox/ ← landrun sandbox profiles (YAML)
|
||||
└── doc/ ← documentation
|
||||
```
|
||||
## Principles That Emerged
|
||||
1. **Filesystem-as-API** — everything is read/write on synthetic files. No
|
||||
custom protocols needed.
|
||||
|
|
@ -444,3 +416,270 @@ Lessons:
|
|||
QTextDocument relayout is O(n) on the full content.
|
||||
- The `loadEarlier` scroll-triggered cascade was creating 500 delegates
|
||||
on startup. Bounded initial window + explicit scroll-back is correct.
|
||||
|
||||
## Phase 14: Multi-Agent Deepening — ID/Name Split & Empty Sessions (Aug 1)
|
||||
|
||||
The multi-agent data structures introduced during the Great Flattening
|
||||
required further refinement to support true multi-agent sessions.
|
||||
|
||||
### ID/Name Split
|
||||
Sessions and agents now have **two identities**:
|
||||
- **`id`** (immutable UUIDv4, set at creation, never changes)
|
||||
- **`name`** (mutable human-readable string, defaults to first UUID segment)
|
||||
|
||||
The sessions map is keyed by Name, not ID. Renaming a session or agent moves
|
||||
the directory in the 9P namespace — `mv` on the session directory calls
|
||||
`RenameFromRoot`, and `write` to the agent's `name` file calls `SetName`.
|
||||
|
||||
Both entities expose `id` and `name` files in their 9P directories.
|
||||
|
||||
### Two-Step Session Creation
|
||||
Creation split into two phases:
|
||||
1. `session/new` — write `name=<name>` to create an empty session (no agent yet)
|
||||
2. `session/{name}/agent/new` — write `cwd=<dir> backend=<backend> model=<model> agent=<agent>` to create the agent
|
||||
|
||||
This enables creating sessions in advance and attaching agents later. Empty
|
||||
sessions (`Core=nil`) are valid — they appear in `session/idx` with empty
|
||||
fields and are fully killable/renameable.
|
||||
|
||||
### Multi-Agent Data Structures
|
||||
- `session.Session` supports multiple agents (`Agents()` returns `[]*agent.Agent`)
|
||||
- `Session.AgentLogs` maps agent IDs to `*AgentLog` instances
|
||||
- Active agent is selectable, each agent has its own prompt/chat/state/log
|
||||
- `session/idx` emits one line per agent, not one line per session
|
||||
|
||||
Key commits: 92 commits across Aug 1 touching session/, agent/, fs/, and kde/.
|
||||
|
||||
## Phase 15: 9P Declarative EDSL (Aug 2 — morning)
|
||||
|
||||
The most significant structural change to the 9P server since its inception:
|
||||
**replaced the imperative filesystem with a declarative EDSL**.
|
||||
|
||||
### Before: Imperative
|
||||
Every 9P operation (stat, walk, read, write, open, create, remove) was
|
||||
hand-coded per node. The `fs/session/` sub-package contained ~3,100 lines
|
||||
of manual tree-building code split across 6 files (`files.go`, `root.go`,
|
||||
`perm.go`, `synth.go`, `create.go`, `persist.go`). Adding a new file meant
|
||||
updating stat, readdir, open, and permission code paths.
|
||||
|
||||
### After: Declarative EDSL
|
||||
The entire namespace is declared in a single `FsNodeDecl` tree in `fs/spec.go`:
|
||||
```go
|
||||
var treeSpec = Dir("/",
|
||||
Leaf("backends", 0444, Read(readBackends), GID("agent")),
|
||||
Leaf("eventwait", 0444, Read(readEventwait), Stream(blockEventwait), GID("agent")),
|
||||
Leaf("complete", 0666, Request(requestComplete), GID("agent")),
|
||||
// ...
|
||||
Dir("session", GID("agent"),
|
||||
Leaf("new", 0666, Request(requestSessionNew)),
|
||||
Leaf("idx", 0444, Read(readSessionIdx)),
|
||||
TemplateDir("{id}", listSessions),
|
||||
),
|
||||
)
|
||||
```
|
||||
|
||||
`BuildTree()` in `fs/builder.go` walks the spec, validates invariants
|
||||
(directories can't have leaf handlers, at most one blocking variant,
|
||||
template nodes must have `List`, ownership inheritance), and produces
|
||||
a fully-wired `*fs.Tree`.
|
||||
|
||||
### Package consolidation
|
||||
- Entire `fs/session/` sub-package flattened into `fs/` — 6 files deleted
|
||||
- `cmd/olliesrv/elevate_tree.go` — elevation tree moved to `fs/elevatefiles.go`
|
||||
- `cmd/olliesrv/server.go` simplified from ~820+ lines to ~200 lines of thin
|
||||
9P protocol handling; all filesystem logic is now in `fs/`
|
||||
- Handler files organized by scope: `rootfiles.go`, `sessionfiles.go`,
|
||||
`agentfiles.go`, `elevatefiles.go`, `procfiles.go`
|
||||
|
||||
### Net result
|
||||
- **−3,152 lines** deleted, **+2,086 lines** added across 24 files
|
||||
- Adding a file = adding one `Leaf()` call. No manual stat/readdir/write.
|
||||
- Validation at `BuildTree` time catches structural errors before serving.
|
||||
- Permission model inlined in the spec via `mode` and `GID()` — no separate `perm.go`.
|
||||
|
||||
## Phase 16: 9P Server Streamlining (Aug 2 — afternoon)
|
||||
|
||||
Following the EDSL conversion, the 9P protocol server itself
|
||||
(`cmd/olliesrv/server.go`) received a focused cleanup over 13 commits:
|
||||
|
||||
### Fid management
|
||||
- **Server owns the fid map**: validates newfids (not already in use, not stale),
|
||||
garbage-collects orphaned fids. No more leaking fids on client disconnect.
|
||||
- **Store opened file on fid**: read/write use `f.entry` directly instead of
|
||||
re-resolving the path on every operation. Eliminates a class of TOCTOU bugs.
|
||||
|
||||
### Method extraction
|
||||
- `attach` and `open` extracted from `*Server` — pass groups and log explicitly
|
||||
- `GroupTable` extracted from `*Server` — permission checking uses bitmask OR
|
||||
instead of a boolean dance (`checkPermBits` → `hasPermBits`)
|
||||
- `checkPerm` extracted to package-level, permissions resolved in `open`
|
||||
- 9 `rootTree`-only methods extracted to package-level functions; switch
|
||||
statement replaced with handler map
|
||||
- Three thin wrapper methods inlined (`readFile`, `writeFile`, `FileTree` alias)
|
||||
- `handle` inlined into `Start`; renamed `Serve` → `Start`, `Shutdown` → `Kill`
|
||||
|
||||
### Result
|
||||
- `server.go` reduced from ~820 to ~200 lines
|
||||
- `main.go` simplified as elevation tree wiring moved to `fs/`
|
||||
- Server is now a thin 9P2000 protocol translator, not a filesystem
|
||||
|
||||
## Phase 17: Eventwait Redesign & Chat Log Format (Aug 2)
|
||||
|
||||
### Global /eventwait
|
||||
Replaced per-session event polling with a global event ring buffer and a
|
||||
single `/eventwait` file. Events carry structured prefixes and delta descriptions:
|
||||
- `S new session/<name>` — session created
|
||||
- `S kill session/<name>` — session destroyed
|
||||
- `S rename session/<old> session/<new>` — session renamed
|
||||
- `A kill session/<name>/agent/<aid>` — agent killed
|
||||
|
||||
The event ring is a fixed-size circular buffer (100 slots) with a `sync.Cond`
|
||||
for blocking reads. Frontends block on `/eventwait` and receive deltas since
|
||||
their last known offset. No polling, no timer, no D-Bus.
|
||||
|
||||
Fixes along the way:
|
||||
- Dangling pointer in `eventRing` cond initialization (caused panics under load)
|
||||
- Double-unlock panic in `WaitEvent`
|
||||
- Spurious `[[[end]]]` markers on reasoning blocks (reasoning events were
|
||||
silently suppressed from chat but still advanced the role-transition state)
|
||||
|
||||
### Chat log: [[[type]]]/[[[end]]] block format
|
||||
The chat log switched from plain text to a structured block format:
|
||||
```
|
||||
[[[assistant:resp-abc123]]]
|
||||
Hello! How can I help?
|
||||
|
||||
[[[end]]]
|
||||
[[[tool:file_read]]]
|
||||
File contents...
|
||||
|
||||
[[[end]]]
|
||||
```
|
||||
|
||||
This enables reliable parsing of multi-turn, multi-role conversations
|
||||
from the flat log. Each event type (`assistant`, `tool`, `call`, `retry`,
|
||||
`error`, `usage`, `info`, `exec`) opens a block and `[[[end]]]` closes it.
|
||||
`reasoning` and `reasoning_think` events are suppressed from the log
|
||||
entirely — they're included in the LLM context but invisible to the user.
|
||||
|
||||
### KDE submodule evolution
|
||||
The KDE frontend received extensive updates across both days:
|
||||
- **StreamFsm refactor** — chat block FSM with proper fence-post handling
|
||||
- **Responsive session tree** — selection/statewait stream management
|
||||
- **Explicit agent selection** — per-agent statewait/chat streams,
|
||||
only `switchAgent` starts streams; session click clears agent selection
|
||||
- **Emoji-labeled states** — visual state indicators (❌ for errors, ⚠️ for warnings)
|
||||
- **Double-click rename** — inline renaming of agent nodes
|
||||
- **Robust session loading** — loads sessions on startup, handles empty state
|
||||
- **Context menu** — kill session, per-session operations
|
||||
- **Eventwait-driven updates** — replaced timer-based polling with blocking `/eventwait`
|
||||
- **Fence fix** — fixed a bug where code fences could eat surrounding content
|
||||
- Removed D-Bus entirely; all communication via `9p` plan9port binary
|
||||
|
||||
### Updated Filesystem Layout (post-refactor)
|
||||
```
|
||||
/ ← root (owned by system user)
|
||||
├── backends backends list
|
||||
├── help help text
|
||||
├── models model list (cached)
|
||||
├── agents agent config list
|
||||
├── ctl root control (invalidate, kill)
|
||||
├── eventwait global event stream (blocking read)
|
||||
├── complete request-response: code completion
|
||||
├── generate request-response: one-shot LLM generation
|
||||
├── route request-response: model routing
|
||||
├── elevate/
|
||||
│ ├── policy global elevation policy
|
||||
│ └── pending/{id} pending elevation requests
|
||||
└── session/
|
||||
├── new write key=value to create session
|
||||
├── idx session index (one line per agent)
|
||||
├── {name}/
|
||||
│ ├── env session environment variables
|
||||
│ ├── ctl session control (kill, save, invalidate)
|
||||
│ ├── plan session-scoped markdown checklist
|
||||
│ ├── id immutable session UUID
|
||||
│ ├── name mutable session name (write to rename)
|
||||
│ ├── elevate per-session elevation policy
|
||||
│ └── agent/
|
||||
│ ├── new write config to create agent (rdwr)
|
||||
│ └── {aid}/
|
||||
│ ├── prompt, prompt.prev, fifo.in, fifo.out
|
||||
│ ├── chat (streaming), log (64KB window)
|
||||
│ ├── state, statewait (blocking)
|
||||
│ ├── cfg, ctl, cwd
|
||||
│ ├── id (immutable), name (mutable)
|
||||
│ ├── offset, usage, cost, ctxsz, models
|
||||
│ ├── systemprompt, context, tail
|
||||
│ ├── tools, toolsrv.loaded, toolsrv.rev
|
||||
│ └── proc/{pid}
|
||||
```
|
||||
|
||||
## Updated Current Topology
|
||||
```
|
||||
ollie/ ← single Go module
|
||||
├── agent/ ← agent loop, history, hooks, commands
|
||||
├── backend/ ← LLM providers (6 backends)
|
||||
├── toolsrv/ ← tool server, registry, remote execution
|
||||
├── session/ ← session lifecycle, config
|
||||
├── fs/ ← 9P filesystem (EDSL-declared, flat package)
|
||||
│ ├── spec.go Namespace declaration (single source of truth)
|
||||
│ ├── fsnode.go FsNodeDecl type + Dir/Leaf/TemplateDir
|
||||
│ ├── builder.go BuildTree — spec → wired *Tree
|
||||
│ ├── rootfiles.go Root-level handlers
|
||||
│ ├── sessionfiles.go Session-level handlers
|
||||
│ ├── agentfiles.go Agent-level handlers
|
||||
│ ├── elevatefiles.go Elevation handlers
|
||||
│ ├── procfiles.go Process handlers
|
||||
│ ├── lifecycle.go Create/kill/rename + event ring
|
||||
│ ├── newroot.go NewRoot constructor
|
||||
│ ├── persist.go Session persistence
|
||||
│ ├── types.go Session/AgentLog types
|
||||
│ ├── tree.go 9P *Tree
|
||||
│ ├── fs.go 9P File/FileConfig
|
||||
│ └── format.go Event formatting
|
||||
├── detach/ ← background process management
|
||||
├── elevate/ ← elevation broker
|
||||
├── sandbox/ ← landrun sandbox config
|
||||
├── env/ ← environment helpers
|
||||
├── log/ ← structured logging
|
||||
├── paths/ ← XDG path resolution
|
||||
├── mount/ ← 9P FUSE mount client (network transparency)
|
||||
├── cmd/ ← binaries (olliesrv, ollie-9p, ollie-remote)
|
||||
├── kde/ ← KDE Plasma (submodule)
|
||||
├── data/tools/ ← tool executables + .meta sidecar files
|
||||
├── data/agents/ ← agent configs (JSON)
|
||||
├── data/prompts/ ← system prompt fragments (markdown)
|
||||
├── data/skills/ ← domain knowledge modules (markdown)
|
||||
├── data/scripts/ ← ollie-remount, ollie-watchdog
|
||||
├── data/services/ ← systemd, xdg-autostart
|
||||
├── prompts/ ← embedded prompt templates
|
||||
├── sandbox/ ← landrun sandbox profiles (YAML)
|
||||
└── doc/ ← documentation
|
||||
```
|
||||
|
||||
## Dead Ends and Reversals (updated)
|
||||
| What | Why it was removed |
|
||||
|------|-------------------|
|
||||
| MCP servers | Too heavyweight; scripts are simpler and faster |
|
||||
| TUI frontend | Replaced by KDE GUI and Emacs (ellie) |
|
||||
| s/sh, s/bfg, s/bbg | Replaced by `generate`/`complete`/`route` 9P files |
|
||||
| `pl/` planning namespace | Over-engineered; plan file is sufficient |
|
||||
| Beads integration | External dependency for something a checklist file handles |
|
||||
| SSH agent proxy | Attack surface not worth the convenience |
|
||||
| Separate ollie-dbus daemon | Consolidated into olliesrv |
|
||||
| FUSE mounts | Replaced by standalone 9P client binary |
|
||||
| Tier routing (FAST/POWER) | Replaced by /route endpoint with real model discovery |
|
||||
| call_tool / pipe | Replaced by native tool registry with tool_load |
|
||||
| execute_code (multi-lang) | Simplified to shell-only; tool scripts handle language choice |
|
||||
| chatwait | Reverted; acme tails chat directly |
|
||||
| Header comment metadata | Replaced by .meta sidecar JSON; decouples metadata from language |
|
||||
| `contrib/` directory | Renamed to `data/`; nothing was community-contributed |
|
||||
| Script namespaces (s/, u/, x/) | Replaced by 9P request-response files |
|
||||
| D-Bus adapter | 9P streaming is superior; no polling, no offset tracking, no frozen GUIs |
|
||||
| `fs/session/` sub-package | Flattened into `fs/` — no more sub-package indirection |
|
||||
| `cmd/olliesrv/elevate_tree.go` | Elevation tree handlers moved to `fs/elevatefiles.go` |
|
||||
| Per-session/agent timestamp IDs | Replaced by UUIDv4 — immutable `id` + mutable `name` |
|
||||
| Single-step session creation | Split into `session/new` + `session/{name}/agent/new` |
|
||||
| D-Bus-driven KDE frontends | Rewritten on pure 9P via plan9port `9p` binary |
|
||||
| Plasmoid/tray KDE components | Deleted — not useful enough to maintain |
|
||||
|
|
|
|||
|
|
@ -6,18 +6,26 @@ ordinary shell scripting against a mounted filesystem.
|
|||
|
||||
## The Namespace
|
||||
|
||||
Every session lives under `/session/{id}/`:
|
||||
Every session lives under `/session/{name}/` where `{name}` is the session's
|
||||
**mutable human-readable name** (not the immutable UUID — see `id` and `name`
|
||||
files). Session directories are keyed by Name; renaming a session via writing
|
||||
to its `name` file moves the entire directory subtree.
|
||||
|
||||
| Path | Purpose |
|
||||
|------|---------|
|
||||
| `/session/new` | Create a session (write `name`, `backend`, `model`, `agent`, `cwd`) |
|
||||
| `/session/{id}/plan` | Session-scoped markdown checklist |
|
||||
| `/session/{id}/env` | Session environment variables |
|
||||
| `/session/{id}/agent/{aid}/prompt` | Submit a prompt (dispatched async on close) |
|
||||
| `/session/{id}/agent/{aid}/chat` | Cumulative conversation history (append-only read) |
|
||||
| `/session/{id}/agent/{aid}/cfg` | Agent config (KV): state, backend, model, agent, cwd, params |
|
||||
| `/session/{id}/agent/{aid}/statewait` | **Blocking read** — returns when state changes |
|
||||
| `/session/{id}/agent/{aid}/ctl` | Control commands: `stop`, `kill`, `compact`, `rn <name>` |
|
||||
| `/session/new` | Create an empty session (write `name=...`, read back name) |
|
||||
| `/session/{name}/agent/new` | Create an agent within a session (write `cwd=... backend=...`) |
|
||||
| `/session/{name}/plan` | Session-scoped markdown checklist |
|
||||
| `/session/{name}/env` | Session environment variables |
|
||||
| `/session/{name}/id` | Immutable session UUID |
|
||||
| `/session/{name}/name` | Mutable session name (write to rename) |
|
||||
| `/session/{name}/agent/{aid}/prompt` | Submit a prompt |
|
||||
| `/session/{name}/agent/{aid}/chat` | Streaming chat log (blocking read) |
|
||||
| `/session/{name}/agent/{aid}/cfg` | Agent config KV: backend, model, agent, cwd, params |
|
||||
| `/session/{name}/agent/{aid}/statewait` | **Blocking read** — returns when state changes |
|
||||
| `/session/{name}/agent/{aid}/ctl` | Control commands: `stop`, `kill`, `compact`, `rn <name>` |
|
||||
| `/session/{name}/agent/{aid}/id` | Immutable agent UUID |
|
||||
| `/session/{name}/agent/{aid}/name` | Mutable agent name (write to rename) |
|
||||
|
||||
Global request-response files provide stateless operations without sessions:
|
||||
|
||||
|
|
@ -32,8 +40,8 @@ Global namespaces are shared across all sessions:
|
|||
| Path | Purpose |
|
||||
|------|---------|
|
||||
| `/session/idx` | Session index (id, state, cwd, backend, model) |
|
||||
| `/session/{id}/plan` | Shared session-scoped checklist |
|
||||
| `/session/{id}/agent/{aid}/chat` | Full chat history (searchable) |
|
||||
| `/session/{name}/plan` | Shared session-scoped checklist |
|
||||
| `/session/{name}/agent/{aid}/chat` | Full chat history (searchable) |
|
||||
|
||||
## Spawning Agents
|
||||
|
||||
|
|
@ -43,9 +51,21 @@ From within an agent session, use `subagent_spawn`:
|
|||
subagent_spawn [-name NAME] [-agent AGENT] [-backend BACKEND] [-model MODEL] [-cwd DIR] <prompt>
|
||||
```
|
||||
|
||||
This creates a child session, writes the prompt, and returns the session path
|
||||
(e.g. `session/my-session`). Child sessions carry the parent session ID in their name,
|
||||
so the spawning tree is recoverable from session IDs alone.
|
||||
This creates a child session + agent atomically, writes the prompt, and returns
|
||||
the session path (e.g. `session/my-session`). Child sessions carry the parent
|
||||
session ID in their name, so the spawning tree is recoverable from session IDs alone.
|
||||
|
||||
### Explicit Two-Step Creation
|
||||
|
||||
Session creation can be split into two phases for advanced orchestration
|
||||
scenarios:
|
||||
|
||||
1. **Create empty session**: `echo "name=my-session" | ollie-9p rdwr session/new`
|
||||
2. **Add agent**: `echo "cwd=/home/user agent=default backend=openai model=gpt-4o" | ollie-9p rdwr session/my-session/agent/new`
|
||||
|
||||
Empty sessions (`Core=nil`) are valid — they appear in `session/idx`, have
|
||||
no running agent, and can be killed or renamed. This enables pre-provisioning
|
||||
session directories before the agent environment is ready.
|
||||
|
||||
## Coordination Patterns
|
||||
|
||||
|
|
@ -64,7 +84,7 @@ flowchart TB
|
|||
```
|
||||
|
||||
A root agent reasons about a task and delegates subtasks to workers. Workers
|
||||
report back via their chat history; the conductor reads `/session/{id}/agent/{aid}/chat` after
|
||||
report back via their chat history; the conductor reads `/session/{name}/agent/{aid}/chat` after
|
||||
`statewait` unblocks.
|
||||
|
||||
```sh
|
||||
|
|
|
|||
|
|
@ -76,7 +76,7 @@ Leaf("chat", 0444, Read(readAgentChat), Stream(blockAgentChat))
|
|||
| `Stream(fn)` | `func(HandlerCtx, ctx, base) (data, base, err)` | `d.Stream` |
|
||||
| `BlockOnce(fn)` | `func(HandlerCtx, ctx, base) (data, base, err)` | `d.BlockOnce` |
|
||||
| `Request(fn)` | `func(HandlerCtx, []byte) ([]byte, error)` | `d.Request` |
|
||||
| `Create(fn)` | `func(HandlerCtx, string) error` | `d.Create` |
|
||||
| `CreateFile(fn)` | `func(HandlerCtx, string) error` | `d.Create` |
|
||||
| `Remove(fn)` | `func(HandlerCtx) error` | `d.Remove` |
|
||||
| `Stat(fn)` | `func() os.FileInfo` | `d.Stat` |
|
||||
| `UID(s)` | string | `d.UID` |
|
||||
|
|
|
|||
Loading…
Reference in New Issue