doc: update architecture diagrams to Mermaid, reflect current codebase
- Convert all ASCII diagrams to Mermaid (gitea-rendered)
- ARCHITECTURE.md: System Overview, Data Flow, Frontends → Mermaid
- CORE.md: Package Map, Per-Iteration Flow → Mermaid
- EVOLUTION.md: Timeline → Gantt, coordination/security/frontends → Mermaid
- MULTI_AGENT.md: coordination patterns → Mermaid
- REMOTE_EXECUTION.md: split-brain architecture → Mermaid
- Update all references: agent.Core → *agent.Agent, execute.Server → toolsrv.Server,
tools.Dispatcher → toolsrv.Runner, s/ → session/ namespace
- Fix namespace layout to show agent/{aid}/ subdirectory hierarchy
- Update package layout to reflect flat Go module (no core/ or 9p/ submodules)
This commit is contained in:
Ollie Agent
parent
df22946249
commit
f63cb3bb37
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
## Philosophy
|
||||
|
||||
ollie's design philosophy is Emacs: a small, extensible core. The Go runtime (`agent.Core`) defines what an agent *is* — an event loop, a backend connection, a message history, and seven built-in primitives (`shell`, `tool_*`, `skill_*`). Everything else — orchestration, scheduling, workflows, UIs — is pushed out to the surrounding environment.
|
||||
ollie's design philosophy is Emacs: a small, extensible core. The Go runtime (`agent.Agent`) defines what an agent *is* — an event loop, a backend connection, a message history, and three built-in primitives (`shell`, tool registry, skill registry). Everything else — orchestration, scheduling, workflows, UIs — is pushed out to the surrounding environment.
|
||||
|
||||
The primary integration philosophy is Plan 9's "everything is a file." `olliesrv` exposes agent state and behaviors as files in a 9P namespace. Any program that can read and write files can drive an agent: shell scripts, editors, web apps, cron, containers.
|
||||
|
||||
|
|
@ -10,117 +10,138 @@ For desktop-native integration, `olliesrv` embeds a D-Bus adapter (`org.ollie.Se
|
|||
|
||||
Design principles:
|
||||
- **Small extensible core.** The agent runtime is minimal; capabilities come from composing external scripts.
|
||||
- **Seven built-in primitives.** `shell`, `tool_load`/`tool_list`/`tool_active`, `skill_load`/`skill_list`/`skill_active` are the only tools compiled into the core. Everything else is a script.
|
||||
- **Three built-in primitives.** `shell`, tool registry (`tool_load`/`tool_list`/`tool_active`), and skill registry (`skill_load`/`skill_list`/`skill_active`) are the only tools compiled into the core. Everything else is a script.
|
||||
- **Two integration paths.** 9P filesystem (canonical) and/or D-Bus (conventional). Same core, different surfaces, selectable via flags.
|
||||
- **No framework lock-in.** Frontends are decoupled via whichever interface they prefer; the core doesn't know or care.
|
||||
|
||||
## Repository Structure
|
||||
|
||||
The project is a monorepo with Git submodules:
|
||||
Single Go module (`ollie`) with two Git submodules for decoupled frontends:
|
||||
|
||||
```
|
||||
ollie/
|
||||
├── core/ → ollie-core Go library: agent loop, backends, tools, config
|
||||
├── 9p/ → ollie-9p 9P filesystem server (olliesrv)
|
||||
├── kde/ → ollie-kde KDE GUI, plasmoid, Kate plugin, KRunner, tray
|
||||
├── httpgw/ → ollie-httpgw HTTP-to-D-Bus gateway
|
||||
├── webui/ → ollie-webui Preact SPA (browser frontend)
|
||||
├── acme/ → ollie-acme Plan 9 acme editor frontend
|
||||
├── el/ → ollie-el Emacs frontend (ellie.el)
|
||||
├── skills/ → agent-skills Domain knowledge files
|
||||
├── agents/ Agent config JSONs (default, coding, orchestrator, worker, ...)
|
||||
├── prompts/ Prompt templates (markdown)
|
||||
├── tools/ Tool scripts (file_read, lsp_*, memory_*, subagent_*, ...)
|
||||
├── scripts/ User-facing scripts organized by namespace (s/, u/, x/)
|
||||
├── sandbox/ Sandbox config YAML
|
||||
└── doc/ Documentation
|
||||
├── agent/ Agent loop, history, hooks, prompt resolution, commands
|
||||
├── backend/ LLM providers (Anthropic, OpenAI, Ollama, Gemini, Copilot, CodeWhisperer)
|
||||
├── toolsrv/ Tool server: sandboxed execution, tool registry, skill management
|
||||
├── session/ Session lifecycle (config, creation, persistence)
|
||||
├── fs/session/ 9P filesystem tree for sessions
|
||||
├── dbus/ D-Bus adapter (org.ollie.SessionManager)
|
||||
├── detach/ Background process management (ring buffer, signal)
|
||||
├── elevate/ Elevation broker (privilege escalation daemon)
|
||||
├── sandbox/ Landlock sandbox config YAML
|
||||
├── env/ Environment variable loading
|
||||
├── log/ Structured logging
|
||||
├── paths/ XDG path resolution
|
||||
├── mount/ 9P FUSE mount client
|
||||
├── cmd/ Binaries:
|
||||
│ ├── olliesrv/ 9P server
|
||||
│ ├── ollie-9p/ 9P client
|
||||
│ ├── ollie-httpgw/ HTTP-to-9P gateway
|
||||
│ └── ollie-remote/ Remote execution server
|
||||
├── kde/ KDE integration (submodule) — plasmoid, GUI, Kate plugin, KRunner, tray
|
||||
├── webui/ Preact SPA browser frontend (submodule)
|
||||
├── agents/ Agent config JSONs (default, coding, orchestrator, worker, ...)
|
||||
├── prompts/ Prompt templates (markdown)
|
||||
├── tools/ Tool scripts (file_read, lsp_*, memory_*, subagent_*, ...)
|
||||
├── skills/ Domain knowledge files (markdown)
|
||||
├── scripts/ User-facing scripts organized by namespace (s/, u/, x/)
|
||||
├── sandbox/ Sandbox config YAML files
|
||||
└── doc/ Documentation
|
||||
```
|
||||
|
||||
## System Overview
|
||||
|
||||
```
|
||||
┌───────────────────────────────────────────────────────────────────────────┐
|
||||
│ Frontends │
|
||||
│ s/sh (terminal) │ acme │ ellie (Emacs) │ KDE GUI/plasmoid │ Web UI │
|
||||
└───────┬──────────┴───┬──┴───────┬───────┴────────┬─────────┴──────┬──────┘
|
||||
│ 9P │ 9P │ 9P │ D-Bus │ D-Bus
|
||||
│ │ │ ▼ ▼
|
||||
│ │ │ ┌──────────────────────────────────┐
|
||||
│ │ │ │ org.ollie.SessionManager (D-Bus)│
|
||||
│ │ │ └──────────────┬───────────────────┘
|
||||
▼ ▼ ▼ │ embedded adapter
|
||||
┌───────────────────────────────────────────────────────▼───────────────────┐
|
||||
│ olliesrv (9P server + D-Bus adapter) │
|
||||
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
|
||||
│ │ Sessions │ │ backends │ │ models │ │
|
||||
│ │ s/ │ │ (file) │ │ (file) │ │
|
||||
│ └────┬─────┘ └──────────┘ └──────────┘ │
|
||||
└───────┼───────────────────────────────────────────────────────────────────┘
|
||||
│ per-session
|
||||
▼
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph Frontends["Frontends"]
|
||||
TUI["s/sh (terminal)"]
|
||||
ACME["acme (Plan 9)"]
|
||||
ELLIE["ellie (Emacs)"]
|
||||
KDE["KDE GUI / Plasmoid"]
|
||||
WEB["Web UI"]
|
||||
end
|
||||
|
||||
D-Bus-only mode: olliesrv -no9p (same interface, no filesystem listener)
|
||||
┌────────────────────────────────────────────────────────────────┐
|
||||
│ agent.Core (per session) │
|
||||
│ ┌─────────────┐ ┌──────────────┐ ┌────────────────────────┐ │
|
||||
│ │ Agent Loop │ │ Session │ │ tools.Dispatcher │ │
|
||||
│ │ (loop.go) │ │ (messages, │ │ ┌──────────────────┐ │ │
|
||||
│ │ │ │ task state, │ │ │ execute.Server │ │ │
|
||||
│ │ stream ←────┼──┤ compaction) │ │ │ (shell, │ │ │
|
||||
│ │ dispatch ───┼──┼───────────────┼──┤ │ tool_*, skill_*)│ │ │
|
||||
│ └─────────────┘ └──────────────┘ │ └────────┬─────────┘ │ │
|
||||
│ └───────────┼────────────┘ │
|
||||
└──────────────────────────────────────────────────┼──────────────┘
|
||||
│
|
||||
┌──────────────────────────────────────────┼──────┐
|
||||
│ Tool Execution │ │
|
||||
│ ┌─────────────┐ ┌───────────────────┐ │ │
|
||||
│ │ landrun │ │ Tool Scripts │ │ │
|
||||
│ │ (sandbox) │ │ (OLLIE_TOOLS_PATH)│ │ │
|
||||
│ └─────────────┘ └───────────────────┘ │ │
|
||||
└──────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ LLM Backends │
|
||||
│ Ollama│OpenAI│ │
|
||||
│ Anthropic│... │
|
||||
└─────────────────┘
|
||||
subgraph Integration["Integration Layer"]
|
||||
direction LR
|
||||
P9["9P Filesystem"]
|
||||
DBUS["org.ollie.SessionManager (D-Bus)"]
|
||||
end
|
||||
|
||||
subgraph Server["olliesrv"]
|
||||
direction TB
|
||||
NS["9P Namespace\nsession/ · backends · models"]
|
||||
DBA["Embedded D-Bus Adapter"]
|
||||
end
|
||||
|
||||
subgraph Core["Agent Engine (per session)"]
|
||||
LOOP["Agent Loop\n(agent/loop.go)"]
|
||||
SESS["Session State\n(agent/history.go)"]
|
||||
TRV["toolsrv.Server\n· shell execution\n· tool registry\n· landlock sandbox"]
|
||||
end
|
||||
|
||||
subgraph Backends["LLM Backends"]
|
||||
OLLAMA["Ollama"]
|
||||
OPENAI["OpenAI / OpenRouter"]
|
||||
ANTHROPIC["Anthropic"]
|
||||
GEMINI["Gemini"]
|
||||
CW["CodeWhisperer"]
|
||||
end
|
||||
|
||||
TUI --> P9
|
||||
ACME --> P9
|
||||
ELLIE --> P9
|
||||
KDE --> DBUS
|
||||
WEB --> DBUS
|
||||
|
||||
P9 --> NS
|
||||
DBUS --> DBA
|
||||
|
||||
NS --> LOOP
|
||||
DBA --> LOOP
|
||||
|
||||
LOOP <--> SESS
|
||||
LOOP --> TRV
|
||||
LOOP <--> Backends
|
||||
|
||||
TRV --> Shell["sandboxed shell\n(landrun)"]
|
||||
TRV --> Tools["tool scripts\n(OLLIE_TOOLS_PATH)"]
|
||||
```
|
||||
|
||||
## Core Library (`core/`)
|
||||
## Core Library
|
||||
|
||||
The core is a Go library (`ollie/`) with no binary. Frontends import it as a dependency.
|
||||
The core is a single Go module (`ollie`) with no binary. Binaries live in `cmd/`.
|
||||
|
||||
### Package Layout
|
||||
|
||||
| Package | Purpose |
|
||||
|---|---|
|
||||
| `agent/` | Core interface, agent loop, compaction, hooks, session persistence, config loading |
|
||||
| `agent/` | Agent struct, loop, history, compaction, hooks, commands, prompt resolution, state |
|
||||
| `backend/` | Backend interface + LLM providers (Anthropic, OpenAI, Ollama, Gemini, Copilot, CodeWhisperer) |
|
||||
| `tools/` | Server/Dispatcher interfaces, tool registry, discovery, schema parsing |
|
||||
| `execute/` | Sandboxed shell execution, promoted tool dispatch, skill dispatch, remote SSH |
|
||||
| `toolsrv/` | Tool server: sandboxed shell execution, named tool dispatch, skill management, result tiering, remote execution |
|
||||
| `session/` | Session lifecycle, config, persistence |
|
||||
| `detach/` | Background process management (ring buffer, signal) |
|
||||
| `skills/` | Skill metadata parsing and per-session registry |
|
||||
| `elevate/` | Elevation broker (privilege escalation daemon, policy) |
|
||||
| `sandbox/` | Landrun sandbox configuration and command wrapping |
|
||||
| `env/` | Session environment helpers |
|
||||
| `log/` | Structured logging |
|
||||
| `paths/` | XDG path resolution |
|
||||
|
||||
### Key Interfaces
|
||||
### Key Types
|
||||
|
||||
**`agent.Core`** — the public API frontends drive:
|
||||
**`agent.Agent`** — the concrete struct frontends drive:
|
||||
```go
|
||||
type Core interface {
|
||||
Submit(ctx context.Context, input string)
|
||||
Interrupt(cause error) bool
|
||||
Queue(prompt string)
|
||||
State() string // "idle" | "thinking" | "calling: <tool>"
|
||||
WaitChange(ctx context.Context, field, current string) (string, bool)
|
||||
Close()
|
||||
// ... model/backend info, CWD, usage, cost, etc.
|
||||
type Agent struct {
|
||||
// unexported fields — no public interface
|
||||
}
|
||||
|
||||
func New(cfg AgentConfig) *Agent
|
||||
func (a *Agent) Submit(ctx context.Context, input string)
|
||||
func (a *Agent) Interrupt(cause error) bool
|
||||
func (a *Agent) Queue(prompt string)
|
||||
func (a *Agent) State() string
|
||||
func (a *Agent) WaitChange(ctx context.Context, field, current string) (string, bool)
|
||||
func (a *Agent) Close()
|
||||
// ... model/backend info, CWD, usage, cost, etc.
|
||||
```
|
||||
|
||||
**`backend.Backend`** — LLM provider contract:
|
||||
|
|
@ -135,21 +156,17 @@ type Backend interface {
|
|||
}
|
||||
```
|
||||
|
||||
**`tools.Server`** — any tool provider:
|
||||
**`toolsrv.Runner`** — minimal tool execution interface:
|
||||
```go
|
||||
type Server interface {
|
||||
type Runner interface {
|
||||
ListTools() ([]ToolInfo, error)
|
||||
CallTool(ctx context.Context, tool string, args json.RawMessage) (json.RawMessage, error)
|
||||
}
|
||||
```
|
||||
|
||||
**`tools.Dispatcher`** — routes tool calls to servers:
|
||||
**`toolsrv.Server`** — the concrete tool server (implements Runner):
|
||||
```go
|
||||
type Dispatcher interface {
|
||||
AddServer(name string, s Server)
|
||||
ListTools() ([]ToolInfo, error)
|
||||
Dispatch(ctx context.Context, server, tool string, args json.RawMessage) (json.RawMessage, error)
|
||||
}
|
||||
type Server struct { /* ... */ }
|
||||
```
|
||||
|
||||
### Agent Loop (`agent/loop.go`)
|
||||
|
|
@ -157,7 +174,7 @@ type Dispatcher interface {
|
|||
The loop is the heart of the system. On each turn:
|
||||
|
||||
1. **Stream** — call `backend.ChatStream()` with messages + tool definitions
|
||||
2. **Dispatch** — if the model returns tool calls, execute them via `tools.Dispatcher`
|
||||
2. **Dispatch** — if the model returns tool calls, execute them via `toolsrv.Server`
|
||||
3. **Update** — append assistant message + tool results to session history
|
||||
4. **Repeat** — loop until the model produces a final text response (no tool calls)
|
||||
|
||||
|
|
@ -169,9 +186,9 @@ Safety mechanisms:
|
|||
- **Transient retry**: up to 3 retries with exponential backoff for rate limits and 5xx
|
||||
- **Context overflow**: triggers automatic compaction and retry
|
||||
|
||||
### Session & Context Management (`agent/session.go`)
|
||||
### Session & Context Management (`agent/history.go`)
|
||||
|
||||
`Session` owns the message history as a flat `[]backend.Message` slice. Key behaviors:
|
||||
`History` owns the message history as a flat `[]backend.Message` slice. Key behaviors:
|
||||
|
||||
- **TaskState**: structured JSON overlay (objective, plan_step, constraints, last_action, next_decision) injected at the top of every turn
|
||||
- **Compaction**: three-zone strategy when context reaches 50% of model limit:
|
||||
|
|
@ -200,50 +217,78 @@ Lifecycle callbacks executed as shell commands with JSON payload on stdin:
|
|||
|
||||
Agent configs declare prompts as a JSON array of shell commands. Each command is executed with `$OLLIE` and `$PWD` available; stdout is concatenated to form the system prompt. This enables dynamic, composable prompts assembled from template fragments.
|
||||
|
||||
## 9P Server (`9p/`)
|
||||
## 9P Server (`cmd/olliesrv/`)
|
||||
|
||||
`olliesrv` is the central daemon. It implements a 9P2000 file server that exposes the entire agent namespace:
|
||||
|
||||
### Namespace Layout
|
||||
|
||||
| Path | Contents |
|
||||
|---|---|
|
||||
| `s/` | Sessions — each session is a directory |
|
||||
| `s/{id}/chat` | Full chat history (streamable) |
|
||||
| `s/{id}/prompt` | Write to submit a prompt |
|
||||
| `s/{id}/ctl` | Control commands (stop, kill, compact, model, ...) |
|
||||
| `s/{id}/cfg` | Session config (key=value) |
|
||||
| `s/{id}/state` | Current state (idle/thinking/calling) |
|
||||
| `s/{id}/statewait` | Blocking read until state changes |
|
||||
| `s/{id}/plan` | Session-scoped markdown checklist |
|
||||
|
||||
```
|
||||
session/
|
||||
├── new write key=value to create session
|
||||
├── idx session index (tab-separated)
|
||||
├── ls exec to list sessions
|
||||
├── kill exec to kill a session
|
||||
├── sh interactive shell
|
||||
├── b batch one-shot (create, submit, wait, print, kill)
|
||||
├── bfg batch foreground (submit to existing, wait, print)
|
||||
├── bbg batch background (submit, return path)
|
||||
├── cleanup kill idle sessions
|
||||
├── {sid}/
|
||||
│ ├── plan session-scoped markdown checklist
|
||||
│ ├── env session environment
|
||||
│ └── agent/
|
||||
│ └── {aid}/
|
||||
│ ├── cfg session config (key=value)
|
||||
│ ├── ctl control commands (stop, kill, compact, model, ...)
|
||||
│ ├── state current state (idle/thinking/calling)
|
||||
│ ├── statewait blocking read until state changes
|
||||
│ ├── chat full chat history (streamable)
|
||||
│ ├── offset byte offset after last user prompt
|
||||
│ ├── prompt write to submit a prompt
|
||||
│ ├── prompt.prev last submitted prompt
|
||||
│ ├── fifo.in queue a prompt (FIFO input)
|
||||
│ ├── fifo.out pop queued prompt (FIFO output)
|
||||
│ ├── context rendered context window
|
||||
│ ├── systemprompt rendered system prompt
|
||||
│ ├── usage token usage stats
|
||||
│ ├── cost estimated cost
|
||||
│ ├── ctxsz context size
|
||||
│ ├── models available models
|
||||
│ ├── tools tool management (read all, write name to load)
|
||||
│ ├── toolsrv.loaded currently loaded tools
|
||||
│ ├── toolsrv.rev revision counter
|
||||
│ ├── tail exec helper
|
||||
│ └── proc/ detached background processes
|
||||
```
|
||||
|
||||
### 9P Interaction Model
|
||||
|
||||
All agent interaction uses the `ollie-9p` client, which auto-discovers the server via `$NAMESPACE` and identifies the caller via `$OLLIE_UNAME`. The 9P filesystem exposes session state, control, and inter-agent communication as files:
|
||||
- Sessions are directories under `s/`
|
||||
- Sessions are directories under `session/`
|
||||
- Prompts, state, chat history, and config are readable/writable files
|
||||
- Permission enforcement uses 9P user principals
|
||||
|
||||
### Permission Model (`9p/fs/perm.go`)
|
||||
### Permission Model (`fs/session/perm.go`)
|
||||
|
||||
All permissions are declared in a single registry (`fs.Perms`). Key design:
|
||||
- `prompt` is mode `0466` — group+other can write, owner (the agent) cannot
|
||||
All permissions are declared in a single registry (`Perms`). Key design:
|
||||
- `prompt` is mode `0666` — group+other can write, owner (the agent) cannot
|
||||
- `chat` is mode `0444` — read-only for everyone
|
||||
- `ctl` is mode `0666` — anyone can send control commands
|
||||
- `plan` is mode `0666` — readable by peer agents
|
||||
|
||||
### Session Manager (`9p/session/`)
|
||||
### Session Management (`fs/session/`)
|
||||
|
||||
The `Manager` handles session lifecycle:
|
||||
- Creating sessions (write to `s/new`)
|
||||
- Killing sessions (delete `s/{id}` or write "kill" to `s/{id}/ctl`)
|
||||
- Renaming sessions
|
||||
- Assigning unique numeric UIDs for 9P principals
|
||||
- Wiring up `agent.Core` with backends and tool servers
|
||||
- Persisting/restoring sessions across restarts
|
||||
The `*fs.Tree` IS the session collection. Package functions manage lifecycle:
|
||||
- `NewRoot(cfg)` — create the root tree
|
||||
- `Lookup(tree, id)` — find a session by ID
|
||||
- `All(tree)` — list all sessions
|
||||
- `CreateFromRoot(tree, args)` — create a new session
|
||||
- `KillFromRoot(tree, id)` — kill a session
|
||||
- `RenameFromRoot(tree, old, new)` — rename a session
|
||||
- `Shutdown(tree)` — clean shutdown
|
||||
|
||||
### Embedded D-Bus Adapter (`9p/dbus.go`)
|
||||
### Embedded D-Bus Adapter
|
||||
|
||||
`olliesrv` includes an embedded D-Bus adapter that claims `org.ollie.SessionManager` on the session bus. This provides:
|
||||
- The same method/signal interface used by KDE frontends
|
||||
|
|
@ -253,9 +298,9 @@ The `Manager` handles session lifecycle:
|
|||
|
||||
The adapter is best-effort: if no session bus is available (e.g., headless/container), `olliesrv` continues functioning as a pure 9P server.
|
||||
|
||||
## D-Bus Daemon (`dbus/`)
|
||||
## D-Bus Adapter (`dbus/`)
|
||||
|
||||
For D-Bus-only deployments (no 9P), run `olliesrv -no9p`. This disables the filesystem listener and per-session FUSE mounts while keeping the full D-Bus interface. Session persistence uses the same `~/.local/share/ollie/sessions/active/` directory.
|
||||
The D-Bus adapter is embedded in `olliesrv` (no separate daemon). For D-Bus-only deployments, run `olliesrv -no9p`. This disables the filesystem listener while keeping the full D-Bus interface. Session persistence uses the same `~/.local/share/ollie/sessions/active/` directory.
|
||||
|
||||
## KDE Integration (`kde/`)
|
||||
|
||||
|
|
@ -275,13 +320,15 @@ All KDE components use `QDBusServiceWatcher` to handle daemon restarts gracefull
|
|||
|
||||
### Architecture
|
||||
|
||||
The tool system has exactly one built-in server (`execute.Server`) that exposes three executors:
|
||||
The tool system has exactly one built-in server (`toolsrv.Server`) that exposes three primitives:
|
||||
|
||||
| Primitive | Purpose |
|
||||
|---|---|
|
||||
| `shell` | Run a bash command in a sandbox |
|
||||
| `tool_load`/`tool_list`/`tool_active` | Dynamic tool registry |
|
||||
| `skill_load`/`skill_list`/`skill_active` | Dynamic skill registry |
|
||||
|
||||
All other capabilities are **tool scripts** — executable files discovered from `OLLIE_TOOLS_PATH` (default: `~/.config/ollie/tools`).
|
||||
All other capabilities are **tool scripts** — executable files discovered from `OLLIE_TOOLS_PATH` (default: `~/.config/ollie/tools/`).
|
||||
|
||||
### Tool Scripts
|
||||
|
||||
|
|
@ -293,10 +340,11 @@ Tool scripts are plain executables with a structured header comment declaring th
|
|||
- `reasoning_think` — scratchpad for externalized reasoning
|
||||
- `subagent_spawn`, `subagent_generate` — sub-agent lifecycle
|
||||
- `route` — orchestrator dispatch
|
||||
- `gui_*` — KDE desktop automation (screenshot, windows, clipboard, input, ...)
|
||||
|
||||
Scripts annotated `ollie:parallel read` can be fanned out concurrently by the agent loop.
|
||||
|
||||
### Sandboxing (`internal/sandbox/`)
|
||||
### Sandboxing
|
||||
|
||||
Every code execution step is wrapped with [landrun](https://github.com/landlock-lsm/landrun) (Landlock LSM). Configuration is YAML-based:
|
||||
|
||||
|
|
@ -323,6 +371,9 @@ All backends implement the streaming-only `Backend` interface:
|
|||
| `OllamaBackend` | `/api/chat` | Local models, context length from `/api/show` |
|
||||
| `OpenAIBackend` | `/v1/chat/completions` | OpenAI, OpenRouter, any compatible API |
|
||||
| `AnthropicBackend` | `/v1/messages` | Extended thinking, prompt caching |
|
||||
| `GeminiBackend` | Gemini API | Google Gemini models |
|
||||
| `CopilotBackend` | GitHub Copilot API | Copilot Chat |
|
||||
| `CodeWhispererBackend` | CodeWhisperer API | AWS CodeWhisperer |
|
||||
|
||||
Shared infrastructure:
|
||||
- `streamRequest()` — common HTTP streaming with error classification
|
||||
|
|
@ -335,10 +386,10 @@ Multi-agent coordination is built entirely on the filesystem primitive:
|
|||
|
||||
### Mechanism
|
||||
|
||||
1. **Spawn**: write session spec to `s/new` → new session directory appears
|
||||
2. **Prompt**: write task to `s/{id}/prompt`
|
||||
3. **Observe**: read `s/{id}/state` or block on `s/{id}/statewait`
|
||||
4. **Result**: child writes to `s/{parent_id}/prompt` when done
|
||||
1. **Spawn**: write session spec to `session/new` → new session directory appears
|
||||
2. **Prompt**: write task to `session/{id}/prompt`
|
||||
3. **Observe**: read `session/{id}/state` or block on `session/{id}/statewait`
|
||||
4. **Result**: child writes to `session/{parent_id}/prompt` when done
|
||||
|
||||
### Delegation Tools
|
||||
|
||||
|
|
@ -359,26 +410,53 @@ ARTIFACTS: <paths>
|
|||
|
||||
The parent never actively waits — results arrive as queued prompts.
|
||||
|
||||
## HTTP Gateway (`httpgw/`)
|
||||
## HTTP Gateway (`cmd/ollie-httpgw/`)
|
||||
|
||||
`ollie-httpgw` bridges HTTP clients (like the web UI) to the D-Bus session manager. It translates REST-style requests into `org.ollie.SessionManager` method calls and streams `ChatUpdated` signals back as Server-Sent Events.
|
||||
`ollie-httpgw` bridges HTTP clients (like the web UI) to the 9P filesystem. It translates REST-style requests into 9P filesystem operations and streams events back as Server-Sent Events.
|
||||
|
||||
This is the secondary integration path for non-LLM consumers (web UIs, dashboards, external tools) that want a conventional HTTP API without speaking 9P or D-Bus directly.
|
||||
This is the secondary integration path for non-LLM consumers (web UIs, dashboards, external tools) that want a conventional HTTP API without speaking 9P directly.
|
||||
|
||||
## Frontends
|
||||
|
||||
Frontends connect via either 9P (filesystem) or D-Bus, depending on preference:
|
||||
|
||||
| Frontend | Technology | Connection |
|
||||
|---|---|---|
|
||||
| `s/sh` | Terminal (readline) | 9P |
|
||||
| `ollie-acme` | Plan 9 acme editor | 9P |
|
||||
| `ellie` | Emacs Lisp | 9P |
|
||||
| KDE GUI | Qt/QML standalone app | D-Bus |
|
||||
| Plasmoid | Plasma widget | D-Bus |
|
||||
| Kate plugin | KTextEditor plugin | D-Bus |
|
||||
| KRunner | Runner plugin | D-Bus |
|
||||
| Web UI | Preact SPA | D-Bus (via `httpgw`) |
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph Core["agent.Agent"]
|
||||
AG["Agent Engine"]
|
||||
end
|
||||
|
||||
subgraph Surfaces["Integration Surfaces"]
|
||||
P9["9P Filesystem\n(session/ namespace)"]
|
||||
DB["D-Bus\n(org.ollie.SessionManager)"]
|
||||
end
|
||||
|
||||
subgraph Frontends
|
||||
SH["s/sh (terminal)"]
|
||||
ACME["acme (Plan 9)"]
|
||||
EL["ellie (Emacs)"]
|
||||
KG["KDE GUI"]
|
||||
KP["KDE Plasmoid"]
|
||||
KK["Kate Plugin"]
|
||||
KR["KRunner"]
|
||||
WEB["Web UI"]
|
||||
HTTP["curl / scripts"]
|
||||
end
|
||||
|
||||
AG --> P9
|
||||
AG --> DB
|
||||
|
||||
SH --> P9
|
||||
ACME --> P9
|
||||
EL --> P9
|
||||
HTTP --> P9
|
||||
|
||||
KG --> DB
|
||||
KP --> DB
|
||||
KK --> DB
|
||||
KR --> DB
|
||||
WEB --> DB
|
||||
```
|
||||
|
||||
Frontends are fully decoupled — the core doesn't know which interface they use.
|
||||
|
||||
|
|
@ -419,28 +497,42 @@ The `prompt` array is the key innovation: each element is a shell command whose
|
|||
|
||||
## Data Flow
|
||||
|
||||
```
|
||||
User input (via any frontend)
|
||||
│
|
||||
▼ write to s/{id}/prompt
|
||||
olliesrv
|
||||
│
|
||||
▼ agent.Core.Submit()
|
||||
agent loop (loop.go)
|
||||
│
|
||||
├── backend.ChatStream() → LLM API call (streaming)
|
||||
│
|
||||
├── tools.Dispatcher.Dispatch() → execute.Server
|
||||
│ │
|
||||
│ ├── {code} steps → landrun sandbox → subprocess
|
||||
│ └── {tool} steps → landrun sandbox → tool script
|
||||
│
|
||||
├── session.update() → append to message history
|
||||
│
|
||||
└── loop (until no tool calls)
|
||||
│
|
||||
▼ event bus → session log
|
||||
Frontend reads s/{id}/chat (streaming)
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant F as Frontend
|
||||
participant S as olliesrv
|
||||
participant A as agent.Agent
|
||||
participant B as LLM Backend
|
||||
participant T as toolsrv.Server
|
||||
participant SH as sandbox (landrun)
|
||||
|
||||
F->>S: write to session/{id}/prompt
|
||||
S->>A: Submit()
|
||||
activate A
|
||||
|
||||
loop Agent Loop
|
||||
A->>B: ChatStream(messages, tools)
|
||||
activate B
|
||||
B-->>A: stream events
|
||||
deactivate B
|
||||
|
||||
alt has tool calls
|
||||
A->>T: Dispatch(tool, args)
|
||||
activate T
|
||||
T->>SH: sandboxed execution
|
||||
SH-->>T: result
|
||||
T-->>A: ToolResult
|
||||
deactivate T
|
||||
A->>A: append to history
|
||||
else no tool calls
|
||||
A->>A: turn complete
|
||||
break
|
||||
end
|
||||
end
|
||||
|
||||
A-->>S: turn complete
|
||||
deactivate A
|
||||
S-->>F: readable via session/{id}/chat
|
||||
```
|
||||
|
||||
## Extension Points
|
||||
|
|
@ -449,7 +541,7 @@ Frontend reads s/{id}/chat (streaming)
|
|||
|---|---|
|
||||
| Add an LLM backend | Implement `backend.Backend` |
|
||||
| Add a tool | Drop an executable into `OLLIE_TOOLS_PATH` |
|
||||
| Add a tool server | Implement `tools.Server`, register in `NewDispatcherFunc` |
|
||||
| Add a tool server | Implement `toolsrv.Runner`, register in `NewToolServer` |
|
||||
| Add a frontend | Connect via 9P (filesystem) or D-Bus (method calls) |
|
||||
| Add a skill | Drop a markdown file into `skills/` |
|
||||
| Add an agent persona | Create a JSON config in `agents/` |
|
||||
|
|
@ -458,7 +550,7 @@ Frontend reads s/{id}/chat (streaming)
|
|||
## Key Design Decisions
|
||||
|
||||
1. **Two integration surfaces** — 9P filesystem (canonical, Unix philosophy) and D-Bus (conventional, for GUI/web consumers). Both are thin adapters over the same core.
|
||||
2. **Seven built-in primitives** — `shell`, `tool_*`, `skill_*` are the only compiled-in capabilities. This keeps the core small and makes all other tools equal citizens.
|
||||
2. **Three built-in primitives** — `shell`, tool registry, skill registry. Everything else is a script. This keeps the core small and makes all other tools equal citizens.
|
||||
3. **Mandatory sandboxing** — no unsandboxed fallback. Security is not optional.
|
||||
4. **Streaming-only backends** — simplifies the interface; blocking APIs wrap as single-event streams.
|
||||
5. **Fire-and-forget sub-agents** — no active waiting, no polling. Results arrive as filesystem writes.
|
||||
|
|
|
|||
416
doc/CORE.md
416
doc/CORE.md
|
|
@ -1,115 +1,139 @@
|
|||
# ollie-core: Detailed Architecture
|
||||
|
||||
This document covers the internal design of the `core/` library — the Go package that implements the agent engine. It is the dependency imported by all consumers (olliesrv, frontends, tests).
|
||||
This document covers the internal design of the Go library — the packages that implement the agent engine. It is the dependency imported by all consumers (olliesrv, frontends, tests).
|
||||
|
||||
## Package Map
|
||||
|
||||
```
|
||||
core/
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph Agent["agent/ package"]
|
||||
AGENT["agent.go\nAgent struct, New()"]
|
||||
LOOP["loop.go\nrun() — agent loop"]
|
||||
HISTORY["history.go\nHistory, TaskState, compaction"]
|
||||
TURN["turn.go\nper-turn state machine"]
|
||||
HOOKS["hooks.go\nlifecycle callbacks"]
|
||||
CMDS["commands.go\n/command dispatch"]
|
||||
BUILD["build_runtime.go\nBuildRuntime()"]
|
||||
PROMPT["prompt_resolver.go\nsystem prompt assembly"]
|
||||
STATE["state.go\nResultTier, toolResult"]
|
||||
COST["cost.go\npricing table"]
|
||||
FIFO["fifo.go\nprompt queue"]
|
||||
COMPLETE["complete.go\ncode completion"]
|
||||
CONFIG["agent_config.go\nConfig schema"]
|
||||
CONFIG_PATHS["config_paths.go\nXDG config discovery"]
|
||||
RUNTIME["runtime.go\nRuntime struct"]
|
||||
NEW["new.go\nNew() factory"]
|
||||
USAGE["usage_log.go\nusage tracking"]
|
||||
end
|
||||
|
||||
│ ├── agent/ The agent engine
|
||||
│ │ ├── types.go Core interface, Event, PromptFIFO
|
||||
│ │ ├── runtime.go Runtime struct (swappable per-agent config)
|
||||
│ │ ├── core.go AgentCoreConfig, agent struct, NewAgentCore, BuildRuntime
|
||||
│ │ ├── loop.go The agent loop (run function)
|
||||
│ │ ├── session.go Session, TaskState, compaction, persistence
|
||||
│ │ ├── state.go state interface, ResultTier, toolResult
|
||||
│ │ ├── hooks.go Hooks map, HookResult, hook execution
|
||||
│ │ ├── commands.go Slash-command dispatch (/model, /agent, /compact, ...)
|
||||
│ │ ├── cost.go Model pricing table, cost computation
|
||||
│ │ └── prompt_resolver.go System prompt assembly from shell commands
|
||||
│ ├── backend/ LLM provider implementations
|
||||
│ │ ├── backend.go Backend interface, shared types, streamRequest
|
||||
│ │ ├── openai.go OpenAI/OpenRouter/compatible APIs
|
||||
│ │ ├── anthropic.go Anthropic Messages API
|
||||
│ │ └── ollama.go Ollama local inference
|
||||
│ ├── config/ Agent config JSON schema and loader
|
||||
│ ├── env/ Environment variable loading
|
||||
│ ├── log/ Structured logger
|
||||
│ ├── paths/ XDG directory resolution
|
||||
│ └── tools/ Tool system
|
||||
│ ├── tools.go Server, Dispatcher, classifier interfaces
|
||||
│ └── execute/ The execute.Server
|
||||
│ ├── server.go Server struct, Decl factory, ListTools, options
|
||||
│ ├── code.go Step resolution, validation, dispatch routing
|
||||
│ ├── tool.go Named tool script discovery and execution
|
||||
│ ├── flock.go Advisory locking, parallelism classification
|
||||
│ └── tier.go Result tier classification (hot/warm/cold)
|
||||
└── internal/
|
||||
└── sandbox/ Landlock sandbox wrapper
|
||||
├── config.go YAML config, path expansion, access checking
|
||||
└── wrapper.go landrun command construction
|
||||
subgraph Backend["backend/ package"]
|
||||
BACKEND["backend.go\nBackend interface"]
|
||||
OPENAI["openai.go"]
|
||||
ANTHROPIC["anthropic.go"]
|
||||
OLLAMA["ollama.go"]
|
||||
GEMINI["gemini.go"]
|
||||
COPILOT["copilot.go"]
|
||||
CW["codewhisperer.go"]
|
||||
POOL["pool.go\nbackend pool"]
|
||||
end
|
||||
|
||||
subgraph Toolsrv["toolsrv/ package"]
|
||||
SERVER["server.go\nServer struct"]
|
||||
SHELL["shell.go\nshell execution"]
|
||||
TOOLS["tools.go\nRunner interface"]
|
||||
REGISTRY["registry.go\ntool discovery"]
|
||||
DISCOVER["discover.go\ntool script scanning"]
|
||||
SCHEMA["schema.go\nJSON Schema parsing"]
|
||||
TIER["tier.go\nresult tiering"]
|
||||
SKILLS["skills.go\nskill management"]
|
||||
STREAM["stream.go\noutput streaming"]
|
||||
REMOTE["remote.go\nRemoteServer"]
|
||||
DIAL["dial.go\nSSH bootstrap"]
|
||||
ACCESSORS["accessors.go\nIsParallelRead, ResultTier"]
|
||||
BOOTSTRAP["bootstrap.sh\nSSH bootstrap script"]
|
||||
end
|
||||
|
||||
subgraph Session["session/ package"]
|
||||
SESS["session.go\nSession, Config"]
|
||||
end
|
||||
|
||||
subgraph FsSession["fs/session/ package"]
|
||||
TREE["tree.go\n*fs.Tree"]
|
||||
ROOT["root.go\nNewRoot()"]
|
||||
CREATE["create.go\nCreateFromRoot()"]
|
||||
FILES["files.go\nfile operations"]
|
||||
PERM["perm.go\npermission registry"]
|
||||
PERSIST["persist.go\nsession persistence"]
|
||||
FSGO["fs.go\n9P filesystem"]
|
||||
FORMAT["format.go\nevent formatting"]
|
||||
SYNTH["synth.go\nsynthetic files"]
|
||||
TYPES["types.go\nSession struct"]
|
||||
end
|
||||
|
||||
subgraph Support["Supporting Packages"]
|
||||
DETACH["detach/\nprocess management"]
|
||||
ELEVATE["elevate/\nelevation broker"]
|
||||
SANDBOX["sandbox/\nLandlock config"]
|
||||
ENV["env/\nenvironment"]
|
||||
LOG["log/\nstructured logging"]
|
||||
PATHS["paths/\nXDG resolution"]
|
||||
end
|
||||
|
||||
Agent --> Backend
|
||||
Agent --> Toolsrv
|
||||
Agent --> Session
|
||||
FsSession --> Session
|
||||
Toolsrv --> SANDBOX
|
||||
Toolsrv --> DETACH
|
||||
Toolsrv --> ELEVATE
|
||||
```
|
||||
|
||||
## The Core Interface (`agent/types.go`)
|
||||
## The Agent Struct (`agent/agent.go`)
|
||||
|
||||
`Core` is the boundary between the engine and any consumer. It is deliberately narrow:
|
||||
`Agent` is the boundary between the engine and any consumer. It is a concrete struct, not an interface — the interface was removed during the Great Flattening because no consumer ever implemented it independently.
|
||||
|
||||
```go
|
||||
type Core interface {
|
||||
// Turn lifecycle
|
||||
Submit(ctx context.Context, input string) // process user input (commands or prompts)
|
||||
Interrupt(cause error) bool // cancel current turn
|
||||
Inject(prompt string) // append to next tool result
|
||||
Queue(prompt string) // FIFO for deferred prompts
|
||||
PopQueue() (string, bool) // drain one queued prompt
|
||||
|
||||
// Observation
|
||||
IsRunning() bool
|
||||
State() string // "idle" | "thinking" | "calling: <tool>"
|
||||
Reply() string // last assistant text
|
||||
Bus() *pubsub.Bus // event stream
|
||||
WaitChange(ctx context.Context, field, current string) (string, bool)
|
||||
|
||||
// Configuration
|
||||
AgentName() string
|
||||
BackendName() string
|
||||
ModelName() string
|
||||
CWD() string
|
||||
SetCWD(dir string) error
|
||||
SetSessionID(newID string) error
|
||||
SetEnv(key, value string)
|
||||
SetGenerationParams(params backend.GenerationParams) error
|
||||
|
||||
// Introspection
|
||||
Context() []backend.Message
|
||||
SystemPrompt() string
|
||||
GenerationParams() backend.GenerationParams
|
||||
CtxSz() string
|
||||
Usage() string
|
||||
Cost() string
|
||||
ListModels() string
|
||||
ToolCallCount() int64
|
||||
|
||||
// Cleanup
|
||||
Close()
|
||||
type Agent struct {
|
||||
history *History
|
||||
runtime *Runtime
|
||||
cfg agentConfig
|
||||
agentName string
|
||||
agentsDir string
|
||||
baseLayers []string
|
||||
promptEnvExtra []string
|
||||
newToolServer func() toolsrv.Runner
|
||||
newBackend func(string) (backend.Backend, error)
|
||||
currentAction atomic.Pointer[actionHandle]
|
||||
// ... execution state, sync primitives, session metadata
|
||||
}
|
||||
|
||||
func New(cfg AgentConfig) *Agent
|
||||
```
|
||||
|
||||
All output flows through the event bus as `Event{Role, Name, Content}` values. Roles include: `assistant`, `tool`, `call`, `info`, `exec`, `error`, `usage`, `reasoning`, `limitretry`.
|
||||
|
||||
## Agent Construction (`agent/core.go`)
|
||||
## Agent Construction (`agent/new.go`)
|
||||
|
||||
### AgentCoreConfig
|
||||
### AgentConfig
|
||||
|
||||
The full construction config:
|
||||
|
||||
| Field | Purpose |
|
||||
|---|---|
|
||||
| `Backend` | Active LLM backend instance |
|
||||
| `ModelName` | Override backend's default model |
|
||||
| `History` | Pre-existing session history to restore (nil = new) |
|
||||
| `Runtime` | Pre-built *Runtime (tools, hooks, prompt, params) |
|
||||
| `AgentName` | Name of the active agent config |
|
||||
| `AgentsDir` | Path to agent JSON directory |
|
||||
| `SessionsDir` | Path to persisted session files |
|
||||
| `SessionID` | Unique session identifier |
|
||||
| `Uname` | Immutable 9P user principal |
|
||||
| `AgentID` | Immutable agent principal |
|
||||
| `CWD` | Working directory for tools and prompt |
|
||||
| `Session` | Pre-existing session to restore (nil = new) |
|
||||
| `Runtime` | Pre-built *Runtime (tools, hooks, prompt, params) |
|
||||
| `NewDispatcher` | Factory for fresh Dispatcher on agent switch |
|
||||
| `NewToolServer` | Factory for fresh toolsrv.Runner on agent switch |
|
||||
| `NewBackend` | Factory for backend by name |
|
||||
| `Log` | Logger instance |
|
||||
| `MaxSteps` | Override agent config's maxSteps |
|
||||
| `ReadPlanStep` | Function to read next unchecked plan step |
|
||||
| `ListHandlers` | Additional list handlers |
|
||||
|
||||
### Runtime
|
||||
|
||||
|
|
@ -118,67 +142,68 @@ The `Runtime` struct holds all per-agent configuration that changes on an `/agen
|
|||
```go
|
||||
type Runtime struct {
|
||||
Backend backend.Backend
|
||||
Dispatcher tools.Dispatcher
|
||||
Runner toolsrv.Runner
|
||||
Hooks Hooks
|
||||
Preamble string // compiled system prompt
|
||||
Tools []backend.Tool
|
||||
Exec toolExecutor
|
||||
ClassifyTool toolClassifier
|
||||
ClassifyTier func(string) ResultTier
|
||||
ClassifyTool func(string) bool // parallelism check
|
||||
ClassifyTier func(string, json.RawMessage) string
|
||||
GenParams backend.GenerationParams
|
||||
MaxSteps int
|
||||
ToolResultMaxBytes int // 0 means use default (128KB)
|
||||
CfgBackend string // backend override from agent JSON
|
||||
CfgModel string // model override from agent JSON
|
||||
Messages []string // non-fatal warnings from construction
|
||||
ToolResultMaxBytes int
|
||||
CfgBackend string
|
||||
CfgModel string
|
||||
Messages []string // non-fatal warnings
|
||||
}
|
||||
```
|
||||
|
||||
### BuildRuntime
|
||||
|
||||
`BuildRuntime(cfg, dispatcher, cwd, env)` wires everything together:
|
||||
`BuildRuntime(cfg, runner, cwd, env)` wires everything together:
|
||||
|
||||
1. Lists all tools from the dispatcher
|
||||
1. Lists all tools from the runner
|
||||
2. Resolves the system prompt by executing the config's `prompt` commands
|
||||
3. Extracts generation parameters from the config
|
||||
4. Builds the tool executor closure (routes calls through the dispatcher)
|
||||
5. Extracts parallelism and tier classifiers from the execute server
|
||||
4. Builds the tool executor closure (routes calls through the runner)
|
||||
5. Extracts parallelism and tier classifiers
|
||||
6. Applies tool/executor restrictions if configured
|
||||
7. Returns a `*Runtime` struct ready for `NewAgentCore`
|
||||
7. Returns a `*Runtime` struct ready for `New`
|
||||
|
||||
### NewAgentCore
|
||||
### New
|
||||
|
||||
Creates the `agent` struct from `AgentCoreConfig`:
|
||||
Creates the `Agent` struct from `AgentConfig`:
|
||||
1. Sweeps stale tmpdirs from previous crashes
|
||||
2. Stores the provided `Runtime` (or creates an empty one)
|
||||
3. Sets the backend model if `ModelName` is specified
|
||||
4. Creates `/tmp/ollie/{sessionID}` for this session
|
||||
5. Sets up the plan file reader (reads `s/{id}/plan` for the next unchecked step)
|
||||
5. Sets up the plan file reader
|
||||
6. Wires the `turnError` hook with a detached context
|
||||
7. Pushes `OLLIE_SESSION_ID` and `OLLIE_UNAME` into the execute server env
|
||||
7. Pushes `OLLIE_SESSION_ID` and `OLLIE_AGENT_ID` into the runner env
|
||||
8. Sets the flock directory for parallel coordination
|
||||
|
||||
## The Agent Loop (`agent/loop.go`)
|
||||
|
||||
The `run()` function is the core execution engine. It takes an `agentConfig` and a `state` interface, and loops until the model stops calling tools.
|
||||
|
||||
The `agentConfig` struct bundles per-turn dependencies: `Backend`, `Tools`, `Exec` (tool executor closure), `ClassifyTool`/`ClassifyTier` (parallelism and retention classifiers), `ToolResultMaxBytes`, `Output` (event handler), `preamble`, `GenerationParams`, and lifecycle callbacks: `PopInject`, `AutoCompact`, `SaveSession`, `PreTool`, `PostTool`, `TurnError`, `MaxSteps`, `ReadPlanStep`, and `IncrToolCallCount`.
|
||||
|
||||
### Per-Iteration Flow
|
||||
|
||||
```
|
||||
1. Context gate: strip cold material if over budget
|
||||
2. Build message history:
|
||||
- If TaskState exists: system prompt + task state + hot tail only
|
||||
- Otherwise: system prompt + full history
|
||||
3. Stream from backend (with retry logic)
|
||||
4. If no tool calls: break (turn complete)
|
||||
5. Execute tool calls (serial or parallel)
|
||||
6. Update state (append messages + results)
|
||||
7. Infer TaskState updates from what happened
|
||||
8. Safety checks (error limits, stall, replan, max steps)
|
||||
9. Auto-compact if needed
|
||||
10. Loop
|
||||
```mermaid
|
||||
flowchart TB
|
||||
START["Start turn"]
|
||||
GATE["Context gate:\nstrip cold material\nif over budget"]
|
||||
BUILD["Build message history:\nsystem prompt + task state + hot tail"]
|
||||
STREAM["Stream from backend\n(with retry logic)"]
|
||||
CHECK{"Tool calls\nin response?"}
|
||||
EXEC["Execute tool calls\n(serial or parallel)"]
|
||||
UPDATE["Update state:\nappend messages + results"]
|
||||
INFER["Infer TaskState\nupdates"]
|
||||
SAFETY["Safety checks:\nerror limits, stall,\nreplan, max steps"]
|
||||
COMPACT["Auto-compact\nif needed"]
|
||||
DONE["Turn complete"]
|
||||
|
||||
START --> GATE --> BUILD --> STREAM --> CHECK
|
||||
CHECK -- yes --> EXEC --> UPDATE --> INFER --> SAFETY --> COMPACT --> BUILD
|
||||
CHECK -- no --> DONE
|
||||
```
|
||||
|
||||
### Streaming & Retry
|
||||
|
|
@ -187,9 +212,9 @@ The streaming loop handles three failure modes:
|
|||
|
||||
| Failure | Detection | Recovery |
|
||||
|---|---|---|
|
||||
| Rate limit (429) | `*RateLimitError` | Exponential backoff (5×2^attempt seconds), up to 3 retries |
|
||||
| Rate limit (429) | `*RateLimitError` | Exponential backoff (5x2^attempt seconds), up to 3 retries |
|
||||
| Transient (5xx, network) | `*TransientError` | Same retry logic |
|
||||
| Stream drop | `Done` never received, channel closes | Retry with 2×2^attempt second delay |
|
||||
| Stream drop | `Done` never received, channel closes | Retry with 2x2^attempt second delay |
|
||||
| Context overflow | `*ContextOverflowError` | Trigger compaction, retry |
|
||||
| Tool unsupported | `*ToolUnsupportedError` | Propagate to turnError hook |
|
||||
|
||||
|
|
@ -231,13 +256,13 @@ After each tool round, `inferTaskStateUpdate` updates the structured overlay:
|
|||
- `LastAction` ← most recent tool call + truncated result
|
||||
- `PlanStep` ← next unchecked item from the plan file (or heuristic from assistant text)
|
||||
|
||||
## Session & Context (`agent/session.go`)
|
||||
## Session & Context (`agent/history.go`)
|
||||
|
||||
### Message History
|
||||
|
||||
`Session` holds a flat `[]backend.Message` slice. No rolling window — grows until compaction.
|
||||
`History` holds a flat `[]backend.Message` slice. No rolling window — grows until compaction.
|
||||
|
||||
**Checkpoint**: `Session.Checkpoint(ts TaskState)` forks the session — returns a new Session that inherits the given TaskState but starts with a clean message history. Used for narrow-context sub-agents that know what to do without inheriting parent message noise.
|
||||
**Checkpoint**: `History.Checkpoint(ts TaskState)` forks the session — returns a new History that inherits the given TaskState but starts with a clean message history. Used for narrow-context sub-agents that know what to do without inheriting parent message noise.
|
||||
|
||||
Token tracking per session:
|
||||
- `TotalInputTokens`, `TotalOutputTokens`, `TotalRequests`
|
||||
|
|
@ -275,7 +300,7 @@ Tool results are classified at execution time:
|
|||
| Cold | Immediately collapsed to one-line summary | `file_read`, `memory_recall`, `lsp_*` |
|
||||
|
||||
Classification sources (in priority order):
|
||||
1. Built-in table in `execute/tier.go` (`defaultColdTools`)
|
||||
1. Built-in table in `toolsrv/tier.go` (`defaultColdTools`)
|
||||
2. `ollie:tier` annotation in the tool script header
|
||||
3. Default: hot
|
||||
|
||||
|
|
@ -291,7 +316,7 @@ Sessions are saved as `PersistedSession` JSON:
|
|||
}
|
||||
```
|
||||
|
||||
Saved after every tool round (`SaveSession` callback) and on clean exit.
|
||||
Saved after every tool round (`saveSession` callback) and on clean exit.
|
||||
|
||||
## Hooks (`agent/hooks.go`)
|
||||
|
||||
|
|
@ -341,52 +366,38 @@ Multiple commands per hook run sequentially. If any exits 2, execution stops imm
|
|||
| `/sp` | Show rendered system prompt |
|
||||
| `/i <prompt>` | Inject into running turn |
|
||||
| `/irw <prompt>` | Inject-rewrite |
|
||||
| `/queued [pop\|clear]` | Manage queued prompts |
|
||||
| `/queued [pop|clear]` | Manage queued prompts |
|
||||
| `/help` | Show command help |
|
||||
|
||||
### Agent Switching
|
||||
|
||||
When `/agent <name>` is issued:
|
||||
1. Load the new agent config JSON
|
||||
2. Create a fresh dispatcher via `newDispatcher()`
|
||||
2. Create a fresh runner via `newToolServer()`
|
||||
3. Call `BuildRuntime` with the new config
|
||||
4. If the config specifies a backend/model, switch those too
|
||||
5. Replace the entire `Runtime` pointer on the agent struct
|
||||
6. Clear the session (fresh context for the new persona)
|
||||
|
||||
## Tool System (`tools/`)
|
||||
## Tool System (`toolsrv/`)
|
||||
|
||||
### Dispatcher
|
||||
### Runner Interface
|
||||
|
||||
The `Dispatcher` is a simple name→Server router:
|
||||
The `Runner` is the minimal interface satisfied by any tool server (local or remote):
|
||||
|
||||
```go
|
||||
type Dispatcher interface {
|
||||
AddServer(name string, s Server)
|
||||
GetServer(name string) (Server, bool)
|
||||
type Runner interface {
|
||||
ListTools() ([]ToolInfo, error)
|
||||
Dispatch(ctx context.Context, server, tool string, args json.RawMessage) (json.RawMessage, error)
|
||||
CallTool(ctx context.Context, tool string, args json.RawMessage) (json.RawMessage, error)
|
||||
}
|
||||
```
|
||||
|
||||
`NewDispatcherFunc` returns a factory that creates a fresh dispatcher with all servers instantiated. Called on agent switches to get clean server state.
|
||||
### Server
|
||||
|
||||
### Optional Server Interfaces
|
||||
|
||||
Servers may implement additional interfaces for runtime configuration:
|
||||
|
||||
| Interface | Purpose |
|
||||
|---|---|
|
||||
| `CWDSetter` | Update working directory |
|
||||
| `EnvSetter` | Inject session env vars |
|
||||
| `LockDirSetter` | Set flock directory for parallel coordination |
|
||||
| `ToolRestrictionSetter` | Restrict available executors/tools |
|
||||
| `ParallelClassifier` | Report if a tool is read-safe |
|
||||
| `TierClassifier` | Report result retention tier |
|
||||
|
||||
### execute.Server (`execute/`)
|
||||
|
||||
The built-in server exposes seven primitives: `shell`, `tool_load`, `tool_list`, `tool_active`, `skill_load`, `skill_list`, `skill_active`.
|
||||
The concrete `Server` struct implements `Runner` and provides:
|
||||
- **shell**: sandboxed bash command execution
|
||||
- **tool registry**: `tool_load`, `tool_list`, `tool_active` — lazy tool discovery
|
||||
- **skill registry**: `skill_load`, `skill_list`, `skill_active` — skill management
|
||||
|
||||
#### shell
|
||||
|
||||
|
|
@ -414,7 +425,7 @@ Before execution, inline code is checked against dangerous patterns:
|
|||
|
||||
Validation failures are rate-limited: 5 failures within 1 minute triggers a 5-minute block.
|
||||
|
||||
### Parallelism & Locking (`execute/flock.go`)
|
||||
### Parallelism & Locking
|
||||
|
||||
Tool scripts declare their concurrency class via header annotations:
|
||||
|
||||
|
|
@ -426,7 +437,7 @@ Tool scripts declare their concurrency class via header annotations:
|
|||
|
||||
Lock files live in `/tmp/ollie/{sessionID}/` and are acquired via `flock(2)`.
|
||||
|
||||
### Result Tier Classification (`execute/tier.go`)
|
||||
### Result Tier Classification (`toolsrv/tier.go`)
|
||||
|
||||
Built-in cold tools (results consumed immediately, don't need verbatim retention):
|
||||
- `file_read`, `memory_recall`, `web_fetch`, `web_search`
|
||||
|
|
@ -438,9 +449,29 @@ Custom tools declare their tier via `ollie:tier cold|warm|hot` in the script hea
|
|||
|
||||
Steps marked `elevated: true` bypass the sandbox entirely. They are dispatched to the elevation backend (`x/elevate`) which runs outside landrun. Only bash is supported for elevated steps.
|
||||
|
||||
### FUSE Mount
|
||||
### Remote Execution
|
||||
|
||||
When `WithMount()` is set, the execute server spawns `ollie-9p-mount -u {uname}` to create a per-session FUSE mount. This gives the agent its own 9P identity for permission enforcement. The mount path is exposed as `$OLLIE` in subprocess environments.
|
||||
When a session is configured with `remote=user@host`, the runner is a `RemoteServer` that forwards tool calls over SSH to an `ollie-remote` binary:
|
||||
|
||||
```go
|
||||
type RemoteServer struct {
|
||||
conn *rpc.Conn // JSON-RPC over SSH stdin/stdout
|
||||
}
|
||||
|
||||
func (r *RemoteServer) CallTool(ctx context.Context, tool string, args json.RawMessage) (json.RawMessage, error) {
|
||||
return r.conn.Call(ctx, "execute", tool, args)
|
||||
}
|
||||
```
|
||||
|
||||
The agent loop doesn't know or care whether execution is local or remote. The `NewToolServer` factory returns a `RemoteServer` instead of a local `Server` when the session is configured for remote execution.
|
||||
|
||||
SSH bootstrap:
|
||||
1. Open SSH connection to target host
|
||||
2. Send bootstrap script over stdin
|
||||
3. Bootstrap checks for cached `ollie-remote` binary (by SHA-256 hash)
|
||||
4. If not cached: receive binary over stdin (gzipped + base64)
|
||||
5. Launch `ollie-remote` with the working directory
|
||||
6. Switch to JSON-RPC over stdin/stdout
|
||||
|
||||
## Backend System (`backend/`)
|
||||
|
||||
|
|
@ -493,33 +524,42 @@ Shared HTTP streaming infrastructure used by all backends:
|
|||
- 200k context window (hardcoded)
|
||||
- SSE streaming with content_block_delta events
|
||||
|
||||
**GeminiBackend** (Gemini API):
|
||||
- Google Gemini models
|
||||
|
||||
**CopilotBackend** (GitHub Copilot API):
|
||||
- Copilot Chat integration
|
||||
|
||||
**CodeWhispererBackend** (AWS CodeWhisperer API):
|
||||
- AWS CodeWhisperer integration
|
||||
|
||||
## Config System (`agent/`)
|
||||
|
||||
### Agent Config Schema
|
||||
|
||||
```go
|
||||
type Config struct {
|
||||
Hooks map[string]HookCmds // lifecycle callbacks
|
||||
Prompt Prompt // system prompt (string or command array)
|
||||
Backend string // backend override
|
||||
Model string // model override
|
||||
Tools *bool // enable/disable tools (default: true)
|
||||
AllowExecutors []string // restrict executors
|
||||
AllowTools []string // restrict tool scripts
|
||||
MaxTokens int // generation limit
|
||||
MaxCompletionTokens int // OpenAI o-series
|
||||
MaxSteps int // tool rounds per turn
|
||||
Hooks map[string]HookCmds
|
||||
Prompt Prompt
|
||||
Backend string
|
||||
Model string
|
||||
Tools *bool
|
||||
AllowExecutors []string
|
||||
AllowTools []string
|
||||
MaxTokens int
|
||||
MaxCompletionTokens int
|
||||
MaxSteps int
|
||||
Temperature *float64
|
||||
TopP, TopK, MinP, TopA *float64/*int
|
||||
FrequencyPenalty *float64
|
||||
PresencePenalty *float64
|
||||
RepetitionPenalty *float64
|
||||
Reasoning int // Anthropic thinking budget
|
||||
ReasoningEffort string // OpenAI: low/medium/high
|
||||
IncludeReasoning *bool // OpenRouter
|
||||
ResponseFormat string // json_object, text
|
||||
Stop []string // stop sequences
|
||||
Verbosity string // output detail level
|
||||
Reasoning int
|
||||
ReasoningEffort string
|
||||
IncludeReasoning *bool
|
||||
ResponseFormat string
|
||||
Stop []string
|
||||
Verbosity string
|
||||
}
|
||||
```
|
||||
|
||||
|
|
@ -563,32 +603,32 @@ When the backend doesn't report cost directly (only OpenRouter does), cost is co
|
|||
- Cache creation tokens charged at 125% of input rate
|
||||
- Unknown/local models return zero cost
|
||||
|
||||
## Sandbox (`internal/sandbox/`)
|
||||
## Sandbox (`sandbox/`)
|
||||
|
||||
### Config Format
|
||||
|
||||
```yaml
|
||||
general:
|
||||
best_effort: true # don't fail if kernel doesn't support all features
|
||||
log_level: "" # landrun log level
|
||||
best_effort: true
|
||||
log_level: ""
|
||||
|
||||
filesystem:
|
||||
ro: ["/usr", "/lib", "/etc"] # read-only
|
||||
rox: ["/usr/bin", "/usr/local/bin"] # read + execute
|
||||
rw: ["{CWD}", "{TMPDIR}"] # read + write
|
||||
rwx: [] # read + write + execute
|
||||
ro: ["/usr", "/lib", "/etc"]
|
||||
rox: ["/usr/bin", "/usr/local/bin"]
|
||||
rw: ["{CWD}", "{TMPDIR}"]
|
||||
rwx: []
|
||||
|
||||
network:
|
||||
enabled: true
|
||||
unrestricted: true # all TCP allowed
|
||||
bind_tcp: [] # specific bind ports
|
||||
connect_tcp: [] # specific connect targets
|
||||
unrestricted: true
|
||||
bind_tcp: []
|
||||
connect_tcp: []
|
||||
|
||||
env: [] # extra environment variables
|
||||
env: []
|
||||
|
||||
advanced:
|
||||
ldd: false # resolve shared library paths
|
||||
add_exec: false # add execute to all read paths
|
||||
ldd: false
|
||||
add_exec: false
|
||||
```
|
||||
|
||||
### Template Variables
|
||||
|
|
@ -606,32 +646,28 @@ advanced:
|
|||
| `{OLLIE_CFG_PATH}` | ollie config directory |
|
||||
| `{OLLIE_DATA_PATH}` | ollie data directory |
|
||||
|
||||
### Path Validation
|
||||
|
||||
`checkPath(cfg, path, write, cwd, getenv)` validates that a given absolute path falls within the sandbox's allowed paths. Symlinks are resolved before checking. Used by file tools to pre-validate paths before execution.
|
||||
|
||||
## Event System
|
||||
|
||||
The agent communicates exclusively through events published on a `pubsub.Bus`:
|
||||
|
||||
```go
|
||||
type Event struct {
|
||||
Role string // event type
|
||||
Name string // tool name (for call/tool events)
|
||||
Content string // payload
|
||||
Role string
|
||||
Name string
|
||||
Content string
|
||||
}
|
||||
```
|
||||
|
||||
| Role | When |
|
||||
|---|---|
|
||||
| `assistant` | Streaming text delta from the model |
|
||||
| `reasoning` | Thinking/reasoning delta (wrapped in `<think>` tags) |
|
||||
| `reasoning` | Thinking/reasoning delta |
|
||||
| `call` | Tool call initiated (Name=tool, Content=args JSON) |
|
||||
| `tool` | Tool result returned (Name=tool, Content=result) |
|
||||
| `info` | Informational message (command output, status) |
|
||||
| `exec` | Shell command output (`!cmd`) |
|
||||
| `error` | Error message |
|
||||
| `usage` | Token usage report (space-separated numbers) |
|
||||
| `usage` | Token usage report |
|
||||
| `limitretry` | Rate limit retry initiated |
|
||||
|
||||
Consumers subscribe to the `"event"` topic on the bus. The 9P server's session manager subscribes to build the chat log; frontends subscribe to render output.
|
||||
|
|
@ -650,4 +686,4 @@ Consumers subscribe to the `"event"` topic on the bus. The 9P server's session m
|
|||
<unix-nanoseconds>-<6-hex-chars>
|
||||
```
|
||||
|
||||
Lexicographically sortable by creation time. Unique even within the same nanosecond due to the random suffix.
|
||||
Lexicographically sortable by creation time. Unique even within the same nanosecond due to the random suffix.
|
||||
310
doc/EVOLUTION.md
310
doc/EVOLUTION.md
|
|
@ -5,44 +5,48 @@ How the Ollie system-of-systems emerged and evolved.
|
|||
|
||||
## Timeline
|
||||
|
||||
```
|
||||
Apr May Jun Jul
|
||||
|----|--------|--------|--------|--------|
|
||||
Foundation
|
||||
Monorepo + submodules ===
|
||||
9P filesystem . ============
|
||||
```mermaid
|
||||
gantt
|
||||
title Ollie Evolution
|
||||
dateFormat YYYY-MM-DD
|
||||
axisFormat %b %d
|
||||
|
||||
Core Primitives
|
||||
Session lifecycle (s/) ==================
|
||||
Tool scripts ============
|
||||
Skills system ========
|
||||
section Foundation
|
||||
Monorepo + submodules :done, 2026-04-11, 14d
|
||||
9P filesystem :done, 2026-04-14, 28d
|
||||
|
||||
Execution
|
||||
execute_code ==========
|
||||
Batch jobs (b/→s/bfg) ==================
|
||||
Sandbox (landrun) ====================================
|
||||
section Core Primitives
|
||||
Session lifecycle (session/) :done, 2026-04-20, 30d
|
||||
Tool scripts :done, 2026-05-01, 20d
|
||||
Skills system :done, 2026-05-10, 14d
|
||||
|
||||
Multi-Agent
|
||||
Subagent spawn ==========
|
||||
Cascade orchestrator ==========
|
||||
JIT agent generation ====
|
||||
section Execution
|
||||
execute_code / shell :done, 2026-05-08, 20d
|
||||
Batch jobs (b/ → session/bfg) :done, 2026-05-15, 25d
|
||||
Sandbox (landrun) :done, 2026-05-20, 40d
|
||||
|
||||
Frontends
|
||||
TUI + Emacs (ellie) =====================================
|
||||
Acme (Plan 9) ============================
|
||||
Web UI + httpgw ====================
|
||||
KDE/Plasma ========================
|
||||
section Multi-Agent
|
||||
Subagent spawn :done, 2026-05-25, 15d
|
||||
Cascade orchestrator :done, 2026-06-05, 20d
|
||||
JIT agent generation :done, 2026-07-10, 10d
|
||||
|
||||
Control Planes
|
||||
D-Bus adapter (ollied) ===============
|
||||
D-Bus embedded in srv =====
|
||||
section Frontends
|
||||
TUI + Emacs (ellie) :done, 2026-04-15, 45d
|
||||
Acme (Plan 9) :done, 2026-05-20, 30d
|
||||
Web UI + httpgw :done, 2026-06-01, 25d
|
||||
KDE/Plasma :done, 2026-06-10, 40d
|
||||
|
||||
Late Evolution
|
||||
Remote execution ==========
|
||||
Elevation broker ==========
|
||||
Tool registry (lazy) ======
|
||||
Reactions / feedback =====
|
||||
FUSE → 9P client ====
|
||||
section Control Planes
|
||||
D-Bus adapter (ollied) :done, 2026-06-25, 20d
|
||||
D-Bus embedded in srv :done, 2026-07-15, 10d
|
||||
|
||||
section Late Evolution
|
||||
Remote execution :done, 2026-07-05, 15d
|
||||
Elevation broker :done, 2026-07-10, 15d
|
||||
Tool registry (lazy) :done, 2026-07-20, 10d
|
||||
Reactions / feedback :done, 2026-07-22, 8d
|
||||
FUSE → 9P client :done, 2026-07-25, 5d
|
||||
The Great Flattening :done, 2026-07-29, 2d
|
||||
```
|
||||
|
||||
## Phase 1: Monorepo Bootstrap (Apr 11)
|
||||
|
|
@ -50,8 +54,8 @@ Late Evolution
|
|||
Started as independent git repos unified under a monorepo with submodules.
|
||||
Initial components:
|
||||
|
||||
- **core** — agent loop (Go): backend dispatch, tool execution, session state
|
||||
- **9p** — 9P file server exposing agent sessions as a synthetic filesystem
|
||||
- **agent** — agent loop (Go): backend dispatch, tool execution, session state
|
||||
- **fs/session** — 9P file server exposing agent sessions as a synthetic filesystem
|
||||
- **tui** — terminal UI (later removed)
|
||||
- **el** — Emacs integration (ellie.el)
|
||||
|
||||
|
|
@ -61,13 +65,13 @@ Build system was `mkfile` (Plan 9 make), later `Makefile`, finally `justfile`.
|
|||
|
||||
The defining architectural choice: **every agent primitive is a file**.
|
||||
|
||||
Sessions are directories. Sending a prompt is writing to `s/{id}/prompt`.
|
||||
Reading state is `cat s/{id}/state`. This made the control plane
|
||||
Sessions are directories. Sending a prompt is writing to `session/{id}/prompt`.
|
||||
Reading state is `cat session/{id}/state`. This made the control plane
|
||||
protocol-agnostic — any program that can read/write files can be a frontend.
|
||||
|
||||
Key files crystallized: `prompt`, `chat`, `state`, `ctl`, `cfg`, `statewait`.
|
||||
|
||||
The `s/new` file (write key=value pairs, read back session ID) became the
|
||||
The `session/new` file (write key=value pairs, read back session ID) became the
|
||||
session factory.
|
||||
|
||||
## Phase 3: Tool System (Apr 28 – May 10)
|
||||
|
|
@ -95,7 +99,7 @@ Planning went through the most iterations of any subsystem:
|
|||
3. **Beads** integration (external issue tracker) — made optional, then dropped
|
||||
4. `task_add` / `task_check` tools
|
||||
5. Inline markdown plans forbidden
|
||||
6. **Final form**: a single `s/{id}/plan` file (markdown checklist), written by
|
||||
6. **Final form**: a single `session/{id}/plan` file (markdown checklist), written by
|
||||
the agent, persisted across context compaction
|
||||
|
||||
The lesson: planning needed to be simple, agent-controlled, and not bureaucratic.
|
||||
|
|
@ -109,91 +113,100 @@ Three patterns emerged:
|
|||
- **cascade** — script-driven fan-out with `-max-workers`, `-retries`, optional
|
||||
synthesis step
|
||||
|
||||
```
|
||||
┌──────────────────┐
|
||||
│ User prompt │
|
||||
└────────┬─────────┘
|
||||
│
|
||||
┌────────▼─────────┐
|
||||
│ Parent Session │
|
||||
└──┬──────┬──────┬─┘
|
||||
subagent_ │ │ │ subagent_
|
||||
spawn │ │ │ spawn
|
||||
┌────────▼┐ ┌──▼───┐ ┌▼────────┐
|
||||
│Worker 1 │ │Wkr 2 │ │Worker 3 │
|
||||
└────┬────┘ └──┬───┘ └────┬────┘
|
||||
│ │ │
|
||||
└──────────┼───────────┘
|
||||
write to parent's prompt
|
||||
```mermaid
|
||||
flowchart TB
|
||||
UP["User prompt"]
|
||||
PARENT["Parent Session"]
|
||||
W1["Worker 1"]
|
||||
W2["Worker 2"]
|
||||
W3["Worker 3"]
|
||||
CASCADE["u/cascade\n(spawn + throttle)"]
|
||||
CW1["Cascade Worker"]
|
||||
CW2["Cascade Worker"]
|
||||
|
||||
┌────────▼─────────┐
|
||||
│ Parent Session │──── cascade ───┐
|
||||
└──────────────────┘ │
|
||||
┌─────────▼──────────┐
|
||||
│ u/cascade │
|
||||
│ (spawn + throttle) │
|
||||
└───┬──────────┬─────┘
|
||||
│ │
|
||||
┌────▼───┐ ┌───▼────┐
|
||||
│Cascade │ │Cascade │
|
||||
│Worker │ │Worker │
|
||||
└────────┘ └────────┘
|
||||
UP --> PARENT
|
||||
PARENT -- subagent_spawn --> W1
|
||||
PARENT -- subagent_spawn --> W2
|
||||
PARENT -- subagent_spawn --> W3
|
||||
W1 --> PARENT
|
||||
W2 --> PARENT
|
||||
W3 --> PARENT
|
||||
PARENT -- cascade --> CASCADE
|
||||
CASCADE --> CW1
|
||||
CASCADE --> CW2
|
||||
```
|
||||
|
||||
Inter-agent communication uses the filesystem: agents write to each other's
|
||||
`prompt` file or use bidirectional `peer/` links.
|
||||
`prompt` file.
|
||||
|
||||
## Phase 6: Frontend Proliferation
|
||||
|
||||
```
|
||||
┌─────────────────┐
|
||||
│ agent.Core │
|
||||
└───┬─────────┬───┘
|
||||
│ │
|
||||
┌──────▼───┐ ┌───▼──────┐
|
||||
│ 9P Server│ │D-Bus Svc │
|
||||
└──┬──┬──┬┘ └──┬─────┬──┘
|
||||
│ │ │ │ │
|
||||
┌─────────┘ │ └──┐ │ └──────────┐
|
||||
│ │ │ │ │
|
||||
┌─────▼───┐ ┌────▼──┐ │ ├────────┐ ┌────▼────┐
|
||||
│ s/sh │ │ acme │ │ │ KDE │ │ Web UI │
|
||||
└─────────┘ └───────┘ │ │Plasmoid│ │(httpgw) │
|
||||
┌────▼──┤KRunner │ └─────────┘
|
||||
│ellie │Kate │
|
||||
└───────┤GUI │
|
||||
└────────┘
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph Core["agent.Agent"]
|
||||
AG["Agent Engine"]
|
||||
end
|
||||
|
||||
subgraph Surfaces["Integration Surfaces"]
|
||||
P9["9P Filesystem\n(session/ namespace)"]
|
||||
DB["D-Bus\n(org.ollie.SessionManager)"]
|
||||
end
|
||||
|
||||
subgraph Frontends
|
||||
SH["s/sh (terminal)"]
|
||||
ACME["acme (Plan 9)"]
|
||||
EL["ellie (Emacs)"]
|
||||
KG["KDE GUI"]
|
||||
KP["KDE Plasmoid"]
|
||||
KK["Kate Plugin"]
|
||||
KR["KRunner"]
|
||||
WEB["Web UI"]
|
||||
HTTP["curl / scripts"]
|
||||
end
|
||||
|
||||
AG --> P9
|
||||
AG --> DB
|
||||
|
||||
SH --> P9
|
||||
ACME --> P9
|
||||
EL --> P9
|
||||
HTTP --> P9
|
||||
|
||||
KG --> DB
|
||||
KP --> DB
|
||||
KK --> DB
|
||||
KR --> DB
|
||||
WEB --> DB
|
||||
```
|
||||
|
||||
- **s/sh** — shell script frontend (bash, briefly rc, back to bash)
|
||||
- **acme** — Plan 9 editor integration with `Oi` (quick query), `Kmpl` (completion)
|
||||
- **ellie.el** — Emacs with ghost-text completion
|
||||
- **httpgw** — HTTP→D-Bus gateway
|
||||
- **httpgw** — HTTP→9P gateway
|
||||
- **webui** — browser frontend via httpgw
|
||||
- **ollie-kde** — full Plasma integration: plasmoid, KRunner, Kate plugin, standalone GUI, GUI automation tools
|
||||
- **TUI** — removed (Jul) in favor of s/sh and richer GUIs
|
||||
|
||||
## Phase 7: Security Model (May – Jul)
|
||||
|
||||
```
|
||||
9P Permissions Execution Sandbox
|
||||
────────────── ─────────────────
|
||||
Owner → rw Agent
|
||||
Agent → restricted │
|
||||
Peers → read-only ▼
|
||||
landrun sandbox (restricted fs)
|
||||
│
|
||||
▼
|
||||
Tool scripts
|
||||
│ needs escape?
|
||||
▼
|
||||
x/elevate (socket broker)
|
||||
│
|
||||
▼
|
||||
Integrated Broker
|
||||
│ user confirms
|
||||
▼
|
||||
Privileged Action
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph Permissions["9P Permissions"]
|
||||
OWNER["Owner → rw"]
|
||||
AGENT["Agent → restricted"]
|
||||
PEERS["Peers → read-only"]
|
||||
end
|
||||
|
||||
subgraph Execution["Execution Sandbox"]
|
||||
AGT["Agent"]
|
||||
SANDBOX["landrun sandbox\n(restricted fs)"]
|
||||
TOOLS["Tool scripts"]
|
||||
ELEV["x/elevate\n(socket broker)"]
|
||||
PRIV["Privileged Action"]
|
||||
end
|
||||
|
||||
AGT --> SANDBOX --> TOOLS
|
||||
TOOLS -- needs escape --> ELEV --> PRIV
|
||||
```
|
||||
|
||||
Evolution:
|
||||
|
|
@ -210,7 +223,7 @@ Evolution:
|
|||
Originally everything was 9P-only. D-Bus was added for desktop integration:
|
||||
|
||||
1. Separate `ollie-dbus` daemon (ollied)
|
||||
2. Embedded directly into `olliesrv` (9P server got `-nodbus` flag)
|
||||
2. Embedded directly into `olliesrv` (9P server got `-no9p` flag)
|
||||
3. KDE uses D-Bus exclusively; acme/sh/el use 9P
|
||||
4. **9P and D-Bus do not share sessions** — independent stores, same core
|
||||
|
||||
|
|
@ -242,25 +255,24 @@ This solved system prompt bloat — only load what's needed per task.
|
|||
Largest single-day structural change: **−4,573 lines net** across the codebase.
|
||||
Eliminated accumulated abstractions that no longer served a purpose.
|
||||
|
||||
### Core repo (−4,400 lines)
|
||||
### Core (−4,400 lines)
|
||||
|
||||
- **Killed `agent.Core` interface** — concrete `*agent.Agent` used directly.
|
||||
Nobody else implemented Core; the interface just added indirection.
|
||||
- **Killed `session.Core` interface** — same pattern, same outcome.
|
||||
- **Killed `Dispatcher` indirection** in toolsrv — tools.Server called directly.
|
||||
- **Killed `Dispatcher` indirection** in toolsrv — `toolsrv.Server` called directly.
|
||||
- **Merged `execute/` into `tools/`** — separate package for 3 functions was
|
||||
pointless overhead.
|
||||
- **Renamed packages for clarity**: `agent.Session` → `agent.History`,
|
||||
- **Renamed for clarity**: `agent.Session` → `agent.History`,
|
||||
`session.Core` → `session.Session`, `NewAgentCore` → `New`.
|
||||
- **Extracted `agent/` package** from the monolithic session package.
|
||||
- **Threaded context.Context** properly through daemon → session → agent
|
||||
(replaced ad-hoc interrupt mechanisms with cancellation).
|
||||
- Dead code removal: `GlobalToolNames`, `ExtractReturnSchema`.
|
||||
|
||||
### 9P server repo (−214 lines)
|
||||
### 9P server (−214 lines)
|
||||
|
||||
- **Killed `mgr/` package entirely** — replaced `Manager` struct with package
|
||||
functions in `session/`. The `fs.Tree` IS the session collection; `rootState`
|
||||
functions in `fs/session/`. The `*fs.Tree` IS the session collection; `rootState`
|
||||
(unexported) lives in `tree.Data`. CRUD via `NewRoot`, `Lookup`, `Create`,
|
||||
`Kill`, `Rename`, `Shutdown`.
|
||||
- **Fixed infinite recursion** in `session/{sid}/agent/` directory — `pathType()`
|
||||
|
|
@ -291,23 +303,30 @@ session/
|
|||
│ ├── env session environment
|
||||
│ └── agent/
|
||||
│ └── {aid}/
|
||||
│ ├── chat conversation log
|
||||
│ ├── prompt submit prompts
|
||||
│ ├── cfg agent configuration
|
||||
│ ├── ctl control commands
|
||||
│ ├── state current state
|
||||
│ ├── statewait blocks until state changes
|
||||
│ ├── usage token stats
|
||||
│ ├── cost cost estimate
|
||||
│ ├── ctxsz context size
|
||||
│ ├── cfg session configuration (key=value)
|
||||
│ ├── ctl control commands
|
||||
│ ├── state current state
|
||||
│ ├── statewait blocks until state changes
|
||||
│ ├── chat conversation log
|
||||
│ ├── prompt submit prompts
|
||||
│ ├── prompt.prev last submitted prompt
|
||||
│ ├── offset byte offset after last user prompt
|
||||
│ ├── fifo.in queue a prompt (FIFO input)
|
||||
│ ├── fifo.out pop queued prompt (FIFO output)
|
||||
│ ├── context rendered context window
|
||||
│ ├── systemprompt rendered system prompt
|
||||
│ ├── tools tool management
|
||||
│ ├── tools.loaded currently loaded tools
|
||||
│ └── proc/
|
||||
│ └── {pid} detached process output
|
||||
│ ├── usage token stats
|
||||
│ ├── cost cost estimate
|
||||
│ ├── ctxsz context size
|
||||
│ ├── models available models
|
||||
│ ├── tools tool management
|
||||
│ ├── toolsrv.loaded currently loaded tools
|
||||
│ ├── toolsrv.rev revision counter
|
||||
│ ├── tail exec helper
|
||||
│ └── proc/ detached process output
|
||||
```
|
||||
|
||||
### Architecture after
|
||||
### Architecture after (single Go module, no submodules for core/9p)
|
||||
|
||||
```
|
||||
*fs.Tree (root) ← IS the session collection
|
||||
|
|
@ -328,8 +347,8 @@ Package functions (no Manager):
|
|||
|
||||
| Component | Lines |
|
||||
|-----------|-------|
|
||||
| core (minus backend) | 10,437 |
|
||||
| 9p server | 7,925 |
|
||||
| agent + backend + toolsrv + session | 10,437 |
|
||||
| fs/session + cmd/olliesrv | 7,925 |
|
||||
| kde gui | 12,331 |
|
||||
| el + acme + httpgw | 1,845 |
|
||||
| **Total** | **32,538** |
|
||||
|
|
@ -339,30 +358,37 @@ Zero dead exported functions remain (verified via LSP + grep across all repos).
|
|||
## Current Topology
|
||||
|
||||
```
|
||||
ollie/ ← monorepo root
|
||||
├── core/ ← agent loop, backends, tool dispatch (Go)
|
||||
├── 9p/ ← 9P server + embedded D-Bus adapter (Go)
|
||||
├── acme/ ← Plan 9 acme frontend (Go)
|
||||
├── el/ ← Emacs integration (elisp)
|
||||
├── kde/ ← KDE Plasma: plasmoid, GUI, Kate, KRunner (C++/QML)
|
||||
├── httpgw/ ← HTTP→D-Bus gateway (Go)
|
||||
├── webui/ ← browser frontend (JS)
|
||||
├── remote/ ← split-brain execution server (Go)
|
||||
ollie/ ← single Go module
|
||||
├── agent/ ← agent loop, history, hooks, commands
|
||||
├── backend/ ← LLM providers (6 backends)
|
||||
├── toolsrv/ ← tool server, registry, remote execution
|
||||
├── session/ ← session lifecycle, config
|
||||
├── fs/session/ ← 9P filesystem tree
|
||||
├── dbus/ ← D-Bus adapter
|
||||
├── detach/ ← background process management
|
||||
├── elevate/ ← elevation broker
|
||||
├── sandbox/ ← landrun sandbox config
|
||||
├── env/ ← environment helpers
|
||||
├── log/ ← structured logging
|
||||
├── paths/ ← XDG path resolution
|
||||
├── mount/ ← 9P FUSE mount client
|
||||
├── cmd/ ← binaries (olliesrv, ollie-9p, ollie-httpgw, ollie-remote)
|
||||
├── kde/ ← KDE Plasma (submodule)
|
||||
├── webui/ ← browser frontend (submodule)
|
||||
├── tools/ ← 37 tool scripts (Python)
|
||||
├── scripts/{s,u,x}/ ← filesystem-hosted scripts
|
||||
├── scripts/{s,u,x}/ ← filesystem-hosted scripts
|
||||
├── agents/ ← agent configs (JSON)
|
||||
├── prompts/ ← system prompt fragments (markdown)
|
||||
├── skills/ ← domain knowledge modules (markdown)
|
||||
├── sandbox/ ← landrun sandbox profiles (YAML)
|
||||
├── services/ ← init/service files
|
||||
└── experiments/ ← multi-agent trial docs
|
||||
└── doc/ ← documentation
|
||||
```
|
||||
|
||||
## Principles That Emerged
|
||||
|
||||
1. **Filesystem-as-API** — everything is read/write on synthetic files. No
|
||||
custom protocols needed.
|
||||
2. **Submodule independence** — each component has its own repo, tests, build.
|
||||
2. **Submodule independence** — KDE and webui have their own repos, tests, build.
|
||||
Monorepo coordinates versions.
|
||||
3. **Scripts over servers** — MCP servers removed early. Simple scripts won.
|
||||
4. **Progressive disclosure** — lazy tool/skill loading keeps system prompts
|
||||
|
|
@ -388,4 +414,4 @@ ollie/ ← monorepo root
|
|||
| Tier routing (FAST/POWER) | Replaced by /route endpoint with real model discovery |
|
||||
| call_tool / pipe | Replaced by native tool registry with tool_load |
|
||||
| execute_code (multi-lang) | Simplified to shell-only; tool scripts handle language choice |
|
||||
| chatwait | Reverted; acme tails chat directly |
|
||||
| chatwait | Reverted; acme tails chat directly |
|
||||
|
|
@ -11,11 +11,13 @@ Every session lives under `/session/{id}/`:
|
|||
| Path | Purpose |
|
||||
|------|---------|
|
||||
| `/session/new` | Create a session (write `name`, `backend`, `model`, `agent`, `cwd`) |
|
||||
| `/session/{id}/prompt` | Submit a prompt (dispatched async on close) |
|
||||
| `/session/{id}/chat` | Cumulative conversation history (append-only read) |
|
||||
| `/session/{id}/cfg` | Session config (KV): state, backend, model, agent, cwd, params |
|
||||
| `/session/{id}/statewait` | **Blocking read** — returns when state changes |
|
||||
| `/session/{id}/ctl` | Control commands: `stop`, `kill`, `compact`, `rn <name>` |
|
||||
| `/session/{id}/plan` | Session-scoped markdown checklist |
|
||||
| `/session/{id}/env` | Session environment variables |
|
||||
| `/session/{id}/agent/{aid}/prompt` | Submit a prompt (dispatched async on close) |
|
||||
| `/session/{id}/agent/{aid}/chat` | Cumulative conversation history (append-only read) |
|
||||
| `/session/{id}/agent/{aid}/cfg` | Agent config (KV): state, backend, model, agent, cwd, params |
|
||||
| `/session/{id}/agent/{aid}/statewait` | **Blocking read** — returns when state changes |
|
||||
| `/session/{id}/agent/{aid}/ctl` | Control commands: `stop`, `kill`, `compact`, `rn <name>` |
|
||||
|
||||
Utility files at the session root (`/session/`) provide higher-level operations:
|
||||
|
||||
|
|
@ -31,9 +33,8 @@ Global namespaces are shared across all sessions:
|
|||
| Path | Purpose |
|
||||
|------|---------|
|
||||
| `/session/idx` | Session index (id, state, cwd, backend, model) |
|
||||
| `/session/{id}/prompt` | Submit a prompt to a peer session |
|
||||
| `/session/{id}/plan` | Shared session-scoped checklist |
|
||||
| `/session/{id}/chat` | Full chat history (searchable) |
|
||||
| `/session/{id}/agent/{aid}/chat` | Full chat history (searchable) |
|
||||
|
||||
## Spawning Agents
|
||||
|
||||
|
|
@ -44,15 +45,27 @@ subagent_spawn [-name NAME] [-agent AGENT] [-backend BACKEND] [-model MODEL] [-c
|
|||
```
|
||||
|
||||
This creates a child session, writes the prompt, and returns the session path
|
||||
(e.g. `s/my-session`). Child sessions carry the parent session ID in their name,
|
||||
(e.g. `session/my-session`). Child sessions carry the parent session ID in their name,
|
||||
so the spawning tree is recoverable from session IDs alone.
|
||||
|
||||
## Coordination Patterns
|
||||
|
||||
### Hub-and-Spoke (Conductor)
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
ROOT["Root Agent\n(reasoning)"]
|
||||
W1["Worker A\n(cheap model)"]
|
||||
W2["Worker B\n(cheap model)"]
|
||||
|
||||
ROOT -- "subagent_spawn" --> W1
|
||||
ROOT -- "subagent_spawn" --> W2
|
||||
W1 -- "statewait → read chat" --> ROOT
|
||||
W2 -- "statewait → read chat" --> ROOT
|
||||
```
|
||||
|
||||
A root agent reasons about a task and delegates subtasks to workers. Workers
|
||||
report back via their chat history; the conductor reads `/session/{id}/chat` after
|
||||
report back via their chat history; the conductor reads `/session/{id}/agent/{aid}/chat` after
|
||||
`statewait` unblocks.
|
||||
|
||||
```sh
|
||||
|
|
@ -73,6 +86,13 @@ to cheaper models; reasoning stays on the expensive one.
|
|||
|
||||
### Pipeline
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
A["Step 1\nExtract TODOs"]
|
||||
B["Step 2\nPrioritise"]
|
||||
A -->|"/tmp/todos.txt"| B
|
||||
```
|
||||
|
||||
Agent A produces output; agent B consumes it. Composition via `/tmp/` or direct
|
||||
path passing.
|
||||
|
||||
|
|
@ -84,6 +104,20 @@ subagent_spawn "prioritise the TODOs in /tmp/todos.txt and produce a plan"
|
|||
|
||||
### Scatter-Gather (Swarm)
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
P["Parent"]
|
||||
S1["Worker 1"]
|
||||
S2["Worker 2"]
|
||||
S3["Worker N"]
|
||||
P --> S1
|
||||
P --> S2
|
||||
P --> S3
|
||||
S1 --> P
|
||||
S2 --> P
|
||||
S3 --> P
|
||||
```
|
||||
|
||||
Spawn N workers in parallel, collect when all reach `idle`. Because `statewait`
|
||||
blocks until state changes, no polling loop is needed.
|
||||
|
||||
|
|
@ -124,7 +158,7 @@ This enables event-driven pipelines without polling:
|
|||
|
||||
```sh
|
||||
# Block until agent finishes, then trigger downstream work
|
||||
cat /s/worker-123/statewait
|
||||
cat /session/worker-123/statewait
|
||||
# agent is now idle — proceed
|
||||
```
|
||||
|
||||
|
|
@ -143,4 +177,4 @@ environment — shell scripts, cron, systemd, containers, or another agent.
|
|||
|
||||
This is intentional. The filesystem namespace is the API. Any tool that can
|
||||
read and write files can participate in a multi-agent workflow, which means the
|
||||
full Unix toolchain is available as an orchestration layer.
|
||||
full Unix toolchain is available as an orchestration layer.
|
||||
|
|
@ -15,37 +15,32 @@ The split-brain model keeps the **brain local** (agent loop, LLM, D-Bus, GUI) an
|
|||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Local machine Remote host (via SSH)
|
||||
┌──────────────────────────────┐ ┌──────────────────────────────┐
|
||||
│ olliesrv / ollied │ │ ollie-remote │
|
||||
│ │ │ │
|
||||
│ ┌────────────────────────┐ │ SSH │ ┌────────────────────────┐ │
|
||||
│ │ agent.Core │ │────────▶│ │ execute.Server │ │
|
||||
│ │ - LLM streaming │ │ JSON │ │ - landrun sandbox │ │
|
||||
│ │ - prompt assembly │ │ RPC │ │ - bash/python/perl │ │
|
||||
│ │ - event bus │ │◀────────│ │ - tool scripts │ │
|
||||
│ │ - compaction │ │ │ │ - file I/O │ │
|
||||
│ └────────────────────────┘ │ │ └────────────────────────┘ │
|
||||
│ │ │ │
|
||||
│ ┌────────────────────────┐ │ │ Working directory: │
|
||||
│ │ D-Bus adapter │ │ │ /home/user/project/ │
|
||||
│ │ - SessionCreated │ │ │ │
|
||||
│ │ - StateChanged │ │ │ Sandbox config: │
|
||||
│ │ - ChatUpdated │ │ │ remote host's landrun │
|
||||
│ └────────────────────────┘ │ │ │
|
||||
│ ▲ │ └──────────────────────────────┘
|
||||
│ │ D-Bus signals │
|
||||
│ ▼ │
|
||||
│ ┌────────────────────────┐ │
|
||||
│ │ KDE GUI / Web UI │ │
|
||||
│ └────────────────────────┘ │
|
||||
└──────────────────────────────┘
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph Local["Local Machine"]
|
||||
SRV["olliesrv"]
|
||||
AG["agent.Agent\n- LLM streaming\n- prompt assembly\n- event bus\n- compaction"]
|
||||
DBUS["D-Bus adapter\n- SessionCreated\n- StateChanged\n- ChatUpdated"]
|
||||
GUI["KDE GUI / Web UI"]
|
||||
end
|
||||
|
||||
subgraph Remote["Remote Host (via SSH)"]
|
||||
REMOTE_BIN["ollie-remote"]
|
||||
EXEC["toolsrv.Server\n- landrun sandbox\n- shell execution\n- tool scripts\n- file I/O"]
|
||||
CWD["Working directory:\n/home/user/project/"]
|
||||
SANDCFG["Sandbox config:\nremote host's landrun"]
|
||||
end
|
||||
|
||||
AG -- "SSH / JSON-RPC" --> REMOTE_BIN
|
||||
REMOTE_BIN --> EXEC
|
||||
EXEC --> CWD
|
||||
EXEC --> SANDCFG
|
||||
DBUS -.->|"D-Bus signals"| GUI
|
||||
```
|
||||
|
||||
## Split Point
|
||||
|
||||
The natural split is at `tools.Server`. Today, `agent.Core` calls `execute.Server` in-process. The remote mode replaces this with a `RemoteServer` that forwards calls over an SSH-backed RPC channel:
|
||||
The natural split is at `toolsrv.Runner`. Today, `agent.Agent` calls `toolsrv.Server` in-process. The remote mode replaces this with a `RemoteServer` that forwards calls over an SSH-backed RPC channel:
|
||||
|
||||
```go
|
||||
// Local side
|
||||
|
|
@ -53,12 +48,12 @@ type RemoteServer struct {
|
|||
conn *rpc.Conn // JSON-RPC over SSH stdin/stdout
|
||||
}
|
||||
|
||||
func (r *RemoteServer) Call(ctx context.Context, req tools.Request) (tools.Response, error) {
|
||||
return r.conn.Call(ctx, "execute", req)
|
||||
func (r *RemoteServer) CallTool(ctx context.Context, tool string, args json.RawMessage) (json.RawMessage, error) {
|
||||
return r.conn.Call(ctx, "execute", tool, args)
|
||||
}
|
||||
```
|
||||
|
||||
The agent loop doesn't know or care whether execution is local or remote. The `NewDispatcherFunc` in session creation just returns a `RemoteServer` instead of a local `execute.Server` when the session is configured for remote execution.
|
||||
The agent loop doesn't know or care whether execution is local or remote. The `NewToolServer` factory in session creation just returns a `RemoteServer` instead of a local `toolsrv.Server` when the session is configured for remote execution.
|
||||
|
||||
## SSH Bootstrap
|
||||
|
||||
|
|
@ -90,7 +85,7 @@ exec "$BIN" serve --cwd "@@CWD@@"
|
|||
|
||||
A minimal binary containing:
|
||||
|
||||
- `execute.Server` (the same one used locally)
|
||||
- `toolsrv.Server` (the same one used locally)
|
||||
- Sandbox enforcement (landrun)
|
||||
- Tool script discovery from a bundled or remote-local `OLLIE_TOOLS_PATH`
|
||||
- JSON-RPC server over stdin/stdout
|
||||
|
|
@ -106,7 +101,7 @@ It does NOT contain:
|
|||
// cmd/ollie-remote/main.go
|
||||
func main() {
|
||||
cwd := flag.Arg(0)
|
||||
srv := execute.NewServer(cwd, toolsPath, sandboxConfig)
|
||||
srv := toolsrv.NewServer(cwd, toolsPath, sandboxConfig)
|
||||
rpc.ServeStdio(srv)
|
||||
}
|
||||
```
|
||||
|
|
@ -143,7 +138,7 @@ A session targeting a remote host:
|
|||
|
||||
```bash
|
||||
# Via 9P
|
||||
printf 'cwd=/home/user/project\nremote=devbox\n' > $OLLIE/s/new
|
||||
printf 'cwd=/home/user/project\nremote=devbox\n' > $OLLIE/session/new
|
||||
|
||||
# Via D-Bus
|
||||
dbus-send ... org.ollie.SessionManager.CreateSession \
|
||||
|
|
@ -159,7 +154,7 @@ remote=user@devbox.internal
|
|||
The `remote` field is an SSH target. When set, the session manager:
|
||||
1. Bootstraps `ollie-remote` on the target
|
||||
2. Creates a `RemoteServer` connected over the SSH channel
|
||||
3. Passes it to `agent.Core` as the tool dispatcher
|
||||
3. Passes it to `agent.Agent` as the tool runner
|
||||
|
||||
## Tool Scripts
|
||||
|
||||
|
|
@ -214,19 +209,19 @@ They complement each other. You might use split-brain for active development (lo
|
|||
All five phases are implemented:
|
||||
|
||||
### Phase 1: ollie-remote binary ✓
|
||||
- `remote/` submodule with `main.go`
|
||||
- Wraps `execute.Server` with JSON-RPC stdio interface
|
||||
- `cmd/ollie-remote/` with `main.go`
|
||||
- Wraps `toolsrv.Server` with JSON-RPC stdio interface
|
||||
- Embeds `sandbox-remote.yaml` and tool scripts via `go:embed`
|
||||
- RPC methods: `shell`, `list_tools`, `host_info`, `ping`
|
||||
|
||||
### Phase 2: SSH bootstrap ✓
|
||||
- `core/execute/bootstrap.sh` embedded template
|
||||
- `toolsrv/bootstrap.sh` embedded template
|
||||
- SHA-256 hash-based caching on the remote
|
||||
- Binary transfer: gzip + base64 over stdin
|
||||
- `Dial()` handles full lifecycle: bootstrap → transfer → exec → JSON-RPC
|
||||
|
||||
### Phase 3: RemoteServer adapter ✓
|
||||
- `core/execute/remote.go` implements `tools.Server`
|
||||
- `toolsrv/remote.go` implements `toolsrv.Runner`
|
||||
- `ListTools()` calls remote `list_tools` RPC
|
||||
- `CallTool()` passes tool name as RPC method (routes shell)
|
||||
- `HostInfo` fetched after connect (platform, arch, is_git_repo)
|
||||
|
|
@ -368,14 +363,14 @@ Remote → local streaming uses JSON-RPC notifications (no `id` field):
|
|||
{"jsonrpc":"2.0","result":{"data":"compiling...\n"}}
|
||||
```
|
||||
|
||||
`remote.Server.CallTool` reads these in a loop, calling `tools.StreamOutput(ctx, data)` for each, until the actual response (with matching `id`) arrives.
|
||||
`remote.Server.CallTool` reads these in a loop, calling `toolsrv.StreamOutput(ctx, data)` for each, until the actual response (with matching `id`) arrives.
|
||||
|
||||
### Agent Loop Integration
|
||||
|
||||
The agent loop wraps the execution context with a streaming callback:
|
||||
|
||||
```go
|
||||
streamCtx := tools.WithOutputStream(ctx, func(data string) {
|
||||
streamCtx := toolsrv.WithOutputStream(ctx, func(data string) {
|
||||
emit(cfg, Event{Role: "tool", Name: tc.Name, Content: data})
|
||||
})
|
||||
```
|
||||
|
|
@ -384,7 +379,7 @@ On first chunk, an empty-content tool event is emitted to signal stream start. I
|
|||
|
||||
### Execute Server
|
||||
|
||||
`limitedWriter` has a `stream func(string)` field, wired to `tools.StreamFunc(ctx)`. Every `Write()` call flushes the chunk both to the output buffer (for the full result) and to the stream callback (for real-time display).
|
||||
`limitedWriter` has a `stream func(string)` field, wired to `toolsrv.StreamFunc(ctx)`. Every `Write()` call flushes the chunk both to the output buffer (for the full result) and to the stream callback (for real-time display).
|
||||
|
||||
### Chat Log
|
||||
|
||||
|
|
@ -393,4 +388,4 @@ On first chunk, an empty-content tool event is emitted to signal stream start. I
|
|||
- Subsequent tool events → content appended directly (no repeated header)
|
||||
- Non-tool event → closes the stream with a newline
|
||||
|
||||
For non-streamed tools (fast results), the single tool event renders as before.
|
||||
For non-streamed tools (fast results), the single tool event renders as before.
|
||||
Loading…
Reference in New Issue