ollie/doc/architecture-core.md

# Core Architecture

This document describes the agent-side core. For the tool-execution service, its 9P namespace, registry, metadata, sandbox, process lifecycle, and cancellation path, see [`architecture-toolsrv.md`](architecture-toolsrv.md).

This document covers the internal Go packages that implement Ollie's agent runtime. The code lives under `cmd/olliesrv/internal/`; it is not a separately imported public library. `olliesrv` composes the agent, backend, session, filesystem, and tool-client packages.

## Package Map

```mermaid
flowchart TB
    subgraph Agent["cmd/olliesrv/internal/agent/"]
        AGENT["agent.go — Agent struct, identity, events"]
        LOOP["loop.go — main agent loop, streaming"]
        TURN["turn.go — turn orchestration, Submit"]
        DISPATCH["dispatch.go — tool execution, batching"]
        HISTORY["history.go — message history, usage"]
        COMPACT["compact.go — compaction, cold summarization"]
        CACHE["cache.go — tool result caching"]
        RETRY["retry.go — error tracking, retry logic"]
        STATE["state.go, chat.go, peer.go, subagent.go"]
        RUNTIME["runtime.go — Runtime, preamble, tool dispatch"]
        CONFIG["agent_config.go — profile configuration"]
        PROMPT["prompt_resolver.go — prompt resolution"]
    end

    subgraph Backend["cmd/olliesrv/internal/backend/"]
        BACKEND["backend.go — Backend interface and streaming"]
        PROVIDERS["openai, anthropic, ollama, gemini, copilot, kiro"]
        POOL["pool.go — backend pool"]
    end

    subgraph ToolClient["cmd/olliesrv/internal/toolclient/"]
        CLIENT["9P toolsrv client"]
        SPAWN["spawn.go — local/remote toolsrv lifecycle"]
    end

    subgraph Session["cmd/olliesrv/internal/session/"]
        SESS["session.go, registry.go — sessions and agents"]
        PERSIST["persist.go — session persistence"]
    end

    subgraph Fs["cmd/olliesrv/internal/fs/"]
        ROOT["newroot.go — NewRoot, RestoreSessions"]
        SPEC["spec.go — 9P namespace declaration"]
        SUPPORT["support.go — namespace support"]
    end

    subgraph Toolsrv["cmd/toolsrv/internal/"]
        SERVER["server — 9P tool server and proc handlers"]
        REGISTRY["registry — per-agent loaded tools"]
        EXEC["exec — tool process execution"]
        SANDBOX["sandbox — native Landlock policy"]
    end

    Agent --> Backend
    Agent --> ToolClient
    Agent --> Session
    ToolClient --> Toolsrv
    Fs --> Session
    Toolsrv --> REGISTRY
    Toolsrv --> EXEC
    EXEC --> SANDBOX
```

## The Agent Struct (`cmd/olliesrv/internal/agent/agent.go`)

`Agent` owns one agent identity inside a session. Its runtime is swappable: `/agent` loads a new profile, creates a fresh toolsrv connection, rebuilds the runtime, and clears history. The session remains the stable host.

```go
type Agent struct {
    history       *History
    runtime       *Runtime
    name          string
    profile       string
    agentsDir     string
    currentAction atomic.Pointer[actionHandle]
    fifo          Fifo
    Feed          Feed
    output        EventHandler
    sessionID     string
    id            string
    parentID      string
    cwd           string
    chatLog       []byte
    // synchronization, persistence, and observation state
}

func NewAgent(cfg AgentParams) *Agent
```

All output flows through the agent's `EventHandler` as `Event{Role, Name, Content, ResponseID, OutputFormat}` values. Events include `user`, `assistant`, `reasoning`, `call`, `tool`, `usage`, `state`, `limitretry`, `retry`, `maxsteps`, `error`, and `info`. Session-level events use a separate topic-based pubsub bus for 9P observers.

## Agent Construction (`cmd/olliesrv/internal/agent/agent.go`, `runtime.go`, `agent_config.go`)

### AgentConfig

The full construction config:

| Field | Purpose |
|---|---|
| `History` | Pre-existing session history to restore (nil = new) |
| `Runtime` | Pre-built *Runtime (tools, hooks, prompt, params) |
| `Profile` | Name of the active agent configuration profile |
| `ID` | Immutable agent ID (`uname`) |
| `Name` | Mutable display name |
| `AgentsDir` | Path to agent JSON directory |
| `SessionsDir` | Path to persisted session files |
| `SessionID` | Unique session identifier |
| `AgentID` | Immutable agent principal |
| `CWD` | Working directory for tools and prompt |
| `NewToolServer` | Factory for fresh `ToolsrvConn` on agent switch |
| `NewBackend` | Factory for backend by name |
| `Log` | Logger instance |
| `MaxSteps` | Override agent config's maxSteps |
| `ReadPlanStep` | Function to read next unchecked plan step |
| `ListHandlers` | Additional list handlers |

### `Runtime`

`Runtime` contains the active backend, authenticated 9P toolsrv connection, rendered preamble, tool metadata, executor closure, generation parameters, max-step limit, compaction model, resolved user prompt, and startup messages. `BuildRuntime` lists loaded tools, resolves agent and user prompts, injects dispatch flags into tool schemas, and routes calls through `ToolsrvConn.CallTool(ctx, ...)`.

```go
type Runtime struct {
    Backend         backend.Backend
    ToolServer      *toolclient.ToolsrvConn
    Preamble        *Preamble
    Tools           []backend.Tool
    ToolMeta        map[string]protocol.ToolInfo
    ToolRevision    uint64
    Exec            toolExecutor
    GenParams       backend.GenerationParams
    MaxSteps        int
    CompactionModel string
    UserPrompt      string
    Messages        []string
}
```

### BuildRuntime

`BuildRuntime(cfg, runner, cwd, env)` wires everything together:

1. Lists the agent’s currently loaded tools from the runner and records the agent-scoped registry revision
2. Resolves the system prompt by executing the config's `prompt` commands
3. Extracts generation parameters from the config
4. Builds the tool executor closure (routes calls through the runner)
5. Extracts parallelism and tier classifiers
6. Applies tool/executor restrictions if configured
7. Returns a `*Runtime` struct ready for `New`

### Agent construction

`NewAgent(AgentParams)` creates an agent with identity, session, profile, history, runtime, working directory, persistence callbacks, event handler state, chat-log state, and session environment. The configuration type on disk is `AgentConfig`; construction dependencies are carried separately by `AgentParams`.

## The Agent Loop (`cmd/olliesrv/internal/agent/`)

The agent package is organized by concern:

| File | Responsibility |
|------|---------------|
| `loop.go` | Main loop: stream LLM, execute tools, update history |
| `turn.go` | Submit entry point, turn orchestration |
| `dispatch.go` | Tool execution, batching, conflict detection |
| `history.go` | Message history, usage tracking |
| `compact.go` | Context compaction, cold summarization |
| `cache.go` | Tool result caching with file staleness |
| `retry.go` | Error tracking, transient retry logic |
| `state.go` | State management, signals, WaitChange |
| `chat.go` | Chat log, streaming output |
| `peer.go` | Peer agent management |
| `subagent.go` | Sub-agent depth tracking |

The `run()` function in `loop.go` is the core execution engine. It loops until the model stops calling tools.

### Per-Iteration Flow

```mermaid
flowchart TB
    START["Start turn"]
    GATE["Context gate:\nstrip cold material\nif over budget"]
    BUILD["Build message history:\nsystem prompt + task state + hot tail"]
    STREAM["Stream from backend\n(with retry logic)"]
    CHECK{"Tool calls\nin response?"}
    EXEC["Execute tool calls\n(serial or parallel)"]
    UPDATE["Update state:\nappend messages + results"]
    INFER["Infer TaskState\nupdates"]
    SAFETY["Safety checks:\nerror limits, stall,\nreplan, max steps"]
    COMPACT["Auto-compact\nif needed"]
    DONE["Turn complete"]

    START --> GATE --> BUILD --> STREAM --> CHECK
    CHECK -- yes --> EXEC --> UPDATE --> INFER --> SAFETY --> COMPACT --> BUILD
    CHECK -- no --> DONE
```

### Streaming & Retry

The streaming loop handles three failure modes:

| Failure | Detection | Recovery |
|---|---|---|
| Rate limit (429) | `*RateLimitError` | Retry using provider `Retry-After` when available, otherwise exponential backoff |
| Transient (5xx/network) | `*TransientError` | Retry with exponential backoff |
| Stream drop | Stream closes without completion | Retry |
| Context overflow | `*ContextOverflowError` | Compact and retry once |
| Tool unsupported | `*ToolUnsupportedError` | Return through normal turn error handling |

On the first error, the `turnError` hook fires. If it exits 0 (handled), the loop returns immediately — the hook is responsible for recovery (e.g., switching models via `freeloader`).

### Tool Execution

Tool calls are dispatched with parallelism awareness. Tool schemas are
loaded progressively: the default profile starts with `client_9p`, and semantic
tool hints tell the agent to issue `tool_load <name>` through its agent `ctl`
file.

After a tool round, the agent compares the toolsrv agent-scoped registry
revision with `Runtime.ToolRevision`. If the revision changed, it lists the
loaded tools, rebuilds backend schemas and metadata, and updates the rendered
tool preamble before the next model request.

1. Classify each call from the tool metadata and dispatch flags.
2. Batch read-safe calls when the metadata permits parallel execution.
3. Execute calls through the runtime executor, with per-turn caching for eligible file reads.
4. Hard-limit model-facing tool output to 32 KiB.
5. Assign each result a hot, warm, or cold memory tier for context management.

Tool calls use the current turn context, so interrupting a turn cancels in-flight tool execution.

### Safety Mechanisms

| Mechanism | Threshold | Action |
|---|---|---|
| Consecutive errors (soft) | 5 rounds | Inject nudge: "try a different approach" |
| Consecutive errors (hard) | 10 rounds | Abort the turn |
| Replan gate | 8 rounds without PLAN: block | Inject replan nudge |
| Stall detection | 5 rounds with same LastAction | Inject stall nudge |
| Max steps | Configurable per-agent | Soft exit with nudge |
| Plan re-injection | Every 10 tool rounds | Re-inject task state |

### TaskState Inference

After each tool round, `inferTaskStateUpdate` updates the structured overlay:
- `LastAction` ← most recent tool call + truncated result
- `PlanStep` ← next unchecked item from the plan file (or heuristic from assistant text)

## Session & Context (`cmd/olliesrv/internal/agent/history.go`, `compact.go`)

### Message History

`History` holds a flat `[]backend.Message` slice. No rolling window — grows until compaction.

**Checkpoint**: `History.Checkpoint(ts TaskState)` forks the session — returns a new History that inherits the given TaskState but starts with a clean message history. Used for narrow-context sub-agents that know what to do without inheriting parent message noise.

Token tracking per session:
- `TotalInputTokens`, `TotalOutputTokens`, `TotalRequests`
- `TotalCachedInputTokens`, `TotalCacheCreationTokens`
- Per-turn accumulators reset at turn start
- `LastTurnCostUSD`, `SessionCostUSD`

### Compaction

Triggered when estimated tokens reach 50% of model context length.

**Three-zone strategy:**

| Zone | Content | Size |
|---|---|---|
| Cold | Structured TaskState JSON | 1 message |
| Warm | One-line decision index | ~10 messages |
| Hot | Verbatim recent messages | 8 messages |

Process:
1. Flatten tool call/result pairs to plain text
2. Send full history + compaction prompt to the LLM
3. Parse structured JSON response into TaskState
4. Rebuild history as cold + warm + hot zones
5. Write pre-compaction snapshot to `{id}.compaction.jsonl` for recovery

### Result Tiers

Tool results are classified at execution time:

| Tier | Behavior | Examples |
|---|---|---|
| Hot | Stays verbatim in messages | `file_write`, `file_edit`, `reasoning_think` |
| Warm | Summarized on next compaction | (custom tools) |
| Cold | Immediately collapsed to one-line summary | `file_read`, `memory_recall`, `lsp_*` |

1. The tool's `tier` metadata (`cold`, `warm`, or `hot`)
2. `hot` when no tier metadata is present

### Persistence

Sessions are saved as `PersistedSession` JSON:
```json
{
  "id": "1716300000000-abc123",
  "agent": "coding",
  "messages": [...],
  "taskState": {...}
}
```

Saved after every tool round (`saveSession` callback) and on clean exit.

## Hooks and lifecycle callbacks

Hooks are shell commands executed with:
- **stdin**: JSON payload with context (session_id, cwd, model, error, etc.)
- **env**: `OLLIE_*` variables derived from the payload map
- **timeout**: 60 seconds (configurable in tests)
- **process group**: `Setpgid=true` for clean kill on timeout

### Exit Code Semantics

| Code | Meaning |
|---|---|
| 0 | Success. Stdout appended to context. |
| 2 | Block. Stops the action (e.g., blocks a tool call, prevents compaction). |
| Other | Non-blocking warning. Execution continues. |

### Hook Execution Order

Multiple commands per hook run sequentially. If any exits 2, execution stops immediately and returns `Blocked=true`.

## Commands (`cmd/olliesrv/internal/agent/commands.go`)

`HandleCommand` forwards slash-command text to the agent `ctl` file over 9P. The current command set is defined by the filesystem control handler; examples include profile/backend/model changes, compaction, context and usage inspection, session lifecycle, tool/skill listing, prompt injection, queue management, and working-directory changes. `Inject` queues an interruption for the running turn; `InjectRewrite` replaces the pending interruption.

### Agent Switching

When `/agent <name>` is issued:
1. Load the new agent config JSON
2. Create a fresh runner via `newToolServer()`
3. Call `BuildRuntime` with the new config
4. If the config specifies a backend/model, switch those too
5. Replace the entire `Runtime` pointer on the agent struct
6. Clear the session (fresh context for the new persona)

## Backend System (`cmd/olliesrv/internal/backend/`)

### Shared Types

| Type | Purpose |
|---|---|
| `Message` | Conversation turn (role, content, tool_calls, tool_call_id) |
| `Tool` | Function declaration (name, description, JSON Schema params) |
| `ToolCall` | Model's request to invoke a function |
| `StreamEvent` | Incremental streaming delta (content, reasoning, tool_calls, done, usage) |
| `Usage` | Token counts + optional cost |
| `GenerationParams` | Unified sampling parameters |

### Error Types

| Error | HTTP Code | Loop Behavior |
|---|---|---|
| `RateLimitError` | 429 | Retry with backoff |
| `TransientError` | 5xx, network | Retry with backoff |
| `ContextOverflowError` | 400 (specific) | Compact and retry |
| `ToolUnsupportedError` | 400/422 (specific) | Propagate to turnError hook |

### streamRequest

Shared HTTP streaming infrastructure used by all backends:
1. Execute the HTTP request
2. Classify response status into typed errors
3. Spawn a goroutine that parses SSE/NDJSON into `StreamEvent` channel
4. Return the channel for the loop to consume

### Backend Implementations

**OllamaBackend** (`/api/chat`):
- Local inference, no API key
- Context length from `/api/show` (cached)
- Model list from `/api/tags`
- NDJSON streaming

**OpenAIBackend** (`/v1/chat/completions`):
- Works with OpenAI, OpenRouter, any compatible API
- Configurable base URL, name, extra headers
- Context length from model list endpoint (cached)
- SSE streaming (`data: {...}` lines)
- Handles both `tool_calls` and legacy `function_call`

**AnthropicBackend** (`/v1/messages`):
- Extended thinking (reasoning budget via `ThinkingBudget`)
- Prompt caching (cache_control blocks)
- 200k context window (hardcoded)
- SSE streaming with content_block_delta events

**GeminiBackend** (Gemini API):
- Google Gemini models

**CopilotBackend** (GitHub Copilot API):
- Copilot Chat integration

**CodeWhispererBackend** (AWS CodeWhisperer API):
- AWS CodeWhisperer integration

## Config System (`cmd/olliesrv/internal/agent/agent_config.go`)

### Agent Config Schema

The on-disk `AgentConfig` currently contains `prompt`, `userPrompts`, `backend`, `model`, `tools`, `autoLoad`, `maxSteps`, `compactionModel`, `systemPrompt`, and embedded backend `GenerationParams`. Tool and prompt resolution are performed while building the runtime.

### Prompt Type

Dual-mode unmarshaling:
- **JSON string**: treated as literal text (with `!command`, file path, or env expansion)
- **JSON array**: each element is a shell command; stdout concatenated

The array form is preferred — it enables composable prompt assembly from template fragments.

### HookCmds Type

Dual-mode: a single string `"cmd"` or an array `["cmd1", "cmd2"]`. Multiple commands run sequentially.

## Prompt Resolution

Prompt assembly and resolution are documented in [`architecture-prompting.md`](architecture-prompting.md). The agent runtime consumes the resulting preamble and per-turn user prompt.

## Cost Tracking (`cmd/olliesrv/internal/agent/cost.go`)

When the backend doesn't report cost directly (only OpenRouter does), cost is computed from a built-in pricing table:

- Matches model name prefix (case-insensitive)
- Covers Anthropic Claude, OpenAI GPT/o-series, Google Gemini
- Cached input tokens charged at discount (10% Anthropic, 50% others)
- Cache creation tokens charged at 125% of input rate
- Unknown/local models return zero cost

## Sandbox

Sandbox policy is owned and enforced by toolsrv. See [`architecture-toolsrv.md`](architecture-toolsrv.md#sandbox).

## Event System

The agent emits events through its `EventHandler`. Session-level observers use `session.PublishEvent`, `SubscribeEvents`, and topic wildcards. The filesystem layer converts session events into chat and blocking event files.

## Concurrency Model

- **One turn at a time**: `submitMu` serializes `Submit()` calls
- **Interrupt**: `currentAction` holds a `CancelCauseFunc`; `Interrupt()` calls it
- **Queue drain**: after each turn completes, queued prompts are drained sequentially
- **State observation**: agent state and chat changes signal 9P waiters through change channels and condition variables
- **Tool parallelism**: read-safe tools fan out within a turn when metadata permits; result caching and `/tmp/ollie/{sessionID}` locks coordinate execution
- **Peer messaging**: writing to `peer/{name}` triggers an async `Submit()` on the target agent; delivery is fire-and-forget within the session

## Session ID Format

```
<unix-nanoseconds>-<6-hex-chars>
```

Lexicographically sortable by creation time. Unique even within the same nanosecond due to the random suffix.