ollie/doc/architecture-core.md

# Core Architecture

This document describes the agent-side core. For the tool-execution service, its 9P namespace, registry, metadata, sandbox, process lifecycle, and cancellation path, see [`architecture-toolsrv.md`](architecture-toolsrv.md).

This document covers the internal Go packages that implement Ollie's agent runtime. The code lives under `cmd/olliesrv/internal/`; it is not a separately imported public library. `olliesrv` composes the agent, backend, session, filesystem, and tool-client packages.

## Package Map

```mermaid
flowchart TB
    subgraph Agent["cmd/olliesrv/internal/agent/"]
        AGENT["agent.go — Agent, events, interrupt"]
        LOOP["loop.go — agent loop"]
        HISTORY["history.go — history, task state, compaction"]
        TURN["turn.go — per-turn state and cancellation"]
        CMDS["commands.go — command dispatch"]
        RUNTIME["runtime.go — Runtime, preamble, tool dispatch"]
        CONFIG["agent_config.go — profile configuration"]
        PROMPT["prompt_resolver.go — prompt resolution"]
    end

    subgraph Backend["cmd/olliesrv/internal/backend/"]
        BACKEND["backend.go — Backend interface and streaming"]
        PROVIDERS["openai, anthropic, ollama, gemini, copilot, kiro, codewhisperer"]
        POOL["pool.go — backend pool"]
    end

    subgraph ToolClient["cmd/olliesrv/internal/toolclient/"]
        CLIENT["9P toolsrv client"]
        SPAWN["spawn.go — local/remote toolsrv lifecycle"]
    end

    subgraph Session["cmd/olliesrv/internal/session/"]
        SESS["session.go, registry.go — sessions and agents"]
        PERSIST["persist.go — session persistence"]
    end

    subgraph Fs["cmd/olliesrv/internal/fs/"]
        ROOT["newroot.go — NewRoot, RestoreSessions"]
        SPEC["spec.go — 9P namespace declaration"]
        SUPPORT["support.go — namespace support"]
    end

    subgraph Toolsrv["cmd/toolsrv/internal/"]
        SERVER["server — 9P tool server and proc handlers"]
        REGISTRY["registry — per-agent loaded tools"]
        EXEC["exec — tool process execution"]
        SANDBOX["sandbox — native Landlock policy"]
    end

    Agent --> Backend
    Agent --> ToolClient
    Agent --> Session
    ToolClient --> Toolsrv
    Fs --> Session
    Toolsrv --> REGISTRY
    Toolsrv --> EXEC
    EXEC --> SANDBOX
```

## The Agent Struct (`cmd/olliesrv/internal/agent/agent.go`)

`Agent` owns one agent identity inside a session. Its runtime is swappable: `/agent` loads a new profile, creates a fresh toolsrv connection, rebuilds the runtime, and clears history. The session remains the stable host.

```go
type Agent struct {
    history       *History
    runtime       *Runtime
    name          string
    profile       string
    agentsDir     string
    currentAction atomic.Pointer[actionHandle]
    fifo          Fifo
    Feed          Feed
    output        EventHandler
    sessionID     string
    id            string
    parentID      string
    cwd           string
    chatLog       []byte
    // synchronization, persistence, and observation state
}

func NewAgent(cfg AgentParams) *Agent
```

All output flows through the agent's `EventHandler` as `Event{Role, Name, Content, ResponseID, OutputFormat}` values. Events include `user`, `assistant`, `reasoning`, `call`, `tool`, `usage`, `state`, `limitretry`, `retry`, `maxsteps`, `error`, and `info`. Session-level events use a separate topic-based pubsub bus for 9P observers.

## Agent Construction (`cmd/olliesrv/internal/agent/agent.go`, `runtime.go`, `agent_config.go`)

### AgentConfig

The full construction config:

| Field | Purpose |
|---|---|
| `History` | Pre-existing session history to restore (nil = new) |
| `Runtime` | Pre-built *Runtime (tools, hooks, prompt, params) |
| `Profile` | Name of the active agent configuration profile |
| `ID` | Immutable agent ID (`uname`) |
| `Name` | Mutable display name |
| `AgentsDir` | Path to agent JSON directory |
| `SessionsDir` | Path to persisted session files |
| `SessionID` | Unique session identifier |
| `AgentID` | Immutable agent principal |
| `CWD` | Working directory for tools and prompt |
| `NewToolServer` | Factory for fresh `ToolsrvConn` on agent switch |
| `NewBackend` | Factory for backend by name |
| `Log` | Logger instance |
| `MaxSteps` | Override agent config's maxSteps |
| `ReadPlanStep` | Function to read next unchecked plan step |
| `ListHandlers` | Additional list handlers |

### `Runtime`

`Runtime` contains the active backend, authenticated 9P toolsrv connection, rendered preamble, tool metadata, executor closure, generation parameters, max-step limit, compaction model, resolved user prompt, and startup messages. `BuildRuntime` lists loaded tools, resolves agent and user prompts, injects dispatch flags into tool schemas, and routes calls through `ToolsrvConn.CallTool(ctx, ...)`.

```go
type Runtime struct {
    Backend         backend.Backend
    ToolServer      *toolclient.ToolsrvConn
    Preamble        *Preamble
    Tools           []backend.Tool
    ToolMeta        map[string]protocol.ToolInfo
    Exec            toolExecutor
    GenParams       backend.GenerationParams
    MaxSteps        int
    CompactionModel string
    UserPrompt      string
    Messages        []string
}
```

### BuildRuntime

`BuildRuntime(cfg, runner, cwd, env)` wires everything together:

1. Lists all tools from the runner
2. Resolves the system prompt by executing the config's `prompt` commands
3. Extracts generation parameters from the config
4. Builds the tool executor closure (routes calls through the runner)
5. Extracts parallelism and tier classifiers
6. Applies tool/executor restrictions if configured
7. Returns a `*Runtime` struct ready for `New`

### Agent construction

`NewAgent(AgentParams)` creates an agent with identity, session, profile, history, runtime, working directory, persistence callbacks, event handler state, chat-log state, and session environment. The configuration type on disk is `AgentConfig`; construction dependencies are carried separately by `AgentParams`.

## The Agent Loop (`cmd/olliesrv/internal/agent/loop.go`, `turn.go`)

The `run()` function is the core execution engine. It takes an `agentConfig` and a `state` interface, and loops until the model stops calling tools.

### Per-Iteration Flow

```mermaid
flowchart TB
    START["Start turn"]
    GATE["Context gate:\nstrip cold material\nif over budget"]
    BUILD["Build message history:\nsystem prompt + task state + hot tail"]
    STREAM["Stream from backend\n(with retry logic)"]
    CHECK{"Tool calls\nin response?"}
    EXEC["Execute tool calls\n(serial or parallel)"]
    UPDATE["Update state:\nappend messages + results"]
    INFER["Infer TaskState\nupdates"]
    SAFETY["Safety checks:\nerror limits, stall,\nreplan, max steps"]
    COMPACT["Auto-compact\nif needed"]
    DONE["Turn complete"]

    START --> GATE --> BUILD --> STREAM --> CHECK
    CHECK -- yes --> EXEC --> UPDATE --> INFER --> SAFETY --> COMPACT --> BUILD
    CHECK -- no --> DONE
```

### Streaming & Retry

The streaming loop handles three failure modes:

| Failure | Detection | Recovery |
|---|---|---|
| Rate limit (429) | `*RateLimitError` | Retry using provider `Retry-After` when available, otherwise exponential backoff |
| Transient (5xx/network) | `*TransientError` | Retry with exponential backoff |
| Stream drop | Stream closes without completion | Retry |
| Context overflow | `*ContextOverflowError` | Compact and retry once |
| Tool unsupported | `*ToolUnsupportedError` | Return through normal turn error handling |

On the first error, the `turnError` hook fires. If it exits 0 (handled), the loop returns immediately — the hook is responsible for recovery (e.g., switching models via `freeloader`).

### Tool Execution

Tool calls are dispatched with parallelism awareness:

1. Classify each call from the tool metadata and dispatch flags.
2. Batch read-safe calls when the metadata permits parallel execution.
3. Execute calls through the runtime executor, with per-turn caching for eligible file reads.
4. Hard-limit model-facing tool output to 32 KiB.
5. Assign each result a hot, warm, or cold memory tier for context management.

Tool calls use the current turn context, so interrupting a turn cancels in-flight tool execution.

### Safety Mechanisms

| Mechanism | Threshold | Action |
|---|---|---|
| Consecutive errors (soft) | 5 rounds | Inject nudge: "try a different approach" |
| Consecutive errors (hard) | 10 rounds | Abort the turn |
| Replan gate | 8 rounds without PLAN: block | Inject replan nudge |
| Stall detection | 5 rounds with same LastAction | Inject stall nudge |
| Max steps | Configurable per-agent | Soft exit with nudge |
| Plan re-injection | Every 10 tool rounds | Re-inject task state |

### TaskState Inference

After each tool round, `inferTaskStateUpdate` updates the structured overlay:
- `LastAction` ← most recent tool call + truncated result
- `PlanStep` ← next unchecked item from the plan file (or heuristic from assistant text)

## Session & Context (`cmd/olliesrv/internal/agent/history.go`)

### Message History

`History` holds a flat `[]backend.Message` slice. No rolling window — grows until compaction.

**Checkpoint**: `History.Checkpoint(ts TaskState)` forks the session — returns a new History that inherits the given TaskState but starts with a clean message history. Used for narrow-context sub-agents that know what to do without inheriting parent message noise.

Token tracking per session:
- `TotalInputTokens`, `TotalOutputTokens`, `TotalRequests`
- `TotalCachedInputTokens`, `TotalCacheCreationTokens`
- Per-turn accumulators reset at turn start
- `LastTurnCostUSD`, `SessionCostUSD`

### Compaction

Triggered when estimated tokens reach 50% of model context length.

**Three-zone strategy:**

| Zone | Content | Size |
|---|---|---|
| Cold | Structured TaskState JSON | 1 message |
| Warm | One-line decision index | ~10 messages |
| Hot | Verbatim recent messages | 8 messages |

Process:
1. Flatten tool call/result pairs to plain text
2. Send full history + compaction prompt to the LLM
3. Parse structured JSON response into TaskState
4. Rebuild history as cold + warm + hot zones
5. Write pre-compaction snapshot to `{id}.compaction.jsonl` for recovery

### Result Tiers

Tool results are classified at execution time:

| Tier | Behavior | Examples |
|---|---|---|
| Hot | Stays verbatim in messages | `file_write`, `file_edit`, `reasoning_think` |
| Warm | Summarized on next compaction | (custom tools) |
| Cold | Immediately collapsed to one-line summary | `file_read`, `memory_recall`, `lsp_*` |

1. The tool's `tier` metadata (`cold`, `warm`, or `hot`)
2. `hot` when no tier metadata is present

### Persistence

Sessions are saved as `PersistedSession` JSON:
```json
{
  "id": "1716300000000-abc123",
  "agent": "coding",
  "messages": [...],
  "taskState": {...}
}
```

Saved after every tool round (`saveSession` callback) and on clean exit.

## Hooks and lifecycle callbacks

Hooks are shell commands executed with:
- **stdin**: JSON payload with context (session_id, cwd, model, error, etc.)
- **env**: `OLLIE_*` variables derived from the payload map
- **timeout**: 60 seconds (configurable in tests)
- **process group**: `Setpgid=true` for clean kill on timeout

### Exit Code Semantics

| Code | Meaning |
|---|---|
| 0 | Success. Stdout appended to context. |
| 2 | Block. Stops the action (e.g., blocks a tool call, prevents compaction). |
| Other | Non-blocking warning. Execution continues. |

### Hook Execution Order

Multiple commands per hook run sequentially. If any exits 2, execution stops immediately and returns `Blocked=true`.

## Commands (`cmd/olliesrv/internal/agent/commands.go`)

`HandleCommand` forwards slash-command text to the agent `ctl` file over 9P. The current command set is defined by the filesystem control handler; examples include profile/backend/model changes, compaction, context and usage inspection, session lifecycle, tool/skill listing, prompt injection, queue management, and working-directory changes. `Inject` queues an interruption for the running turn; `InjectRewrite` replaces the pending interruption.

### Agent Switching

When `/agent <name>` is issued:
1. Load the new agent config JSON
2. Create a fresh runner via `newToolServer()`
3. Call `BuildRuntime` with the new config
4. If the config specifies a backend/model, switch those too
5. Replace the entire `Runtime` pointer on the agent struct
6. Clear the session (fresh context for the new persona)

## Backend System (`cmd/olliesrv/internal/backend/`)

### Shared Types

| Type | Purpose |
|---|---|
| `Message` | Conversation turn (role, content, tool_calls, tool_call_id) |
| `Tool` | Function declaration (name, description, JSON Schema params) |
| `ToolCall` | Model's request to invoke a function |
| `StreamEvent` | Incremental streaming delta (content, reasoning, tool_calls, done, usage) |
| `Usage` | Token counts + optional cost |
| `GenerationParams` | Unified sampling parameters |

### Error Types

| Error | HTTP Code | Loop Behavior |
|---|---|---|
| `RateLimitError` | 429 | Retry with backoff |
| `TransientError` | 5xx, network | Retry with backoff |
| `ContextOverflowError` | 400 (specific) | Compact and retry |
| `ToolUnsupportedError` | 400/422 (specific) | Propagate to turnError hook |

### streamRequest

Shared HTTP streaming infrastructure used by all backends:
1. Execute the HTTP request
2. Classify response status into typed errors
3. Spawn a goroutine that parses SSE/NDJSON into `StreamEvent` channel
4. Return the channel for the loop to consume

### Backend Implementations

**OllamaBackend** (`/api/chat`):
- Local inference, no API key
- Context length from `/api/show` (cached)
- Model list from `/api/tags`
- NDJSON streaming

**OpenAIBackend** (`/v1/chat/completions`):
- Works with OpenAI, OpenRouter, any compatible API
- Configurable base URL, name, extra headers
- Context length from model list endpoint (cached)
- SSE streaming (`data: {...}` lines)
- Handles both `tool_calls` and legacy `function_call`

**AnthropicBackend** (`/v1/messages`):
- Extended thinking (reasoning budget via `ThinkingBudget`)
- Prompt caching (cache_control blocks)
- 200k context window (hardcoded)
- SSE streaming with content_block_delta events

**GeminiBackend** (Gemini API):
- Google Gemini models

**CopilotBackend** (GitHub Copilot API):
- Copilot Chat integration

**CodeWhispererBackend** (AWS CodeWhisperer API):
- AWS CodeWhisperer integration

## Config System (`cmd/olliesrv/internal/agent/agent_config.go`)

### Agent Config Schema

The on-disk `AgentConfig` currently contains `prompt`, `userPrompts`, `backend`, `model`, `tools`, `autoLoad`, `maxSteps`, `compactionModel`, `systemPrompt`, and embedded backend `GenerationParams`. Tool and prompt resolution are performed while building the runtime.

### Prompt Type

Dual-mode unmarshaling:
- **JSON string**: treated as literal text (with `!command`, file path, or env expansion)
- **JSON array**: each element is a shell command; stdout concatenated

The array form is preferred — it enables composable prompt assembly from template fragments.

### HookCmds Type

Dual-mode: a single string `"cmd"` or an array `["cmd1", "cmd2"]`. Multiple commands run sequentially.

## Prompt Resolution

Prompt assembly and resolution are documented in [`architecture-prompting.md`](architecture-prompting.md). The agent runtime consumes the resulting preamble and per-turn user prompt.

## Cost Tracking (`cmd/olliesrv/internal/agent/cost.go`)

When the backend doesn't report cost directly (only OpenRouter does), cost is computed from a built-in pricing table:

- Matches model name prefix (case-insensitive)
- Covers Anthropic Claude, OpenAI GPT/o-series, Google Gemini
- Cached input tokens charged at discount (10% Anthropic, 50% others)
- Cache creation tokens charged at 125% of input rate
- Unknown/local models return zero cost

## Sandbox

Sandbox policy is owned and enforced by toolsrv. See [`architecture-toolsrv.md`](architecture-toolsrv.md#sandbox).

## Event System

The agent emits events through its `EventHandler`. Session-level observers use `session.PublishEvent`, `SubscribeEvents`, and topic wildcards. The filesystem layer converts session events into chat and blocking event files.

## Concurrency Model

- **One turn at a time**: `submitMu` serializes `Submit()` calls
- **Interrupt**: `currentAction` holds a `CancelCauseFunc`; `Interrupt()` calls it
- **Queue drain**: after each turn completes, queued prompts are drained sequentially
- **State observation**: agent state and chat changes signal 9P waiters through change channels and condition variables
- **Tool parallelism**: read-safe tools fan out within a turn when metadata permits; result caching and `/tmp/ollie/{sessionID}` locks coordinate execution

## Session ID Format

```
<unix-nanoseconds>-<6-hex-chars>
```

Lexicographically sortable by creation time. Unique even within the same nanosecond due to the random suffix.