ollie/doc/CORE.md

686 lines
26 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# ollie-core: Detailed Architecture
This document covers the internal design of the `core/` library — the Go package that implements the agent engine. It is the dependency imported by all consumers (olliesrv, frontends, tests).
## Package Map
```
core/
├── pkg/
│ ├── agent/ The agent engine
│ │ ├── types.go Core interface, Event, PromptFIFO
│ │ ├── runtime.go Runtime struct (swappable per-agent config)
│ │ ├── core.go AgentCoreConfig, agent struct, NewAgentCore, BuildRuntime
│ │ ├── loop.go The agent loop (run function)
│ │ ├── session.go Session, TaskState, compaction, persistence
│ │ ├── state.go state interface, ResultTier, toolResult
│ │ ├── hooks.go Hooks map, HookResult, hook execution
│ │ ├── commands.go Slash-command dispatch (/model, /agent, /compact, ...)
│ │ ├── cost.go Model pricing table, cost computation
│ │ └── prompt_resolver.go System prompt assembly from shell commands
│ ├── backend/ LLM provider implementations
│ │ ├── backend.go Backend interface, shared types, streamRequest
│ │ ├── openai.go OpenAI/OpenRouter/compatible APIs
│ │ ├── anthropic.go Anthropic Messages API
│ │ └── ollama.go Ollama local inference
│ ├── config/ Agent config JSON schema and loader
│ ├── env/ Environment variable loading
│ ├── log/ Structured logger
│ ├── paths/ XDG directory resolution
│ └── tools/ Tool system
│ ├── tools.go Server, Dispatcher, classifier interfaces
│ └── execute/ The execute.Server
│ ├── server.go Server struct, Decl factory, ListTools, options
│ ├── code.go Step resolution, validation, dispatch routing
│ ├── tool.go Named tool script discovery and execution
│ ├── pipe.go Pipeline stage execution, parallel fan-out
│ ├── flock.go Advisory locking, parallelism classification
│ └── tier.go Result tier classification (hot/warm/cold)
└── internal/
└── sandbox/ Landlock sandbox wrapper
├── config.go YAML config, path expansion, access checking
└── wrapper.go landrun command construction
```
## The Core Interface (`pkg/agent/types.go`)
`Core` is the boundary between the engine and any consumer. It is deliberately narrow:
```go
type Core interface {
// Turn lifecycle
Submit(ctx context.Context, input string) // process user input (commands or prompts)
Interrupt(cause error) bool // cancel current turn
Inject(prompt string) // append to next tool result
Queue(prompt string) // FIFO for deferred prompts
PopQueue() (string, bool) // drain one queued prompt
// Observation
IsRunning() bool
State() string // "idle" | "thinking" | "calling: <tool>"
Reply() string // last assistant text
Bus() *pubsub.Bus // event stream
WaitChange(ctx context.Context, field, current string) (string, bool)
// Configuration
AgentName() string
BackendName() string
ModelName() string
CWD() string
SetCWD(dir string) error
SetSessionID(newID string) error
SetEnv(key, value string)
SetGenerationParams(params backend.GenerationParams) error
// Introspection
Context() []backend.Message
SystemPrompt() string
GenerationParams() backend.GenerationParams
CtxSz() string
Usage() string
Cost() string
ListModels() string
ToolCallCount() int64
// Cleanup
Close()
}
```
All output flows through the event bus as `Event{Role, Name, Content}` values. Roles include: `assistant`, `tool`, `call`, `info`, `exec`, `error`, `usage`, `reasoning`, `limitretry`.
## Agent Construction (`pkg/agent/core.go`)
### AgentCoreConfig
The full construction config:
| Field | Purpose |
|---|---|
| `Backend` | Active LLM backend instance |
| `ModelName` | Override backend's default model |
| `AgentName` | Name of the active agent config |
| `AgentsDir` | Path to agent JSON directory |
| `SessionsDir` | Path to persisted session files |
| `SessionID` | Unique session identifier |
| `Uname` | Immutable 9P user principal |
| `CWD` | Working directory for tools and prompt |
| `Session` | Pre-existing session to restore (nil = new) |
| `Runtime` | Pre-built *Runtime (tools, hooks, prompt, params) |
| `NewDispatcher` | Factory for fresh Dispatcher on agent switch |
| `NewBackend` | Factory for backend by name |
| `Log` | Logger instance |
| `MaxSteps` | Override agent config's maxSteps |
### Runtime
The `Runtime` struct holds all per-agent configuration that changes on an `/agent` switch but remains stable across turns. The agent struct stores a pointer to the active Runtime; switching agents replaces it atomically.
```go
type Runtime struct {
Backend backend.Backend
Dispatcher tools.Dispatcher
Hooks Hooks
Preamble string // compiled system prompt
Tools []backend.Tool
Exec toolExecutor
ClassifyTool toolClassifier
ClassifyTier func(string) ResultTier
GenParams backend.GenerationParams
MaxSteps int
ToolResultMaxBytes int // 0 means use default (128KB)
CfgBackend string // backend override from agent JSON
CfgModel string // model override from agent JSON
Messages []string // non-fatal warnings from construction
}
```
### BuildRuntime
`BuildRuntime(cfg, dispatcher, cwd, env)` wires everything together:
1. Lists all tools from the dispatcher
2. Resolves the system prompt by executing the config's `prompt` commands
3. Extracts generation parameters from the config
4. Builds the tool executor closure (routes calls through the dispatcher)
5. Extracts parallelism and tier classifiers from the execute server
6. Applies tool/executor restrictions if configured
7. Returns a `*Runtime` struct ready for `NewAgentCore`
### NewAgentCore
Creates the `agent` struct from `AgentCoreConfig`:
1. Sweeps stale tmpdirs from previous crashes
2. Stores the provided `Runtime` (or creates an empty one)
3. Sets the backend model if `ModelName` is specified
4. Creates `/tmp/ollie/{sessionID}` for this session
5. Sets up the plan file reader (reads `s/{id}/plan` for the next unchecked step)
6. Wires the `turnError` hook with a detached context
7. Pushes `OLLIE_SESSION_ID` and `OLLIE_UNAME` into the execute server env
8. Sets the flock directory for parallel coordination
## The Agent Loop (`pkg/agent/loop.go`)
The `run()` function is the core execution engine. It takes an `agentConfig` and a `state` interface, and loops until the model stops calling tools.
The `agentConfig` struct bundles per-turn dependencies: `Backend`, `Tools`, `Exec` (tool executor closure), `ClassifyTool`/`ClassifyTier` (parallelism and retention classifiers), `ToolResultMaxBytes`, `Output` (event handler), `preamble`, `GenerationParams`, and lifecycle callbacks: `PopInject`, `AutoCompact`, `SaveSession`, `PreTool`, `PostTool`, `TurnError`, `MaxSteps`, `ReadPlanStep`, and `IncrToolCallCount`.
### Per-Iteration Flow
```
1. Context gate: strip cold material if over budget
2. Build message history:
- If TaskState exists: system prompt + task state + hot tail only
- Otherwise: system prompt + full history
3. Stream from backend (with retry logic)
4. If no tool calls: break (turn complete)
5. Execute tool calls (serial or parallel)
6. Update state (append messages + results)
7. Infer TaskState updates from what happened
8. Safety checks (error limits, stall, replan, max steps)
9. Auto-compact if needed
10. Loop
```
### Streaming & Retry
The streaming loop handles three failure modes:
| Failure | Detection | Recovery |
|---|---|---|
| Rate limit (429) | `*RateLimitError` | Exponential backoff (5×2^attempt seconds), up to 3 retries |
| Transient (5xx, network) | `*TransientError` | Same retry logic |
| Stream drop | `Done` never received, channel closes | Retry with 2×2^attempt second delay |
| Context overflow | `*ContextOverflowError` | Trigger compaction, retry |
| Tool unsupported | `*ToolUnsupportedError` | Propagate to turnError hook |
On the first error, the `turnError` hook fires. If it exits 0 (handled), the loop returns immediately — the hook is responsible for recovery (e.g., switching models via `freeloader`).
### Tool Execution
Tool calls are dispatched with parallelism awareness:
1. **Classify** each tool call via `ClassifyTool(name)` (reads `ollie:parallel` annotation)
2. **Batch** consecutive parallel-read-safe calls into a group
3. **Fan out** the batch concurrently with deduplication (identical calls share one execution)
4. **Serial** calls run one at a time
Per-call lifecycle:
```
preTool hook → execute → postTool hook → inject check → truncation → cache (if read-safe)
```
- `preTool` exit 2 blocks execution
- `postTool` exit 2 replaces the result
- Results exceeding `ToolResultMaxBytes` are hard-truncated
- Read-safe results are cached in a `sync.Map` for the duration of the turn
### Safety Mechanisms
| Mechanism | Threshold | Action |
|---|---|---|
| Consecutive errors (soft) | 5 rounds | Inject nudge: "try a different approach" |
| Consecutive errors (hard) | 10 rounds | Abort the turn |
| Replan gate | 8 rounds without PLAN: block | Inject replan nudge |
| Stall detection | 5 rounds with same LastAction | Inject stall nudge |
| Max steps | Configurable per-agent | Soft exit with nudge |
| Plan re-injection | Every 10 tool rounds | Re-inject task state |
### TaskState Inference
After each tool round, `inferTaskStateUpdate` updates the structured overlay:
- `LastAction` ← most recent tool call + truncated result
- `PlanStep` ← next unchecked item from the plan file (or heuristic from assistant text)
## Session & Context (`pkg/agent/session.go`)
### Message History
`Session` holds a flat `[]backend.Message` slice. No rolling window — grows until compaction.
**Checkpoint**: `Session.Checkpoint(ts TaskState)` forks the session — returns a new Session that inherits the given TaskState but starts with a clean message history. Used for narrow-context sub-agents that know what to do without inheriting parent message noise.
Token tracking per session:
- `TotalInputTokens`, `TotalOutputTokens`, `TotalRequests`
- `TotalCachedInputTokens`, `TotalCacheCreationTokens`
- Per-turn accumulators reset at turn start
- `LastTurnCostUSD`, `SessionCostUSD`
### Compaction
Triggered when estimated tokens reach 50% of model context length.
**Three-zone strategy:**
| Zone | Content | Size |
|---|---|---|
| Cold | Structured TaskState JSON | 1 message |
| Warm | One-line decision index | ~10 messages |
| Hot | Verbatim recent messages | 8 messages |
Process:
1. Flatten tool call/result pairs to plain text
2. Send full history + compaction prompt to the LLM
3. Parse structured JSON response into TaskState
4. Rebuild history as cold + warm + hot zones
5. Write pre-compaction snapshot to `{id}.compaction.jsonl` for recovery
### Result Tiers
Tool results are classified at execution time:
| Tier | Behavior | Examples |
|---|---|---|
| Hot | Stays verbatim in messages | `file_write`, `file_edit`, `reasoning_think` |
| Warm | Summarized on next compaction | (custom tools) |
| Cold | Immediately collapsed to one-line summary | `file_read`, `memory_recall`, `lsp_*` |
Classification sources (in priority order):
1. Built-in table in `execute/tier.go` (`defaultColdTools`)
2. `ollie:tier` annotation in the tool script header
3. Default: hot
### Persistence
Sessions are saved as `PersistedSession` JSON:
```json
{
"id": "1716300000000-abc123",
"agent": "coding",
"messages": [...],
"taskState": {...}
}
```
Saved after every tool round (`SaveSession` callback) and on clean exit.
## Hooks (`pkg/agent/hooks.go`)
Hooks are shell commands executed with:
- **stdin**: JSON payload with context (session_id, cwd, model, error, etc.)
- **env**: `OLLIE_*` variables derived from the payload map
- **timeout**: 60 seconds (configurable in tests)
- **process group**: `Setpgid=true` for clean kill on timeout
### Exit Code Semantics
| Code | Meaning |
|---|---|
| 0 | Success. Stdout appended to context. |
| 2 | Block. Stops the action (e.g., blocks a tool call, prevents compaction). |
| Other | Non-blocking warning. Execution continues. |
### Hook Execution Order
Multiple commands per hook run sequentially. If any exits 2, execution stops immediately and returns `Blocked=true`.
## Commands (`pkg/agent/commands.go`)
`Submit()` first checks for commands before starting an agent turn:
| Prefix | Dispatch |
|---|---|
| `!<cmd>` | Shell command in session CWD |
| `/backend [name]` | Show or switch LLM backend |
| `/model [name]` | Show or switch model |
| `/models` | List available models |
| `/agent [name]` | Show or switch agent config (rebuilds Runtime) |
| `/agents` | List available agents |
| `/maxsteps [n]` | Get/set max steps (0 = unlimited) |
| `/compact` | Manual compaction |
| `/context` | Show context size and message breakdown |
| `/usage` | Show token usage and context percentage |
| `/cost` | Show last turn and session cost |
| `/history` | Dump bounded message history |
| `/sessions` | List saved sessions |
| `/clear` | Clear session (reset context) |
| `/kill` | Kill session |
| `/rn <name>` | Rename session |
| `/cwd [path]` | Show or change working directory |
| `/skills` | List available skills |
| `/tools` | List available tools |
| `/sp` | Show rendered system prompt |
| `/i <prompt>` | Inject into running turn |
| `/irw <prompt>` | Inject-rewrite |
| `/queued [pop\|clear]` | Manage queued prompts |
| `/help` | Show command help |
### Agent Switching
When `/agent <name>` is issued:
1. Load the new agent config JSON
2. Create a fresh dispatcher via `newDispatcher()`
3. Call `BuildRuntime` with the new config
4. If the config specifies a backend/model, switch those too
5. Replace the entire `Runtime` pointer on the agent struct
6. Clear the session (fresh context for the new persona)
## Tool System (`pkg/tools/`)
### Dispatcher
The `Dispatcher` is a simple name→Server router:
```go
type Dispatcher interface {
AddServer(name string, s Server)
GetServer(name string) (Server, bool)
ListTools() ([]ToolInfo, error)
Dispatch(ctx context.Context, server, tool string, args json.RawMessage) (json.RawMessage, error)
}
```
`NewDispatcherFunc` returns a factory that creates a fresh dispatcher with all servers instantiated. Called on agent switches to get clean server state.
### Optional Server Interfaces
Servers may implement additional interfaces for runtime configuration:
| Interface | Purpose |
|---|---|
| `CWDSetter` | Update working directory |
| `EnvSetter` | Inject session env vars |
| `LockDirSetter` | Set flock directory for parallel coordination |
| `ToolRestrictionSetter` | Restrict available executors/tools |
| `ParallelClassifier` | Report if a tool is read-safe |
| `TierClassifier` | Report result retention tier |
### execute.Server (`pkg/tools/execute/`)
The single built-in server exposes three executors:
#### execute_code
Runs inline code steps sequentially (or parallel via `{parallel: [...]}`).
**Input schema:**
```json
{
"steps": [{"code": "...", "language": "bash"}],
"timeout": 30,
"sandbox": "default"
}
```
**Supported languages:** bash, python3, perl, lua, awk, sed, jq, ed, expect, bc.
#### call_tool
Runs named tool scripts from `OLLIE_TOOLS_PATH`.
**Input schema:**
```json
{
"calls": [{"tool": "file_read", "args": ["/path"]}],
"timeout": 30,
"sandbox": "default"
}
```
Parallel fan-out is automatic for consecutive calls to `ollie:parallel read` tools.
#### pipe
Sequential pipeline — each stage's stdout feeds the next stage's stdin.
**Input schema:**
```json
{
"stages": [
{"tool": "file_grep", "args": ["TODO", "--path=src/"]},
{"code": "wc -l"}
],
"timeout": 30
}
```
### Code Validation
Before execution, inline code is checked against dangerous patterns:
- **Universal**: `mkfs`, `dd if=/dev/`, `sudo/su`, `/etc/shadow`
- **Bash**: recursive force-delete of system paths, fork bombs, `/dev/sd` writes, eval injection
- **Python**: `shutil.rmtree("/")`, subprocess rm -rf, os.remove of system paths
- **Perl**: system/exec rm -rf, backtick rm, unlink system paths
- **Lua**: os.execute rm -rf, os.remove system paths
Validation failures are rate-limited: 5 failures within 1 minute triggers a 5-minute block.
### Parallelism & Locking (`execute/flock.go`)
Tool scripts declare their concurrency class via header annotations:
| Annotation | Lock Class | Behavior |
|---|---|---|
| `ollie:parallel read` | Read | Shared lock (LOCK_SH); concurrent with other reads |
| `ollie:parallel write` | Write | Exclusive lock (LOCK_EX); serialized |
| (none) | Global | Serialized with everything |
Lock files live in `/tmp/ollie/{sessionID}/` and are acquired via `flock(2)`.
### Result Tier Classification (`execute/tier.go`)
Built-in cold tools (results consumed immediately, don't need verbatim retention):
- `file_read`, `memory_recall`, `web_fetch`, `web_search`
- `lsp_definition`, `lsp_references`, `lsp_hover`
Custom tools declare their tier via `ollie:tier cold|warm|hot` in the script header.
### Elevation
Steps marked `elevated: true` bypass the sandbox entirely. They are dispatched to the elevation backend (`x/elevate`) which runs outside landrun. Only bash is supported for elevated steps.
### FUSE Mount
When `WithMount()` is set, the execute server spawns `ollie-9p-mount -u {uname}` to create a per-session FUSE mount. This gives the agent its own 9P identity for permission enforcement. The mount path is exposed as `$OLLIE` in subprocess environments.
## Backend System (`pkg/backend/`)
### Shared Types
| Type | Purpose |
|---|---|
| `Message` | Conversation turn (role, content, tool_calls, tool_call_id) |
| `Tool` | Function declaration (name, description, JSON Schema params) |
| `ToolCall` | Model's request to invoke a function |
| `StreamEvent` | Incremental streaming delta (content, reasoning, tool_calls, done, usage) |
| `Usage` | Token counts + optional cost |
| `GenerationParams` | Unified sampling parameters |
### Error Types
| Error | HTTP Code | Loop Behavior |
|---|---|---|
| `RateLimitError` | 429 | Retry with backoff |
| `TransientError` | 5xx, network | Retry with backoff |
| `ContextOverflowError` | 400 (specific) | Compact and retry |
| `ToolUnsupportedError` | 400/422 (specific) | Propagate to turnError hook |
### streamRequest
Shared HTTP streaming infrastructure used by all backends:
1. Execute the HTTP request
2. Classify response status into typed errors
3. Spawn a goroutine that parses SSE/NDJSON into `StreamEvent` channel
4. Return the channel for the loop to consume
### Backend Implementations
**OllamaBackend** (`/api/chat`):
- Local inference, no API key
- Context length from `/api/show` (cached)
- Model list from `/api/tags`
- NDJSON streaming
**OpenAIBackend** (`/v1/chat/completions`):
- Works with OpenAI, OpenRouter, any compatible API
- Configurable base URL, name, extra headers
- Context length from model list endpoint (cached)
- SSE streaming (`data: {...}` lines)
- Handles both `tool_calls` and legacy `function_call`
**AnthropicBackend** (`/v1/messages`):
- Extended thinking (reasoning budget via `ThinkingBudget`)
- Prompt caching (cache_control blocks)
- 200k context window (hardcoded)
- SSE streaming with content_block_delta events
## Config System (`pkg/config/`)
### Agent Config Schema
```go
type Config struct {
Hooks map[string]HookCmds // lifecycle callbacks
Prompt Prompt // system prompt (string or command array)
Backend string // backend override
Model string // model override
Tools *bool // enable/disable tools (default: true)
AllowExecutors []string // restrict executors
AllowTools []string // restrict tool scripts
MaxTokens int // generation limit
MaxCompletionTokens int // OpenAI o-series
MaxSteps int // tool rounds per turn
Temperature *float64
TopP, TopK, MinP, TopA *float64/*int
FrequencyPenalty *float64
PresencePenalty *float64
RepetitionPenalty *float64
Reasoning int // Anthropic thinking budget
ReasoningEffort string // OpenAI: low/medium/high
IncludeReasoning *bool // OpenRouter
ResponseFormat string // json_object, text
Stop []string // stop sequences
Verbosity string // output detail level
}
```
### Prompt Type
Dual-mode unmarshaling:
- **JSON string**: treated as literal text (with `!command`, file path, or env expansion)
- **JSON array**: each element is a shell command; stdout concatenated
The array form is preferred — it enables composable prompt assembly from template fragments.
### HookCmds Type
Dual-mode: a single string `"cmd"` or an array `["cmd1", "cmd2"]`. Multiple commands run sequentially.
## Prompt Resolution (`pkg/agent/prompt_resolver.go`)
Two paths based on `Prompt.IsExec`:
**Exec mode** (array):
- Each command runs via `sh -c` with 30s timeout
- CWD set to session working directory
- Extra env vars injected (OLLIE_SESSION_ID, OLLIE mount path)
- Non-zero exit skips that command (no error propagation)
- All stdout concatenated with newlines
**String mode** (legacy):
- Environment variables expanded
- If contains newline: literal text
- If starts with `!`: execute as shell command
- If names an existing file: read file contents
- Otherwise: use as-is
## Cost Tracking (`pkg/agent/cost.go`)
When the backend doesn't report cost directly (only OpenRouter does), cost is computed from a built-in pricing table:
- Matches model name prefix (case-insensitive)
- Covers Anthropic Claude, OpenAI GPT/o-series, Google Gemini
- Cached input tokens charged at discount (10% Anthropic, 50% others)
- Cache creation tokens charged at 125% of input rate
- Unknown/local models return zero cost
## Sandbox (`internal/sandbox/`)
### Config Format
```yaml
general:
best_effort: true # don't fail if kernel doesn't support all features
log_level: "" # landrun log level
filesystem:
ro: ["/usr", "/lib", "/etc"] # read-only
rox: ["/usr/bin", "/usr/local/bin"] # read + execute
rw: ["{CWD}", "{TMPDIR}"] # read + write
rwx: [] # read + write + execute
network:
enabled: true
unrestricted: true # all TCP allowed
bind_tcp: [] # specific bind ports
connect_tcp: [] # specific connect targets
env: [] # extra environment variables
advanced:
ldd: false # resolve shared library paths
add_exec: false # add execute to all read paths
```
### Template Variables
| Variable | Expansion |
|---|---|
| `{CWD}` | Session working directory |
| `{HOME}` | User home directory |
| `{TMPDIR}` | Temp directory (default: /tmp) |
| `{XDG_CONFIG_HOME}` | XDG config (default: ~/.config) |
| `{XDG_DATA_HOME}` | XDG data (default: ~/.local/share) |
| `{XDG_CACHE_HOME}` | XDG cache (default: ~/.cache) |
| `{XDG_STATE_HOME}` | XDG state (default: ~/.local/state) |
| `{XDG_RUNTIME_DIR}` | XDG runtime (default: /run/user/UID) |
| `{OLLIE_CFG_PATH}` | ollie config directory |
| `{OLLIE_DATA_PATH}` | ollie data directory |
### Path Validation
`checkPath(cfg, path, write, cwd, getenv)` validates that a given absolute path falls within the sandbox's allowed paths. Symlinks are resolved before checking. Used by file tools to pre-validate paths before execution.
## Event System
The agent communicates exclusively through events published on a `pubsub.Bus`:
```go
type Event struct {
Role string // event type
Name string // tool name (for call/tool events)
Content string // payload
}
```
| Role | When |
|---|---|
| `assistant` | Streaming text delta from the model |
| `reasoning` | Thinking/reasoning delta (wrapped in `<think>` tags) |
| `call` | Tool call initiated (Name=tool, Content=args JSON) |
| `tool` | Tool result returned (Name=tool, Content=result) |
| `info` | Informational message (command output, status) |
| `exec` | Shell command output (`!cmd`) |
| `error` | Error message |
| `usage` | Token usage report (space-separated numbers) |
| `limitretry` | Rate limit retry initiated |
Consumers subscribe to the `"event"` topic on the bus. The 9P server's session manager subscribes to build the chat log; frontends subscribe to render output.
## Concurrency Model
- **One turn at a time**: `submitMu` serializes `Submit()` calls
- **Interrupt**: `currentAction` holds a `CancelCauseFunc`; `Interrupt()` calls it
- **Queue drain**: after each turn completes, queued prompts are drained sequentially
- **State observation**: `WaitChange()` uses a condition variable (`changeCond`) that broadcasts on any state/usage/cwd change
- **Tool parallelism**: within a turn, read-safe tools fan out via goroutines with `sync.Map` caching and `flock` coordination
## Session ID Format
```
<unix-nanoseconds>-<6-hex-chars>
```
Lexicographically sortable by creation time. Unique even within the same nanosecond due to the random suffix.