ollie/doc/CORE.md

26 KiB
Raw Blame History

ollie-core: Detailed Architecture

This document covers the internal design of the core/ library — the Go package that implements the agent engine. It is the dependency imported by all consumers (olliesrv, frontends, tests).

Package Map

core/
├── pkg/
│   ├── agent/           The agent engine
│   │   ├── types.go       Core interface, Event, PromptFIFO
│   │   ├── runtime.go     Runtime struct (swappable per-agent config)
│   │   ├── core.go        AgentCoreConfig, agent struct, NewAgentCore, BuildRuntime
│   │   ├── loop.go        The agent loop (run function)
│   │   ├── session.go     Session, TaskState, compaction, persistence
│   │   ├── state.go       state interface, ResultTier, toolResult
│   │   ├── hooks.go       Hooks map, HookResult, hook execution
│   │   ├── commands.go    Slash-command dispatch (/model, /agent, /compact, ...)
│   │   ├── cost.go        Model pricing table, cost computation
│   │   └── prompt_resolver.go  System prompt assembly from shell commands
│   ├── backend/         LLM provider implementations
│   │   ├── backend.go     Backend interface, shared types, streamRequest
│   │   ├── openai.go      OpenAI/OpenRouter/compatible APIs
│   │   ├── anthropic.go   Anthropic Messages API
│   │   └── ollama.go      Ollama local inference
│   ├── config/          Agent config JSON schema and loader
│   ├── env/             Environment variable loading
│   ├── log/             Structured logger
│   ├── paths/           XDG directory resolution
│   └── tools/           Tool system
│       ├── tools.go       Server, Dispatcher, classifier interfaces
│       └── execute/       The execute.Server
│           ├── server.go  Server struct, Decl factory, ListTools, options
│           ├── code.go    Step resolution, validation, dispatch routing
│           ├── tool.go    Named tool script discovery and execution
│           ├── pipe.go    Pipeline stage execution, parallel fan-out
│           ├── flock.go   Advisory locking, parallelism classification
│           └── tier.go    Result tier classification (hot/warm/cold)
└── internal/
    └── sandbox/         Landlock sandbox wrapper
        ├── config.go    YAML config, path expansion, access checking
        └── wrapper.go   landrun command construction

The Core Interface (pkg/agent/types.go)

Core is the boundary between the engine and any consumer. It is deliberately narrow:

type Core interface {
    // Turn lifecycle
    Submit(ctx context.Context, input string)  // process user input (commands or prompts)
    Interrupt(cause error) bool                // cancel current turn
    Inject(prompt string)                      // append to next tool result
    Queue(prompt string)                       // FIFO for deferred prompts
    PopQueue() (string, bool)                  // drain one queued prompt

    // Observation
    IsRunning() bool
    State() string                             // "idle" | "thinking" | "calling: <tool>"
    Reply() string                             // last assistant text
    Bus() *pubsub.Bus                          // event stream
    WaitChange(ctx context.Context, field, current string) (string, bool)

    // Configuration
    AgentName() string
    BackendName() string
    ModelName() string
    CWD() string
    SetCWD(dir string) error
    SetSessionID(newID string) error
    SetEnv(key, value string)
    SetGenerationParams(params backend.GenerationParams) error

    // Introspection
    Context() []backend.Message
    SystemPrompt() string
    GenerationParams() backend.GenerationParams
    CtxSz() string
    Usage() string
    Cost() string
    ListModels() string
    ToolCallCount() int64

    // Cleanup
    Close()
}

All output flows through the event bus as Event{Role, Name, Content} values. Roles include: assistant, tool, call, info, exec, error, usage, reasoning, limitretry.

Agent Construction (pkg/agent/core.go)

AgentCoreConfig

The full construction config:

Field Purpose
Backend Active LLM backend instance
ModelName Override backend's default model
AgentName Name of the active agent config
AgentsDir Path to agent JSON directory
SessionsDir Path to persisted session files
SessionID Unique session identifier
Uname Immutable 9P user principal
CWD Working directory for tools and prompt
Session Pre-existing session to restore (nil = new)
Runtime Pre-built *Runtime (tools, hooks, prompt, params)
NewDispatcher Factory for fresh Dispatcher on agent switch
NewBackend Factory for backend by name
Log Logger instance
MaxSteps Override agent config's maxSteps

Runtime

The Runtime struct holds all per-agent configuration that changes on an /agent switch but remains stable across turns. The agent struct stores a pointer to the active Runtime; switching agents replaces it atomically.

type Runtime struct {
    Backend      backend.Backend
    Dispatcher   tools.Dispatcher
    Hooks        Hooks
    Preamble     string              // compiled system prompt
    Tools        []backend.Tool
    Exec         toolExecutor
    ClassifyTool toolClassifier
    ClassifyTier func(string) ResultTier
    GenParams    backend.GenerationParams
    MaxSteps           int
    ToolResultMaxBytes int           // 0 means use default (128KB)
    CfgBackend   string              // backend override from agent JSON
    CfgModel     string              // model override from agent JSON
    Messages     []string            // non-fatal warnings from construction
}

BuildRuntime

BuildRuntime(cfg, dispatcher, cwd, env) wires everything together:

  1. Lists all tools from the dispatcher
  2. Resolves the system prompt by executing the config's prompt commands
  3. Extracts generation parameters from the config
  4. Builds the tool executor closure (routes calls through the dispatcher)
  5. Extracts parallelism and tier classifiers from the execute server
  6. Applies tool/executor restrictions if configured
  7. Returns a *Runtime struct ready for NewAgentCore

NewAgentCore

Creates the agent struct from AgentCoreConfig:

  1. Sweeps stale tmpdirs from previous crashes
  2. Stores the provided Runtime (or creates an empty one)
  3. Sets the backend model if ModelName is specified
  4. Creates /tmp/ollie/{sessionID} for this session
  5. Sets up the plan file reader (reads s/{id}/plan for the next unchecked step)
  6. Wires the turnError hook with a detached context
  7. Pushes OLLIE_SESSION_ID and OLLIE_UNAME into the execute server env
  8. Sets the flock directory for parallel coordination

The Agent Loop (pkg/agent/loop.go)

The run() function is the core execution engine. It takes an agentConfig and a state interface, and loops until the model stops calling tools.

The agentConfig struct bundles per-turn dependencies: Backend, Tools, Exec (tool executor closure), ClassifyTool/ClassifyTier (parallelism and retention classifiers), ToolResultMaxBytes, Output (event handler), preamble, GenerationParams, and lifecycle callbacks: PopInject, AutoCompact, SaveSession, PreTool, PostTool, TurnError, MaxSteps, ReadPlanStep, and IncrToolCallCount.

Per-Iteration Flow

1. Context gate: strip cold material if over budget
2. Build message history:
   - If TaskState exists: system prompt + task state + hot tail only
   - Otherwise: system prompt + full history
3. Stream from backend (with retry logic)
4. If no tool calls: break (turn complete)
5. Execute tool calls (serial or parallel)
6. Update state (append messages + results)
7. Infer TaskState updates from what happened
8. Safety checks (error limits, stall, replan, max steps)
9. Auto-compact if needed
10. Loop

Streaming & Retry

The streaming loop handles three failure modes:

Failure Detection Recovery
Rate limit (429) *RateLimitError Exponential backoff (5×2^attempt seconds), up to 3 retries
Transient (5xx, network) *TransientError Same retry logic
Stream drop Done never received, channel closes Retry with 2×2^attempt second delay
Context overflow *ContextOverflowError Trigger compaction, retry
Tool unsupported *ToolUnsupportedError Propagate to turnError hook

On the first error, the turnError hook fires. If it exits 0 (handled), the loop returns immediately — the hook is responsible for recovery (e.g., switching models via freeloader).

Tool Execution

Tool calls are dispatched with parallelism awareness:

  1. Classify each tool call via ClassifyTool(name) (reads ollie:parallel annotation)
  2. Batch consecutive parallel-read-safe calls into a group
  3. Fan out the batch concurrently with deduplication (identical calls share one execution)
  4. Serial calls run one at a time

Per-call lifecycle:

preTool hook → execute → postTool hook → inject check → truncation → cache (if read-safe)
  • preTool exit 2 blocks execution
  • postTool exit 2 replaces the result
  • Results exceeding ToolResultMaxBytes are hard-truncated
  • Read-safe results are cached in a sync.Map for the duration of the turn

Safety Mechanisms

Mechanism Threshold Action
Consecutive errors (soft) 5 rounds Inject nudge: "try a different approach"
Consecutive errors (hard) 10 rounds Abort the turn
Replan gate 8 rounds without PLAN: block Inject replan nudge
Stall detection 5 rounds with same LastAction Inject stall nudge
Max steps Configurable per-agent Soft exit with nudge
Plan re-injection Every 10 tool rounds Re-inject task state

TaskState Inference

After each tool round, inferTaskStateUpdate updates the structured overlay:

  • LastAction ← most recent tool call + truncated result
  • PlanStep ← next unchecked item from the plan file (or heuristic from assistant text)

Session & Context (pkg/agent/session.go)

Message History

Session holds a flat []backend.Message slice. No rolling window — grows until compaction.

Checkpoint: Session.Checkpoint(ts TaskState) forks the session — returns a new Session that inherits the given TaskState but starts with a clean message history. Used for narrow-context sub-agents that know what to do without inheriting parent message noise.

Token tracking per session:

  • TotalInputTokens, TotalOutputTokens, TotalRequests
  • TotalCachedInputTokens, TotalCacheCreationTokens
  • Per-turn accumulators reset at turn start
  • LastTurnCostUSD, SessionCostUSD

Compaction

Triggered when estimated tokens reach 50% of model context length.

Three-zone strategy:

Zone Content Size
Cold Structured TaskState JSON 1 message
Warm One-line decision index ~10 messages
Hot Verbatim recent messages 8 messages

Process:

  1. Flatten tool call/result pairs to plain text
  2. Send full history + compaction prompt to the LLM
  3. Parse structured JSON response into TaskState
  4. Rebuild history as cold + warm + hot zones
  5. Write pre-compaction snapshot to {id}.compaction.jsonl for recovery

Result Tiers

Tool results are classified at execution time:

Tier Behavior Examples
Hot Stays verbatim in messages file_write, file_edit, reasoning_think
Warm Summarized on next compaction (custom tools)
Cold Immediately collapsed to one-line summary file_read, memory_recall, lsp_*

Classification sources (in priority order):

  1. Built-in table in execute/tier.go (defaultColdTools)
  2. ollie:tier annotation in the tool script header
  3. Default: hot

Persistence

Sessions are saved as PersistedSession JSON:

{
  "id": "1716300000000-abc123",
  "agent": "coding",
  "messages": [...],
  "taskState": {...}
}

Saved after every tool round (SaveSession callback) and on clean exit.

Hooks (pkg/agent/hooks.go)

Hooks are shell commands executed with:

  • stdin: JSON payload with context (session_id, cwd, model, error, etc.)
  • env: OLLIE_* variables derived from the payload map
  • timeout: 60 seconds (configurable in tests)
  • process group: Setpgid=true for clean kill on timeout

Exit Code Semantics

Code Meaning
0 Success. Stdout appended to context.
2 Block. Stops the action (e.g., blocks a tool call, prevents compaction).
Other Non-blocking warning. Execution continues.

Hook Execution Order

Multiple commands per hook run sequentially. If any exits 2, execution stops immediately and returns Blocked=true.

Commands (pkg/agent/commands.go)

Submit() first checks for commands before starting an agent turn:

Prefix Dispatch
!<cmd> Shell command in session CWD
/backend [name] Show or switch LLM backend
/model [name] Show or switch model
/models List available models
/agent [name] Show or switch agent config (rebuilds Runtime)
/agents List available agents
/maxsteps [n] Get/set max steps (0 = unlimited)
/compact Manual compaction
/context Show context size and message breakdown
/usage Show token usage and context percentage
/cost Show last turn and session cost
/history Dump bounded message history
/sessions List saved sessions
/clear Clear session (reset context)
/kill Kill session
/rn <name> Rename session
/cwd [path] Show or change working directory
/skills List available skills
/tools List available tools
/sp Show rendered system prompt
/i <prompt> Inject into running turn
/irw <prompt> Inject-rewrite
/queued [pop|clear] Manage queued prompts
/help Show command help

Agent Switching

When /agent <name> is issued:

  1. Load the new agent config JSON
  2. Create a fresh dispatcher via newDispatcher()
  3. Call BuildRuntime with the new config
  4. If the config specifies a backend/model, switch those too
  5. Replace the entire Runtime pointer on the agent struct
  6. Clear the session (fresh context for the new persona)

Tool System (pkg/tools/)

Dispatcher

The Dispatcher is a simple name→Server router:

type Dispatcher interface {
    AddServer(name string, s Server)
    GetServer(name string) (Server, bool)
    ListTools() ([]ToolInfo, error)
    Dispatch(ctx context.Context, server, tool string, args json.RawMessage) (json.RawMessage, error)
}

NewDispatcherFunc returns a factory that creates a fresh dispatcher with all servers instantiated. Called on agent switches to get clean server state.

Optional Server Interfaces

Servers may implement additional interfaces for runtime configuration:

Interface Purpose
CWDSetter Update working directory
EnvSetter Inject session env vars
LockDirSetter Set flock directory for parallel coordination
ToolRestrictionSetter Restrict available executors/tools
ParallelClassifier Report if a tool is read-safe
TierClassifier Report result retention tier

execute.Server (pkg/tools/execute/)

The single built-in server exposes three executors:

execute_code

Runs inline code steps sequentially (or parallel via {parallel: [...]}).

Input schema:

{
  "steps": [{"code": "...", "language": "bash"}],
  "timeout": 30,
  "sandbox": "default"
}

Supported languages: bash, python3, perl, lua, awk, sed, jq, ed, expect, bc.

call_tool

Runs named tool scripts from OLLIE_TOOLS_PATH.

Input schema:

{
  "calls": [{"tool": "file_read", "args": ["/path"]}],
  "timeout": 30,
  "sandbox": "default"
}

Parallel fan-out is automatic for consecutive calls to ollie:parallel read tools.

pipe

Sequential pipeline — each stage's stdout feeds the next stage's stdin.

Input schema:

{
  "stages": [
    {"tool": "file_grep", "args": ["TODO", "--path=src/"]},
    {"code": "wc -l"}
  ],
  "timeout": 30
}

Code Validation

Before execution, inline code is checked against dangerous patterns:

  • Universal: mkfs, dd if=/dev/, sudo/su, /etc/shadow
  • Bash: recursive force-delete of system paths, fork bombs, /dev/sd writes, eval injection
  • Python: shutil.rmtree("/"), subprocess rm -rf, os.remove of system paths
  • Perl: system/exec rm -rf, backtick rm, unlink system paths
  • Lua: os.execute rm -rf, os.remove system paths

Validation failures are rate-limited: 5 failures within 1 minute triggers a 5-minute block.

Parallelism & Locking (execute/flock.go)

Tool scripts declare their concurrency class via header annotations:

Annotation Lock Class Behavior
ollie:parallel read Read Shared lock (LOCK_SH); concurrent with other reads
ollie:parallel write Write Exclusive lock (LOCK_EX); serialized
(none) Global Serialized with everything

Lock files live in /tmp/ollie/{sessionID}/ and are acquired via flock(2).

Result Tier Classification (execute/tier.go)

Built-in cold tools (results consumed immediately, don't need verbatim retention):

  • file_read, memory_recall, web_fetch, web_search
  • lsp_definition, lsp_references, lsp_hover

Custom tools declare their tier via ollie:tier cold|warm|hot in the script header.

Elevation

Steps marked elevated: true bypass the sandbox entirely. They are dispatched to the elevation backend (x/elevate) which runs outside landrun. Only bash is supported for elevated steps.

FUSE Mount

When WithMount() is set, the execute server spawns ollie-9p-mount -u {uname} to create a per-session FUSE mount. This gives the agent its own 9P identity for permission enforcement. The mount path is exposed as $OLLIE in subprocess environments.

Backend System (pkg/backend/)

Shared Types

Type Purpose
Message Conversation turn (role, content, tool_calls, tool_call_id)
Tool Function declaration (name, description, JSON Schema params)
ToolCall Model's request to invoke a function
StreamEvent Incremental streaming delta (content, reasoning, tool_calls, done, usage)
Usage Token counts + optional cost
GenerationParams Unified sampling parameters

Error Types

Error HTTP Code Loop Behavior
RateLimitError 429 Retry with backoff
TransientError 5xx, network Retry with backoff
ContextOverflowError 400 (specific) Compact and retry
ToolUnsupportedError 400/422 (specific) Propagate to turnError hook

streamRequest

Shared HTTP streaming infrastructure used by all backends:

  1. Execute the HTTP request
  2. Classify response status into typed errors
  3. Spawn a goroutine that parses SSE/NDJSON into StreamEvent channel
  4. Return the channel for the loop to consume

Backend Implementations

OllamaBackend (/api/chat):

  • Local inference, no API key
  • Context length from /api/show (cached)
  • Model list from /api/tags
  • NDJSON streaming

OpenAIBackend (/v1/chat/completions):

  • Works with OpenAI, OpenRouter, any compatible API
  • Configurable base URL, name, extra headers
  • Context length from model list endpoint (cached)
  • SSE streaming (data: {...} lines)
  • Handles both tool_calls and legacy function_call

AnthropicBackend (/v1/messages):

  • Extended thinking (reasoning budget via ThinkingBudget)
  • Prompt caching (cache_control blocks)
  • 200k context window (hardcoded)
  • SSE streaming with content_block_delta events

Config System (pkg/config/)

Agent Config Schema

type Config struct {
    Hooks               map[string]HookCmds  // lifecycle callbacks
    Prompt              Prompt               // system prompt (string or command array)
    Backend             string               // backend override
    Model               string               // model override
    Tools               *bool                // enable/disable tools (default: true)
    AllowExecutors      []string             // restrict executors
    AllowTools          []string             // restrict tool scripts
    MaxTokens           int                  // generation limit
    MaxCompletionTokens int                  // OpenAI o-series
    MaxSteps            int                  // tool rounds per turn
    Temperature         *float64
    TopP, TopK, MinP, TopA  *float64/*int
    FrequencyPenalty    *float64
    PresencePenalty     *float64
    RepetitionPenalty   *float64
    Reasoning           int                  // Anthropic thinking budget
    ReasoningEffort     string               // OpenAI: low/medium/high
    IncludeReasoning    *bool                // OpenRouter
    ResponseFormat      string               // json_object, text
    Stop                []string             // stop sequences
    Verbosity           string               // output detail level
}

Prompt Type

Dual-mode unmarshaling:

  • JSON string: treated as literal text (with !command, file path, or env expansion)
  • JSON array: each element is a shell command; stdout concatenated

The array form is preferred — it enables composable prompt assembly from template fragments.

HookCmds Type

Dual-mode: a single string "cmd" or an array ["cmd1", "cmd2"]. Multiple commands run sequentially.

Prompt Resolution (pkg/agent/prompt_resolver.go)

Two paths based on Prompt.IsExec:

Exec mode (array):

  • Each command runs via sh -c with 30s timeout
  • CWD set to session working directory
  • Extra env vars injected (OLLIE_SESSION_ID, OLLIE mount path)
  • Non-zero exit skips that command (no error propagation)
  • All stdout concatenated with newlines

String mode (legacy):

  • Environment variables expanded
  • If contains newline: literal text
  • If starts with !: execute as shell command
  • If names an existing file: read file contents
  • Otherwise: use as-is

Cost Tracking (pkg/agent/cost.go)

When the backend doesn't report cost directly (only OpenRouter does), cost is computed from a built-in pricing table:

  • Matches model name prefix (case-insensitive)
  • Covers Anthropic Claude, OpenAI GPT/o-series, Google Gemini
  • Cached input tokens charged at discount (10% Anthropic, 50% others)
  • Cache creation tokens charged at 125% of input rate
  • Unknown/local models return zero cost

Sandbox (internal/sandbox/)

Config Format

general:
  best_effort: true       # don't fail if kernel doesn't support all features
  log_level: ""           # landrun log level

filesystem:
  ro:  ["/usr", "/lib", "/etc"]           # read-only
  rox: ["/usr/bin", "/usr/local/bin"]     # read + execute
  rw:  ["{CWD}", "{TMPDIR}"]             # read + write
  rwx: []                                 # read + write + execute

network:
  enabled: true
  unrestricted: true      # all TCP allowed
  bind_tcp: []            # specific bind ports
  connect_tcp: []         # specific connect targets

env: []                   # extra environment variables

advanced:
  ldd: false              # resolve shared library paths
  add_exec: false         # add execute to all read paths

Template Variables

Variable Expansion
{CWD} Session working directory
{HOME} User home directory
{TMPDIR} Temp directory (default: /tmp)
{XDG_CONFIG_HOME} XDG config (default: ~/.config)
{XDG_DATA_HOME} XDG data (default: ~/.local/share)
{XDG_CACHE_HOME} XDG cache (default: ~/.cache)
{XDG_STATE_HOME} XDG state (default: ~/.local/state)
{XDG_RUNTIME_DIR} XDG runtime (default: /run/user/UID)
{OLLIE_CFG_PATH} ollie config directory
{OLLIE_DATA_PATH} ollie data directory

Path Validation

checkPath(cfg, path, write, cwd, getenv) validates that a given absolute path falls within the sandbox's allowed paths. Symlinks are resolved before checking. Used by file tools to pre-validate paths before execution.

Event System

The agent communicates exclusively through events published on a pubsub.Bus:

type Event struct {
    Role    string  // event type
    Name    string  // tool name (for call/tool events)
    Content string  // payload
}
Role When
assistant Streaming text delta from the model
reasoning Thinking/reasoning delta (wrapped in <think> tags)
call Tool call initiated (Name=tool, Content=args JSON)
tool Tool result returned (Name=tool, Content=result)
info Informational message (command output, status)
exec Shell command output (!cmd)
error Error message
usage Token usage report (space-separated numbers)
limitretry Rate limit retry initiated

Consumers subscribe to the "event" topic on the bus. The 9P server's session manager subscribes to build the chat log; frontends subscribe to render output.

Concurrency Model

  • One turn at a time: submitMu serializes Submit() calls
  • Interrupt: currentAction holds a CancelCauseFunc; Interrupt() calls it
  • Queue drain: after each turn completes, queued prompts are drained sequentially
  • State observation: WaitChange() uses a condition variable (changeCond) that broadcasts on any state/usage/cwd change
  • Tool parallelism: within a turn, read-safe tools fan out via goroutines with sync.Map caching and flock coordination

Session ID Format

<unix-nanoseconds>-<6-hex-chars>

Lexicographically sortable by creation time. Unique even within the same nanosecond due to the random suffix.