22 KiB

Raw Blame History

AGENTS.md

Project-level context for AI agents working in this repository.

⚠️ IMPORTANT: All work must be done in THIS source directory (~/src/ollie/). Never edit files under ~/.config/ollie/ — that is an install target. Changes made there are overwritten on the next make install-data. Edit source files here, then run make to build and install.

Project Overview

Ollie is an AI agent runtime inspired by Plan 9: agent state and behaviors are exposed as files in a 9P namespace. Orchestration, scheduling, and UIs are external — shell scripts, editors, web apps. The core is minimal; capabilities come from composing scripts.

Repository Layout

Single Go module. KDE integration is part of the repository under kde/.

ollie/                              ← you are here
├── cmd/
│   ├── olliesrv/                   9P server: sessions, agents, backends
│   │   └── internal/
│   │       ├── agent/              Agent loop, history, compaction, prompts
│   │       ├── backend/            LLM provider implementations
│   │       ├── bypass/             Sandbox bypass broker
│   │       ├── fs/                 9P namespace and handlers
│   │       ├── prompts/            System-prompt resolution
│   │       ├── session/            Session lifecycle and persistence
│   │       └── toolclient/         Local/remote toolsrv process management
│   ├── toolsrv/                    Sandboxed tool execution server
│   │   └── internal/
│   │       ├── exec/               Tool execution
│   │       ├── registry/            Per-agent tool registry
│   │       ├── sandbox/             Landlock configuration and enforcement
│   │       └── server/              Namespace specification and process state
│   └── ollie-9p/                   9P client CLI
├── tools/                          Compiled tool implementations
│   ├── codeintel/                  Tree-sitter code-intelligence tools
│   ├── filetools/                  Go file tools
│   ├── lsp/                        LSP bridge and client tools
│   └── web/                        Web-fetch tool
├── toolsrv/                        toolsrv 9P client library and registry types
├── virtfs/                         Virtual filesystem declaration EDSL
├── lib9p/                          9P protocol library and native client
├── env/, format/, log/, paths/     Shared Go packages
├── kde/                            KDE GUI, Kate, KRunner, and KIO integration
├── contrib/elisp/                  Emacs frontend (`ellie.el`)
├── data/agents/                    Agent configuration JSON
├── data/prompts/                   Prompt templates
├── data/tools/                     Script tools and `.meta` files
├── data/skills/                    Domain knowledge modules
├── data/scripts/                   CLI and integration scripts
├── data/services/                  User service files
├── cmd/toolsrv/internal/sandbox/   Installed sandbox configuration source
├── doc/                            Architecture and usage documentation
└── experiments/                    Experimental code

Canonical source for prompts, tools, and skills

Do not edit ~/.config/ollie/ directly. It is an install target. Runtime data is copied from data/ by make install-data; compiled tools are built into the same runtime tools directory by their build targets. The installed configuration also contains backends.conf, sandbox.yaml, agents, prompts, skills, scripts, and tools.

Build System

The default make target builds, tests, and installs. Build and install are separate phases.

make                 # build + test + install
make build           # build core, 9P, client, tools, and KDE
make core            # go build ./...
make ninep           # olliesrv and ollie-9p
make client          # native lib9p shared library and header
make tools           # compiled tools (code-intel, file, LSP, web)
make kde             # KDE KF6 integration
make install-data    # runtime configuration, prompts, skills, scripts, and tools
make test            # core and lib9p tests
make test-core       # cmd/olliesrv, cmd/toolsrv, shared packages
make test-9p         # lib9p tests
make clean           # remove build artifacts

Requires GNU Make.

Testing

The supported test entry points are:

make test
make test-core
make test-9p

For direct Go testing, use the packages covered by make test-core and make test-9p.

Language & Conventions

Go (root module): Go 1.25+, standard library preferred, minimal dependencies.
C++20/Qt6/KF6 (kde): CMake build.
Elisp (el): single file ellie.el.
Tool scripts: Python 3, Bash, or compiled binaries. Must be executable. Metadata lives in a .meta sidecar JSON file (see data/tools/*.meta).

Code style

Go: gofmt, short variable names, error returns (no panics), table-driven tests.
Tool scripts: emit structured output (STATUS=ok, STATUS=error). Image/LSP tools return JSON content blocks.
Prompts: markdown, concise, example-driven. Follow the pattern in existing tools-*.md files.

Stability & compatibility

Ollie is experimental and unstable software. The features and API are approaching stability but are not there yet. Optimize for a clean, minimal codebase over preserving existing behavior.

Do not add backward-compatibility code. No shims, deprecation aliases, legacy fallbacks, dual-format parsers, or "keep the old path working" branches. When something changes, change it fully and delete the old form.
Prefer subtraction. Removing code is a feature. If a rename, refactor, or new design lets you delete the old thing, delete it — don't leave both.
Break callers freely. Renaming a field, changing a wire format, or altering a ctl verb is fine; update all call sites in the same change. There are no external consumers to protect.
Extreme minimalism. No pointless indirection, no defensive code for cases that can't happen, no configuration knobs "just in case."

This is a standing preference, not a per-task instruction. Apply it without asking.

Architecture (key concepts)

One integration surface: olliesrv exposes sessions and agents through a 9P2000 filesystem. Frontends include o, ollie-9p, and KDE. The event file provides real-time streaming with optional filtering via the streaming rdwr pattern.
Session and tool processes: Each session owns an agent runtime in olliesrv and a separate toolsrv process. They communicate over an authenticated Unix socket using 9P. toolclient can respawn local toolsrv processes and can deploy/start toolsrv remotely over SSH with socket forwarding.
Agent loop (cmd/olliesrv/internal/agent/): The agent package is organized by concern:
- loop.go: Main loop — stream LLM, execute tools, update history
- turn.go: Submit entry point, turn orchestration
- dispatch.go: Tool execution, batching, conflict detection
- history.go: Message history, usage tracking
- compact.go: Context compaction, cold summarization
- cache.go: Tool result caching with file staleness detection
- retry.go: Error tracking, transient retry logic
- state.go, chat.go, peer.go, subagent.go: Agent state and coordination
Dynamic tools: Tools are external executables described by .meta files. toolsrv owns discovery and per-agent registries. olliesrv refreshes the registry, injects common dispatch flags (bypass, timeout, sandbox, background), and calls tools through toolsrv.Conn.
Parallel and background dispatch: Non-conflicting tool calls in one turn run concurrently. Metadata scopes schedule read, write, and global operations; shell-like global operations serialize. Any tool may run in the background, producing a process ID whose output is injected when the process changes or exits.
Sandbox (cmd/toolsrv/internal/sandbox/): Native Landlock policies control filesystem access in a short-lived child helper. The installed sandbox.yaml is sourced from cmd/toolsrv/internal/sandbox/sandbox.yaml. The toolsrv namespace and process state live under cmd/toolsrv/p9.go and cmd/toolsrv/internal/server/. The bypass broker provides policy-controlled escape requests, approval, persistence, and rate limiting.
Backends (cmd/olliesrv/internal/backend/): Supported names are ollama, openai, openrouter, anthropic, copilot, kiro, and gemini. Configuration is read from ~/.config/ollie/backends.conf; environment variables are fallback inputs.
Prompt assembly: cmd/olliesrv/internal/prompts/system_prompt.md is embedded as the default system prompt. An agent's systemPrompt can override it with a filesystem path. prompt and userPrompts entries resolve files, expand environment variables, and support legacy !command entries. The runtime combines system, environment, agent, and tool sections.
Context management: History tracks messages, usage, costs, cache statistics, and structured task state. Cold/warm/hot result tiers and automatic compaction preserve recent context while summarizing older material.
Sub-agents: subagent_spawn creates a transient child session with an independent runtime and context. The child receives a one-time parent-history snapshot and returns only its final reply. Parent/child IDs are retained for tracing; concurrent children are supported.
Peers: Persistent agents in the same session can be linked via peeradd. Links are bidirectional. Agents communicate by writing to peer/{name}, which delivers to the target's prompt handler. Only declared peers can be messaged — the peer/ directory is the access control surface.
9P namespace declaration: cmd/olliesrv/internal/fs/spec.go declares the olliesrv namespace. The toolsrv namespace is declared by cmd/toolsrv/p9.go using cmd/toolsrv/internal/server.Spec; process state and handlers are in cmd/toolsrv/internal/server/. Both use the virtfs EDSL and virtfs.BuildTree().
Working directory (default + override): Session cwd is required at session creation and is the inheritance root for every agent — the sane default so many agents can work in one directory with zero per-agent config. Agent cwd is an optional per-agent override (empty = inherit the session cwd). Because the per-session toolsrv sets cmd.Dir per tool call, per-agent cwd needs no extra process: toolsrv keeps an in-memory agentCWD map resolved per call from the agent= field (cmd/toolsrv/internal/server/proc.go), falling back to the session-level global cwd. The override is set only over the controlled olliesrv→toolsrv ctl channel (agentcwd <id> <dir>), never embedded in a tool-call payload, so the model cannot influence where its own tools run. The map is process-local, so Agent.SyncCwdToToolServer re-pushes the override on every (re)connect — profile switch, resume, restore — matching how env and tools are resynced. Set/clear via the agent cfg (cwd=...) or ctl (cwd [<dir>|-]); the GUI Agent Settings dialog shows (inherit: <sessionCwd>) as the placeholder.
Chat log files: Each agent exposes views of its conversation history via the 9P namespace:
- chat.raw — authoritative live JSONL stream. On open it replays the finalized history (one line per block), then streams live deltas: the current in-flight partial block whenever it changes, and finalized blocks as they are appended. GUI clients consume this and must not poll log.raw. Partials are delivered out-of-band (never persisted) and collapse by block ID on the client, so reconnects stay O(history), not O(streaming chunks).
- log.raw — JSONL snapshot (one JSON object per line) of finalized blocks only. One-shot read: returns the full history and EOFs. Used for explicit history loads (initial populate, bookmark reload), never polled for live updates. Each line is {"role":"...","id":"...","content":"..."} with optional name and format fields.
- log — Rendered text snapshot (last 64KB). Non-blocking; suitable for one-shot inspection. Filters out reasoning, call, and tool blocks; shows only user/assistant/context content.
- chat — Rendered text stream. Blocking; suitable for TUI live tailing. Same filtering as log.
- block — Rdwr lookup: write a block ID (8-char hex), read the matching block as JSON. Returns the full block object or an error if not found. Block IDs are 8-char hex strings generated deterministically from sha256(sessionID + agentID + counter).

Where to Start

Entry points for understanding different parts of the codebase:

Agent execution

cmd/olliesrv/internal/agent/loop.go — Main loop: stream LLM, execute tools, update history
cmd/olliesrv/internal/agent/turn.go — Turn orchestration; Submit is the entry point
cmd/olliesrv/internal/agent/dispatch.go — Tool batching and conflict detection

9P namespace

cmd/olliesrv/internal/fs/spec.go — All handlers inline, no chasing
cmd/toolsrv/p9.go — toolsrv namespace; internal/server/proc.go owns process state

Backend integration

cmd/olliesrv/internal/backend/backend.go — Interface and shared types
Pick a concrete backend (e.g., anthropic.go, openai.go) to see implementation

Context management

cmd/olliesrv/internal/agent/history.go — Message storage, token tracking
cmd/olliesrv/internal/agent/compact.go — Compaction logic, cold summarization

Tool discovery

cmd/olliesrv/internal/agent/tool_match.go — Semantic matching
cmd/olliesrv/internal/agent/runtime.go — Preamble and tool schema assembly

Sandbox and execution

cmd/toolsrv/internal/sandbox/ — Landlock policy configuration
cmd/toolsrv/internal/exec/exec.go — Tool execution wrapper
cmd/toolsrv/internal/server/proc.go — Process lifecycle

Key Files

What	Where
9P namespace (olliesrv)	`cmd/olliesrv/internal/fs/spec.go`
9P namespace (toolsrv)	`cmd/toolsrv/p9.go`, `cmd/toolsrv/internal/server/server.go`
virtfs EDSL	`virtfs/decl.go`, `virtfs/builder.go`
Agent core and identity	`cmd/olliesrv/internal/agent/agent.go`
Agent loop	`cmd/olliesrv/internal/agent/loop.go`
Turn orchestration	`cmd/olliesrv/internal/agent/turn.go`
Tool dispatch and batching	`cmd/olliesrv/internal/agent/dispatch.go`
Message history	`cmd/olliesrv/internal/agent/history.go`
Context compaction	`cmd/olliesrv/internal/agent/compact.go`
Tool result caching	`cmd/olliesrv/internal/agent/cache.go`
Error retry logic	`cmd/olliesrv/internal/agent/retry.go`
Runtime and prompt assembly	`cmd/olliesrv/internal/agent/runtime.go`, `prompt_resolver.go`
Text-based tool parsing	`cmd/olliesrv/internal/agent/text_parse.go`
Semantic tool/skill matching	`cmd/olliesrv/internal/agent/tool_match.go`, `skill_match.go`
Chat output formatting	`cmd/olliesrv/internal/agent/chatlog.go`
Local summarization	`cmd/olliesrv/internal/agent/local_summary.go`
Workflow classification	`cmd/olliesrv/internal/agent/workflow.go`
Tool server binary	`cmd/toolsrv/`
Tool server client	`cmd/olliesrv/internal/toolclient/`
Process lifecycle	`cmd/toolsrv/internal/server/proc.go`
Sandbox enforcement	`cmd/toolsrv/internal/sandbox/`
Bypass broker	`cmd/olliesrv/internal/bypass/`
Session management	`cmd/olliesrv/internal/session/`
Embedded system prompt	`cmd/olliesrv/internal/prompts/system_prompt.md`
Agent configs	`data/agents/*.json`
Backend configuration	`data/backends.conf`, `~/.config/ollie/backends.conf`
Compiled tools	`tools/codeintel/`, `tools/filetools/`, `tools/lsp/`, `tools/web/`
KDE GUI	`kde/gui/`
Kate plugin	`kde/kate/`

Environment

Runtime configuration is read from ~/.config/ollie/ (or $XDG_CONFIG_HOME/ollie/). The backend configuration file is backends.conf, not env.

backend = ... in backends.conf selects the default backend; a session backend=... can override it.
OLLIE_BACKEND is a fallback when no configured backend or session backend is selected.
OLLIE_MODEL is a legacy/environment model fallback; configured backend sections can set model = ....
The runtime path helpers honor XDG_CONFIG_HOME, XDG_DATA_HOME, and XDG_RUNTIME_DIR.
The Makefile install targets use ~/.config/ollie, ~/.local/share/ollie, and ~/.local/bin directly; non-default XDG locations require adjusting the install targets or copying the installed files manually.
Runtime tools are discovered from $XDG_CONFIG_HOME/ollie/tools (default: ~/.config/ollie/tools).
OptMem runtime data is under $XDG_DATA_HOME/ollie/optmem (default: ~/.local/share/ollie/optmem).

Adding a new tool

Script-based tool (Python/Bash)

Create an executable script in data/tools/<name>

Create data/tools/<name>.meta with JSON metadata:

{"description":"...","prompt":"...","args":{...},"tier":"hot","readOnly":false}

Run make install-data to install

Compiled tool (Go)

Create a package under tools/<family>/cmd/<name>/main.go
- Read JSON args from stdin, write result to stdout, exit 0/1
- Share library code in tools/<family>/ (e.g. tools/lsp/)
Create <name>.meta alongside main.go in the same cmd/<name>/ directory (same format as above)
Add a build target in the Makefile that compiles to $(CFG)/tools/<name> and installs the .meta file alongside it
Run make to build and install

The .meta file lives with the code that produces the tool, not in data/tools/. See the tools target in the Makefile for the canonical pattern.

Both paths produce the same result: an executable + .meta in $XDG_CONFIG_HOME/ollie/tools. The registry doesn't distinguish between scripts and binaries.

Adding a new prompt

Write the markdown file in data/prompts/
If it should be loaded by default, reference it in data/agents/default.json
Run make install-data to install

KDE development

KDE integration is part of this repository under kde/. Build and install it through the root Makefile target (make kde).

Key Lessons (Aug 14–17 session)

Unix permissions ARE the enforcement mechanism. Sub-agents get GID "subagent"; top-level agents get GID "agent". File modes control access. Don't invent authorization layers when chmod works.
Context cancellation propagates automatically. Interrupting a parent kills all sub-agents at arbitrary depth through Go's context.Context chain. No explicit cleanup code needed.
New features should be wiring, not construction. If a feature requires more than ~50 lines, you're probably building infrastructure that already exists. Sub-agents: 43 lines. Goals: ~40 lines of handler. The rest is prompt.
No pointless indirection. Thin wrappers, thin delegations, adapter functions that just call another function — these are banned. Call the real thing directly. Move the code, don't wrap it.
Shared code goes in shared packages. ollie/toolsrv is importable by both olliesrv and toolsrv binaries. Don't duplicate functions across internal packages.
The plan file is per-agent, NOT per-session. Each agent owns its own plan at session/{s}/agent/{a}/plan.
The goal file is per-session. Writing to session/{s}/goal triggers a workflow (default: conductor). The goal text is NEVER overwritten by status changes.
Subtraction > addition. Removing maxSteps was -52 lines. The timeout on sub-agents replaced it with 4 lines. Always look for what to remove first.
The 9P namespace is the API. Every capability is a file. Read, Write, or Rdwr. BlockOnce and Stream are special cases of Read. That's the entire interface.
Rdwr is an atomic operation — not a variant of read or write. It's write-then-read as one unit. Sub-agents, session creation, tool execution, and generation all use this primitive.
Persisted state can contain garbage. When debugging impossible errors, check if the data itself is corrupted. A failed command's stderr captured and stored as state will return that error on every subsequent read — the bug isn't in the code, it's in the data.
Environment variables don't always propagate. Tools run through toolsrv, which sets specific env vars. If a tool calls another binary that expects $USER or $OLLIE_UNAME, verify those are actually set in the execution context. Provide explicit fallbacks.
Don't blame the build system. When a "fixed" bug keeps appearing, the code is probably fine. Check if you're reading stale data, hitting a different code path, or misunderstanding the actual error source. The build cache, the compiler, and the linker are rarely at fault.
Separate index files for separate concerns. Don't cram session and agent data into one line. session/idx lists sessions; session/{s}/agent/idx lists agents per session. Simpler parsing, fewer race conditions, cleaner code.
Split large files by concern, not by size. A 900-line file with mixed responsibilities is worse than three 300-line files with clear boundaries. Name files by what they do (dispatch.go, compact.go, retry.go), not by their parent type (agent_dispatch.go).
Never shell out to git in agent plumbing. A repo's own .git/config can name commands (core.fsmonitor, hooks, filters) that run as the user on any git invocation — silently, before any prompt (GitSpawn class, Sep 2026). Repo detection stays os.Stat(.git) (util/paths.go, cmd/toolsrv/internal/server/proc.go). If a feature needs git context, run git -c core.fsmonitor=false ... inside the sandbox, or read the files directly. Sandboxed tool env already forces core.fsmonitor=false via GIT_CONFIG_* (cmd/toolsrv/internal/exec/exec.go).

22 KiB Raw Blame History Unescape Escape