update AGENTS.md and architecture-core.md for agent package reorganization

AGENTS.md:
- Update Architecture section with detailed agent package file list
- Update Key Files table with new agent package files
- Add lesson 15: split by concern, not size

architecture-core.md:
- Update Package Map diagram with new files
- Add file responsibility table in Agent Loop section
- Reference compact.go in Session & Context section
This commit is contained in:
Levi Neely 2026-08-27 10:15:31 +02:00
parent c8ccc2c450
commit 1159bd700a
2 changed files with 49 additions and 15 deletions

View File

@ -94,7 +94,15 @@ For direct Go testing, use the packages covered by `make test-core` and `make te
## Architecture (key concepts)
1. **One integration surface**: `olliesrv` exposes sessions and agents through a 9P2000 filesystem. Frontends include `o`, `ollie-9p`, KDE, Kate, Emacs, and scripts. Reads from `chat`, `statewait`, `eventwait`, and `feed` provide blocking/event-driven synchronization.
2. **Session and tool processes**: Each session owns an agent runtime in `olliesrv` and a separate `toolsrv` process. They communicate over an authenticated Unix socket using 9P. `toolclient` can respawn local toolsrv processes and can deploy/start toolsrv remotely over SSH with socket forwarding.
3. **Agent loop** (`cmd/olliesrv/internal/agent/loop.go` and `turn.go`): Stream an LLM response, parse native or text tool calls, execute tools, update history, and repeat until a final response, cancellation, or a configured limit. It includes transient retries, context-overflow compaction, error/stall/replan controls, and tool-result caching.
3. **Agent loop** (`cmd/olliesrv/internal/agent/`): The agent package is organized by concern:
- `loop.go`: Main loop — stream LLM, execute tools, update history
- `turn.go`: Submit entry point, turn orchestration
- `dispatch.go`: Tool execution, batching, conflict detection
- `history.go`: Message history, usage tracking
- `compact.go`: Context compaction, cold summarization
- `cache.go`: Tool result caching with file staleness detection
- `retry.go`: Error tracking, transient retry logic
- `state.go`, `chat.go`, `peer.go`, `subagent.go`: Agent state and coordination
4. **Dynamic tools**: Tools are external executables described by `.meta` files. `toolsrv` owns discovery and per-agent registries. `olliesrv` refreshes the registry, injects common dispatch flags (`bypass`, `timeout`, `sandbox`, `background`), and calls tools through `toolsrv.Conn`.
5. **Parallel and background dispatch**: Non-conflicting tool calls in one turn run concurrently. Metadata scopes schedule `read`, `write`, and global operations; shell-like global operations serialize. Any tool may run in the background, producing a process ID whose output is injected when the process changes or exits.
6. **Sandbox** (`cmd/toolsrv/internal/sandbox/`): Native Landlock policies control filesystem access in a short-lived child helper. The installed `sandbox.yaml` is sourced from `cmd/toolsrv/internal/sandbox/sandbox.yaml`. The toolsrv namespace and process state live under `cmd/toolsrv/p9.go` and `cmd/toolsrv/internal/server/`. The bypass broker provides policy-controlled escape requests, approval, persistence, and rate limiting.
@ -110,15 +118,19 @@ For direct Go testing, use the packages covered by `make test-core` and `make te
| 9P namespace (olliesrv) | `cmd/olliesrv/internal/fs/spec.go` |
| 9P namespace (toolsrv) | `cmd/toolsrv/p9.go`, `cmd/toolsrv/internal/server/server.go` |
| virtfs EDSL | `virtfs/decl.go`, `virtfs/builder.go` |
| Agent loop and dispatch | `cmd/olliesrv/internal/agent/loop.go`, `turn.go` |
| Agent core and identity | `cmd/olliesrv/internal/agent/agent.go` |
| Agent loop | `cmd/olliesrv/internal/agent/loop.go` |
| Turn orchestration | `cmd/olliesrv/internal/agent/turn.go` |
| Tool dispatch and batching | `cmd/olliesrv/internal/agent/dispatch.go` |
| Message history | `cmd/olliesrv/internal/agent/history.go` |
| Context compaction | `cmd/olliesrv/internal/agent/compact.go` |
| Runtime and prompt assembly | `cmd/olliesrv/internal/agent/runtime.go`, `prompt_resolver.go` |
| Tool server binary | `cmd/toolsrv/` |
| Tool server client and process lifecycle | `toolsrv/client9p.go`, `cmd/olliesrv/internal/toolclient/` |
| Remote tool execution | `cmd/olliesrv/internal/toolclient/spawn.go` |
| Tool server client | `cmd/olliesrv/internal/toolclient/` |
| Sandbox enforcement | `cmd/toolsrv/internal/sandbox/` |
| Bypass broker | `cmd/olliesrv/internal/bypass/` |
| Session management and persistence | `cmd/olliesrv/internal/session/` |
| Embedded default system prompt | `cmd/olliesrv/internal/prompts/system_prompt.md` |
| Session management | `cmd/olliesrv/internal/session/` |
| Embedded system prompt | `cmd/olliesrv/internal/prompts/system_prompt.md` |
| Agent configs | `data/agents/*.json` |
| Backend configuration | `data/backends.conf`, `~/.config/ollie/backends.conf` |
| Compiled tools | `tools/codeintel/`, `tools/filetools/`, `tools/lsp/`, `tools/web/` |
@ -191,3 +203,5 @@ KDE integration is part of this repository under `kde/`. Build and install it th
13. **Don't blame the build system.** When a "fixed" bug keeps appearing, the code is probably fine. Check if you're reading stale data, hitting a different code path, or misunderstanding the actual error source. The build cache, the compiler, and the linker are rarely at fault.
14. **Separate index files for separate concerns.** Don't cram session and agent data into one line. `session/idx` lists sessions; `session/{s}/agent/idx` lists agents per session. Simpler parsing, fewer race conditions, cleaner code.
15. **Split large files by concern, not by size.** A 900-line file with mixed responsibilities is worse than three 300-line files with clear boundaries. Name files by what they do (`dispatch.go`, `compact.go`, `retry.go`), not by their parent type (`agent_dispatch.go`).

View File

@ -9,11 +9,15 @@ This document covers the internal Go packages that implement Ollie's agent runti
```mermaid
flowchart TB
subgraph Agent["cmd/olliesrv/internal/agent/"]
AGENT["agent.go — Agent, events, interrupt"]
LOOP["loop.go — agent loop"]
HISTORY["history.go — history, task state, compaction"]
TURN["turn.go — per-turn state and cancellation"]
CMDS["commands.go — command dispatch"]
AGENT["agent.go — Agent struct, identity, events"]
LOOP["loop.go — main agent loop, streaming"]
TURN["turn.go — turn orchestration, Submit"]
DISPATCH["dispatch.go — tool execution, batching"]
HISTORY["history.go — message history, usage"]
COMPACT["compact.go — compaction, cold summarization"]
CACHE["cache.go — tool result caching"]
RETRY["retry.go — error tracking, retry logic"]
STATE["state.go, chat.go, peer.go, subagent.go"]
RUNTIME["runtime.go — Runtime, preamble, tool dispatch"]
CONFIG["agent_config.go — profile configuration"]
PROMPT["prompt_resolver.go — prompt resolution"]
@ -21,7 +25,7 @@ flowchart TB
subgraph Backend["cmd/olliesrv/internal/backend/"]
BACKEND["backend.go — Backend interface and streaming"]
PROVIDERS["openai, anthropic, ollama, gemini, copilot, kiro, codewhisperer"]
PROVIDERS["openai, anthropic, ollama, gemini, copilot, kiro"]
POOL["pool.go — backend pool"]
end
@ -148,9 +152,25 @@ type Runtime struct {
`NewAgent(AgentParams)` creates an agent with identity, session, profile, history, runtime, working directory, persistence callbacks, event handler state, chat-log state, and session environment. The configuration type on disk is `AgentConfig`; construction dependencies are carried separately by `AgentParams`.
## The Agent Loop (`cmd/olliesrv/internal/agent/loop.go`, `turn.go`)
## The Agent Loop (`cmd/olliesrv/internal/agent/`)
The `run()` function is the core execution engine. It takes an `agentConfig` and a `state` interface, and loops until the model stops calling tools.
The agent package is organized by concern:
| File | Responsibility |
|------|---------------|
| `loop.go` | Main loop: stream LLM, execute tools, update history |
| `turn.go` | Submit entry point, turn orchestration |
| `dispatch.go` | Tool execution, batching, conflict detection |
| `history.go` | Message history, usage tracking |
| `compact.go` | Context compaction, cold summarization |
| `cache.go` | Tool result caching with file staleness |
| `retry.go` | Error tracking, transient retry logic |
| `state.go` | State management, signals, WaitChange |
| `chat.go` | Chat log, streaming output |
| `peer.go` | Peer agent management |
| `subagent.go` | Sub-agent depth tracking |
The `run()` function in `loop.go` is the core execution engine. It loops until the model stops calling tools.
### Per-Iteration Flow
@ -224,7 +244,7 @@ After each tool round, `inferTaskStateUpdate` updates the structured overlay:
- `LastAction` ← most recent tool call + truncated result
- `PlanStep` ← next unchecked item from the plan file (or heuristic from assistant text)
## Session & Context (`cmd/olliesrv/internal/agent/history.go`)
## Session & Context (`cmd/olliesrv/internal/agent/history.go`, `compact.go`)
### Message History