add file-level doc comments and improve AGENTS.md navigation

Agent package files now have descriptive header comments explaining
their purpose:
- dispatch.go: tool execution, batching, conflict detection
- turn.go: turn orchestration and Submit entry point
- state.go: agent state machine and notifications
- history.go: message history and token tracking
- compact.go: context compaction and cold summarization
- cache.go: tool result caching with staleness detection
- retry.go: error tracking and transient retry logic
- runtime.go: preamble assembly and tool schema management
- text_parse.go: text-based tool call parsing
- chatlog.go: chat output formatting
- chat.go: chat log storage and streaming
- workflow.go: workflow classification for discovery
- skill_match.go: semantic skill discovery
- tool_match.go: semantic tool discovery
- commands.go: slash command interception
- feed.go: feed value storage with dedup
- fifo.go: buffered prompt queue
- peer.go: peer agent management
- subagent.go: sub-agent depth and child tracking
- cost.go: cost calculation and audit logging
- prompt_resolver.go: prompt file resolution
- local_summary.go: non-LLM text summarization
- agent_config.go: configuration types

AGENTS.md improvements:
- Add 'Where to Start' section with entry points by concern
- Expand Key Files table with cache, retry, text_parse, tool_match,
  skill_match, chatlog, local_summary, workflow, and proc files
This commit is contained in:
Levi Neely 2026-08-27 10:23:26 +02:00
parent 1159bd700a
commit 765e8b9d05
23 changed files with 173 additions and 0 deletions

View File

@ -112,6 +112,37 @@ For direct Go testing, use the packages covered by `make test-core` and `make te
10. **Sub-agents**: `subagent_spawn` creates a transient child session with an independent runtime and context. The child receives a one-time parent-history snapshot and returns only its final reply. Parent/child IDs are retained for tracing; concurrent children are supported.
11. **Peers**: Persistent agents in the same session can be linked via `peeradd`. Links are bidirectional. Agents communicate by writing to `peer/{name}`, which delivers to the target's prompt handler. Only declared peers can be messaged — the `peer/` directory is the access control surface.
12. **9P namespace declaration**: `cmd/olliesrv/internal/fs/spec.go` declares the olliesrv namespace. The toolsrv namespace is declared by `cmd/toolsrv/p9.go` using `cmd/toolsrv/internal/server.Spec`; process state and handlers are in `cmd/toolsrv/internal/server/`. Both use the `virtfs` EDSL and `virtfs.BuildTree()`.
## Where to Start
Entry points for understanding different parts of the codebase:
**Agent execution**
1. `cmd/olliesrv/internal/agent/loop.go` — Main loop: stream LLM, execute tools, update history
2. `cmd/olliesrv/internal/agent/turn.go` — Turn orchestration; `Submit` is the entry point
3. `cmd/olliesrv/internal/agent/dispatch.go` — Tool batching and conflict detection
**9P namespace**
1. `cmd/olliesrv/internal/fs/spec.go` — All handlers inline, no chasing
2. `cmd/toolsrv/p9.go` — toolsrv namespace; `internal/server/proc.go` owns process state
**Backend integration**
1. `cmd/olliesrv/internal/backend/backend.go` — Interface and shared types
2. Pick a concrete backend (e.g., `anthropic.go`, `openai.go`) to see implementation
**Context management**
1. `cmd/olliesrv/internal/agent/history.go` — Message storage, token tracking
2. `cmd/olliesrv/internal/agent/compact.go` — Compaction logic, cold summarization
**Tool discovery**
1. `cmd/olliesrv/internal/agent/tool_match.go` — Semantic matching
2. `cmd/olliesrv/internal/agent/runtime.go` — Preamble and tool schema assembly
**Sandbox and execution**
1. `cmd/toolsrv/internal/sandbox/` — Landlock policy configuration
2. `cmd/toolsrv/internal/exec/exec.go` — Tool execution wrapper
3. `cmd/toolsrv/internal/server/proc.go` — Process lifecycle
## Key Files
| What | Where |
|------|-------|
@ -124,9 +155,17 @@ For direct Go testing, use the packages covered by `make test-core` and `make te
| Tool dispatch and batching | `cmd/olliesrv/internal/agent/dispatch.go` |
| Message history | `cmd/olliesrv/internal/agent/history.go` |
| Context compaction | `cmd/olliesrv/internal/agent/compact.go` |
| Tool result caching | `cmd/olliesrv/internal/agent/cache.go` |
| Error retry logic | `cmd/olliesrv/internal/agent/retry.go` |
| Runtime and prompt assembly | `cmd/olliesrv/internal/agent/runtime.go`, `prompt_resolver.go` |
| Text-based tool parsing | `cmd/olliesrv/internal/agent/text_parse.go` |
| Semantic tool/skill matching | `cmd/olliesrv/internal/agent/tool_match.go`, `skill_match.go` |
| Chat output formatting | `cmd/olliesrv/internal/agent/chatlog.go` |
| Local summarization | `cmd/olliesrv/internal/agent/local_summary.go` |
| Workflow classification | `cmd/olliesrv/internal/agent/workflow.go` |
| Tool server binary | `cmd/toolsrv/` |
| Tool server client | `cmd/olliesrv/internal/toolclient/` |
| Process lifecycle | `cmd/toolsrv/internal/server/proc.go` |
| Sandbox enforcement | `cmd/toolsrv/internal/sandbox/` |
| Bypass broker | `cmd/olliesrv/internal/bypass/` |
| Session management | `cmd/olliesrv/internal/session/` |

View File

@ -1,3 +1,8 @@
// agent_config.go — Agent configuration types and JSON unmarshaling.
//
// Defines the Prompt type (list of file paths) and related configuration
// helpers used when loading agent profiles from JSON.
package agent
import (

View File

@ -1,3 +1,9 @@
// cache.go — Tool result caching with file staleness detection.
//
// Caches tool results keyed by (tool name, arguments). File-based tools
// track mtime and size; cache entries are invalidated when the underlying
// file changes. Non-file results never expire within a session.
package agent
import (

View File

@ -1,3 +1,9 @@
// chat.go — Chat log storage and streaming.
//
// The chat log is an append-only buffer of formatted output. Readers can
// stream from any offset; StreamChat returns a read function that blocks
// until data is available.
package agent
import (

View File

@ -1,3 +1,9 @@
// chatlog.go — Chat output formatting and event handling.
//
// Converts agent events (streaming text, tool calls, tool results) into
// formatted output written to the chat log. The output handler is set up
// once at agent construction.
package agent
import (

View File

@ -1,3 +1,9 @@
// commands.go — Slash command interception.
//
// Intercepts /commands in user prompts before they reach the LLM. Each
// slash command maps to a ctl write: /compact → "compact", /clear → "clear".
// Returns true if the input was consumed as a command.
package agent
import (

View File

@ -1,3 +1,9 @@
// cost.go — Cost calculation and audit logging.
//
// Calculates model cost from token usage. auditCost logs tool calls with
// model, parameters, and truncated result for observability. Model pricing
// is looked up from a hardcoded table.
package agent
import (

View File

@ -1,3 +1,9 @@
// dispatch.go — Tool execution, batching, and conflict detection.
//
// execToolCalls processes a list of tool calls from a single LLM turn.
// Non-conflicting calls run in parallel (reads, writes to different paths).
// Writes to the same path and global-scope tools serialize.
package agent
import (

View File

@ -1,3 +1,9 @@
// feed.go — Feed value storage with dedup.
//
// Feed holds the current feed value for BlockOnce-style reads. Writes
// store data and compute a short hash. Readers compare hashes to dedup;
// only changed values are returned.
package agent
import (

View File

@ -1,3 +1,9 @@
// fifo.go — Buffered prompt queue.
//
// The FIFO queue holds pending prompts written to the agent's fifo file.
// Bounded by item count and byte size. PopQueue consumes the next item;
// Submit processes the queue before accepting new work.
package agent
import (

View File

@ -1,3 +1,9 @@
// history.go — Message history management and token tracking.
//
// History stores conversation messages, tracks token usage across turns,
// and implements multi-zone compaction (cold/warm/hot) to manage context
// window limits while preserving recent context and decision summaries.
package agent
import (

View File

@ -1,3 +1,10 @@
// local_summary.go — Local (non-LLM) text summarization.
//
// Summarizes large tool outputs without an LLM call. Preserves lines with
// signal words (error, failed, success, etc.) and caps output to reduce
// context consumption. Falls back to head+tail truncation for outputs
// without clear signal.
package agent
import (

View File

@ -1,3 +1,9 @@
// peer.go — Peer agent management.
//
// Peers are other agents in the same session that can communicate directly.
// Links are bidirectional: adding a peer on one agent adds the reverse link.
// The peer/ directory lists peers; writing to peer/{name} sends a message.
package agent
import "slices"

View File

@ -1,3 +1,9 @@
// prompt_resolver.go — Prompt file resolution and environment expansion.
//
// Reads prompt file paths, expands environment variables in both paths and
// content, and joins results. Supports legacy !command entries that execute
// and capture output.
package agent
import (

View File

@ -1,3 +1,10 @@
// retry.go — Error tracking and transient retry logic.
//
// Tracks consecutive tool errors and repeated error messages. At the soft
// limit, nudges the model to try a different approach. At the hard limit,
// aborts the turn. Handles transient errors (rate limits, network) with
// exponential backoff.
package agent
import (

View File

@ -1,3 +1,10 @@
// runtime.go — Runtime state, preamble assembly, and tool schema management.
//
// Runtime holds the active preamble (system prompt sections), tool schemas,
// and backend reference. BuildRuntime constructs it from agent config and
// toolsrv state. The preamble is rebuilt on profile switches or when the
// tool registry changes.
package agent
import (

View File

@ -1,3 +1,9 @@
// skill_match.go — Semantic skill discovery.
//
// Matches user requests to skills using embedding similarity. Returns
// high-scoring skills (above threshold) up to a limit. Uses the shared
// embedding model for cosine similarity ranking.
package agent
import (

View File

@ -1,3 +1,9 @@
// state.go — Agent state machine and state-change notifications.
//
// The agent state tracks execution progress: idle, running, completed, error.
// WaitChange blocks until a field changes, enabling 9P clients to observe
// state transitions via statewait.
package agent
import "context"

View File

@ -1,3 +1,9 @@
// subagent.go — Sub-agent depth and child tracking.
//
// Sub-agents are transient child agents with independent context windows.
// This file provides accessors for depth (0 = top-level) and active child
// count, used to enforce spawn limits and hierarchy tracking.
package agent
// Depth returns the sub-agent depth (0=top-level).

View File

@ -1,3 +1,9 @@
// text_parse.go — Text-based tool call parsing.
//
// Extracts tool calls from assistant text when the model emits them as
// plain text instead of using the function calling API. Supports JSON
// object format and shorthand key=value pairs.
package agent
import (

View File

@ -1,3 +1,9 @@
// tool_match.go — Semantic tool discovery.
//
// Matches user requests to tools using embedding similarity. Returns
// high-scoring tools (above threshold) up to a limit. Injects matching
// tool hints into the preamble without loading all tool schemas.
package agent
import (

View File

@ -1,3 +1,9 @@
// turn.go — Turn orchestration and main execution entry point.
//
// Submit queues a user prompt and starts a turn. executeTurn is the core
// execution path: resolve tools, call the LLM, handle tool calls, and loop
// until the model produces a final response or an error limit is reached.
package agent
import (

View File

@ -1,3 +1,9 @@
// workflow.go — Workflow classification for capability discovery.
//
// Extracts a Workflow from a user request: domain, operation, target, scope,
// and needs. Used by tool and skill matching to prioritize relevant
// capabilities without requiring exact name matches.
package agent
import "strings"