From 42823dab2e572d5a53d628192d4efdbbfcf58730 Mon Sep 17 00:00:00 2001 From: Ollie Agent Date: Sun, 9 Aug 2026 03:06:05 +0200 Subject: [PATCH] docs: update README, AGENTS.md, architecture, evolution for Aug 9 changes - README: fix CLI commands, repo layout (remove detach/, add fsedsl/lib9p) - AGENTS.md: update architecture section (ctl rdwr, bypass, tool_load) - architecture.md: rewrite namespace tree to match current state - evolution.md: add Aug 9 section documenting all simplifications --- AGENTS.md | 8 ++-- README.md | 17 +++---- doc/resources/architecture.md | 57 +++++++++--------------- doc/resources/evolution.md | 84 +++++++++++++++++++++++++++++++++++ 4 files changed, 119 insertions(+), 47 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 9fcffd5..d6e3354 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -92,13 +92,13 @@ just test-remote - Tool scripts: emit structured output (`STATUS=ok`, `STATUS=error`). Image/LSP tools return JSON content blocks. - Prompts: markdown, concise, example-driven. Follow the pattern in existing `tools-*.md` files. ## Architecture (key concepts) -1. **One integration surface: 9P filesystem (sessions at `session/{sname}/agent/{aname}/`). Tools, skills, memory on physical filesystem via env vars. +1. **One integration surface**: 9P filesystem. Sessions at `session/{sname}/agent/{aname}/`. All control via `ctl` (rdwr: write command, read response). Tools, skills, memory on physical filesystem via env vars. 2. **Agent loop** (`agent/loop.go`): Streaming LLM call → parse tool calls → dispatch → loop until no more tool calls or max steps. -3. **Tool dispatch** (`toolsrv/`): `shell` is built-in. All others are external scripts resolved from `$XDG_CONFIG_HOME/ollie/tools`. -4. **Sandbox** (`sandbox/`): Landlock-based. Config in `sandbox/*.yaml` defines filesystem access per profile. +3. **Tool dispatch** (`toolsrv/`): `shell` is built-in. All others are external scripts resolved from `$XDG_CONFIG_HOME/ollie/tools`. Load via `ctl tool_load `. +4. **Sandbox** (`sandbox/`): Landlock-based. Config in `sandbox/*.yaml` defines filesystem access per profile. Escape via bypass broker (`bypass/`). 5. **Backends** (`backend/`): Ollama, OpenAI-compatible, Anthropic, Copilot, Kiro, Gemini, CodeWhisperer. Selectable per-session. 6. **Prompts assembled at runtime**: Agent JSON `prompt` array specifies which prompt files to concatenate. Static prompt files can be included directly; the base system prompt is embedded in the binary and always prepended. -7. **9P namespace declared via EDSL**: The entire filesystem is a single recursive `FsNodeDecl` tree in `fs/spec.go`, built by `BuildTree()` in `fs/builder.go`. The `fs/` package is flat — no sub-package — handler files (`rootfiles.go`, `sessionfiles.go`, `agentfiles.go`) are organized by scope. See `doc/edsl.md` for the full reference. +7. **9P namespace declared via EDSL**: The entire filesystem is a single recursive `FsNodeDecl` tree in `fs/spec.go`, built by `BuildTree()` in `fs/builder.go`. The `fs/` package is flat — handlers in `handlers.go`, ctl dispatch in `ctl.go`. See `doc/edsl.md` for the full reference. ## Key Files | What | Where | |------|-------| diff --git a/README.md b/README.md index 5b1c73c..5fa8b1d 100644 --- a/README.md +++ b/README.md @@ -43,15 +43,15 @@ Everything lives under `$XDG_CONFIG_HOME/ollie/` (default: `~/.config/ollie/`) | Capability | How | |---|---| -| **CLI** | `o` — unified namespace CLI with `read`, `write`, `readloop`, `ls`, `ctl`, `prompt`, `tail`, `env`, `stop`, `kill` | -| **Terminal TUI** | `o tui` — tmux + two shell commands, no widgets | +| **CLI** | `o` — unified namespace CLI: `read`, `write`, `ls`, `ctl`, `prompt`, `tui`, `env`, `generate`, `complete`, `route` | +| **Terminal TUI** | `o tui` — tmux layout, no widgets | | **One-shot LLM** | `o generate "explain monads"` or `echo "prompt" \| o generate` | | **Code completion** | `o complete '{"file":"main.go","prefix":"func "}'` or `echo '{"file":"..."}' \| o complete` | | **Task routing** | `o route "implement login"` or `echo "task" \| o route` → `backend=X model=Y` | | **Run an agent** | Create a session + agent via 9P — then connect with any frontend ([ellie](doc/ellie.md), KDE (GUI/KRunner/Kate), `o tui`) | | **Remote execution** | Set `remote=user@host` in session config | | **Agents** | Write a JSON config in `data/agents/` with agent prompt in `data/prompts/` | -| **Tools** | Drop an executable + `.meta` in `~/.config/ollie/tools/` — then load at runtime via `tool_load` | +| **Tools** | Drop an executable + `.meta` in `~/.config/ollie/tools/` — load at runtime via `/tool_load` | | **Domain skills** | Load markdown skill modules at runtime | ## Repository layout @@ -60,17 +60,18 @@ Single Go module with one Git submodule (kde). | Directory | Language | Description | |-----------|----------|-------------| -| `agent/` | Go | Agent loop, history, hooks, prompt resolution, commands | +| `agent/` | Go | Agent loop, history, prompt resolution, context compaction | | `backend/` | Go | LLM providers (Anthropic, OpenAI, Ollama, Gemini, Copilot, CodeWhisperer) | | `toolsrv/` | Go | Tool server, dynamic tool dispatch, sandboxed execution, remote execution | | `tools/lsp/` | Go | LSP bridge daemon + client library (gopls, clangd, intelephense) | | `session/` | Go | Session lifecycle, config, persistence | -| `fs/` | Go | 9P filesystem tree for session namespace | -| `detach/` | Go | Background process management | +| `fs/` | Go | 9P filesystem tree (EDSL-declared namespace, ctl dispatch) | +| `fsedsl/` | Go | Filesystem declaration EDSL (generic, reusable) | | `bypass/` | Go | Bypass broker (sandbox escape approval) | | `sandbox/` | Go | Landlock sandbox config | +| `lib9p/` | Go | 9P protocol library + native C client | | `cmd/olliesrv/` | Go | The main binary | -| `cmd/ollie-9p/` | Go | 9P client | +| `cmd/ollie-9p/` | Go | 9P client CLI | | `cmd/ollie-remote/` | Go | Remote execution binary | | `kde/` | C++/Qt6 | KDE plasmoid, GUI, Kate plugin, KRunner *(submodule)* | | `data/agents/` | JSON | Agent configs | @@ -106,7 +107,7 @@ graph TB end subgraph Execution - LOCAL[Tool Scripts
loaded via 9P write
· shell, reasoning_think
· file_*, lsp_*, memory_*
· gui_*, subagent_*] + LOCAL[Tool Scripts
loaded via ctl
· shell, reasoning_think
· file_*, lsp_*, memory_*
· gui_*, subagent_*] REMOTE[ollie-remote
via SSH · no embedded tools] end diff --git a/doc/resources/architecture.md b/doc/resources/architecture.md index 740c860..66d9c3d 100644 --- a/doc/resources/architecture.md +++ b/doc/resources/architecture.md @@ -15,35 +15,30 @@ Design principles: Single Go module (`ollie`) with one Git submodule (`kde/`) for the KDE frontend: ``` ollie/ -├── agent/ Agent loop, history, hooks, prompt resolution, commands +├── agent/ Agent loop, history, prompt resolution, context compaction ├── backend/ LLM providers (Anthropic, OpenAI, Ollama, Gemini, Copilot, CodeWhisperer) ├── toolsrv/ Tool server: sandboxed execution, tool registry, skill management ├── session/ Session lifecycle (config, creation, persistence) ├── fs/ 9P filesystem: EDSL spec + handlers (flat package) │ ├── spec.go Namespace declaration (single source of truth) -│ ├── fsnode.go FsNodeDecl type + Dir/Leaf/TemplateDir constructors -│ ├── builder.go BuildTree — spec -> *Tree wiring -│ ├── rootfiles.go Root-level handlers (backends, models, eventwait, complete, generate, route) -│ ├── sessionfiles.go Session-level handlers (env, ctl, plan, agent/, ...) -│ ├── agentfiles.go Agent-level handlers (prompt, chat, state, cfg, ...) -│ ├── bypassfiles.go Bypass handlers (policy, pending) -│ ├── procfiles.go Process handlers -│ ├── lifecycle.go Session create/kill/rename/shutdown + event ring -│ ├── newroot.go NewRoot — tree construction + persistence restore -│ ├── persist.go Session persistence to disk -│ ├── types.go Session/AgentLog types -│ ├── tree.go 9P *Tree (from fs package) -│ ├── fs.go 9P File/FileConfig implementations -│ └── format.go Event formatting helpers -├── detach/ Background process management (ring buffer, signal) -├── bypass/ Bypass broker (privilege escalation daemon) +│ ├── handlers.go All file handlers (root, session, agent) +│ ├── ctl.go rdwrHandler type + rdwrDispatch (shared by all ctl files) +│ ├── agent_log.go AgentLog (chat streaming, chatCond) +│ ├── session_node.go SessionNode wrapper +│ ├── lifecycle.go RootState, session node cache +│ ├── newroot.go NewRoot — tree construction config +│ └── cache.go ModelCache +├── fsedsl/ Filesystem declaration EDSL (generic, reusable) +├── bypass/ Bypass broker (sandbox escape approval) ├── sandbox/ Landlock sandbox config YAML +├── lib9p/ 9P protocol library + native C client ├── env/ Environment variable loading ├── log/ Structured logging ├── paths/ XDG path resolution +├── format/ Chat log formatting constants ├── cmd/ Binaries: │ ├── olliesrv/ 9P server -│ ├── ollie-9p/ 9P client +│ ├── ollie-9p/ 9P client CLI │ └── ollie-remote/ Remote execution server ├── kde/ KDE integration (submodule) — standalone GUI, Kate plugin, KRunner ├── data/agents/ Agent config JSONs (default, coding, orchestrator, worker, ...) @@ -209,7 +204,7 @@ Agent configs declare prompts as a JSON array of shell commands. Each command is ├── idx session index (one line per agent) ├── {name}/ │ ├── env session environment variables - │ ├── ctl session control (kill, save, invalidate) + │ ├── ctl session control (rdwr: kill, save, invalidate, pause, resume) │ ├── plan session-scoped markdown checklist │ ├── id immutable session UUID │ ├── name mutable session name (write to rename) @@ -218,27 +213,19 @@ Agent configs declare prompts as a JSON array of shell commands. Each command is │ ├── new write config to create agent (rdwr) │ └── {aname}/ │ ├── prompt submit a prompt - │ ├── prompt.prev last submitted prompt - │ ├── fifo.in queue a prompt - │ ├── fifo.out pop queued prompt - │ ├── chat streaming chat log (blocking read) + │ ├── fifo prompt queue (write=enqueue, read=dequeue) + │ ├── chat streaming filtered text (no markers/fences) + │ ├── chat.raw streaming full markup (block markers + fences) │ ├── log last 64KB of chat (non-blocking) - │ ├── state current state (idle/thinking/calling) │ ├── statewait blocking read until state changes │ ├── cfg agent config (key=value) - │ ├── ctl agent control (stop, compact, ...) - │ ├── cwd working directory + │ ├── ctl agent control (rdwr): stop, compact, clear, inject, + │ │ agent, model, models, tools, tool_load, cwd, name, + │ │ backend, systemprompt + │ ├── stats usage=, cost=, ctxsz= (key=value lines) + │ ├── plan agent-scoped markdown checklist │ ├── id immutable agent UUID │ ├── name mutable agent name - │ ├── offset byte offset after last user prompt - │ ├── usage token usage stats - │ ├── cost estimated cost - │ ├── ctxsz context size - │ ├── models available models - │ ├── systemprompt rendered system prompt - │ ├── context rendered context window - │ ├── tail exec helper for tailing chat - │ ├── tools tool management (read = list loaded, write name = load) │ └── proc/{pid} detached process output ``` ### 9P Interaction Model diff --git a/doc/resources/evolution.md b/doc/resources/evolution.md index 2ee6616..e604619 100644 --- a/doc/resources/evolution.md +++ b/doc/resources/evolution.md @@ -952,3 +952,87 @@ The timeline now extends to Aug 4, 2026 — ~3.75 months of development. Note: Aug 3 count included `backend/` (4,366 lines) and had different kde scope. Comparable core (fs + agent + toolsrv + session + fsedsl + kde gui) shrank from ~23k to ~17k. + +--- + +## Aug 9, 2026 — Simplification & Namespace Cleanup + +Major simplification pass: ~1,450 lines removed from the core runtime. + +### Structural + +- **Section-based preamble** — `Preamble` struct with `Set/Get/String` replaces string surgery (-270 lines) +- **Session** — Export immutable fields, extract `buildAgent` helper (-115 lines) +- **Prompt resolver** — Gutted to file-path-only resolution (-127 lines) +- **TaskState subsystem** — Removed entirely (-246 lines) +- **GenerationParams** — Embedded in AgentConfig, JSON tags added (-34 lines) +- **sync.Cond → close-channel** — `WaitChange` uses idiomatic channel signal +- **Dead code** — `session.New()`, env map, prevPrompt field, usage_log.go, contextDebug() + +### Namespace + +The agent namespace went from 17 files to 10. Redundant files merged or moved to ctl: + +| Before | After | +|--------|-------| +| `fifo.in` + `fifo.out` | `fifo` (write=enqueue, read=dequeue) | +| `cost` + `usage` + `ctxsz` | `stats` (key=value lines) | +| `chat` | `chat` (filtered text) + `chat.raw` (full markup) | +| `cwd`, `models`, `tools`, `systemprompt` | ctl commands | +| `connection`, `state`, `context`, `tail`, `offset`, `prompt.prev` | removed | + +### ctl Unification + +All ctl files (root, session, agent) now share: +```go +type rdwrHandler func(ctx HandlerCtx, args []string) ([]byte, error) +``` + +Agent ctl is **rdwr** (request-response): write command, read result. +Commands with no args return current state (e.g., `/model` prints the model). + +Agent ctl commands: `stop`, `compact`, `clear`, `inject`, `agent`, `model`, `models`, `tools`, `tool_load`, `cwd`, `name`, `backend`, `systemprompt`. + +### Command Surface + +`/` prefix in prompts == `o ctl`. No special-cased commands in the agent. +- `/model qwen3:8b` → switches model, returns new model name +- `/tools` → lists loaded tools +- `/inject look at this` → injects prompt mid-turn (overwrites pending) +- `!q` in the REPL → exits the TUI (local-only) + +### elevate → bypass + +The sandbox escape mechanism was renamed from `elevate` to `bypass` throughout (package, namespace, env vars, tool args, all docs). Eliminates confusion with sudo. + +### Streaming Fix + +The TUI streaming regression (line-by-line instead of char-by-char since Aug 4) was caused by piping through `grep -v` — grep is inherently line-buffered. Fixed by: +- `chat` file now serves filtered text (markers + fences stripped server-side via `stripMarkers` state machine) +- `chat.raw` serves full markup for GUIs that parse blocks +- TUI no longer pipes through grep for streaming + +### CLI (`o` script) + +- `readloop` → `read -l` +- `chatstream` → removed (`o read chat` streams directly) +- `stop`, `kill` → removed (use `o ctl stop`, `o ctl kill`) +- `ctl` uses `ollie-9p rdwr` (gets response) +- REPL: `/` → `o ctl`, `!q` → exit TUI +- Background statewait loop trapped on EXIT (fixes zombie) + +### SLOC (Aug 9) + +| Component | Lines (excl. tests, backends, generated) | +|-----------|--------------------:| +| agent | 3,178 | +| toolsrv | 2,497 | +| fs | 1,737 | +| session | 1,402 | +| cmd/olliesrv | 1,165 | +| bypass | 733 | +| lib9p + client | 690 | +| cmd/ollie-9p | 356 | +| sandbox | 293 | +| log + env + paths + format | 363 | +| **Total (core)** | **12,414** |