Consolidate tool architecture documentation

This commit is contained in:
Ollie Agent 2026-08-16 19:44:09 +02:00
parent 670fa717c7
commit 16240b3e12
5 changed files with 185 additions and 418 deletions

183
doc/architecture-tools.md Normal file
View File

@ -0,0 +1,183 @@
# Tools Architecture and Authoring
A tool is declared by a `.meta` file. It may also have an accompanying executable, but neither form is required in every case:
- **Metadata plus binary/script:** place `<name>.meta` beside an executable named `<name>`.
- **Metadata only:** set `cmd` in `<name>.meta` to a shell command. The registry executes that command directly.
Both approaches are valid. The `.meta` file is always the source of discovery, model-facing documentation, and the JSON argument schema.
The registry reads the metadata, presents the tool to the model, and executes the resolved command when called. No integration with the Ollie source tree is required. Put the `.meta` file in `$OLLIE_TOOLS_PATH`; the tool is available on the next discovery or session start.
## Quick start
### Metadata plus an executable
```text
~/.config/ollie/tools/
├── my_tool # executable in any language
└── my_tool.meta # JSON metadata
```
### Metadata-only tool
```json
{
"description": "Extract a value from JSON.",
"prompt": "## json_value\n\nRead `.path` from JSON input.",
"cmd": "input=$(cat); echo \"$input\" | jq -r '.path'",
"args": {
"type": "object",
"required": ["path"],
"properties": {"path": {"type": "string"}}
}
}
```
A metadata-only tool needs no script or binary. Its `cmd` is run through the shell with the tool-call JSON on standard input.
## The `.meta` sidecar
The `.meta` file is the sole source of discovery and schema. The registry never inspects an accompanying executable to infer its interface.
```json
{
"description": "One-liner shown in tool listings.",
"prompt": "## my_tool\n\nFull documentation shown when loaded.",
"args": {
"type": "object",
"required": ["path"],
"properties": {
"path": {"type": "string", "description": "File path"}
}
},
"tier": "hot",
"cmd": "my_tool"
}
```
| Field | Type | Purpose |
|---|---|---|
| `description` | string | One-liner shown in the tool listing. |
| `prompt` | string | Documentation injected when the tool is loaded. |
| `args` | JSON Schema | Input schema exposed to the model. |
| `tier` | `hot`, `warm`, or `cold` | Context retention tier. |
| `cmd` | string | Shell command or executable path/name. |
| `variants` | array | Conditional definitions for heterogeneous hosts. |
| `scope` | string | Tool scope metadata. |
| `outputFormat` | string | Output format metadata. |
| `resetsCounter` | bool | Marks tools that reset the relevant counter. |
## Command resolution
When the registry resolves a tool named `name`, it checks:
1. `$OLLIE_TOOLS_PATH/name` — a co-located executable or script.
2. The resolved `.meta` `cmd` field — an absolute path, bare executable name, or shell command.
3. `name` through `$PATH`.
A co-located executable takes precedence over `cmd`. Metadata-only tools use the `cmd` value directly through the shell. This supports scripts, compiled binaries, wrappers, system commands, and pipelines.
## Variants
The first variant whose conditions all match determines the tool’s schema, documentation, and command. If no variant matches, the tool is hidden.
| Key | Semantics | Example |
|---|---|---|
| `binary` | Binary exists in `$PATH` or at an absolute path. | `journalctl` |
| `file` | File or directory exists. | `/var/log/syslog` |
| `os` | Matches `runtime.GOOS`. | `linux` |
| `arch` | Matches `runtime.GOARCH`. | `amd64` |
| `env` | Environment variable is non-empty. | `DISPLAY` |
| `nenv` | Environment variable is empty or unset. | `SSH_CONNECTION` |
All keys in one `match` object use AND logic. An array value uses OR logic. Variant fields override top-level fields; unset fields inherit. If a remote toolsrv receives the metadata, it evaluates variants against the remote host.
```json
{
"description": "Read system logs.",
"variants": [
{"match": {"binary": "journalctl"}, "cmd": "system_logs_journald"},
{"match": {"file": "/var/log/syslog"}, "cmd": "system_logs_syslog"}
]
}
```
## Input and output contract
Both tool forms use the same contract:
- Input is one JSON object on standard input.
- Successful output is written to standard output and exits with status 0.
- Errors may be written to standard output or standard error and exit non-zero.
- Structured output may be returned as `{"content": [...]}` JSON.
The dispatcher does not require flags, argv conventions, or language-specific integration.
### Executable example
```python
#!/usr/bin/env python3
import json
import sys
args = json.load(sys.stdin)
print(f"result: {args['path']}")
```
### Metadata-only example
```json
{
"description": "Search with ripgrep.",
"cmd": "input=$(cat); p=$(echo \"$input\" | jq -r '.pattern'); rg \"$p\"",
"args": {
"type": "object",
"required": ["pattern"],
"properties": {"pattern": {"type": "string"}}
}
}
```
For subcommand-style tools, parse the JSON with `jq` and construct the command explicitly. Avoid unsafe `eval` when arguments can be passed as arrays or quoted shell parameters.
## Sandboxing and bypass
Tools run inside the toolsrv sandbox by default. The sandbox profile controls filesystem and network access. A tool call that requires bypass goes through the bypass broker and approval path; the command still runs as the current user.
Sandbox and bypass behavior are execution-service concerns. The metadata describes the tool interface and command, not the complete security policy.
## Environment
Tools may receive:
| Variable | Purpose |
|---|---|
| `OLLIE_TOOLS_PATH` | Tool metadata and executable directory. |
| `OLLIE_BYPASS_SOCKET` | Bypass broker socket. |
| `PWD` | Session working directory. |
## Installation and testing
Copy a `.meta` file and, when using the binary approach, its executable to the tools directory:
```sh
cp my_tool.meta ~/.config/ollie/tools/
cp my_tool ~/.config/ollie/tools/
chmod +x ~/.config/ollie/tools/my_tool
```
A metadata-only tool needs only the first command. Test its command contract with JSON on standard input:
```sh
echo '{"pattern":"TODO"}' | bash -c 'YOUR_CMD'
```
Agent profiles can load tools at startup with `autoLoad`. See [`architecture-toolsrv.md`](architecture-toolsrv.md) for discovery, loading, registry, and execution architecture.
## Repository examples
The repository includes executable and metadata-backed tools under `data/tools/`, including filesystem, GUI, LSP, and memory tools. Compiled tools are valid alongside scripts and metadata-only definitions.
See [`no-mcp.md`](resources/no-mcp.md) for the rationale for CLI tool definitions instead of native MCP support.

View File

@ -70,7 +70,7 @@ echo 'file_edit' | 9p rdwr ollie/tools # returns file_edit's full documentatio
Drop an executable and its `.meta` file in `$OLLIE_TOOLS_PATH`.
The next session created will pick it up automatically. No restart required.
See [doc/writing-tools.md](writing-tools.md) for the full specification including
See [architecture-tools.md](../architecture-tools.md) for the full specification including
host-conditional variants and privileged tools.
---

View File

@ -1,150 +0,0 @@
# Tool Definitions
A `.meta` file can define a tool without any companion executable. The `cmd`
field specifies a shell command to run — the registry executes it via shell,
passing the tool's JSON arguments on stdin.
This is the same `.meta` schema used in `writing-tools.md`, but without the
sidecar executable. No script, no binary — just the definition.
## Resolution
When the registry resolves a tool, it checks:
1. `~/.config/ollie/tools/<name>` — local script or binary
2. `.meta` `cmd` field — shell command, executed directly
3. `exec.LookPath(<name>)` — PATH lookup
If a `.meta` file has a `cmd` field and no co-located executable exists, the
registry runs the command via shell.
## Structure
```json
{
"description": "One-liner for listings.",
"prompt": "## my_tool\n\nUsage docs.\n\n```\nmy_tool(arg=\"value\")\n```",
"cmd": "jq -r '.arg' | xargs my-cli",
"args": {
"type": "object",
"required": ["arg"],
"properties": {
"arg": {"type": "string", "description": "Some argument"}
}
},
"scope": "global"
}
```
The `cmd` field is the only addition to the standard `.meta` schema. All other
fields work the same as documented in `writing-tools.md`.
## Parsing arguments
The model sends JSON on stdin. Parse with jq:
```json
{
"cmd": "input=$(cat); arg=$(echo \"$input\" | jq -r '.arg'); my-cli \"$arg\""
}
```
For subcommand-style CLIs with variable arguments:
```json
{
"cmd": "input=$(cat); cmd=$(echo \"$input\" | jq -r '.command'); args=$(echo \"$input\" | jq -r 'del(.command) | to_entries | map(\"--\\(.key)=\\(.value|tostring)\") | join(\" \")'); eval \"my-cli $cmd $args\"",
"args": {
"type": "object",
"required": ["command"],
"properties": {
"command": {"type": "string", "description": "Subcommand"}
},
"additionalProperties": true
}
}
```
This extracts a `command` field, builds `--key=value` flags from remaining
fields, and calls `my-cli <command> --key=value ...`.
## Example: ripgrep
```json
{
"description": "Search file contents with ripgrep.",
"prompt": "## rg_search\n\nSearch files.\n\n**Args**: `pattern` (required), `path`, `glob`, `case_insensitive`\n\n```\nrg_search(pattern=\"TODO\", path=\"src/\", glob=\"*.go\")\n```",
"cmd": "input=$(cat); p=$(echo \"$input\" | jq -r '.pattern'); dir=$(echo \"$input\" | jq -r '.path // \".\"'); glob=$(echo \"$input\" | jq -r '.glob // empty'); ci=$(echo \"$input\" | jq -r '.case_insensitive // empty'); rg ${ci:+-i} ${glob:+-g \"$glob\"} \"$p\" \"$dir\"",
"args": {
"type": "object",
"required": ["pattern"],
"properties": {
"pattern": {"type": "string", "description": "Regex pattern"},
"path": {"type": "string", "description": "Search directory"},
"glob": {"type": "string", "description": "File glob filter"},
"case_insensitive": {"type": "boolean", "description": "Case-insensitive search"}
}
},
"scope": "read"
}
```
## Example: kubectl
```json
{
"description": "Kubernetes cluster operations.",
"prompt": "## k8s\n\nKubernetes operations.\n\n**Args**: `command` (required), plus command-specific args.\n\n| Command | Args |\n|---------|------|\n| `get` | `resource`, `name?`, `namespace?`, `output?` |\n| `describe` | `resource`, `name`, `namespace?` |\n| `logs` | `pod`, `namespace?`, `tail?`, `follow?` |\n| `apply` | `file` |\n| `delete` | `resource`, `name`, `namespace?` |\n\n```\nk8s(command=\"get\", resource=\"pods\", namespace=\"default\")\nk8s(command=\"logs\", pod=\"api-7d4b8c6f9-x2k4m\", tail=100)\nk8s(command=\"apply\", file=\"deploy.yaml\")\n```",
"cmd": "input=$(cat); cmd=$(echo \"$input\" | jq -r '.command'); case $cmd in get) res=$(echo \"$input\" | jq -r '.resource'); name=$(echo \"$input\" | jq -r '.name // empty'); ns=$(echo \"$input\" | jq -r '.namespace // empty'); out=$(echo \"$input\" | jq -r '.output // empty'); kubectl get $res $name ${ns:+-n $ns} ${out:+-o $out} ;; describe) res=$(echo \"$input\" | jq -r '.resource'); name=$(echo \"$input\" | jq -r '.name'); ns=$(echo \"$input\" | jq -r '.namespace // empty'); kubectl describe $res $name ${ns:+-n $ns} ;; logs) pod=$(echo \"$input\" | jq -r '.pod'); ns=$(echo \"$input\" | jq -r '.namespace // empty'); tail=$(echo \"$input\" | jq -r '.tail // empty'); follow=$(echo \"$input\" | jq -r '.follow // empty'); kubectl logs $pod ${ns:+-n $ns} ${tail:+--tail=$tail} ${follow:+-f} ;; apply) file=$(echo \"$input\" | jq -r '.file'); kubectl apply -f \"$file\" ;; delete) res=$(echo \"$input\" | jq -r '.resource'); name=$(echo \"$input\" | jq -r '.name'); ns=$(echo \"$input\" | jq -r '.namespace // empty'); kubectl delete $res $name ${ns:+-n $ns} ;; *) echo \"unknown command: $cmd\" >&2; exit 1 ;; esac",
"args": {
"type": "object",
"required": ["command"],
"properties": {
"command": {"type": "string", "description": "Operation (get, describe, logs, apply, delete)"}
},
"additionalProperties": true
},
"scope": "global"
}
```
## Example: MCP bridge
MCP servers expose tools over JSON-RPC. CLI bridges like `mcp-client-cli` let
you call them from the command line:
```json
{
"description": "GitHub operations via MCP.",
"prompt": "## github\n\nGitHub MCP tools.\n\n**Args**: `tool` (required), plus tool-specific args.\n\n| Tool | Args |\n|------|------|\n| `search_repositories` | `query` |\n| `get_file_contents` | `owner`, `repo`, `path`, `branch?` |\n| `create_issue` | `owner`, `repo`, `title`, `body?` |\n| `list_issues` | `owner`, `repo`, `state?` |\n\n```\ngithub(tool=\"search_repositories\", query=\"language:go stars:>1000\")\ngithub(tool=\"get_file_contents\", owner=\"golang\", repo=\"go\", path=\"README.md\")\n```",
"cmd": "input=$(cat); tool=$(echo \"$input\" | jq -r '.tool'); args=$(echo \"$input\" | jq -c 'del(.tool)'); echo \"$args\" | mcp-client-cli --server=github call \"$tool\"",
"args": {
"type": "object",
"required": ["tool"],
"properties": {
"tool": {"type": "string", "description": "MCP tool name"}
},
"additionalProperties": true
},
"scope": "global"
}
```
This works for any MCP server with a CLI bridge. See `doc/resources/no-mcp.md`
for why ollie uses this pattern instead of implementing MCP natively.
## Installation
Copy the `.meta` file to `~/.config/ollie/tools/`. The tool is live immediately
— no restart needed.
```sh
cp my_tool.meta ~/.config/ollie/tools/
```
## Notes
- `additionalProperties: true` allows pass-through arguments the schema doesn't enumerate.
- Prefix tool names to avoid collisions (`acme_deploy`, not `deploy`).
- Test cmd pipelines: `echo '{"pattern":"TODO"}' | bash -c 'YOUR_CMD'`
- Agent configs can auto-load tools via `autoLoad`.

View File

@ -292,5 +292,4 @@ while :; do git diff HEAD --; sleep 5; done | tee >(o myproj/security write feed
- **acme** — Plan 9 editor (scripts in `data/scripts/acme/`)
For technical details on the raw 9P filesystem, see [`doc/9p.md`](9p.md).
For tool authoring, see [`doc/writing-tools.md`](writing-tools.md).
For meta-only tool definitions (no script needed), see [`doc/tool-definitions.md`](tool-definitions.md).
For tool architecture and authoring, see [`architecture-tools.md`](architecture-tools.md).

View File

@ -1,265 +0,0 @@
# Writing Ollie Tools
A tool is any executable with a `.meta` sidecar file. That's it.
This is **declarative tool registration**. You don't write code to register a
tool, you don't edit config files, you don't restart a server. You write a
`.meta` file that declares:
- **What** the tool does (description, prompt, args schema)
- **Where** to find it (cmd, co-located executable, or PATH)
- **When** it's available (match conditions per variant)
The registry reads the `.meta` file, presents the tool to the model, and
executes the binary when called. The model sees a typed function. The runtime
sees an exec call. The `.meta` bridges the two.
The executable can be a bash script, a Python script, a compiled Go binary, a
Rust binary, a symlink to `/usr/bin/jq` — anything that reads JSON from stdin
and writes results to stdout.
No integration with the ollie source tree is required. Drop a `.meta` file in
`$OLLIE_TOOLS_PATH` and the tool is live on the next session.
## Quick start
```
~/.config/ollie/tools/
├── my_tool ← executable (any language)
└── my_tool.meta ← JSON metadata
```
That's a complete tool. The agent can now `tool_load("my_tool")` and call it.
## The .meta sidecar
The `.meta` file is the sole source of discovery and schema. The registry
reads only `.meta` files — it never inspects the executable itself.
```json
{
"description": "One-liner shown in tool listings.",
"prompt": "## my_tool\n\nFull documentation shown when loaded.\n\n**Args**: `path` (required)\n\n```\nmy_tool(path=\"/foo\")\n```",
"args": {
"type": "object",
"required": ["path"],
"properties": {
"path": {"type": "string", "description": "File path"}
}
},
"tier": "hot",
"readOnly": false
}
```
| Field | Type | Purpose |
|---|---|---|
| `description` | string | One-liner shown in the tool listing (system prompt) |
| `prompt` | string | Full documentation injected when the tool is loaded |
| `args` | JSON Schema | Input schema exposed to the model |
| `tier` | `"hot"\|"warm"\|"cold"` | Context retention tier (default: hot) |
| `readOnly` | bool | Safe for parallel execution with other read tools |
| `cmd` | string | Executable path or name (see Resolution below) |
| `variants` | array | Conditional definitions for heterogeneous hosts (see Variants below) |
## Executable resolution
The `cmd` field is optional. When the dispatcher needs to run a tool, it
resolves the executable in this order:
1. **`$OLLIE_TOOLS_PATH/<name>`** — if an executable with the tool's name
exists in the tools directory, use it. This always wins, allowing local
wrappers to shadow system binaries.
2. **`.meta` `cmd` field** — if set, use it:
- Absolute path → use directly
- Bare name → resolve via `$PATH`
3. **`$PATH` lookup** — final fallback, search PATH for the tool name.
This means:
- **Co-located scripts** (the common case): no `cmd` needed, just put the
executable next to the `.meta`.
- **System binaries**: write a `.meta` with `"cmd": "/usr/bin/rg"` or
`"cmd": "rg"` — no copying or symlinking required.
- **Wrappers**: put a script in `$OLLIE_TOOLS_PATH` that wraps a complex
binary with simpler arguments the model can understand. The wrapper shadows
the system binary automatically.
## Variants: conditional tool definitions
Tools can declare multiple **variants** gated by match conditions. The first
variant where all conditions pass determines the tool's schema, documentation,
and executable. If no variant matches, the tool is hidden from the registry —
it doesn't exist on this host.
This is the mechanism for **network transparency across heterogeneous hosts**.
One `.meta` file works on any machine; the capabilities adapt to what's
actually available.
### Example: system log reader
```json
{
"description": "Read system logs.",
"variants": [
{
"match": {"binary": "journalctl"},
"description": "Read system logs (journald).",
"prompt": "## system_logs\n\nFilter by unit, time, pattern.\n\n```\nsystem_logs(unit=\"sshd\", since=\"1 hour ago\", grep=\"error\")\n```",
"args": {
"type": "object",
"properties": {
"unit": {"type": "string", "description": "Systemd unit name"},
"since": {"type": "string", "description": "Time filter"},
"lines": {"type": "string", "description": "Number of lines"},
"grep": {"type": "string", "description": "Pattern filter"}
}
},
"cmd": "system_logs_journald",
"readOnly": true
},
{
"match": {"file": "/var/log/syslog"},
"description": "Read system logs (syslog).",
"prompt": "## system_logs\n\nTail syslog with optional grep.\n\n```\nsystem_logs(lines=\"50\", grep=\"error\")\n```",
"args": {
"type": "object",
"properties": {
"lines": {"type": "string", "description": "Number of lines"},
"grep": {"type": "string", "description": "Pattern filter"}
}
},
"cmd": "system_logs_syslog",
"readOnly": true
}
]
}
```
On a systemd host: model sees unit filtering, time ranges, grep. Runs `system_logs_journald`.
On Alpine/BSD: model sees tail + grep only. Runs `system_logs_syslog`.
On a host with neither: tool doesn't appear in the registry.
### Match conditions
All conditions in a match must be true (AND logic).
| Key | Semantics | Example |
|-----|-----------|---------|
| `binary` | Binary exists in `$PATH` (or absolute path stat) | `"binary": "journalctl"` |
| `file` | File or directory exists | `"file": "/var/log/syslog"` |
| `os` | `runtime.GOOS` matches | `"os": "linux"` |
| `arch` | `runtime.GOARCH` matches | `"arch": "amd64"` |
| `env` | Environment variable is non-empty | `"env": "DISPLAY"` |
| `nenv` | Environment variable is empty | `"nenv": "SSH_CONNECTION"` |
### Variant field merging
Variant fields override the top-level `.meta` fields where set. Unset variant
fields inherit from the top level. This means you can put common fields
(like `tier` or `readOnly`) at the top level and only override `args`, `prompt`,
and `cmd` per variant.
### Network transparency
When `ollie-remote` deploys to a remote host, it sends the `.meta` files.
The remote's tool registry resolves variants against that host's capabilities.
No configuration, no host-specific tool sets — declarations adapt automatically.
## Input/Output contract
**Input**: JSON object on stdin. Fields match the `args` schema in the `.meta`.
**Output (success)**: print result to stdout, exit 0.
**Output (error)**: print error to stdout or stderr, exit non-zero.
**Structured output**: return `{"content": [...]}` JSON to pass content blocks directly.
That's the entire protocol. The dispatcher pipes the model's tool-call arguments
to stdin and captures stdout. No flags, no argv, no environment negotiation.
## Examples by language
### Bash
```bash
#!/usr/bin/env bash
set -e
input=$(cat)
path=$(echo "$input" | jq -r '.path')
echo "result: $path"
```
### Python
```python
#!/usr/bin/env python3
import json, sys
args = json.load(sys.stdin)
path = args["path"]
print(f"result: {path}")
```
### Go
```go
package main
import (
"encoding/json"
"fmt"
"os"
)
func main() {
var args struct {
Path string `json:"path"`
}
json.NewDecoder(os.Stdin).Decode(&args)
fmt.Printf("result: %s\n", args.Path)
}
```
### Any other language
The contract is language-agnostic. If it's executable and reads JSON from
stdin, it's a valid tool.
## Sandboxing & Privilege Escalation
**Every tool runs inside a Landlock sandbox** by default. The sandbox profile
(`~/.config/ollie/sandbox/default.yaml`) controls filesystem and network access.
### Elevation: escape the sandbox
Tools that need to escape the sandbox declare `"bypass": true` in their
`.meta`, or the agent passes `"bypass": true` in the tool call. The
dispatcher requests approval from the **bypass broker**, which notifies the
user (desktop notification, D-Bus, etc.). If approved, the tool runs outside
Landlock but still as the current user.
```json
{
"description": "Write to a protected path.",
"bypass": true,
"cmd": "/usr/local/bin/my-tool"
}
```
## Environment Variables
Tools receive:
| Variable | Purpose |
|---|---|
| `OLLIE_TOOLS_PATH` | Path to the tools directory |
| `OLLIE_BYPASS_SOCKET` | Unix socket for bypass broker |
| `PWD` | Session's current working directory |
## Bundled examples
See `data/tools/` in the ollie repo:
- `file_read`, `file_edit`, `file_grep` — filesystem I/O (Python)
- `gui_*` — KDE desktop automation (Bash/Python)
- `lsp_*` — language server protocol (Go, compiled from `tools/lsp/cmd/`)
- `memory_*` — persistent OptMem memory tools; records are stored in OptMem's B-tree at `$XDG_DATA_HOME/ollie/optmem` (default: `~/.local/share/ollie/optmem`)