quirks: reject shell calls to native tools

- Add quirks package for stupid model behavior workarounds
- ShellInvokesNativeTool blocks shell(cmd="tool_name") patterns
- Add client_9p tool: native wrapper for ollie-9p operations
- Block ollie-9p in shell — use client_9p instead
- Update all prompts to use client_9p, not shell+ollie-9p
- Clarify 9P namespace is complete (tools are NOT in 9P)
- Registry.All() lists all available tools for validation
This commit is contained in:
Levi Neely 2026-08-20 14:08:53 +02:00
parent ec6db39722
commit cd9da5ed27
13 changed files with 193 additions and 71 deletions

View File

@ -578,6 +578,7 @@ func (ag *Agent) execOne(ctx context.Context, call backend.ToolCall) (toolResult
ag.emit(Event{Role: "tool", Name: call.Name, Content: cr.Content, OutputFormat: toolOutputFormat(rt, call.Name)})
return cr, true
}
ag.emit(Event{Role: "call", Name: call.Name, Content: string(call.Arguments)})
readSafe := rt.ToolMeta[call.Name].Scope == "read"

View File

@ -59,38 +59,20 @@ You have autonomous access to tools and skills. These are two different things:
## Tools
Your loaded tools are visible in your context. Use them without hesitation:
- Need to understand code? Call `file_read`, `code_outline`, `codebase_overview`.
- Need to find something? Call `file_grep`, `file_glob`, `code_symbols`.
- Need to change code? Call `file_edit`.
- Need to run a command? Call `shell`.
Tools are in your dispatch — call them directly by name. They are **not** 9P files; the 9P namespace (below) shows exactly what's in 9P.
If a tool you need isn't loaded, load it via ctl or ask — but prefer using what's available.
**Tool preference** (most specific first):
1. **LSP** — `lsp_rename`, `lsp_definition`, `lsp_references`, `lsp_hover`, `lsp_diagnostics`
2. **Code** — `code_outline`, `code_symbols`, `code_query`, `codebase_overview`
3. **File** — `file_read`, `file_edit`, `file_grep`, `file_glob`
4. **9P** — `client_9p` for plans, ctl, peers, and other namespace operations
5. **Shell** — build, test, git, and project binaries
**Tool preference order** (use the most specific tool available):
1. **LSP tools** — `lsp_rename`, `lsp_definition`, `lsp_references`, `lsp_hover`, `lsp_diagnostics` — semantic, language-aware, cross-file safe
2. **Code tools** — `code_outline`, `code_symbols`, `code_query`, `codebase_overview` — structural, AST-aware, faster than reading files
3. **File tools** — `file_read`, `file_edit`, `file_grep`, `file_glob` — direct file access, use when code tools don't fit
4. **Shell** — `git`, `make`, `npm`, `cargo`, `python`, `ollie-9p`, etc. — for commands with no native tool equivalent
Examples:
- Rename a function → `lsp_rename`, not find-and-replace with `file_edit`
- Find all callers → `lsp_references`, not `file_grep`
- Understand a file's structure → `code_outline`, not `file_read` the whole thing
- Search for a symbol → `code_symbols`, not `grep`
- Read a specific section → `file_read` with line range, not `shell(cmd="sed -n ...")`
Shell is a fallback. If you're shelling out to `cat`, `grep`, `sed`, or `find`, check whether a native tool does it better.
If a native tool exists for an operation, use it. Shell calls that invoke native tools will be rejected.
## Parallel Calls
Make independent tool calls in the same turn whenever possible. Non-conflicting calls execute in parallel automatically:
- Multiple reads (file_read, file_grep, file_glob) — always parallel.
- Multiple writes to different files (file_write, file_edit) — parallel.
- Shell is always sequential (one at a time).
Do NOT serialize calls that could be parallel. If you need to read 3 files, call file_read 3 times in one turn, not across 3 turns.
Independent tool calls in the same turn run in parallel. Don't serialize reads across turns.
## Skills
@ -134,8 +116,8 @@ Background processes have no timeout — they run until they exit, you stop them
**Rules**:
- Use background for long operations (builds, test suites, deployments, log tailing). Do NOT background short commands (< 5s).
- React to `<system-proc-interrupt>` naturally. If a build fails, fix it. If output is irrelevant, stop the process.
- Stop background processes when done: `echo term | ollie-9p write session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/proc/<id>/ctl` (use `kill` instead of `term` to force-kill)
- Read full output: `ollie-9p read session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/proc/<id>/out`
- Stop background processes: `client_9p(op="write", path="session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/proc/<id>/ctl", data="term")` (use `kill` instead of `term` to force-kill)
- Read full output: `client_9p(op="read", path="session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/proc/<id>/out")`
# Security
@ -174,7 +156,9 @@ Parallel sessions on this machine are all you, and may all write memories. A sub
# 9P Filesystem
Your world model is a 9P filesystem. Your session ID is `${OLLIE_SESSION_ID}`. Use `ollie-9p` for all filesystem operations.
Your world model is a 9P filesystem. The tables below show the **complete namespace** — nothing is hidden. If a path isn't listed here, it doesn't exist. Tools are NOT in the 9P namespace; they're in your dispatch.
Use `client_9p` for all 9P operations. Your session ID is `${OLLIE_SESSION_ID}`.
## Root Namespace
@ -215,36 +199,28 @@ Your world model is a 9P filesystem. Your session ID is `${OLLIE_SESSION_ID}`. U
## Operations
```bash
# Read/write your plan
ollie-9p read session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/plan
echo "## Plan\n- [ ] step one" | ollie-9p write session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/plan
Use `client_9p` for all 9P operations — never shell out to `ollie-9p`.
# Load a tool
echo "file_read" | ollie-9p write session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/tools
```
# Read/write your plan
client_9p(op="read", path="session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/plan")
client_9p(op="write", path="session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/plan", data="## Plan\n- [ ] step one")
# Control commands
echo "compact" | ollie-9p write session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/ctl
echo "model gpt-4o" | ollie-9p write session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/ctl
# Spawn a peer session
printf 'name=worker\ncwd=%s\n' "$PWD" | ollie-9p write session/new
# Send a prompt to another agent
echo "fix the bug" | ollie-9p write session/{sname}/agent/{aname}/prompt
client_9p(op="rdwr", path="session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/ctl", data="compact")
client_9p(op="rdwr", path="session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/ctl", data="model gpt-4o")
# One-shot generation (no session)
echo "summarize this" | ollie-9p rdwr generate
client_9p(op="rdwr", path="generate", data="summarize this")
```
## Sub-Agents
Delegate work to a sub-agent by writing to `agent/new` with a `prompt=` key. The call blocks until the sub-agent finishes and returns its reply. The sub-agent is destroyed automatically.
Delegate work to a sub-agent using the `subagent_spawn` tool. The call blocks until the sub-agent finishes and returns its reply. The sub-agent is destroyed automatically.
```bash
# Spawn a sub-agent, block until done, get the reply
printf 'cwd=%s\nprompt=fix the failing tests in auth/\n' "$PWD" \
| ollie-9p rdwr session/$OLLIE_SESSION_ID/agent/new
```
subagent_spawn(prompt="fix the failing tests in auth/")
subagent_spawn(prompt="fix the failing tests in auth/", cwd="/path/to/project")
```
The sub-agent gets its own context window and tool access and runs independently. When spawned from an existing agent, it starts with a snapshot of the parent's conversation history. The snapshot is copied once at spawn time; it is not a live shared context. The child cannot append to, rewrite, or otherwise mutate the parent's history. Its final reply is returned to the parent as the tool result, and the parent decides how to use that reply. Use sub-agents for:
@ -259,15 +235,12 @@ Optional parameters: `profile=`, `backend=`, `model=`, `name=`, `timeout=`, `max
- `max_depth=1` — sub-agents cannot spawn their own sub-agents by default.
- `max_parallel=5` — up to 5 concurrent sub-agents per parent (set to -1 for unlimited).
Multiple sub-agents can run in parallel — writes to the same file path are automatically serialized by the tool server. To run sub-agents concurrently, make multiple shell calls in the same turn:
Multiple sub-agents can run in parallel — writes to the same file path are automatically serialized by the tool server. To run sub-agents concurrently, make multiple `subagent_spawn` calls in the same turn:
```bash
# Each of these blocks independently; call them in parallel tool calls
printf 'cwd=%s\nprompt=fix tests in auth/\n' "$PWD" \
| ollie-9p rdwr session/$OLLIE_SESSION_ID/agent/new
printf 'cwd=%s\nprompt=fix tests in api/\n' "$PWD" \
| ollie-9p rdwr session/$OLLIE_SESSION_ID/agent/new
```
# These run in parallel automatically when called in the same turn
subagent_spawn(prompt="fix tests in auth/")
subagent_spawn(prompt="fix tests in api/")
```
## Peers
@ -276,19 +249,18 @@ Peer agents are persistent agents in the same session that can communicate direc
The `peer/` directory lists agents you can message. You can only write to agents that are your peers — this is the access control mechanism.
```bash
```
# Send a message to a peer
echo "I've finished my analysis. Here are my findings: ..." \
| ollie-9p write session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/peer/{peername}
client_9p(op="write", path="session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/peer/{peername}", data="I've finished my analysis...")
# List your peers
echo "peers" | ollie-9p rdwr session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/ctl
client_9p(op="rdwr", path="session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/ctl", data="peers")
# Add a peer (bidirectional — both agents get the link)
echo "peeradd othername" | ollie-9p rdwr session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/ctl
client_9p(op="rdwr", path="session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/ctl", data="peeradd othername")
# Remove a peer (bidirectional)
echo "peerdel othername" | ollie-9p rdwr session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/ctl
client_9p(op="rdwr", path="session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/ctl", data="peerdel othername")
```
When you receive a message from a peer, it arrives as a prompt. Respond by writing to your peer link for that agent. Peers are constrained to the current session — cross-session communication is not supported.

View File

@ -0,0 +1,51 @@
// Package quirks contains workarounds for stupid model behaviors.
package quirks
import "strings"
// ShellInvokesNativeTool checks if a shell command contains a native tool name.
// Returns the tool name if found, empty string otherwise.
//
// This is a hack for broken models that ignore prompt instructions and shell
// out to tools instead of calling them natively.
func ShellInvokesNativeTool(cmd string, tools []string) string {
// Reject ollie-9p in shell — use client_9p native tool instead.
if strings.Contains(cmd, "ollie-9p") {
return "client_9p"
}
// Check if cmd contains any tool name as a word boundary match.
// Match: "skill_list", "echo foo | skill_list", "skill_list arg"
// Don't match: "my_skill_list" (substring)
for _, name := range tools {
if name == "shell" {
continue // don't match shell itself
}
if matchesWordBoundary(cmd, name) {
return name
}
}
return ""
}
// matchesWordBoundary checks if name appears in cmd at word boundaries.
func matchesWordBoundary(cmd, name string) bool {
idx := strings.Index(cmd, name)
if idx < 0 {
return false
}
// Check left boundary: start of string or non-word char
if idx > 0 && isWordChar(cmd[idx-1]) {
return false
}
// Check right boundary: end of string or non-word char
end := idx + len(name)
if end < len(cmd) && isWordChar(cmd[end]) {
return false
}
return true
}
func isWordChar(c byte) bool {
return (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') || c == '_'
}

View File

@ -118,3 +118,14 @@ func (r *Registry) Revision(agentID string) uint64 {
defer r.mu.RUnlock()
return r.revisions[agentID]
}
// All returns all available tool names (from the tools directory).
func (r *Registry) All() []string {
tools := r.discover()
names := make([]string, 0, len(tools))
for name := range tools {
names = append(names, name)
}
sort.Strings(names)
return names
}

View File

@ -17,6 +17,7 @@ import (
"time"
"ollie/cmd/toolsrv/internal/exec"
"ollie/cmd/toolsrv/internal/quirks"
"ollie/cmd/toolsrv/internal/registry"
"ollie/toolsrv/protocol"
"ollie/util"
@ -285,6 +286,15 @@ func (st *State) NewProc(ctx context.Context, payload string, background bool) (
}
}
// Reject shell calls that invoke a native tool.
// This is a hack for stupid, broken models that ignore prompt instructions
// and shell out to tools instead of calling them natively.
if toolName == "shell" {
if conflicting := quirks.ShellInvokesNativeTool(args["cmd"], reg.All()); conflicting != "" {
return "", 0, fmt.Errorf("STOP. You called shell to run '%s', but '%s' is a native tool. Call it directly: %s(...). NEVER use shell to invoke native tools", conflicting, conflicting, conflicting)
}
}
// Create proc context. For background procs, detach from request context
// so the process continues after the 9P request returns.
var procCtx context.Context

View File

@ -6,7 +6,7 @@ You are an author agent in a review workflow. You produce work to satisfy a goal
2. **Submit to reviewer.** Send your complete work via peer link:
```bash
echo "<your complete work>" | ollie-9p write session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/peer/reviewer
client_9p(op="write", path="session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/peer/reviewer", data="<your complete work>")
```
3. **Incorporate feedback.** When the reviewer sends feedback, revise your work to address every point. If you disagree with a point, explain why in the revised submission.

View File

@ -27,7 +27,7 @@ You are a coding agent.
- Before non-trivial tasks: inspect, plan, act, verify, update plan.
- Verify means: run the build, run affected tests. Not "looks right to me".
- Keep plans short, explicit, and task-focused.
- Use `ollie-9p write` to maintain a plan: `echo "## Plan\n..." | ollie-9p write session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/plan`
- Use `client_9p` to maintain a plan: `client_9p(op="write", path="session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/plan", data="## Plan\n...")`
- Update the plan after major progress, blockers, or changes in approach.
# Constraints

View File

@ -13,7 +13,7 @@ You are a panelist agent in a consensus workflow. You independently analyze a go
4. **Report to the foreman.** When your analysis is complete, send it via your peer link:
```bash
echo "<your full analysis>" | ollie-9p write session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/peer/foreman
client_9p(op="write", path="session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/peer/foreman", data="<your full analysis>")
```
# Constraints

View File

@ -8,7 +8,7 @@ You are a reviewer agent in a review workflow. You evaluate work submitted by th
- If the work satisfies the goal: write "complete" to `session/$OLLIE_SESSION_ID/goalstatus`.
- If the work needs improvement: send specific feedback via peer link:
```bash
echo "<feedback>" | ollie-9p write session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/peer/author
client_9p(op="write", path="session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/peer/author", data="<feedback>")
```
3. **Track rounds.** Count how many times you have sent feedback. Your initial prompt specifies the maximum. On the final allowed round, you MUST approve regardless of remaining issues.

View File

@ -1,3 +1,5 @@
[Style: visual-first (diagram before prose for architecture/flow/state/code), brief responses, direct voice, structured Markdown.]
[Style: visual-first, brief, direct, structured Markdown.]
[Behavior: Load necessary tools or skills autonomously to complete the task. Always run tools natively.]
[Behavior: Load tools/skills autonomously. Call native tools directly — shell calls to native tools are rejected.]
[MANDATORY: Before saying "I can't" or "I don't have access" — call `skill_list()` and check `tools_all` via ctl. A skill or tool probably exists. NEVER give up without checking.]

View File

@ -11,6 +11,7 @@
# Subcommands:
# read <path> Read a file (-l to loop)
# write <path> [data] Write to a file (args or stdin)
# rdwr <path> [data] Write then read (request-response files)
# ls [path] List namespace entries
# new [cwd] Create session (if needed) and agent (agent-level)
# ctl <command> Send a control command
@ -34,7 +35,7 @@ CTX_SESSION="" # Session name if in session/agent context
CTX_AGENT="" # Agent name if in agent context
# ── Known commands (to distinguish from context) ─────────────────
COMMANDS="read write ls new ctl prompt log tui env generate help"
COMMANDS="read write rdwr ls new ctl prompt log tui env generate help"
is_command() {
local arg="$1"
@ -99,6 +100,21 @@ cmd_write() {
fi
}
cmd_rdwr() {
if [[ $# -lt 1 ]]; then
echo "usage: o rdwr <path> [data...]" >&2
exit 1
fi
local file="$1"; shift
local path
path=$(resolve_path "$file")
if [[ $# -gt 0 ]]; then
printf '%s\n' "$*" | ollie-9p rdwr "$path"
else
ollie-9p rdwr "$path"
fi
}
cmd_ls() {
local path="${PREFIX}"
if [[ $# -ge 1 ]]; then
@ -355,6 +371,7 @@ Commands (availability depends on context):
Common:
read [-l] <path> Read a file (-l: loop)
write <path> [data] Write to a file
rdwr <path> [data] Write then read (request-response)
env Print export statements for context
help Show this help
@ -377,6 +394,7 @@ dispatch_cmd() {
case "$cmd" in
read) cmd_read "$@" ;;
write) cmd_write "$@" ;;
rdwr) cmd_rdwr "$@" ;;
ls) cmd_ls "$@" ;;
generate) cmd_generate "$@" ;;
ctl) cmd_ctl "$@" ;;

34
data/tools/client_9p Executable file
View File

@ -0,0 +1,34 @@
#!/bin/bash
# client_9p - 9P client tool wrapping ollie-9p
set -euo pipefail
# Read JSON args from stdin
args=$(cat)
op=$(echo "$args" | jq -r '.op // empty')
path=$(echo "$args" | jq -r '.path // empty')
data=$(echo "$args" | jq -r '.data // empty')
if [[ -z "$op" ]]; then
echo "error: missing 'op' (read, write, rdwr)" >&2
exit 1
fi
if [[ -z "$path" ]]; then
echo "error: missing 'path'" >&2
exit 1
fi
case "$op" in
read)
ollie-9p read "$path"
;;
write)
echo -n "$data" | ollie-9p write "$path"
;;
rdwr)
echo -n "$data" | ollie-9p rdwr "$path"
;;
*)
echo "error: unknown op '$op' (use read, write, rdwr)" >&2
exit 1
;;
esac

23
data/tools/client_9p.meta Normal file
View File

@ -0,0 +1,23 @@
{
"description": "9P filesystem client for reading/writing agent state (plans, ctl, peers, etc.)",
"prompt": "## client_9p\n\nRead and write 9P namespace paths. Use instead of shelling out to ollie-9p.\n\n**Operations:**\n- `read`: read file contents\n- `write`: write data to file\n- `rdwr`: write then read (for request-response files like ctl, generate)\n\n**Examples:**\n```\nclient_9p(op=\"read\", path=\"session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/plan\")\nclient_9p(op=\"write\", path=\"session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/plan\", data=\"## Plan\\n- [ ] step one\")\nclient_9p(op=\"rdwr\", path=\"session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/ctl\", data=\"compact\")\n```",
"args": {
"type": "object",
"required": ["op", "path"],
"properties": {
"op": {
"type": "string",
"enum": ["read", "write", "rdwr"],
"description": "Operation: read, write, or rdwr"
},
"path": {
"type": "string",
"description": "9P path (e.g., session/$OLLIE_SESSION_ID/agent/$OLLIE_UNAME/plan)"
},
"data": {
"type": "string",
"description": "Data to write (required for write/rdwr)"
}
}
}
}