BlockingReadMode had the same 512-byte truncation bug: it returned EOF
on offset>0 before the full content was delivered. Now caches content
on the fid like normal reads.
StreamMode is separated into its own path since it has fundamentally
different semantics (each Tread blocks for the next chunk, no caching).
In 9P, clients issue multiple Treads at increasing offsets to read
files larger than the initial buffer (io.ReadAll starts at 512 bytes).
The server was calling entry.Read() on every Tread, which for Request
files cleared the result after the first call, causing truncation.
Fix: cache the Read() result on the fid (readCache). Subsequent Treads
at higher offsets serve from the cache via readSlice. The cache is
cleared on write (for rdwr files) so the next request-response cycle
gets fresh data. This mirrors how dirCache already works for directories.
The Request read handler was clearing the result after the first Read()
call. But io.ReadAll starts with a 512-byte buffer, so the client's
first Tread has count=512. If the response is larger, the client issues
a second Tread — but the result was already cleared, causing truncation.
Fix: keep the result available across multiple reads. It's replaced on
the next write (new request-response cycle).
One toolsrv per session means the session-scoped map was unnecessary
indirection. Registry now tracks a flat map of loaded tools with no
session ID parameter. This eliminates the class of bugs where sessID
could be empty or mismatched between Load and Loaded calls.
Shell doesn't expand ~ in programmatically-passed arguments, so
toolsrv received literal ~/src/ollie as --cwd. Go's os/exec fails
with ENOENT on chdir before exec, breaking all tool execution.
Expand ~ at two boundaries:
- main.go: immediately after flag parsing
- proc.go SetCWD(): for runtime cwd changes via 9P
Remove all cached tool state from agent:
- Remove toolRegistryRevision from Runtime
- Remove wireToolsChanged (OnToolsChanged callback)
- Remove toolsNeedRefresh (revision-based cache invalidation)
- Remove refreshToolListing (redundant with RefreshTools)
Tools are now fetched fresh from toolsrv at the start of every turn.
The preamble tools section is also rebuilt at turn start. No local
caching means tool_load/unload take effect immediately on the next
turn and /tools always shows the live state from toolsrv.
tool_load was loading on the session connection but tools listed from
the agent connection. Now both use ctx.Agent.ToolServer() so loads are
immediately visible. writeAgentTools also calls RefreshTools after load.
- Background procs get timeout=0 (no deadline) set by toolsrv at
the execution layer — not injected as an arg from olliesrv.
Foreground default remains 30s. User can still override via args.
- Timeout logic simplified: caller sets default, ExecuteTool honors it.
- System prompt: clarify background has no timeout, uses 'stop' not 'kill'
- architecture.md: document lifecycle (connection-based ownership)
- evolution.md: update interrupt section with streaming, lifecycle, id= format
- README.md: add parallel execution to capabilities table
- KillAll(): sends SIGKILL to all running proc groups on clean shutdown
- Pdeathsig: SIGKILL ensures child procs die if toolsrv crashes
- Shell tool: use subshell + set +e for streaming without exit propagation
- Fix integration test to match new error format
This is the connection-based ownership model: toolsrv death = proc death.
The session owns the toolsrv process, so session death cascades to all procs.
- Timeout: timeout=0 means no deadline (was defaulting to 30s)
- Signal: send to process group (-pgid) not just process; SIGTERM no longer
cancels context (only SIGKILL does); cmd.Cancel sends SIGTERM with 5s WaitDelay
- Streaming: background procs stream output in real-time via procWriter;
shell tool no longer buffers all output into a bash variable
- Proc tree: olliesrv exposes proc/{id}/out, proc/{id}/ctl, proc/{id}/status
as proper 9P directory (was broken flat file)
- Connection: proc handlers dial fresh toolsrv conn per request via
Session.DialToolServer() to avoid deadlocking the agent's blocked conn
- Stat format: key=value (exited=true, exit_code=N, id=N) matching client parser
- GC: procs auto-removed 10min after LastRead (exited procs only)
- Rename: PID -> ID throughout (synthetic, not OS PID)
- Ctl commands: term (SIGTERM), kill (SIGKILL), signal <n>, dismiss
- System prompt: correct ollie-9p commands for proc management
Dispatch-level flags are now injected into every tool's JSON schema
at runtime via injectDispatchFlags(). No need to declare them in
individual .meta files — they're universal.
Removed redundant declarations from shell.meta (now injected globally).
When a tool call includes "background": true, it executes via
proc/new.bg and returns immediately with a PID in a
<system-proc-background> tag. The agent tracks active background
processes and injects their output as <system-proc-interrupt> blocks
alongside subsequent tool results.
The model sees background updates without polling. It can react to
build failures, log events, etc. naturally. Kill via PID when done.
Implementation:
- toolsrv client: CallToolBackground (uses proc/new.bg as rdwr)
- toolsrv: proc/new.bg upgraded from write-only to request-response
- agent: bgTracker collects last 20 lines of output per proc
- agent loop: injects interrupts after execToolCalls, before h.update
- system prompt: documents the background mechanism and rules
Only tools that explicitly declare scope="write" get path-based
parallelism. Unset scope is now global (full barrier), avoiding the
"path is a lie" problem where a tool has a path arg but modifies
other files (e.g., lsp_rename).
Tools must opt in to parallelism:
- scope=read: never conflicts
- scope=write: conflicts on same path only
- scope=global (or unset): serialization barrier
Added scope=write to file_edit.meta and file_write.meta.
Rename ToolInfo.ReadOnly bool → ToolInfo.Scope string with three values:
- "read" — path-scoped read, never conflicts (always parallel)
- "write" — path-scoped write, conflicts on same file path only
- "global" — full serialization barrier, runs alone
Tools declare scope in their .meta file. If unset, inferred from path
arg presence (write if path exists, global otherwise).
This correctly classifies lsp_rename as global (cross-file workspace
edits) despite having a path argument. The path arg in lsp_rename is
a symbol coordinate, not a resource scope.
All .meta files updated: readOnly:true → scope:read,
readOnly:false → scope:global, lsp_rename gets scope:global.
Replace binary ReadOnly/not batching with path-based conflict detection.
Tool calls that operate on different file paths now execute in parallel,
while same-path writes and shell (global barrier) serialize.
Conflict rules:
- ReadOnly tools: never conflict (unchanged from before)
- Write tools with path arg: conflict only on same path
- Shell / unknown (no path): global barrier, runs alone
This parallelizes the common refactoring case where the model edits
multiple files in one turn (e.g., 5 file_edits on different files
complete in 1 round-trip instead of 5).
Safety argument:
- LLM API is turn-based: model reads files first, plans edits based
on known content, emits writes in a subsequent turn
- file_edit carries old_string context: self-contained per-file
- Same-path conflicts serialize (detected via extractFilePath)
- Shell is conservative: always a full serialization barrier
- Move toolsrv logic into internal/{fs,exec,registry} packages to match
olliesrv's structure. server.go (9P protocol) stays at top level.
- Deduplicate parseKV/parsePayload: remove dead code from old spec9p.go,
single implementation in internal/fs/parse.go.
- Add structured logging via log/slog (replaces fmt.Fprintf to stderr).
- Standardize qid path hashing to fnv.New64a (matches olliesrv).
- Fix toolsrv process leaks:
- Add Pdeathsig to local spawn so toolsrv dies when olliesrv exits.
- Add idle timeout (30s with no connections after first client seen).
- ProcessKeeper.Dial() kills old process before respawning.
Major changes:
- toolsrv now uses 9P protocol instead of JSON-RPC
- Clean package split: toolsrv/ (client SDK) vs cmd/toolsrv/ (server)
- tool_load is now a built-in tool (not sandbox-restricted)
- New toolclient package for process spawning and lifecycle
- Remote execution via SSH socket forwarding with Tauth authentication
21 commits from toolsrv-9p branch.
Split toolsrv/registry.go:
- cmd/toolsrv/registry.go: Registry type with session-scoped tool state
(Load, Unload, Loaded, Lookup, Revision methods)
- toolsrv/registry.go: shared types only (ToolInfo, ToolsPath, DiscoverTools)
The toolsrv/ package is now a pure client SDK:
- Dial, Conn for 9P connection
- ToolInfo, ToolResult, HostInfo types
- DiscoverTools for listing available tools
Server-only code lives in cmd/toolsrv/:
- Registry for session-scoped tool state
- 9P handlers and execution logic
- Sandbox integration
Move spawn.go from toolsrv/ to cmd/olliesrv/internal/toolclient/:
- Process, ProcessKeeper, Spawn, SpawnRemote now in toolclient package
- toolsrv/ now only contains client code (Dial, Conn, etc.)
This clarifies the architecture:
- toolsrv/ = client SDK for connecting to toolsrv
- cmd/toolsrv/ = the toolsrv server
- cmd/olliesrv/internal/toolclient/ = olliesrv's toolsrv process management
Removed ProcessKeeper tests from toolsrv integration tests since they
now belong to toolclient.
tool_load must be handled specially by the agent runtime because tools
run inside toolsrv's sandbox and cannot load other tools into
themselves. The runtime now intercepts tool_load calls and directly
invokes ToolServer.LoadTool().
This is the only built-in tool - all others are external scripts.
Add /session/{sid}/agent/{aid}/tools file:
- Read: list loaded tools for this agent
- Write: load a tool by name
Also add missing tool_load to justfile install.
HandleCtl was only handling load/unload commands. The client sends
'env KEY=VALUE' and 'cwd PATH' via SetEnv() and SetCWD(), but toolsrv
was returning 'unknown command' errors. This broke tool_load and any
tool that depends on OLLIE_SESSION_ID/OLLIE_UNAME env vars.
Bug: spec9p.go was double-processing payloads - parseKV unescaped
\n to newlines, then rebuilt the payload without re-escaping,
then fs9p.go's parsePayload tried to unescape again. Result:
multiline content like file contents was truncated to first line.
Fix: Pass raw payload data through to fs9p.go. Escaping/unescaping
happens once in parsePayload.
Also:
- Add proper one-pass unescape function (handles \\n correctly)
- Add TestIntegration_MultilineContent to verify fix
- Add InputSchema verification to TestIntegration_ToolExecution
- Pass session ID via --session-id flag (not env var) so tool registry is configured when olliesrv spawns toolsrv
- /tools now returns JSON array with full ToolInfo including InputSchema (fixes Bedrock validation error requiring type: object)
- Implement SpawnRemote: deploy ~/.config/ollie via tarball over SSH, set XDG_CONFIG_HOME so tools/*.meta are found on remote
- Add --no-auth flag for debugging Tauth issues
Tests now skip gracefully in CI environments where ~/.config/ollie/tools
doesn't exist, instead of failing with 'no tool registry configured'.
Also added OLLIE_SESSION_ID to test server startup.
- Parse ToolResult JSON the same way agent/runtime.go does
- Verify Content structure, Type field, and IsError flag
- Add TestIntegration_ToolExecutionError for failed commands
- Tests now verify the wire format matches what olliesrv expects
After Write(), the file offset is at the end of the written data.
For request-response files (like proc/new), we need to read the
result from offset 0, not from the current offset.
Changed CallTool() to use ReadAt(buf, 0) instead of io.ReadAll()
which uses Read() and inherits the wrong offset.
Same fix was already applied to token reading in Dial().
Added TestIntegration_ToolExecution to verify the full tool
execution path works end-to-end.
Replace HMAC-based registration with proper 9P Tauth flow:
- First client to connect sets the secret via Tauth afid write
- Server stores secret in memory, returns session token
- Subsequent clients must provide matching secret
- Secret never in env vars or disk, only transmitted over socket
Changes:
- server9p.go: Authenticate() replaces RegisterAgent/GetAgent
- client9p.go: Dial() does Tauth handshake (write secret, read token)
- spawn.go: generates secret, no longer passes via TOOLSRV_SECRET env
- auth9p.go: simplified to just GenerateSecret()
- spec9p.go: removed /register file, token validation at connection level
- cmd/toolsrv/server.go: handleAuth/handleAuthWrite/handleAuthRead
Security model:
- Session generates secret on Spawn()
- All agents share session's secret via ProcessKeeper
- Socket permissions (local) or SSH (remote) protect transport
- Secret dies with toolsrv process, new secret on respawn
Added integration tests verifying:
- First connection sets secret
- Reconnect with same secret works
- Wrong secret rejected
- ProcessKeeper reconnect/respawn behavior
- Concurrent connections
- server.go: Full 9P2000 protocol handling
- version, auth, attach, walk, open, read, write, clunk, stat
- Uses fsedsl.Tree for all operations
- Handles directories and files
- Supports rdwr files (RequestRespMode)
- main.go: Remove placeholder serve9P, use new implementation
The toolsrv binary now serves a working 9P filesystem with the
namespace defined in toolsrv/spec9p.go:
/register, /ctl, /tools, /proc/*, /info
- spec9p.go: fsedsl-based namespace specification
- ToolsrvCtx with Server, Agent, Proc
- Handlers as package-level functions
- Follows fs/spec.go pattern
- server9p.go: simplified to state management only
- Agent registration with HMAC tokens
- BuildTree() creates fsedsl tree
- Removed redundant 9P protocol handling
The 9P protocol serving will use the same pattern as olliesrv
(cmd/olliesrv/server.go), which takes a *fsedsl.Tree and handles
all 9P message types generically.
Ollie Agent
Levi Neely