Commit Graph

1992 Commits

Author SHA1 Message Date
Levi Neely 565acfb07b toolsrv: improve integration tests to match real usage
- Parse ToolResult JSON the same way agent/runtime.go does
- Verify Content structure, Type field, and IsError flag
- Add TestIntegration_ToolExecutionError for failed commands
- Tests now verify the wire format matches what olliesrv expects
2026-08-10 19:15:56 +02:00
Levi Neely c9789de2bd toolsrv: fix ReadAt offset for request-response files
After Write(), the file offset is at the end of the written data.
For request-response files (like proc/new), we need to read the
result from offset 0, not from the current offset.

Changed CallTool() to use ReadAt(buf, 0) instead of io.ReadAll()
which uses Read() and inherits the wrong offset.

Same fix was already applied to token reading in Dial().

Added TestIntegration_ToolExecution to verify the full tool
execution path works end-to-end.
2026-08-10 19:14:09 +02:00
Levi Neely 11c67a7fb1 toolsrv: implement 9P Tauth authentication
Replace HMAC-based registration with proper 9P Tauth flow:
- First client to connect sets the secret via Tauth afid write
- Server stores secret in memory, returns session token
- Subsequent clients must provide matching secret
- Secret never in env vars or disk, only transmitted over socket

Changes:
- server9p.go: Authenticate() replaces RegisterAgent/GetAgent
- client9p.go: Dial() does Tauth handshake (write secret, read token)
- spawn.go: generates secret, no longer passes via TOOLSRV_SECRET env
- auth9p.go: simplified to just GenerateSecret()
- spec9p.go: removed /register file, token validation at connection level
- cmd/toolsrv/server.go: handleAuth/handleAuthWrite/handleAuthRead

Security model:
- Session generates secret on Spawn()
- All agents share session's secret via ProcessKeeper
- Socket permissions (local) or SSH (remote) protect transport
- Secret dies with toolsrv process, new secret on respawn

Added integration tests verifying:
- First connection sets secret
- Reconnect with same secret works
- Wrong secret rejected
- ProcessKeeper reconnect/respawn behavior
- Concurrent connections
2026-08-10 19:03:10 +02:00
Levi Neely 75f364dd85 cmd/toolsrv: implement 9P protocol server
- server.go: Full 9P2000 protocol handling
  - version, auth, attach, walk, open, read, write, clunk, stat
  - Uses fsedsl.Tree for all operations
  - Handles directories and files
  - Supports rdwr files (RequestRespMode)

- main.go: Remove placeholder serve9P, use new implementation

The toolsrv binary now serves a working 9P filesystem with the
namespace defined in toolsrv/spec9p.go:
  /register, /ctl, /tools, /proc/*, /info
2026-08-10 17:53:01 +02:00
Levi Neely 427cadeb29 toolsrv: remove JSON-RPC, add compat stubs for 9P transition
BREAKING CHANGE: Removes all JSON-RPC toolsrv code.

Deleted files:
- server.go, rpcserver.go, rpcwire.go (JSON-RPC server)
- conn.go, dial.go (JSON-RPC client)
- exec.go, stream.go, detach.go (old execution tied to Server)
- spawn.go, transport.go, processkeeper.go, remote.go (spawning)
- shell_validate.go (tied to old Server)

New/modified:
- compat.go: Stub types to keep agent/session/fs packages compiling
  - Conn, Process, ProcessKeeper with TODO implementations
  - Spawn, SpawnRemote stubs
  - HostInfo, RemoteConfig, Option types
  - All marked TODO for 9P implementation

- cmd/toolsrv/main.go: Updated to use 9P model
  - Requires TOOLSRV_SECRET env var
  - Builds fsedsl tree
  - serve9P placeholder for 9P protocol handling

- exec9p.go: Added ToolResult types, limitedWriter, plan9Namespace

The codebase compiles but toolsrv is non-functional until:
1. serve9P implements 9P protocol
2. Conn stubs are replaced with lib9p client
3. Spawn/ProcessKeeper spawn 9P server
2026-08-10 16:48:40 +02:00
Levi Neely 6fc7cf999b toolsrv: use fsedsl for namespace spec
- spec9p.go: fsedsl-based namespace specification
  - ToolsrvCtx with Server, Agent, Proc
  - Handlers as package-level functions
  - Follows fs/spec.go pattern

- server9p.go: simplified to state management only
  - Agent registration with HMAC tokens
  - BuildTree() creates fsedsl tree
  - Removed redundant 9P protocol handling

The 9P protocol serving will use the same pattern as olliesrv
(cmd/olliesrv/server.go), which takes a *fsedsl.Tree and handles
all 9P message types generically.
2026-08-10 15:52:34 +02:00
Levi Neely d37404ebd3 toolsrv: add 9P server implementation
New files:
- server9p.go: 9P server for tool execution
  - Agent registration with HMAC-signed tokens
  - Token → cwd mapping for secure execution context
  - Full 9P protocol handling (version, attach, walk, open, read, write, stat, clunk)
  - Namespace: /register, /ctl, /tools, /proc/*, /info

- auth9p.go: Authentication helpers
  - GenerateSecret(): 32-byte random secret
  - ComputeRegistrationSig(): HMAC-SHA256 signature for registration

- server9p_test.go: Unit tests for auth and server

Security model:
- olliesrv spawns toolsrv with shared secret (env or file)
- Agent registration: olliesrv signs (agentID, cwd) → token
- Every tool call includes token; toolsrv verifies and looks up cwd
- Prevents unauthorized cwd manipulation from untrusted actors (LLM)
2026-08-10 15:43:20 +02:00
Levi Neely 1cd158926a rename ollie-remote to toolsrv
- cmd/ollie-remote -> cmd/toolsrv
- Update all references in toolsrv package
- Update justfile build targets
- Binary now installed as ~/.local/bin/toolsrv
2026-08-10 15:01:42 +02:00
Levi Neely ced1389549 toolsrv: add 9P-based tool server foundation
New files:
- fs9p.go: FS9P struct with /proc-based execution model
  - /proc/new (rdwr): blocking tool execution
  - /proc/new.bg: background/detached execution
  - /proc/<pid>/out, wait, stat, ctl: process management
  - /tools, /ctl, /info: tool registry and host info
  - Payload format: key=value lines, newline delimited

- exec9p.go: Decoupled tool execution
  - ExecuteTool: standalone function for tool execution
  - executeSandboxed: sandbox execution without Server deps
  - executeBypassDirect: bypass broker without Server deps

- fs9p_test.go: Unit tests for FS9P

Design notes:
- Background execution is the internal model
- Foreground (/proc/new rdwr) blocks until completion
- No per-connection state (cwd/env passed per-call or at attach)
- Session ID scoping via registry, not env propagation

This is the foundation - actual 9P serving and integration pending.
2026-08-10 14:53:13 +02:00
Levi Neely 9c3e935a7e agent: fix tool_load ctl to refresh agent's tool listing
The ctl tool_load command was loading tools into the registry but
not notifying the agent to refresh its preamble. Add RefreshTools()
method and call it after loading a tool via ctl.
2026-08-10 13:56:59 +02:00
Levi Neely 82106e89c2 agent: fix /command routing to use new o syntax
Use 'o session/agent ctl' instead of env vars, which the new
o script no longer supports for root-level command dispatch.
2026-08-10 13:51:47 +02:00
Levi Neely 6c26f15840 o: support root-relative paths with leading /
Paths starting with / are treated as root-relative, ignoring
the current context prefix. This allows accessing root-level
files/directories even when in a session or agent context.

Example: o myproj/coding ls /bypass  # list root bypass/ dir
2026-08-10 13:47:10 +02:00
Levi Neely d621e7322a o: use session/agent slash syntax for context
Simplify context parsing:
- o <cmd>                   Root level
- o <session> <cmd>         Session level
- o <session>/<agent> <cmd> Agent level

The slash disambiguates between session and agent context,
eliminating the need for 'session' and 'agent' keywords.

Examples:
  o myproj ls               # session level
  o myproj/coding prompt    # agent level
2026-08-10 13:40:26 +02:00
Levi Neely 14a52236c8 o: hierarchical context selectors
Refactor o CLI to use explicit context selectors:
- o <cmd>                       Root level
- o session <sname> <cmd>       Session level
- o agent <sname> <aname> <cmd> Agent level

Environment variables $session and $agent can substitute for
positional args for backwards compatibility.

Update acme scripts to use new interface:
  o agent $OLLIE_SESSION $OLLIE_AGENT prompt
2026-08-10 13:37:29 +02:00
Levi Neely 6937dda94a acme: auto-create session/agent like kate plugin
Add acme-ollie-ensure helper that:
- Creates 'acme' session if it doesn't exist
- Creates agent named by cwd hash (like kate's approach)
- Updates agent cwd if it already exists

All acme commands now call acme-ollie-ensure before sending
prompts, so they work without manual session setup.
2026-08-10 11:37:38 +02:00
Levi Neely e667689729 Revert "acme: use o generate instead of o prompt"
This reverts commit 9d7230452c.
2026-08-10 11:36:08 +02:00
Levi Neely 9d7230452c acme: use o generate instead of o prompt
One-shot generation doesn't require session/agent context.
2026-08-10 11:33:57 +02:00
Levi Neely abbd718e6a acme: add ollie integration commands
Add acme scripts for interacting with ollie:
- AskFile: describe the current file
- Explain: explain selected code
- Document: add documentation comments
- Fix: fix selected code
- Refactor: refactor selected code
- AddTests: write unit tests for selection
- SendVerbatim: send selection as-is

Scripts use the acme 9P filesystem to read selections and pipe
prompts to ollie via 'o prompt'.

Also update justfile to install/uninstall the acme scripts.
2026-08-10 11:29:59 +02:00
Levi Neely 46524d37dd kiro: wire session CWD through to EnvState; o: add new command
- backend: add CWD field to GenerationParams for environment state
- kiro: pass session CWD to kiroCurrentEnvState instead of hardcoded '/'
- agent/runtime: set GenParams.CWD from session cwd in BuildRuntime
- o new: create sessions and agents via 9P filesystem
- o prompt: support piped stdin for non-interactive use
2026-08-10 09:10:13 +02:00
Ollie Agent fbe8059cf5 maintainability: docs, contracts, structural cleanup
- doc/boot-sequence.md: init chain from main → fs.NewRoot → session.Init
- Event protocol: comprehensive table on Event struct (all roles, semantics)
- RPC server moved from cmd/ollie-remote into toolsrv/rpcserver.go
  (binary is now a 93-line thin wrapper)
- SetSessionEnv folded into NewAgent (no post-construction wiring needed)
- AgentCfg → AgentParams (distinguish from AgentConfig JSON schema)
- ToolResult/ToolResultContent shared types in toolsrv/rpcwire.go
- toolsrv/shell.go → toolsrv/exec.go (name matches purpose)
- Resume unified: single path, always through Keeper
- AGENTS.md: fix fs/builder.go → fsedsl/builder.go, handlers_*.go
- fsedsl/Tree: document dual-mode (EDSL lazy vs Manual Mount)
- Truncation stack documented at defaultToolResultMaxBytes
- Integration test rewritten to use srv.ServeRPC directly
2026-08-09 21:45:56 +02:00
Ollie Agent 2d74ffb95e prompt audit: trim system prompt, kill dead code, improve maintainability
- System prompt: ~200 → ~80 lines (removed tool-first table, API docs,
  output protocol, dead 9P entries)
- Removed AllowTools entirely (field, RPC, configs)
- Removed PRIME_* env vars, replaced with typed struct fields
- Merged tool listing + docs into single renderTools()
- Killed BuildToolListing, changed OnToolsChanged to func() signal
- Typed Preamble.Section (compile-time safety)
- Added protocol docs on extractToolResult
- Added session/ package godoc
- Moved output protocol to per-agent prompts
- Rewrote data/agents/README.md
- Moved 7 reference docs from doc/resources/ to doc/
- Deleted PromptEnv() dead function
2026-08-09 14:13:52 +02:00
Ollie Agent 1dcde13f26 rm doc 2026-08-09 12:55:02 +02:00
Ollie Agent 2b4d577e39 refactor: architectural cleanup (5 items)
1. Standardize ctl dispatch: hoist rootCtlHandlers and sessionCtlHandlers
   to package-level vars, matching the agent pattern.

2. Fix error wrapping: %v → %w in toolsrv/shell.go (only 2 instances;
   backend/ was already clean).

3. Canonicalize tool server access: LoadTool now only uses
   session.ToolsConn() — removed fallback to agent.ToolServer().

4. Centralize event handler wiring: wireAgentEvents(sessID, ag, al)
   replaces separate SetOutput + wireAgentStateEvents calls. Documents
   the contract.

5. Split toolsrv/shell.go: validation patterns, ValidateCode, and rate
   limiting moved to shell_validate.go (86 lines). shell.go retains
   execution logic (594 lines).
2026-08-09 12:46:36 +02:00
Ollie Agent f5efd90fbe refactor: audit fsedsl exports — remove dead code, unexport internals
Dead code removed:
- CreateFile option + FsNodeDecl.Create field (never set in production)
- Alias option function (Binding.Aliases field retained — used by fs/)
- FileTree interface (unused anywhere)
- Validation for Create-on-non-dir (no longer possible)

Unexported (internal-only plumbing):
- NewTree → newTree, TreeOption → treeOption
- WithLister/Creator/Deleter/Renamer/Opener/Stat/Readdir → with*
- FileConfig → fileConfig
- SyntheticEntry → syntheticEntry
- FileEntry/DirEntry → fileEntry/dirEntry

Retained (used by fs/):
- Tree, BuildTree, FsNodeDecl, File, SyntheticFileInfo
- All active NodeOptions (Read/Write/Stream/BlockOnce/Request/
  RemoveNode/StatOverride/UID/GID/Bind/Doc)
- Perm* constants, Binding, Each, DirNode, FileNode
- GenerateHelp, NodeOption type
2026-08-09 12:34:40 +02:00
Ollie Agent 199acabd61 refactor: code review items 15-17
- Extract paths.IsGitRepo() replacing 3 inline copies
  (agent/agent.go, agent/prompt_resolver.go, cmd/ollie-remote/main.go)

- Extract newStreamScanner() in backend.go replacing 3 identical
  bufio.NewScanner + Buffer setups (ollama, openai, anthropic)

- Replace kiroUserDataDir() with paths.UserDataHome() — deletes 18-line
  platform-switch function (darwin/windows branches were dead code)

- Unexport AnthropicBackend.BaseURL → baseURL for consistency with other
  backends. Add export_test.go with SetBaseURLForTest for external tests.
2026-08-09 12:24:15 +02:00
Ollie Agent da5ce08af6 update kde submodule: fix tool output rendering 2026-08-09 12:14:09 +02:00
Ollie Agent 090ba270f4 update kde submodule: fix tool block rendering 2026-08-09 12:09:48 +02:00
Ollie Agent e1f6981adf update kde submodule 2026-08-09 11:58:00 +02:00
Ollie Agent decd4eb6a0 fix: stripMarkers was suppressing all content inside fences
The inFence state machine was dropping all lines between fence
delimiters, not just the delimiters themselves. Tool output, assistant
code blocks — everything inside fences was being eaten.

Fix: remove only [[[...]]] marker lines and bare fence delimiter lines
(``` / ```lang). Preserve all content between them.
2026-08-09 11:50:52 +02:00
Ollie Agent d861e41d31 fix: chat stream closes when stripMarkers produces empty output
When a streaming chunk contains only markers/fences, stripMarkers
returns empty bytes. The 9P server interprets a zero-length Rread as
EOF, closing the client's FD.

Fix: loop in streamAgentChatText, advancing the offset past
marker-only chunks until real content arrives.
2026-08-09 11:48:29 +02:00
Ollie Agent 61dbf760ac refactor: extract sub-functions from agent/loop.go run()
run() was ~370 lines doing streaming, tool execution, error tracking,
and budget management. Now ~70 lines delegating to:

- streamResponse(): retry loop, event processing, usage tracking
- execToolCalls()/execBatch()/execOne(): dispatch, caching, fan-out, dedup
- trackErrors(): consecutive/repeated error tracking, nudge injection

Added errorState struct for clean state threading.
2026-08-09 11:37:18 +02:00
Ollie Agent 74635b8e00 refactor: items 8-10, 12, 13 from code review
- Move parseRetryAfter to backend.go (where it's called)
- Extract paths.RuntimeDir() replacing 3 divergent XDG_RUNTIME_DIR impls
- Rename CodeWhispererBackend → KiroBackend, NewCodeWhisperer → NewKiro
- Export RPCRequest/RPCError/OutputNotification from toolsrv/rpcwire.go
- Factor tools/lsp/cmdutil: Run() and RunCustom() for 7 LSP binaries
2026-08-09 11:32:31 +02:00
Ollie Agent 9c90cb8954 refactor: reduce redundancy, remove dead code, improve structure
- Fix CodeWhisperer SetModel cache invalidation bug
- Remove dead code: gemini stub, orphaned comments, kiroMetadataEvent
- Extract paths.NewUUID() replacing 3 duplicate UUID implementations
- Export toolsrv.BuildToolListing, deduplicate tool listing render
- Extract baseBackend struct (Name/Model/SetModel) embedded in all backends
- Unify backend factory post-processing (7 if-blocks → 1)
- Split fs/handlers.go (888 lines) into handlers_{root,session,agent}.go
- Strip dead filesystem-backed tree code from fsedsl (WithReadOnly,
  WithIndex, WithResolver, ValidateReadMode, resolve/writePath fallbacks)
2026-08-09 11:23:25 +02:00
Ollie Agent 25924d0909 fs: add /help ctl command — returns command reference 2026-08-09 03:32:21 +02:00
Ollie Agent 05e9c45495 update kde submodule: slash command hints match ctl 2026-08-09 03:15:27 +02:00
Ollie Agent 6147ec00bd docs: update architecture diagram, fix ellie.el for ctl rdwr
README: mermaid diagram now shows o CLI as the foundation layer
between text frontends and 9P. KDE/Kate connect directly to 9P.

ellie.el (both versions): add ellie--9p-rdwr wrapper, use ctl rdwr
for systemprompt (no longer a standalone file).
2026-08-09 03:14:00 +02:00
Ollie Agent aa74e1c683 docs: fix stale references in 9p.md, usage.md, misc.md
- 9p.md: update agent file table to current namespace
- usage.md: readloop→read -l, tail→read chat, command table updated,
  TUI description updated (no grep pipe)
- misc.md: readloop→read -l
2026-08-09 03:10:20 +02:00
Ollie Agent 42823dab2e docs: update README, AGENTS.md, architecture, evolution for Aug 9 changes
- README: fix CLI commands, repo layout (remove detach/, add fsedsl/lib9p)
- AGENTS.md: update architecture section (ctl rdwr, bypass, tool_load)
- architecture.md: rewrite namespace tree to match current state
- evolution.md: add Aug 9 section documenting all simplifications
2026-08-09 03:06:05 +02:00
Ollie Agent cbfc08875b fs: unify ctl — all levels use rdwrHandler + rdwrDispatch
Root, session, and agent ctl files all share the same dispatch
mechanism: rdwrHandler map → rdwrDispatch → Request (rdwr).

- ctlHandler/ctlDispatch (write-only) removed
- Root and session ctl upgraded to Request (rdwr), return 'ok'
- systemprompt moved to agent ctl command
- nopRead removed (dead)
2026-08-09 02:58:09 +02:00
Ollie Agent 0025003aa0 fs: refactor agent ctl to map-based dispatch
Replace switch statement with rdwrHandler map. Each command is a
function value in agentCtlHandlers — easy to extend, test individually,
and introspect.
2026-08-09 02:53:17 +02:00
Ollie Agent 993d5a812a fs: move tools to ctl commands, remove tools file
/tools        — lists loaded tools
/tool_load X  — loads a tool by name

System prompt updated to instruct the agent to use ctl for tool
management instead of writing to a tools file.
2026-08-09 02:52:06 +02:00
Ollie Agent 94088ccc94 fs: move cwd to ctl command, remove cwd file
/cwd returns current directory, /cwd X changes it (with stat check).
One fewer file in the agent namespace.
2026-08-09 02:48:50 +02:00
Ollie Agent 2561e01574 fs: move per-agent models to ctl command, remove models file
/models now lists available models for the agent's backend via ctl.
One fewer file in the agent namespace. The root-level models file
(all backends) remains unchanged.
2026-08-09 02:47:26 +02:00
Ollie Agent c05f4d1ef0 fs: agent ctl becomes rdwr (request-response)
The ctl file is now a Request handler: write a command, read the
response. Commands with no args return current state:

  /model        → prints current model
  /agent        → prints current profile
  /cwd          → prints working directory
  /name         → prints agent name
  /model X      → switches model, returns new model
  /compact      → compacts, returns 'ok'

o ctl now uses 'ollie-9p rdwr' to get the response.
This enables /model (no args) to show the current model from
any frontend.
2026-08-09 02:44:21 +02:00
Ollie Agent a4e5f7c9c5 unify commands: / == o ctl, all operations are ctl commands
The slash prefix in prompts now routes directly to the agent's ctl
file. No special-cased commands in the agent — the ctl handler is the
single command surface:

  /stop, /compact, /clear, /model X, /inject X, /agent X, /cwd X

New ctl commands:
  inject <text>  — overwrites any pending inject (replaces /i and /irw)
  agent <name>   — switch agent profile (moved from commands.go)

Agent methods added: Compact(), Clear(), SwitchProfile() — called by
the ctl handler, testable independently.

HandleCommand is now a thin trampoline: strip /, run 'o ctl <rest>'.
commands.go reduced from 155 to 50 lines.

REPL: / → o ctl, !q/!quit → exit TUI.
2026-08-09 02:40:48 +02:00
Ollie Agent 1bc438771f agent: replace 12 slash commands with ! prefix
! runs 'o' with session/agent context, giving access to the entire
namespace from any frontend (TUI, GUI, Emacs). Examples:
  !ctl compact       — compact context
  !ctl stop          — stop agent
  !read stats        — show usage/cost/ctxsz
  !read models       — list models

Kept: /i (inject mid-turn), /irw (rewrite inject), /agent (profile switch).
Removed: /backend, /models, /model, /maxsteps, /agents, /compact,
/context, /cost, /usage, /history, /clear, /sp.

Also simplified the o prompt REPL: slash commands reduced to /i and
/quit. Everything else via !<cmd>.

Dead code removed: contextDebug() in history.go.
2026-08-09 02:33:03 +02:00
Ollie Agent c7aea0db59 rename elevate→bypass throughout
The sandbox escape mechanism is a bypass, not privilege elevation.
The old name caused the agent to confuse it with sudo.

- elevate/ → bypass/ (package, types, tests)
- elevate_notify.go → bypass_notify.go
- Namespace: /elevate → /bypass, session/*/elevate → session/*/bypass
- Tool arg: "elevated" → "bypass"
- Env: OLLIE_ELEVATE_SOCKET → OLLIE_BYPASS_SOCKET
- File: elevate-policy.yaml → bypass-policy.yaml
- All docs, prompts, and scripts updated
2026-08-09 02:22:39 +02:00
Ollie Agent d1907adfae o script: audit cleanup
- Merge readloop into read -l (loop mode)
- Remove chatstream command (TUI uses 'o read chat' directly)
- Rework ctl: o ctl <cmd> [session] [agent] — dispatches to
  session ctl for kill/pause/resume, agent ctl for the rest
- Add trap EXIT to kill background statewait loop (fixes zombie)
- Remove -1 compatibility flag from generate
- Remove no-op bracketed paste bind from prompt REPL
- Fix /i help text: 'inject prompt (mid-turn or queued)'
- Update all help text to reflect current commands
2026-08-09 02:15:48 +02:00
Ollie Agent 49c93eab5f remove usage_log.go (write-only telemetry), clean AGENT_FILES
- Delete agent/usage_log.go — wrote JSONL to disk but nothing read it
- Remove appendUsageLog call from turn.go
- Remove dead 'offset' from AGENT_FILES tab-completion list
- TUI: grep markers+fences from o log (single read, not streaming)
2026-08-09 02:03:51 +02:00
Ollie Agent 72db6d900d fs: namespace cleanup — merge and remove dead files
- fifo.in + fifo.out → single fifo (write enqueues, read dequeues)
- cost + usage + ctxsz → single stats (key=value lines)
- Remove connection (no consumers, heartbeat handles it)
- Remove dead prevPrompt field and write (prompt.prev gone from spec)
- Clean AGENT_FILES: remove state, context, tail (all already absent
  from spec, were only in the tab-completion list)
2026-08-09 01:57:41 +02:00