Commit Graph

2250 Commits

Author SHA1 Message Date
Ollie Agent 7591b2369b fs: remove connected file, agent tools file; fix session/idx hang
- Removed session/connected (used blocking Ping() on potentially stale conn)
- Removed per-agent tools file (tool management now via toolsrv ctl)
- Fixed session/idx: replaced blocking IsConnected() with non-blocking
  toolsConn != nil check, preserving the field for GUI compatibility
- Removed dead IsConnected() method from session.Session
2026-08-11 21:06:42 +02:00
Ollie Agent d1ef2c946c AGENTS.md: fix duplicate section, update build targets, add toolsrv architecture
- Removed duplicated Submodule/Environment section
- Updated build targets to match actual justfile
- Added two-process model explanation (olliesrv + toolsrv)
- Clarified that ALL tools go through toolsrv
- Fixed data/agents path reference
2026-08-11 20:57:46 +02:00
Ollie Agent e58f76c578 AGENTS.md: update for current architecture
Fix all stale references: fsedsl→virtfs, handlers_*.go→inline spec.go,
flat package layout→cmd/olliesrv/internal/*, etc.
2026-08-11 20:55:48 +02:00
Ollie Agent 066659b8b6 olliesrv/fs: inline all handlers into spec.go
The namespace spec is now fully self-contained. Every handler is a
closure right where it's declared — no jumping between files.

Deleted: handlers_root.go, handlers_session.go, handlers_agent.go, ctl.go
Added: support.go (shared utilities: mergeCtx, streamChat, stripMarkers,
dispatch, wireAgentEvents, help memoization)

spec.go grew to ~920 lines but is now the single source of truth for
the entire 9P namespace. Know where to look.
2026-08-11 20:41:08 +02:00
Ollie Agent a40e1dbed0 virtfs: inline tree options, remove newTree/with* indirection
The with* functional options and newTree constructor were private and
only used in two places (builder and tests). Replaced with direct
field assignment on &Tree{} literals. Less abstraction, same behavior.
2026-08-11 20:28:18 +02:00
Ollie Agent 5a09e603cd virtfs: eliminate Binding type
Each() now returns []FsNodeDecl directly. The Binding type was a
redundant subset of FsNodeDecl with a slightly different Remove
signature. Dynamic entries are now expressed uniformly — Remove,
Rename, Children, Aliases all live on FsNodeDecl like everything else.

Also added: Alias() and RenameNode() options, DirNode accepts
[]FsNodeDecl for passing pre-built child slices.
2026-08-11 20:24:29 +02:00
Ollie Agent 3a875da9f6 virtfs: rewrite README for closure-capture model
Drop all references to generic context type parameter, Applier,
and type/function aliasing patterns. Document the actual API:
plain closures, pre-bound Children in Bindings, no generics.
2026-08-11 17:22:39 +02:00
Ollie Agent ee7426d628 rename fsedsl → virtfs
The package has outgrown its original 'eDSL' framing. It's a virtual
filesystem library. Rename to match.
2026-08-11 17:21:44 +02:00
Ollie Agent d71ff80993 olliesrv/fs: embed *session.Session in SessionNode, kill delegation
SessionNode now embeds *session.Session instead of wrapping it.
All delegation methods (ID(), Name(), Ctx(), Pause(), etc.) are removed.
Handlers access session fields/methods directly through promotion.

Moved pause/resume event publishing into session.Session itself since
the session package already owns PublishEvent.

Eliminated all s.Session.X stutter from handler code.
2026-08-11 17:19:10 +02:00
Ollie Agent 1474eeaaed olliesrv/fs: eliminate Hctx adapter struct
Handlers now accept their actual dependencies directly:
- Root handlers: ModelCache, Broker, context.Context, etc.
- Session handlers: *SessionNode, removeFn, renameFn
- Agent handlers: *agent.Agent, *AgentLog, *SessionNode

Ctl dispatch tables are now per-instance closures built at binding
time rather than package-level vars with generic Hctx.

Also inlined the intermediate internalBinding type — adapt functions
now build fsedsl.Binding directly from raw data.

Removed: Hctx struct, rdwrHandler type, rdwrDispatch func,
bypassBindings helper, processBindings helper, internalBinding type.
2026-08-11 17:15:01 +02:00
Ollie Agent 95eb402c68 olliesrv/fs: migrate to closure-capture fsedsl API
Converted the entire 9P namespace spec from progressive Hctx population
to closure capture. Handlers are now closures that capture their
dependencies (session, agent, etc.) at binding time.

Hctx remains as a transitional adapter — handlers still receive it,
but it's constructed per-call from captured state rather than threaded
through the tree. Will be removed in a follow-up once handlers are
converted to use captured state directly.
2026-08-11 17:04:40 +02:00
Ollie Agent 53b4618561 toolsrv: migrate to closure-capture fsedsl API
Spec() now takes *Server and all handlers are closures that capture it.
Removed Ctx type, type aliases, and separate handler functions.
The tree structure and behavior are declared together in one place.
2026-08-11 16:56:34 +02:00
Ollie Agent f4a6829d89 fsedsl: remove Ctx type parameter, use closure capture
Handlers are now plain closures that capture their target at
construction time. No more progressive context population.

- FsNodeDecl is no longer generic
- Binding carries Children (pre-bound subtree) instead of Applier
- BuildTree takes only the spec (no root context)
- Handler signatures: func() ([]byte, error) instead of func(Ctx) ([]byte, error)

Consumers not yet updated (will not compile).
2026-08-11 16:55:09 +02:00
Ollie Agent 8a5883fcb2 fs: rename HandlerCtx → Hctx, SessionNode.Core → Session 2026-08-11 16:30:05 +02:00
Ollie Agent de5456e4aa toolsrv: fix per-agent tool registry to actually work
The registry was keyed by agent ID but read it from a shared env map
(st.env["OLLIE_UNAME"]) that all agents overwrote — making it
effectively per-session with a race condition.

Fix: pass agent ID explicitly through the protocol at every call site.

- ctl protocol: 'load <agentID> <tool>', 'unload <agentID> <tool>'
- tools file: rdwr (Request) — write agent ID, read filtered list
- proc/new payload: 'agent=<id>' field required
- Client: SetAgentID() stores identity, included in all operations
- Empty agent ID is a hard error everywhere
- Setting OLLIE_UNAME via env ctl is blocked (prevents reintroduction)
2026-08-11 16:26:12 +02:00
Ollie Agent 659f89fcbf toolsrv: per-agent tool registry
Tools are now scoped per-agent within a session. The registry uses
OLLIE_UNAME (agent identity) to partition loaded tools. Each agent
sees only its own tools — load/unload/list/lookup all key on the
agent ID read from the toolsrv environment.
2026-08-11 11:49:19 +02:00
Ollie Agent 7dfd769c25 server: fix BlockingReadMode truncation, separate stream/blocking paths
BlockingReadMode had the same 512-byte truncation bug: it returned EOF
on offset>0 before the full content was delivered. Now caches content
on the fid like normal reads.

StreamMode is separated into its own path since it has fundamentally
different semantics (each Tread blocks for the next chunk, no caching).
2026-08-11 11:45:47 +02:00
Ollie Agent cfc526b0d7 server: cache read content on fid for offset-based paging
In 9P, clients issue multiple Treads at increasing offsets to read
files larger than the initial buffer (io.ReadAll starts at 512 bytes).
The server was calling entry.Read() on every Tread, which for Request
files cleared the result after the first call, causing truncation.

Fix: cache the Read() result on the fid (readCache). Subsequent Treads
at higher offsets serve from the cache via readSlice. The cache is
cleared on write (for rdwr files) so the next request-response cycle
gets fresh data. This mirrors how dirCache already works for directories.
2026-08-11 11:44:30 +02:00
Ollie Agent b270cdc37a fsedsl: fix Request response truncation at 512 bytes
The Request read handler was clearing the result after the first Read()
call. But io.ReadAll starts with a 512-byte buffer, so the client's
first Tread has count=512. If the response is larger, the client issues
a second Tread — but the result was already cleared, causing truncation.

Fix: keep the result available across multiple reads. It's replaced on
the next write (new request-response cycle).
2026-08-11 11:39:42 +02:00
Ollie Agent 83f6211369 toolsrv: simplify registry to flat list, remove session scoping
One toolsrv per session means the session-scoped map was unnecessary
indirection. Registry now tracks a flat map of loaded tools with no
session ID parameter. This eliminates the class of bugs where sessID
could be empty or mismatched between Load and Loaded calls.
2026-08-11 11:30:47 +02:00
Ollie Agent cfc429f5f4 fix: expand ~ in toolsrv CWD to prevent fork/exec ENOENT
Shell doesn't expand ~ in programmatically-passed arguments, so
toolsrv received literal ~/src/ollie as --cwd. Go's os/exec fails
with ENOENT on chdir before exec, breaking all tool execution.

Expand ~ at two boundaries:
- main.go: immediately after flag parsing
- proc.go SetCWD(): for runtime cwd changes via 9P
2026-08-11 11:24:10 +02:00
Ollie Agent 686a560e84 agent: toolsrv is sole owner of tool state
Remove all cached tool state from agent:
- Remove toolRegistryRevision from Runtime
- Remove wireToolsChanged (OnToolsChanged callback)
- Remove toolsNeedRefresh (revision-based cache invalidation)
- Remove refreshToolListing (redundant with RefreshTools)

Tools are now fetched fresh from toolsrv at the start of every turn.
The preamble tools section is also rebuilt at turn start. No local
caching means tool_load/unload take effect immediately on the next
turn and /tools always shows the live state from toolsrv.
2026-08-11 10:59:31 +02:00
Ollie Agent 6b4e97aaf7 fix: tool_load and tools use agent's ToolServer consistently
tool_load was loading on the session connection but tools listed from
the agent connection. Now both use ctx.Agent.ToolServer() so loads are
immediately visible. writeAgentTools also calls RefreshTools after load.
2026-08-11 10:50:38 +02:00
Ollie Agent c3afe1453d ctl: add /i alias for inject 2026-08-11 10:47:58 +02:00
Ollie Agent 573c9f4462 cleanup: fix structural inconsistencies, remove dead code
- backend/openai: fix DefaultModel() dead 'anthropic' case, default to gpt-4o
- agent: SetCWD no longer sets stale envBlock first, uses runtime.GOOS
- backend/kiro: ContextLength cache now checks ctxModel consistently
- agent: AgentConfigPath uses agentsDir as the only search location
- fs: ctl 'tools' handler delegates to readAgentTools (dedup)
- agent: remove redundant TurnCtx field assignments in executeTurn
- agent: remove spending cap feature entirely (OLLIE_MAX_SESSION_COST)
- agent: inline stripColdResults into stripCold, remove wrapper
- agent: remove orphaned doc comment at end of text_parse.go
- agent: remove dead Prompt.MarshalJSON
- agent: cloneMessages now copies ID, Reasoning, ContentBlocks
- agent: fix newHistory comment (Session → History)
- agent: fix orphaned CompactionModel godoc on ListModels
- agent: toolCalls only captured from Done event (fix contract)
- agent: HandleCommand uses lib9p client instead of shelling out
2026-08-11 10:46:38 +02:00
Ollie Agent 9e4c7582a3 doc: remove stale logging section from evolution.md 2026-08-11 10:25:54 +02:00
Ollie Agent 5740333ad2 prompt: instruct agent to make parallel tool calls
Agents now know to batch independent reads and writes in a single
turn rather than serializing them across multiple turns.
2026-08-11 10:24:53 +02:00
Ollie Agent 3a3d471e06 toolsrv: switch from log/slog to ollie/log
Unify logging across the project. toolsrv now uses the same
async non-blocking logger as olliesrv.
2026-08-11 10:23:27 +02:00
Ollie Agent 391465be77 doc+fix: background procs force timeout=0, update docs
- Background procs get timeout=0 (no deadline) set by toolsrv at
  the execution layer — not injected as an arg from olliesrv.
  Foreground default remains 30s. User can still override via args.
- Timeout logic simplified: caller sets default, ExecuteTool honors it.
- System prompt: clarify background has no timeout, uses 'stop' not 'kill'
- architecture.md: document lifecycle (connection-based ownership)
- evolution.md: update interrupt section with streaming, lifecycle, id= format
- README.md: add parallel execution to capabilities table
2026-08-11 10:18:30 +02:00
Ollie Agent 97c208fa39 toolsrv: kill all procs on shutdown, add Pdeathsig
- KillAll(): sends SIGKILL to all running proc groups on clean shutdown
- Pdeathsig: SIGKILL ensures child procs die if toolsrv crashes
- Shell tool: use subshell + set +e for streaming without exit propagation
- Fix integration test to match new error format

This is the connection-based ownership model: toolsrv death = proc death.
The session owns the toolsrv process, so session death cascades to all procs.
2026-08-11 10:02:53 +02:00
Ollie Agent 0d1eeaa46b fix: background process lifecycle, streaming output, and connection deadlocks
- Timeout: timeout=0 means no deadline (was defaulting to 30s)
- Signal: send to process group (-pgid) not just process; SIGTERM no longer
  cancels context (only SIGKILL does); cmd.Cancel sends SIGTERM with 5s WaitDelay
- Streaming: background procs stream output in real-time via procWriter;
  shell tool no longer buffers all output into a bash variable
- Proc tree: olliesrv exposes proc/{id}/out, proc/{id}/ctl, proc/{id}/status
  as proper 9P directory (was broken flat file)
- Connection: proc handlers dial fresh toolsrv conn per request via
  Session.DialToolServer() to avoid deadlocking the agent's blocked conn
- Stat format: key=value (exited=true, exit_code=N, id=N) matching client parser
- GC: procs auto-removed 10min after LastRead (exited procs only)
- Rename: PID -> ID throughout (synthetic, not OS PID)
- Ctl commands: term (SIGTERM), kill (SIGKILL), signal <n>, dismiss
- System prompt: correct ollie-9p commands for proc management
2026-08-11 09:44:08 +02:00
Ollie Agent 12babf2f92 doc: document parallel execution, background interrupts, and dispatch flags 2026-08-11 08:30:31 +02:00
Ollie Agent 945ce69984 agent: add tests for dispatch flag injection and background helpers 2026-08-11 08:28:54 +02:00
Ollie Agent a098fbff0c agent: inject dispatch flags (bypass, timeout, sandbox, background) into all tool schemas
Dispatch-level flags are now injected into every tool's JSON schema
at runtime via injectDispatchFlags(). No need to declare them in
individual .meta files — they're universal.

Removed redundant declarations from shell.meta (now injected globally).
2026-08-11 08:26:14 +02:00
Ollie Agent 308ebbbd22 shell: add background flag to tool schema 2026-08-11 08:24:47 +02:00
Ollie Agent aa270525a5 agent: background process interrupts — auto-inject output at safe points
When a tool call includes "background": true, it executes via
proc/new.bg and returns immediately with a PID in a
<system-proc-background> tag. The agent tracks active background
processes and injects their output as <system-proc-interrupt> blocks
alongside subsequent tool results.

The model sees background updates without polling. It can react to
build failures, log events, etc. naturally. Kill via PID when done.

Implementation:
- toolsrv client: CallToolBackground (uses proc/new.bg as rdwr)
- toolsrv: proc/new.bg upgraded from write-only to request-response
- agent: bgTracker collects last 20 lines of output per proc
- agent loop: injects interrupts after execToolCalls, before h.update
- system prompt: documents the background mechanism and rules
2026-08-11 08:23:00 +02:00
Ollie Agent 3e6d1aa7d5 doc: update concurrency section with parallel fan-out and protocol constraints 2026-08-11 08:06:38 +02:00
Ollie Agent 2870da593d agent: default unset scope to global (safe by default)
Only tools that explicitly declare scope="write" get path-based
parallelism. Unset scope is now global (full barrier), avoiding the
"path is a lie" problem where a tool has a path arg but modifies
other files (e.g., lsp_rename).

Tools must opt in to parallelism:
- scope=read: never conflicts
- scope=write: conflicts on same path only
- scope=global (or unset): serialization barrier

Added scope=write to file_edit.meta and file_write.meta.
2026-08-11 07:57:22 +02:00
Ollie Agent 586edc1332 agent: replace ReadOnly with explicit scope field (read/write/global)
Rename ToolInfo.ReadOnly bool → ToolInfo.Scope string with three values:
- "read"  — path-scoped read, never conflicts (always parallel)
- "write" — path-scoped write, conflicts on same file path only
- "global" — full serialization barrier, runs alone

Tools declare scope in their .meta file. If unset, inferred from path
arg presence (write if path exists, global otherwise).

This correctly classifies lsp_rename as global (cross-file workspace
edits) despite having a path argument. The path arg in lsp_rename is
a symbol coordinate, not a resource scope.

All .meta files updated: readOnly:true → scope:read,
readOnly:false → scope:global, lsp_rename gets scope:global.
2026-08-11 07:56:00 +02:00
Ollie Agent 3dccb36e82 agent: resource-based parallel tool execution
Replace binary ReadOnly/not batching with path-based conflict detection.
Tool calls that operate on different file paths now execute in parallel,
while same-path writes and shell (global barrier) serialize.

Conflict rules:
- ReadOnly tools: never conflict (unchanged from before)
- Write tools with path arg: conflict only on same path
- Shell / unknown (no path): global barrier, runs alone

This parallelizes the common refactoring case where the model edits
multiple files in one turn (e.g., 5 file_edits on different files
complete in 1 round-trip instead of 5).

Safety argument:
- LLM API is turn-based: model reads files first, plans edits based
  on known content, emits writes in a subsequent turn
- file_edit carries old_string context: self-contained per-file
- Same-path conflicts serialize (detected via extractFilePath)
- Shell is conservative: always a full serialization barrier
2026-08-11 07:48:53 +02:00
Ollie Agent bb58c42f4f doc: note concurrency as unsolved problem in toolsrv evolution 2026-08-11 07:32:38 +02:00
Ollie Agent 13a03a440a doc: shorten toolsrv mermaid box to fix overlap 2026-08-11 07:28:12 +02:00
Ollie Agent 41969ed4b2 doc: clarify toolsrv lifecycle for local vs SSH-remote case 2026-08-11 07:27:38 +02:00
Ollie Agent 4b7d5cdd79 doc: update architecture, evolution, README for toolsrv 9P migration
- README: toolsrv shown as separate child process in architecture diagram,
  cmd/toolsrv added to repository layout table
- architecture.md: repository structure reflects cmd/{olliesrv,toolsrv}
  internal packages, package table updated for two-server model
- evolution.md: add Phase 20 (toolsrv 9P migration & process isolation),
  update topology diagram, add to gantt chart
2026-08-11 07:24:53 +02:00
Ollie Agent df74c3fa60 toolsrv: restructure into internal packages, fix process lifecycle
- Move toolsrv logic into internal/{fs,exec,registry} packages to match
  olliesrv's structure. server.go (9P protocol) stays at top level.

- Deduplicate parseKV/parsePayload: remove dead code from old spec9p.go,
  single implementation in internal/fs/parse.go.

- Add structured logging via log/slog (replaces fmt.Fprintf to stderr).

- Standardize qid path hashing to fnv.New64a (matches olliesrv).

- Fix toolsrv process leaks:
  - Add Pdeathsig to local spawn so toolsrv dies when olliesrv exits.
  - Add idle timeout (30s with no connections after first client seen).
  - ProcessKeeper.Dial() kills old process before respawning.
2026-08-11 07:21:14 +02:00
Levi Neely 5a03499c66 Merge branch 'toolsrv-9p': 9P protocol and package restructure
Major changes:
- toolsrv now uses 9P protocol instead of JSON-RPC
- Clean package split: toolsrv/ (client SDK) vs cmd/toolsrv/ (server)
- tool_load is now a built-in tool (not sandbox-restricted)
- New toolclient package for process spawning and lifecycle
- Remote execution via SSH socket forwarding with Tauth authentication

21 commits from toolsrv-9p branch.
2026-08-10 21:05:32 +02:00
Levi Neely e8239927de toolsrv: move Registry to cmd/toolsrv, keep only client SDK in shared package
Split toolsrv/registry.go:
- cmd/toolsrv/registry.go: Registry type with session-scoped tool state
  (Load, Unload, Loaded, Lookup, Revision methods)
- toolsrv/registry.go: shared types only (ToolInfo, ToolsPath, DiscoverTools)

The toolsrv/ package is now a pure client SDK:
- Dial, Conn for 9P connection
- ToolInfo, ToolResult, HostInfo types
- DiscoverTools for listing available tools

Server-only code lives in cmd/toolsrv/:
- Registry for session-scoped tool state
- 9P handlers and execution logic
- Sandbox integration
2026-08-10 20:57:32 +02:00
Levi Neely a0315a558a refactor: move toolsrv spawn/process management to toolclient
Move spawn.go from toolsrv/ to cmd/olliesrv/internal/toolclient/:
- Process, ProcessKeeper, Spawn, SpawnRemote now in toolclient package
- toolsrv/ now only contains client code (Dial, Conn, etc.)

This clarifies the architecture:
- toolsrv/ = client SDK for connecting to toolsrv
- cmd/toolsrv/ = the toolsrv server
- cmd/olliesrv/internal/toolclient/ = olliesrv's toolsrv process management

Removed ProcessKeeper tests from toolsrv integration tests since they
now belong to toolclient.
2026-08-10 20:49:22 +02:00
Levi Neely d292f2aa59 feat: make tool_load a built-in tool
tool_load must be handled specially by the agent runtime because tools
run inside toolsrv's sandbox and cannot load other tools into
themselves. The runtime now intercepts tool_load calls and directly
invokes ToolServer.LoadTool().

This is the only built-in tool - all others are external scripts.
2026-08-10 20:37:37 +02:00
Levi Neely be4f6aaf09 feat: add tools file to agent namespace
Add /session/{sid}/agent/{aid}/tools file:
- Read: list loaded tools for this agent
- Write: load a tool by name

Also add missing tool_load to justfile install.
2026-08-10 20:29:04 +02:00