Route() now queries all configured backends for their model lists,
presents them numbered to the classifier, and parses the number.
No more hardcoded OLLIE_ROUTE_MODEL_FAST/POWER env vars.
The management.*.kiro.dev endpoint was slower/less reliable. Switch to
the same q.<region>.amazonaws.com endpoint that kiro-cli uses, deriving
the region from the profile ARN or runtime endpoint.
Models() now caches at the backend level (shared across sessions) with
a 24h TTL, eliminating repeated expensive HTTP calls to AWS.
The sqliteKiroAuth.load() now caches token reads for 60s, avoiding
shelling out to sqlite3 on every API call (previously 3x per Models
fetch).
- Remove IsSocial guard from sqliteKiroAuth.ProfileARN(); the Kiro API
now requires profileArn on all requests including personal accounts
- Add OLLIE_KIRO_PROFILE_ARN env var for static token auth
- Add repeated error detection (3x identical error triggers corrective nudge)
- Include expected format in tool dispatch error messages
- Switch endpoint from q.*.amazonaws.com to runtime.*.kiro.dev
- Use management.*.kiro.dev for ListModels
- Fix content encoding: plain string instead of json.RawMessage
- Fix tool results: always use text field (upstream rejects bare JSON primitives)
- Normalize tool call arguments (deduplicate keys)
- Expand retry conditions: add 502, 503, 408, InsufficientModelCapacity
- Add SQLite busy-timeout and Go-level retry for lock contention
- Add 30s refresh timeout and fullLoad before refresh
- Add SSO cache fallback for token refresh
- Add jitter to token expiry checks
- Add sqliteHasNewUsableAccessToken for cross-process recovery
- Add envState to current message
- Update user-agent strings to 1.3.15/2.5.1
- Set X-Amzn-Codewhisperer-Optout to true
- Add reasoning/thinking support (additionalModelRequestFields)
- Add counter-thinking tags on user messages before assistants
- Handle reasoningContentEvent in stream
The Kiro/CodeWhisperer API started rejecting tool-use requests from
older API versions (0.1.14474) with InternalServerException. Update
to 0.1.16551 and appVersion 0.12.333 to match kiro-cli.
Also add retry with linear backoff (1-5s) for transient 500 errors
which still occur intermittently even with the correct version.
DeepSeek requires reasoning_content to be echoed back on assistant
messages. Store reasoning from stream events on Message and serialize
it as reasoning_content in the OpenAI wire format.
TestBuildAnthropicMessages_UserImageBlocks: user message with ContentBlocks
TestBuildAnthropicMessages_ToolImageBlocks: tool result with image blocks
TestBuildAnthropicMessages_UserPlainTextFallback: nil ContentBlocks uses Content
TestEncodeMessages_UserWithImageContentBlocks: OpenAI image_url encoding
TestEncodeMessages_UserWithImageContentBlocks_JSON: wire format verification
Add canonical ContentBlock and ImageSource types to backend.go.
When Message.ContentBlocks is set, it takes precedence over Content.
Anthropic: new anthropicImageSource type, anthropicContentBlocks() helper
converts canonical blocks to wire format. buildAnthropicMessages handles
ContentBlocks on user and tool messages.
OpenAI: new openAIImageURL type, openAIContentBlocks() helper converts
canonical blocks to wire format (image_url with data: URIs).
encodeOpenAIMessages uses ContentBlocks when present.
Retry up to 5 times with exponential backoff (5s, 10s, 20s, 40s, 80s)
when the Kiro backend returns ThrottlingException or HTTP 429.
Previously these errors were sent as a fatal StopReason, causing the
agent loop to abort immediately.
Copilot API has no /v1 prefix. Add pathPrefix field to OpenAIBackend
(defaults to /v1) and apply extraHeaders in fetchModels. Copilot sets
pathPrefix to / so it hits /models and /chat/completions directly.
- backend: add HTTP 404 and "no endpoints found" to isToolUnsupported
so OpenRouter 404s are correctly classified as tool_unsupported
- core: drain FIFO on error return from executeTurn so freeloader's
resubmitted prompt (queued via fifo.in while running) is picked up
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- backend: ToolUnsupportedError type, isToolUnsupported detector (400/422)
- backend: classifyError maps error types to hook payload strings
- agent: HookTurnError constant; TurnError func on agentConfig set at init
- loop: fire TurnError hook on first backend error; if ran, skip retries
- store: MemoryStore — in-memory KV Store implementation
- store: Session.prevPrompt field; prompt.prev read-only session file
- hooks: inject payload map as OLLIE_* env vars in runHookCmd
- agent: agentSpawn payload includes model field
- agent: limitretry state emitted on RateLimitError backoff
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
loop_prune_test.go — three new prune tests:
- ReadWriteReadCycle: pre-write read pruned, post-write read kept (primary invariant)
- AssistantMessagesUnaffected: only role:"tool" messages are dropped, never assistant messages
- NonExecuteCodeUnaffected: non-execute_code tool calls don't confuse the path tracker
loop_cache_test.go — one new cache test:
- IdenticalParallelReads: two goroutines simultaneously miss the cache, both execute,
results are consistent and no panic (sync.Map race safety)
cost_test.go — six computeCostUSD unit tests:
- BaseTokens, ClaudeCacheReadDiscount (10%), NonClaudeCacheReadDiscount (50%),
CacheCreationSurcharge (125%), AllFieldsCombined, UnknownModelZero
openai_encode_test.go — four OpenRouter cache injection tests:
- OpenRouterClaude_SystemCacheControl: system content encoded as array with cache_control
- OpenRouterClaude_LastToolCacheControl: last tool gets cache_control:ephemeral
- OpenRouterNonClaude_NoCacheControl: non-Claude model on openrouter gets no injection
- OpenAI_NoCacheControl: plain openai backend never injects cache_control
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- loop_parallel_test.go: four tests covering concurrent execution (barrier
pattern), serial tool breaking a batch, cancellation filling all results,
and nil ClassifyTool falling back to serial
- loop.go: emit "tool" events for all cancelled tool calls (early-ctx-err
path in execOne, top-level ctx check, and post-batch fill) so every
dispatched tool call has a visible result regardless of cancellation timing
- anthropic_stream_test.go: five tests covering tool schema cache_control
placement (last tool only, single-tool edge case) and tool result caching
(large gets cache, small does not, last of multiple large results wins)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Tool schema caching:
- Mark the last tool with cache_control:ephemeral before each Anthropic
request; Anthropic caches all preceding tools as a prefix
- Same for OpenRouter+Claude via openAITool.CacheControl extension
Large tool result caching (Anthropic only):
- In buildAnthropicMessages, scan backwards through the encoded output
and mark the last tool_result block ≥4096 bytes with cache_control
- 4096 bytes ≈ 1024 tokens, Anthropic's minimum cache entry size
- OpenAI wire format has no block structure for tool results; skip
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When OpenRouter routes to a Claude model, encode the system message as
a content-block array with cache_control:ephemeral instead of a plain
string. OpenRouter passes this through to Anthropic, enabling the same
90% cache-read discount as the native Anthropic backend.
Detection: b.name=="openrouter" && model contains "claude-".
Encoding: MarshalJSON on openAIMessage outputs content as array when
ContentBlocks is set, preserving the existing *string path for all
other cases with no test changes required.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Anthropic system prompt sent as a cacheable content block
(cache_control: ephemeral) so repeated turns reuse the cached prefix
- Parse cache_read_input_tokens and cache_creation_input_tokens from
Anthropic message_start; parse cached_tokens from OpenAI
prompt_tokens_details
- Normalise Usage so CachedInputTokens is always excluded from
InputTokens regardless of provider
- computeCostUSD now takes backend.Usage and applies correct cache
discounts: 10% read / 125% creation for Anthropic, 50% read for OpenAI
- Session tracks TotalCachedInputTokens / TotalCacheCreationTokens;
/usage shows cached token count when non-zero
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
On ContextOverflowError (400 with context-length body), restore pre-turn
snapshot, compact, and retry the turn once. Covers runaway context growth
that slips past the proactive 75% threshold.
OLLIE_MAX_SESSION_COST (USD) blocks turn start and interrupts mid-turn
via the usage event handler, which fires between every tool round.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Usage events now carry a 4th field (costUSD) so OpenRouter's reported
cost flows through unchanged. For backends that only report token counts
(Anthropic, direct OpenAI, Ollama), cost is computed from a prefix-keyed
pricing table. The higher-fidelity source always wins.
Per turn:
- turnInputTokens/turnOutputTokens/turnCostUSD reset before each turn
- recordTurnCost() stores LastTurnCostUSD, adds to SessionCostUSD
- "costLast=$X.XXXX" emitted as an info event at turn end
Session file "cost" returns costLast/costSession key=value pairs.
Agent interface gains Cost() string.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previously only HTTP 429 triggered a retry. Network failures, 5xx
responses, and mid-stream drops all hard-failed the turn.
- Add TransientError for 5xx and network-level failures; streamRequest
wraps client.Do errors and >=500 responses in it
- Merge pre-stream and mid-stream retry into one loop (maxTransientRetries=3)
covering rate limits, transient errors, and pure stream drops
- On stream drop with ctx cancelled: record partial state and return
(existing pause behaviour preserved)
- On stream drop without cancellation: close any open <think> block,
emit retry countdown, restart ChatStream from the same history
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- NewWithName and newFromEnv now error when OLLIE_BACKEND is unset
instead of falling back to ollama silently
- createSession reads OLLIE_MODEL as fallback when model= is not given
- Update test to assert error on missing OLLIE_BACKEND
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
DeepSeek's native tool-call format uses <｜DSML｜invoke> tokens. When
OpenRouter fails to convert them to standard tool_calls chunks, they
arrive as raw content text. Detect the marker, buffer the content, and
parse it into ToolCall structs at [DONE] if no structured tool calls
accumulated.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Test all buildAnthropicMessages branches: system concat, assistant
tool-calls-only, nil arguments default, tool batching, unknown role,
full multi-turn conversation
- Test streamAnthropicSSE done re-entry: events after message_stop and
error are ignored
- Remove unreachable done guard in process closure
- sandbox: remove dead code (DefaultConfig, Load/Save, LayeredConfig, Merge,
Version, SystemDefaults, validateName, ConfigPath, OpenSandbox); unexport
internal symbols; LoadSandbox now takes io.Reader; loadSandboxConfig owns
file opening; 83% package coverage via new sandbox_test.go
- backend: add BaseURL field to AnthropicBackend for testability; new
backend_test.go runs checkContract against all five implementations
(Ollama, OpenAI, Copilot, Anthropic, CodeWhisperer) via httptest servers
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace all hard-coded ~/.config/ollie and ~/.local/share/ollie paths
with a new paths package that reads OLLIE_CFG_PATH and OLLIE_DATA_PATH,
falling back to the previous defaults when unset.
Bare JSON scalars (0, true, null, "string") and arrays pass
json.Valid() but the Kiro API rejects them in the json field with
ValidationException. Once such a tool result is in session history,
every subsequent request fails, making the session unrecoverable.
Restrict the json path to content starting with '{' so only actual
JSON objects are sent that way; everything else uses the text field.
loadEnvFile now strips " # ..." suffixes from values so inline comments
in ~/.config/ollie/env don't leak into environment variables. Also add
kiro to the supported-backends error message and fix two raw-string
backtick escaping errors in tool descriptions.
- Add ContextLength(ctx) int to Backend interface; each backend fetches
from its API (OpenAI /v1/models, Ollama /api/show, CodeWhisperer
ListAvailableModels) with caching. Falls back to 128k default.
- Add Models(ctx) []string to Backend interface for model discovery.
- Add /models command to list available models.
- Auto-compact before each turn when estimated tokens >= 75% of context length.
Move internal/backend and internal/config to pkg/ so external consumers
can import them. Remove cmd/ollie and internal/tui — they now live in
the separate ollie-tui repo which imports ollie as a library.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ollie
Levi Neely
Levi Neely
Levi Neely