Update virtfs architecture documentation
This commit is contained in:
Ollie Agent
parent
b1ef9dc68c
commit
0bab759f56
|
|
@ -0,0 +1,157 @@
|
|||
# virtfs Architecture
|
||||
|
||||
`virtfs` is the embedded Go library that defines Ollie’s synthetic filesystem. It separates namespace declaration from protocol transport: `cmd/olliesrv/internal/fs/spec.go` declares the Ollie namespace, `virtfs.BuildTree` materializes it as an in-memory tree, and the 9P server exposes that tree to clients.
|
||||
|
||||
The library is installed as part of the normal Go build. It is not a separately installed runtime service or configuration directory. The repository package at `virtfs/` is compiled into `olliesrv` by the `ninep` and `core` build targets in `justfile`.
|
||||
|
||||
## Design
|
||||
|
||||
Declarations are built with constructors and options. Handlers are plain closures that capture their target state; there is no handler-context object or later wiring pass. Dynamic directories use `Each`, whose callback returns fully bound child declarations.
|
||||
|
||||
The resulting `*virtfs.Tree` supports the filesystem operations needed by the 9P adapter and can also be used directly in tests.
|
||||
|
||||
## Core Type
|
||||
|
||||
```go
|
||||
type FsNodeDecl struct {
|
||||
Name string
|
||||
Aliases []string // alternate lookup names
|
||||
UID string // "" = inherit from parent
|
||||
GID string // "" = inherit from parent
|
||||
Mode os.FileMode // 0 = inherit parent default
|
||||
Desc string // help description
|
||||
|
||||
Stat func() os.FileInfo
|
||||
Children []FsNodeDecl
|
||||
Bindings func() ([]FsNodeDecl, error)
|
||||
|
||||
Read func() ([]byte, error)
|
||||
Write func([]byte) error
|
||||
BlockOnce func(context.Context, string) ([]byte, string, error)
|
||||
Stream func(context.Context, string) ([]byte, string, error)
|
||||
Rdwr func(context.Context, []byte) ([]byte, error)
|
||||
Remove func() error
|
||||
Rename func(string) error
|
||||
}
|
||||
```
|
||||
|
||||
A declaration is a directory when it has `Children` or `Bindings`; otherwise it is a file. `BlockOnce`, `Stream`, and `Rdwr` are mutually exclusive. Handlers capture their target state in closures.
|
||||
|
||||
### `DirNode(name, children_or_options...)`
|
||||
|
||||
Declares a static directory. Arguments may be child declarations, slices of declarations, or `NodeOption` functions:
|
||||
|
||||
```go
|
||||
virtfs.DirNode("session",
|
||||
virtfs.FileNode("new", 0666, virtfs.Rdwr(requestSessionNew)),
|
||||
virtfs.FileNode("idx", 0444, virtfs.Read(readSessionIdx)),
|
||||
virtfs.GID("agent"),
|
||||
)
|
||||
```
|
||||
|
||||
### `Each(name, bindingsFn, options...)`
|
||||
|
||||
Declares a dynamic directory. The callback returns the currently available child declarations. Each returned child is already bound to its target through closures:
|
||||
|
||||
```go
|
||||
virtfs.Each("{sname}", listSessions)
|
||||
```
|
||||
|
||||
### `FileNode(name, mode, options...)`
|
||||
|
||||
Declares a file node. Options provide handlers, metadata, ownership, aliases, and help text:
|
||||
|
||||
```go
|
||||
virtfs.FileNode("chat", 0444,
|
||||
virtfs.Doc("Agent chat log"),
|
||||
virtfs.Stream(streamAgentChat, agentSignal),
|
||||
)
|
||||
```
|
||||
|
||||
## Node Options
|
||||
|
||||
| Option | Signature | Purpose |
|
||||
|---|---|---|
|
||||
| `Read(fn)` | `func() ([]byte, error)` | Non-blocking read |
|
||||
| `Write(fn)` | `func([]byte) error` | Write handler |
|
||||
| `Stream(fn, signal)` | read callback plus signal channel | Blocking streaming read |
|
||||
| `StreamRaw(fn)` | `func(context.Context, string) ...` | Custom streaming read |
|
||||
| `BlockOnce(fn, signal)` | read callback plus signal channel | Return one changed value per open |
|
||||
| `BlockOnceRaw(fn)` | `func(context.Context, string) ...` | Custom blocking read |
|
||||
| `Rdwr(fn)` | `func(context.Context, []byte) ([]byte, error)` | Atomic write-then-read |
|
||||
| `RemoveNode(fn)` | `func() error` | Remove handler |
|
||||
| `RenameNode(fn)` | `func(string) error` | Rename handler |
|
||||
| `StatOverride(fn)` | `func() os.FileInfo` | Custom stat |
|
||||
| `UID(uid)` / `GID(gid)` | `string` | Ownership |
|
||||
| `Alias(names...)` | `...string` | Alternate lookup names |
|
||||
| `Doc(desc)` | `string` | Help description |
|
||||
|
||||
## Dynamic Declarations
|
||||
|
||||
The Ollie namespace uses `Each` for runtime collections such as sessions and agents. The callback builds declarations for the current objects and captures each object in its handlers. It does not receive a context or return a generic template that is wired later.
|
||||
|
||||
```go
|
||||
virtfs.Each("{aname}", func() ([]virtfs.FsNodeDecl, error) {
|
||||
var out []virtfs.FsNodeDecl
|
||||
for _, a := range agents {
|
||||
agent := a
|
||||
out = append(out, virtfs.DirNode(agent.Name(),
|
||||
virtfs.Alias(agent.ID()),
|
||||
virtfs.FileNode("id", 0444, virtfs.Read(func() ([]byte, error) {
|
||||
return []byte(agent.ID() + "\n"), nil
|
||||
})),
|
||||
))
|
||||
}
|
||||
return out, nil
|
||||
})
|
||||
```
|
||||
|
||||
## Building and Installation
|
||||
|
||||
`cmd/olliesrv/internal/fs.NewRoot` initializes session state, calls `buildTreeSpec`, and passes the resulting declaration to `virtfs.BuildTree`. The returned tree is mounted into the 9P server. `virtfs` is therefore a linked package, not a separately installed daemon.
|
||||
|
||||
The repository build uses:
|
||||
|
||||
```sh
|
||||
just build # includes core and the 9P server
|
||||
just install # installs the resulting runtime and data files
|
||||
```
|
||||
|
||||
The install phase copies runtime data under `~/.config/ollie` and `~/.local/share/ollie`; it does not copy the `virtfs` package or install a separate virtfs directory.
|
||||
|
||||
## Tree API
|
||||
|
||||
```go
|
||||
tree := virtfs.BuildTree(spec)
|
||||
tree.List()
|
||||
tree.Stat(path)
|
||||
tree.Open(path)
|
||||
tree.Create(path)
|
||||
tree.Delete(path)
|
||||
tree.Rename(oldPath, newPath)
|
||||
tree.Readdir(path)
|
||||
tree.MkdirAll(path, mode)
|
||||
```
|
||||
|
||||
`virtfs.GenerateHelp(spec)` renders the `Doc` annotations used by the server’s help file.
|
||||
|
||||
## Rules
|
||||
|
||||
1. Directories use `Children` or `Bindings`; file handlers belong on leaf declarations.
|
||||
2. `BlockOnce`, `Stream`, and `Rdwr` are mutually exclusive.
|
||||
3. A template-style name such as `{sname}` must have a `Bindings` callback.
|
||||
4. Ownership inherits through empty `UID` and `GID` values.
|
||||
5. A zero mode inherits from the parent. Root directories default to `0755`; root files default to `0444`.
|
||||
|
||||
## Aliases
|
||||
|
||||
A node may declare alternate names with `Alias(...)`. During path walks, if no child matches `Name`, the tree checks `Aliases`. This lets clients address a node by immutable ID while listings use a mutable display name.
|
||||
|
||||
```go
|
||||
virtfs.DirNode(session.Name(),
|
||||
virtfs.Alias(session.ID()),
|
||||
// ...
|
||||
)
|
||||
```
|
||||
|
||||
Aliases are invisible in directory listings. Session and agent declarations use IDs as stable aliases where needed. Clients that maintain long-lived paths should use the ID form.
|
||||
179
doc/edsl.md
179
doc/edsl.md
|
|
@ -1,179 +0,0 @@
|
|||
# EDSL for 9P Filesystem Specification
|
||||
|
||||
The entire 9P namespace is declared as a single tree using an embedded
|
||||
domain-specific language (EDSL) in Go. The declaration lives in
|
||||
`fs/spec.go` and is the single source of truth for every file
|
||||
and directory in the system.
|
||||
|
||||
## Core Type
|
||||
|
||||
```go
|
||||
type FsNodeDecl struct {
|
||||
Name string
|
||||
Aliases []string // alternate walk names (invisible in listings)
|
||||
UID string // "" = inherit from parent
|
||||
GID string // "" = inherit from parent
|
||||
Mode os.FileMode // 0 = inherit parent default
|
||||
|
||||
Stat func() os.FileInfo // metadata override
|
||||
Children []FsNodeDecl // static children (implies directory)
|
||||
List func(HandlerCtx) ([]FsNodeDecl, error) // dynamic children
|
||||
|
||||
Read func(HandlerCtx) ([]byte, error)
|
||||
Write func(HandlerCtx, []byte) error
|
||||
BlockOnce func(HandlerCtx, context.Context, string) ([]byte, string, error)
|
||||
Stream func(HandlerCtx, context.Context, string) ([]byte, string, error)
|
||||
Rdwr func(HandlerCtx, []byte) ([]byte, error)
|
||||
Create func(HandlerCtx, string) error
|
||||
Remove func(HandlerCtx) error
|
||||
}
|
||||
```
|
||||
|
||||
Nodes are either **directories** (have `Children` or `List`) or **leaf files**
|
||||
(have read/write/blocking handlers). At most one blocking variant
|
||||
(`BlockOnce`, `Stream`, `Rdwr`) may be set per node.
|
||||
|
||||
## Constructors
|
||||
|
||||
### `Dir(name, children_or_options...)`
|
||||
|
||||
Declares a static directory. Accepts a mix of child `FsNodeDecl` values
|
||||
and `NodeOption` functions:
|
||||
|
||||
```go
|
||||
Dir("session",
|
||||
Leaf("new", 0666, Rdwr(requestSessionNew)),
|
||||
Leaf("idx", 0444, Read(readSessionIdx)),
|
||||
GID("agent"),
|
||||
)
|
||||
```
|
||||
|
||||
### `TemplateDir(name, listFn)`
|
||||
|
||||
Declares a dynamic directory whose children are resolved at runtime by
|
||||
`listFn`. The list function receives a `HandlerCtx` and returns fully
|
||||
built `[]FsNodeDecl`:
|
||||
|
||||
```go
|
||||
TemplateDir("{id}", listSessions)
|
||||
```
|
||||
|
||||
### `Leaf(name, mode, options...)`
|
||||
|
||||
Declares a file node. Options specify handlers and ownership:
|
||||
|
||||
```go
|
||||
Leaf("env", 0444, Read(readSessionEnv))
|
||||
Leaf("ctl", 0666, Read(nopRead), Write(writeSessionCtl))
|
||||
Leaf("chat", 0444, Read(readAgentChat), Stream(blockAgentChat))
|
||||
```
|
||||
|
||||
## Node Options
|
||||
|
||||
| Option | Signature | Sets |
|
||||
|---|---|---|
|
||||
| `Read(fn)` | `func(HandlerCtx) ([]byte, error)` | `d.Read` |
|
||||
| `Write(fn)` | `func(HandlerCtx, []byte) error` | `d.Write` |
|
||||
| `Stream(fn)` | `func(HandlerCtx, ctx, base) (data, base, err)` | `d.Stream` |
|
||||
| `BlockOnce(fn)` | `func(HandlerCtx, ctx, base) (data, base, err)` | `d.BlockOnce` |
|
||||
| `Rdwr(fn)` | `func(HandlerCtx, []byte) ([]byte, error)` | `d.Rdwr` |
|
||||
| `CreateFile(fn)` | `func(HandlerCtx, string) error` | `d.Create` |
|
||||
| `Remove(fn)` | `func(HandlerCtx) error` | `d.Remove` |
|
||||
| `Stat(fn)` | `func() os.FileInfo` | `d.Stat` |
|
||||
| `UID(s)` | string | `d.UID` |
|
||||
| `GID(s)` | string | `d.GID` |
|
||||
| `Alias(names...)` | `...string` | `d.Aliases` |
|
||||
|
||||
## Handler Context
|
||||
|
||||
All handler functions receive a `HandlerCtx` that carries runtime
|
||||
dependencies. The context is populated per scope by the wire step:
|
||||
|
||||
```go
|
||||
type HandlerCtx struct {
|
||||
// Root-level (always available)
|
||||
Root *Tree
|
||||
Log *olog.Logger
|
||||
ToolReg *toolsrv.Registry
|
||||
Skills *skills.Registry
|
||||
Models *ModelCache
|
||||
Elevate *bypass.Broker
|
||||
Invalidate func()
|
||||
Shutdown func()
|
||||
|
||||
// Session scope
|
||||
Session *Session
|
||||
Kill func()
|
||||
Rename func(string) error
|
||||
SaveFn func([]byte) error
|
||||
|
||||
// Agent scope
|
||||
Agent *agent.Agent
|
||||
AgentLog *AgentLog
|
||||
}
|
||||
```
|
||||
|
||||
## Template Directories (Factories)
|
||||
|
||||
Template directories use the `List` function as a factory. The factory
|
||||
receives the current `HandlerCtx`, builds fully-wired subtrees using the
|
||||
EDSL constructors, and returns them:
|
||||
|
||||
```go
|
||||
func listSessions(ctx HandlerCtx) ([]FsNodeDecl, error) {
|
||||
rs := ctx.Root.Data.(*rootState)
|
||||
for _, sess := range rs.sessions {
|
||||
// capture session deps
|
||||
sessionDir := Dir(sess.Name(),
|
||||
UID(sess.Uname()), GID("agent"),
|
||||
Leaf("env", 0444, Read(func(_ HandlerCtx) ([]byte, error) {
|
||||
return []byte(sessionEnvContent(sess)), nil
|
||||
})),
|
||||
// ... more files
|
||||
)
|
||||
out = append(out, sessionDir)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
```
|
||||
|
||||
## Rules
|
||||
|
||||
1. **Directories cannot have leaf handlers.** A node with `Children` or
|
||||
`List` must not set `Read`, `Write`, `BlockOnce`, `Stream`, or `Rdwr`.
|
||||
2. **At most one blocking handler.** `BlockOnce`, `Stream`, and `Rdwr`
|
||||
are mutually exclusive.
|
||||
3. **Template nodes must have a `List` function.** Names starting with `{`
|
||||
and ending with `}` require a dynamic child resolver.
|
||||
4. **`Create` and `Remove` are directory-only.**
|
||||
5. **Ownership inherits.** Empty `UID`/`GID` values inherit from the
|
||||
parent node, all the way up to root (set from the system user at
|
||||
`NewRoot` time).
|
||||
6. **Mode defaults.** `0` inherits from parent; if still 0 at the root,
|
||||
directories default to `0755`, files to `0444`.
|
||||
|
||||
## Aliases
|
||||
|
||||
A node may declare alternate names via `Aliases []string`. During path
|
||||
walks (`findChild`), if no child matches by `Name`, aliases are checked.
|
||||
This allows nodes to be **addressed by immutable ID** even though they are
|
||||
**listed by mutable display name**.
|
||||
|
||||
```go
|
||||
sessionDir := Dir(s.Name(),
|
||||
UID(s.Uname()), GID("agent"),
|
||||
Alias(s.RunnableID()), // walk accepts UUID
|
||||
// ...
|
||||
)
|
||||
```
|
||||
|
||||
Aliases are invisible — they do not appear in directory listings (`ls`).
|
||||
They only affect path resolution during `Twalk`. This means:
|
||||
|
||||
- `ls session/` → shows display names (`myproject`, `scratch`)
|
||||
- `read session/a1b2c3d4-uuid/id` → resolves via alias, works
|
||||
- `read session/myproject/id` → resolves via name, works
|
||||
|
||||
Session directories alias their immutable `RunnableID()`. Agent directories
|
||||
alias their immutable `ID()`. Clients that need stable paths (long-lived
|
||||
connections, bookmarks) should use the ID form.
|
||||
Loading…
Reference in New Issue