Revise 9P design: nested hierarchy, EDSL sketch
- Sections nest as directories mirroring org structure - One file per property (title, keyword, tags, etc.) - ID from CUSTOM_ID > ID > slug+hash - EDSL pattern from ollie's virtfs for declarative namespace
This commit is contained in:
Levi Neely
parent
030ac3beeb
commit
0c5f56aad9
398
doc/9P_DESIGN.md
398
doc/9P_DESIGN.md
|
|
@ -1,270 +1,184 @@
|
||||||
# Orkmode 9P Server Design
|
# Orkmode 9P Namespace Design v2
|
||||||
|
|
||||||
Status: **Draft — needs decisions**
|
## Structure
|
||||||
|
|
||||||
## Goal
|
|
||||||
|
|
||||||
A 9P server that exposes org documents so any frontend (CLI, KDE app, web) can read/write by simple file operations. No client library needed.
|
|
||||||
|
|
||||||
## Core Questions
|
|
||||||
|
|
||||||
### 1. Section Hierarchy
|
|
||||||
|
|
||||||
Org sections nest: `* A` contains `** B` contains `*** C`.
|
|
||||||
|
|
||||||
**Option A: Nested directories**
|
|
||||||
```
|
```
|
||||||
sections/
|
ork/
|
||||||
└── a/
|
├── ctl # rdwr: "reload", "quit", "rescan"
|
||||||
├── title
|
├── idx # read: list of org files (name\tpath\ttitle)
|
||||||
├── todo
|
├── keywords # read: configured TODO keywords
|
||||||
└── children/
|
├── capture # write: append "* TODO text" to inbox.org
|
||||||
└── b/
|
│
|
||||||
|
├── agenda/ # computed views
|
||||||
|
│ ├── today # read: items scheduled/deadline today
|
||||||
|
│ ├── week # read: items this week
|
||||||
|
│ └── todos # read: all open TODOs across files
|
||||||
|
│
|
||||||
|
└── {doc}/ # e.g., "todo" for todo.org
|
||||||
|
├── raw # read/write: full org file content
|
||||||
|
├── meta # read: title, author, filetags (tsv)
|
||||||
|
│
|
||||||
|
└── {h1-id}/ # section by ID (nested hierarchy)
|
||||||
|
├── headline # read: full headline text "* TODO [#A] Title :tag:"
|
||||||
|
├── title # read/write: plain title
|
||||||
|
├── keyword # read/write: TODO/DONE/etc or empty
|
||||||
|
├── priority # read/write: A/B/C or empty
|
||||||
|
├── tags # read/write: colon-separated "work:urgent"
|
||||||
|
├── scheduled # read/write: "<2025-01-15 Wed>"
|
||||||
|
├── deadline # read/write: "<2025-01-20 Mon>"
|
||||||
|
├── properties # read/write: "KEY=value\n" lines
|
||||||
|
├── body # read/write: section body (content only, not children)
|
||||||
|
│
|
||||||
|
└── {h2-id}/ # nested children, same structure
|
||||||
|
├── headline
|
||||||
├── title
|
├── title
|
||||||
└── children/
|
└── ...
|
||||||
└── c/
|
|
||||||
```
|
```
|
||||||
- Pro: Reflects actual structure
|
|
||||||
- Con: Deep paths (`sections/a/children/b/children/c/title`), awkward to enumerate all
|
|
||||||
|
|
||||||
**Option B: Flat with parent refs**
|
## Operations by File
|
||||||
```
|
|
||||||
sections/
|
| Path | Mode | Read | Write | Notes |
|
||||||
├── idx # a\t1\tTitle A\tTODO\t\t(parent empty = root)
|
|------|------|------|-------|-------|
|
||||||
│ # b\t2\tTitle B\t\ta (parent = a)
|
| `ctl` | rdwr | — | command | "reload" rescans, "quit" exits |
|
||||||
│ # c\t3\tTitle C\t\tb (parent = b)
|
| `idx` | r | file list | — | `name\tpath\ttitle` per line |
|
||||||
├── a/
|
| `keywords` | r | keyword config | — | `TODO NEXT \| DONE` |
|
||||||
│ ├── title
|
| `capture` | w | — | org text | Appends to inbox.org |
|
||||||
│ ├── parent # (empty)
|
| `agenda/*` | r | computed | — | Filtered section lists |
|
||||||
│ └── children # b
|
| `{doc}/raw` | rw | full content | replace all | Escape hatch |
|
||||||
├── b/
|
| `{doc}/meta` | r | metadata | — | |
|
||||||
│ ├── title
|
| `{doc}/{id}/headline` | r | full line | — | Reconstructed |
|
||||||
│ ├── parent # a
|
| `{doc}/{id}/title` | rw | plain text | update | |
|
||||||
│ └── children # c
|
| `{doc}/{id}/keyword` | rw | keyword | update | |
|
||||||
└── c/
|
| `{doc}/{id}/priority` | rw | A/B/C/empty | update | |
|
||||||
├── title
|
| `{doc}/{id}/tags` | rw | colon-sep | replace | Write "work:urgent" |
|
||||||
├── parent # b
|
| `{doc}/{id}/scheduled` | rw | timestamp | update | Write "<2025-01-15>" |
|
||||||
└── children # (empty)
|
| `{doc}/{id}/deadline` | rw | timestamp | update | |
|
||||||
```
|
| `{doc}/{id}/properties` | rw | key=val lines | replace | |
|
||||||
- Pro: Easy to list all sections, reconstruct tree client-side
|
| `{doc}/{id}/body` | rw | content | replace | Text between headline and children |
|
||||||
- Con: Hierarchy implicit, extra `parent`/`children` files
|
|
||||||
|
## ID Generation
|
||||||
|
|
||||||
|
1. `CUSTOM_ID` property if present
|
||||||
|
2. Else `ID` property if present
|
||||||
|
3. Else slugified title + `-` + short hash of position
|
||||||
|
|
||||||
|
IDs are stable unless CUSTOM_ID/ID changes or section is deleted.
|
||||||
|
|
||||||
|
## Nested Hierarchy
|
||||||
|
|
||||||
|
The hierarchy mirrors org structure:
|
||||||
|
|
||||||
**Option C: Hierarchical IDs**
|
|
||||||
```
|
```
|
||||||
sections/
|
todo/
|
||||||
├── idx # 1\tTitle A\tTODO
|
├── project-alpha/ # * TODO Project Alpha
|
||||||
│ # 1.1\tTitle B
|
│ ├── title # "Project Alpha"
|
||||||
│ # 1.1.1\tTitle C
|
│ ├── keyword # "TODO"
|
||||||
├── 1/
|
│ ├── design-doc/ # ** TODO Design doc
|
||||||
│ └── title
|
│ │ ├── title
|
||||||
├── 1.1/
|
│ │ └── review/ # *** Review
|
||||||
│ └── title
|
│ │ └── title
|
||||||
└── 1.1.1/
|
│ └── implementation/ # ** Implementation
|
||||||
|
│ └── title
|
||||||
|
└── daily-standup/ # * TODO Daily standup
|
||||||
└── title
|
└── title
|
||||||
```
|
```
|
||||||
- Pro: ID encodes position, no extra files
|
|
||||||
- Con: IDs change when sections move, not stable for references
|
|
||||||
|
|
||||||
**Leaning toward:** Option B (flat + parent refs). Stable IDs (from CUSTOM_ID or hash), easy enumeration.
|
Listing `todo/project-alpha/` returns children: `design-doc`, `implementation` (plus property files).
|
||||||
|
|
||||||
---
|
## Write Semantics
|
||||||
|
|
||||||
### 2. Property Granularity
|
**Simple properties** (title, keyword, priority):
|
||||||
|
- Write replaces value
|
||||||
|
- Server updates .org file atomically
|
||||||
|
|
||||||
How to expose section attributes?
|
**Tags**:
|
||||||
|
- Write replaces all tags: `echo "work:urgent" > tags`
|
||||||
|
- To add: read, append, write back (client responsibility)
|
||||||
|
- Or: future `ctl` commands: `echo "tag +urgent" > ctl`
|
||||||
|
|
||||||
**Option A: One file per property**
|
**Timestamps** (scheduled, deadline):
|
||||||
```
|
- Write org timestamp format: `<2025-01-15 Wed>`
|
||||||
sections/review-pr/
|
- Or simplified: `2025-01-15` (server adds day name)
|
||||||
├── title # Review PR
|
- Write empty to remove
|
||||||
├── todo # TODO
|
|
||||||
├── priority # A
|
|
||||||
├── tags # work:urgent
|
|
||||||
├── properties # CUSTOM_ID=review-pr\nEFFORT=2h
|
|
||||||
├── planning # DEADLINE: <2025-01-20 Mon>
|
|
||||||
└── body # The actual content...
|
|
||||||
```
|
|
||||||
- Pro: `echo "DONE" > todo` just works, Unix composable
|
|
||||||
- Con: Many small files, multiple reads to get full section
|
|
||||||
|
|
||||||
**Option B: Single structured file per section**
|
**Body**:
|
||||||
```
|
- Write replaces section body content
|
||||||
sections/review-pr # title: Review PR
|
- Does NOT affect children (only text between headline and first child)
|
||||||
# todo: TODO
|
|
||||||
# priority: A
|
|
||||||
# tags: work urgent
|
|
||||||
# deadline: 2025-01-20
|
|
||||||
# body:
|
|
||||||
# The actual content...
|
|
||||||
```
|
|
||||||
- Pro: One read gets everything
|
|
||||||
- Con: Parsing needed, writes replace whole thing
|
|
||||||
|
|
||||||
**Option C: Both**
|
**Properties**:
|
||||||
```
|
- Write replaces entire drawer
|
||||||
sections/review-pr/
|
- Format: `KEY=value\n` lines
|
||||||
├── data # Structured (all properties)
|
|
||||||
├── title # Just title (convenience)
|
|
||||||
├── todo # Just todo (convenience)
|
|
||||||
└── raw # Original org text for this section
|
|
||||||
```
|
|
||||||
- Pro: Flexible — use what fits
|
|
||||||
- Con: Redundancy, sync complexity
|
|
||||||
|
|
||||||
**Leaning toward:** Option A (one file per property). Matches Unix philosophy. Reads are cheap.
|
## EDSL Example (Rust)
|
||||||
|
|
||||||
---
|
Porting ollie's virtfs pattern:
|
||||||
|
|
||||||
### 3. Change Notification
|
```rust
|
||||||
|
fn build_namespace(state: &OrkState) -> FsNode {
|
||||||
|
dir("/",
|
||||||
|
file("ctl", 0o222, rdwr(|data| state.handle_ctl(data))),
|
||||||
|
file("idx", 0o444, read(|| state.list_docs())),
|
||||||
|
file("keywords", 0o444, read(|| state.keywords())),
|
||||||
|
file("capture", 0o222, write(|data| state.capture(data))),
|
||||||
|
|
||||||
|
dir("agenda",
|
||||||
|
file("today", 0o444, read(|| state.agenda_today())),
|
||||||
|
file("week", 0o444, read(|| state.agenda_week())),
|
||||||
|
file("todos", 0o444, read(|| state.all_todos())),
|
||||||
|
),
|
||||||
|
|
||||||
|
each("{doc}", || state.doc_names(), |doc| {
|
||||||
|
dir(doc,
|
||||||
|
file("raw", 0o644,
|
||||||
|
read(|| state.doc_raw(doc)),
|
||||||
|
write(|data| state.write_doc_raw(doc, data))),
|
||||||
|
file("meta", 0o444, read(|| state.doc_meta(doc))),
|
||||||
|
|
||||||
|
each("{section}", || state.section_ids(doc), |id| {
|
||||||
|
section_node(state, doc, id)
|
||||||
|
}),
|
||||||
|
)
|
||||||
|
}),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
How does a GUI know when to refresh?
|
fn section_node(state: &OrkState, doc: &str, id: &str) -> FsNode {
|
||||||
|
dir(id,
|
||||||
**Option A: No notification (poll)**
|
file("headline", 0o444, read(|| state.headline(doc, id))),
|
||||||
- Client re-reads periodically or on user action
|
file("title", 0o644,
|
||||||
- Simple, works everywhere
|
read(|| state.title(doc, id)),
|
||||||
- Latency for external changes
|
write(|data| state.set_title(doc, id, data))),
|
||||||
|
file("keyword", 0o644,
|
||||||
**Option B: Blocking read (statewait pattern)**
|
read(|| state.keyword(doc, id)),
|
||||||
- `cat sections/idx.wait` blocks until change, returns new content
|
write(|data| state.set_keyword(doc, id, data))),
|
||||||
- Client loops: read → update UI → read again
|
// ... etc
|
||||||
- Complex server state, client must handle reconnects
|
|
||||||
|
// Nested children
|
||||||
**Option C: External mechanism**
|
each("{child}", || state.child_ids(doc, id), |child_id| {
|
||||||
- Server writes to a Unix socket / named pipe on change
|
section_node(state, doc, child_id)
|
||||||
- Or: client uses inotify on .org files directly
|
}),
|
||||||
- Separates concerns
|
)
|
||||||
|
}
|
||||||
**Option D: Skip for v1**
|
|
||||||
- CLI doesn't need it
|
|
||||||
- GUI can poll or watch .org files with inotify
|
|
||||||
- Add later if needed
|
|
||||||
|
|
||||||
**Leaning toward:** Option D for now. Don't over-engineer before we have a GUI.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 4. Write Semantics
|
|
||||||
|
|
||||||
How do writes flow back to .org files?
|
|
||||||
|
|
||||||
**Option A: Direct property writes**
|
|
||||||
```bash
|
|
||||||
echo "DONE" > sections/review-pr/todo
|
|
||||||
# Server rewrites work.org with updated TODO state
|
|
||||||
```
|
|
||||||
- Pro: Intuitive
|
|
||||||
- Con: Partial writes tricky (tags append vs replace?)
|
|
||||||
|
|
||||||
**Option B: Command-based writes**
|
|
||||||
```bash
|
|
||||||
echo "todo DONE" > sections/review-pr/ctl
|
|
||||||
echo "tag +urgent" > sections/review-pr/ctl
|
|
||||||
echo "tag -work" > sections/review-pr/ctl
|
|
||||||
```
|
|
||||||
- Pro: Clear semantics, atomic operations
|
|
||||||
- Con: Less intuitive than direct file writes
|
|
||||||
|
|
||||||
**Option C: Document-level raw write**
|
|
||||||
```bash
|
|
||||||
# Edit raw org content
|
|
||||||
cat docs/work.org/raw > /tmp/edit.org
|
|
||||||
vim /tmp/edit.org
|
|
||||||
cat /tmp/edit.org > docs/work.org/raw
|
|
||||||
# Server reparses and updates index
|
|
||||||
```
|
|
||||||
- Pro: Full control, no semantic mismatch
|
|
||||||
- Con: Client does all the work
|
|
||||||
|
|
||||||
**Leaning toward:** Option A for simple cases (todo, priority), Option B (ctl) for complex ops (tag add/remove). Option C always available as escape hatch.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 5. Agenda / Query Views
|
|
||||||
|
|
||||||
How to handle cross-document queries?
|
|
||||||
|
|
||||||
**Option A: Computed directories**
|
|
||||||
```
|
|
||||||
agenda/
|
|
||||||
├── today # Sections with deadline/scheduled today
|
|
||||||
├── week # This week
|
|
||||||
├── todos # All open TODOs
|
|
||||||
└── tags/
|
|
||||||
└── {tag} # Sections with this tag
|
|
||||||
```
|
|
||||||
- Reads compute result on demand
|
|
||||||
- Simple, no index maintenance
|
|
||||||
|
|
||||||
**Option B: Search file (rdwr)**
|
|
||||||
```
|
|
||||||
echo "todo:TODO tag:work" > search
|
|
||||||
cat search
|
|
||||||
# Returns matching section IDs
|
|
||||||
```
|
|
||||||
- More flexible queries
|
|
||||||
- Single entry point
|
|
||||||
|
|
||||||
**Option C: Both**
|
|
||||||
- Common queries as directories
|
|
||||||
- `search` for custom queries
|
|
||||||
|
|
||||||
**Leaning toward:** Start with Option A (fixed views). Add search later.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Proposed Structure (v1)
|
|
||||||
|
|
||||||
```
|
|
||||||
/ork/
|
|
||||||
├── ctl # Commands: reload, quit
|
|
||||||
├── docs/
|
|
||||||
│ ├── idx # doc\tpath\ttitle\ttodos\tdone (per line)
|
|
||||||
│ └── {doc}/
|
|
||||||
│ ├── raw # Full .org content (r/w)
|
|
||||||
│ ├── meta # title, author, etc. (read-only)
|
|
||||||
│ └── sections/
|
|
||||||
│ ├── idx # id\tlevel\ttitle\ttodo\tpriority\ttags\tparent
|
|
||||||
│ └── {id}/
|
|
||||||
│ ├── title
|
|
||||||
│ ├── todo
|
|
||||||
│ ├── priority
|
|
||||||
│ ├── tags
|
|
||||||
│ ├── planning
|
|
||||||
│ ├── properties
|
|
||||||
│ ├── body
|
|
||||||
│ ├── parent
|
|
||||||
│ ├── children
|
|
||||||
│ └── ctl # Commands: todo X, tag +x, tag -x
|
|
||||||
├── agenda/
|
|
||||||
│ ├── today
|
|
||||||
│ ├── week
|
|
||||||
│ └── todos
|
|
||||||
└── capture # Write "* TODO text" to append to inbox
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## Open Questions
|
## Questions Resolved
|
||||||
|
|
||||||
1. **ID stability** — If CUSTOM_ID doesn't exist, generate from title hash? What if title changes?
|
1. **Hierarchy**: Nested directories reflecting org structure ✓
|
||||||
|
2. **IDs**: CUSTOM_ID > ID > slug+hash ✓
|
||||||
|
3. **Operations**: Read/write per property, rdwr for ctl ✓
|
||||||
|
4. **Tags**: Simple replace (add/remove via read-modify-write or future ctl)
|
||||||
|
5. **Body**: Section content only, not children ✓
|
||||||
|
|
||||||
2. **Concurrency** — Multiple writers (emacs + orkmode)? Last-write-wins, or detect conflicts?
|
## Not Needed (for v1)
|
||||||
|
|
||||||
3. **Large files** — Stream body content, or load all in memory?
|
- `statewait` / blocking reads — GUI can poll or use inotify on .org files
|
||||||
|
- `search` file — agenda views sufficient for now
|
||||||
|
- Symlinks — just return data directly
|
||||||
|
|
||||||
4. **Symbolic links** — Should `agenda/today` entries be symlinks to actual section dirs?
|
## Implementation Order
|
||||||
|
|
||||||
5. **Error handling** — Write fails (invalid todo keyword) — return error how? (9P has Rerror)
|
1. Port virtfs EDSL to Rust
|
||||||
|
2. Build 9P server using `nine` crate
|
||||||
---
|
3. Implement read-only namespace first
|
||||||
|
4. Add write operations
|
||||||
## Implementation Plan
|
5. File watching for external changes
|
||||||
|
|
||||||
1. **ork-server crate** — 9P server using `nine` crate
|
|
||||||
2. **VirtFS layer** — Map namespace to org-parser operations
|
|
||||||
3. **File watcher** — Detect external .org changes, reparse
|
|
||||||
4. **CLI update** — `ork` talks to server instead of parsing directly (optional)
|
|
||||||
|
|
||||||
## References
|
|
||||||
|
|
||||||
- [nine crate](https://crates.io/crates/nine) — Rust 9P2000
|
|
||||||
- [9P protocol](http://9p.io/sys/man/5/INDEX.html) — Plan 9 manual
|
|
||||||
- ~/src/ollie — Working 9P implementation (Go)
|
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue