Revise 9P design: nested hierarchy, EDSL sketch
- Sections nest as directories mirroring org structure - One file per property (title, keyword, tags, etc.) - ID from CUSTOM_ID > ID > slug+hash - EDSL pattern from ollie's virtfs for declarative namespace
This commit is contained in:
Levi Neely
parent
030ac3beeb
commit
0c5f56aad9
398
doc/9P_DESIGN.md
398
doc/9P_DESIGN.md
|
|
@ -1,270 +1,184 @@
|
|||
# Orkmode 9P Server Design
|
||||
# Orkmode 9P Namespace Design v2
|
||||
|
||||
Status: **Draft — needs decisions**
|
||||
## Structure
|
||||
|
||||
## Goal
|
||||
|
||||
A 9P server that exposes org documents so any frontend (CLI, KDE app, web) can read/write by simple file operations. No client library needed.
|
||||
|
||||
## Core Questions
|
||||
|
||||
### 1. Section Hierarchy
|
||||
|
||||
Org sections nest: `* A` contains `** B` contains `*** C`.
|
||||
|
||||
**Option A: Nested directories**
|
||||
```
|
||||
sections/
|
||||
└── a/
|
||||
├── title
|
||||
├── todo
|
||||
└── children/
|
||||
└── b/
|
||||
ork/
|
||||
├── ctl # rdwr: "reload", "quit", "rescan"
|
||||
├── idx # read: list of org files (name\tpath\ttitle)
|
||||
├── keywords # read: configured TODO keywords
|
||||
├── capture # write: append "* TODO text" to inbox.org
|
||||
│
|
||||
├── agenda/ # computed views
|
||||
│ ├── today # read: items scheduled/deadline today
|
||||
│ ├── week # read: items this week
|
||||
│ └── todos # read: all open TODOs across files
|
||||
│
|
||||
└── {doc}/ # e.g., "todo" for todo.org
|
||||
├── raw # read/write: full org file content
|
||||
├── meta # read: title, author, filetags (tsv)
|
||||
│
|
||||
└── {h1-id}/ # section by ID (nested hierarchy)
|
||||
├── headline # read: full headline text "* TODO [#A] Title :tag:"
|
||||
├── title # read/write: plain title
|
||||
├── keyword # read/write: TODO/DONE/etc or empty
|
||||
├── priority # read/write: A/B/C or empty
|
||||
├── tags # read/write: colon-separated "work:urgent"
|
||||
├── scheduled # read/write: "<2025-01-15 Wed>"
|
||||
├── deadline # read/write: "<2025-01-20 Mon>"
|
||||
├── properties # read/write: "KEY=value\n" lines
|
||||
├── body # read/write: section body (content only, not children)
|
||||
│
|
||||
└── {h2-id}/ # nested children, same structure
|
||||
├── headline
|
||||
├── title
|
||||
└── children/
|
||||
└── c/
|
||||
└── ...
|
||||
```
|
||||
- Pro: Reflects actual structure
|
||||
- Con: Deep paths (`sections/a/children/b/children/c/title`), awkward to enumerate all
|
||||
|
||||
**Option B: Flat with parent refs**
|
||||
```
|
||||
sections/
|
||||
├── idx # a\t1\tTitle A\tTODO\t\t(parent empty = root)
|
||||
│ # b\t2\tTitle B\t\ta (parent = a)
|
||||
│ # c\t3\tTitle C\t\tb (parent = b)
|
||||
├── a/
|
||||
│ ├── title
|
||||
│ ├── parent # (empty)
|
||||
│ └── children # b
|
||||
├── b/
|
||||
│ ├── title
|
||||
│ ├── parent # a
|
||||
│ └── children # c
|
||||
└── c/
|
||||
├── title
|
||||
├── parent # b
|
||||
└── children # (empty)
|
||||
```
|
||||
- Pro: Easy to list all sections, reconstruct tree client-side
|
||||
- Con: Hierarchy implicit, extra `parent`/`children` files
|
||||
## Operations by File
|
||||
|
||||
| Path | Mode | Read | Write | Notes |
|
||||
|------|------|------|-------|-------|
|
||||
| `ctl` | rdwr | — | command | "reload" rescans, "quit" exits |
|
||||
| `idx` | r | file list | — | `name\tpath\ttitle` per line |
|
||||
| `keywords` | r | keyword config | — | `TODO NEXT \| DONE` |
|
||||
| `capture` | w | — | org text | Appends to inbox.org |
|
||||
| `agenda/*` | r | computed | — | Filtered section lists |
|
||||
| `{doc}/raw` | rw | full content | replace all | Escape hatch |
|
||||
| `{doc}/meta` | r | metadata | — | |
|
||||
| `{doc}/{id}/headline` | r | full line | — | Reconstructed |
|
||||
| `{doc}/{id}/title` | rw | plain text | update | |
|
||||
| `{doc}/{id}/keyword` | rw | keyword | update | |
|
||||
| `{doc}/{id}/priority` | rw | A/B/C/empty | update | |
|
||||
| `{doc}/{id}/tags` | rw | colon-sep | replace | Write "work:urgent" |
|
||||
| `{doc}/{id}/scheduled` | rw | timestamp | update | Write "<2025-01-15>" |
|
||||
| `{doc}/{id}/deadline` | rw | timestamp | update | |
|
||||
| `{doc}/{id}/properties` | rw | key=val lines | replace | |
|
||||
| `{doc}/{id}/body` | rw | content | replace | Text between headline and children |
|
||||
|
||||
## ID Generation
|
||||
|
||||
1. `CUSTOM_ID` property if present
|
||||
2. Else `ID` property if present
|
||||
3. Else slugified title + `-` + short hash of position
|
||||
|
||||
IDs are stable unless CUSTOM_ID/ID changes or section is deleted.
|
||||
|
||||
## Nested Hierarchy
|
||||
|
||||
The hierarchy mirrors org structure:
|
||||
|
||||
**Option C: Hierarchical IDs**
|
||||
```
|
||||
sections/
|
||||
├── idx # 1\tTitle A\tTODO
|
||||
│ # 1.1\tTitle B
|
||||
│ # 1.1.1\tTitle C
|
||||
├── 1/
|
||||
│ └── title
|
||||
├── 1.1/
|
||||
│ └── title
|
||||
└── 1.1.1/
|
||||
todo/
|
||||
├── project-alpha/ # * TODO Project Alpha
|
||||
│ ├── title # "Project Alpha"
|
||||
│ ├── keyword # "TODO"
|
||||
│ ├── design-doc/ # ** TODO Design doc
|
||||
│ │ ├── title
|
||||
│ │ └── review/ # *** Review
|
||||
│ │ └── title
|
||||
│ └── implementation/ # ** Implementation
|
||||
│ └── title
|
||||
└── daily-standup/ # * TODO Daily standup
|
||||
└── title
|
||||
```
|
||||
- Pro: ID encodes position, no extra files
|
||||
- Con: IDs change when sections move, not stable for references
|
||||
|
||||
**Leaning toward:** Option B (flat + parent refs). Stable IDs (from CUSTOM_ID or hash), easy enumeration.
|
||||
Listing `todo/project-alpha/` returns children: `design-doc`, `implementation` (plus property files).
|
||||
|
||||
---
|
||||
## Write Semantics
|
||||
|
||||
### 2. Property Granularity
|
||||
**Simple properties** (title, keyword, priority):
|
||||
- Write replaces value
|
||||
- Server updates .org file atomically
|
||||
|
||||
How to expose section attributes?
|
||||
**Tags**:
|
||||
- Write replaces all tags: `echo "work:urgent" > tags`
|
||||
- To add: read, append, write back (client responsibility)
|
||||
- Or: future `ctl` commands: `echo "tag +urgent" > ctl`
|
||||
|
||||
**Option A: One file per property**
|
||||
```
|
||||
sections/review-pr/
|
||||
├── title # Review PR
|
||||
├── todo # TODO
|
||||
├── priority # A
|
||||
├── tags # work:urgent
|
||||
├── properties # CUSTOM_ID=review-pr\nEFFORT=2h
|
||||
├── planning # DEADLINE: <2025-01-20 Mon>
|
||||
└── body # The actual content...
|
||||
```
|
||||
- Pro: `echo "DONE" > todo` just works, Unix composable
|
||||
- Con: Many small files, multiple reads to get full section
|
||||
**Timestamps** (scheduled, deadline):
|
||||
- Write org timestamp format: `<2025-01-15 Wed>`
|
||||
- Or simplified: `2025-01-15` (server adds day name)
|
||||
- Write empty to remove
|
||||
|
||||
**Option B: Single structured file per section**
|
||||
```
|
||||
sections/review-pr # title: Review PR
|
||||
# todo: TODO
|
||||
# priority: A
|
||||
# tags: work urgent
|
||||
# deadline: 2025-01-20
|
||||
# body:
|
||||
# The actual content...
|
||||
```
|
||||
- Pro: One read gets everything
|
||||
- Con: Parsing needed, writes replace whole thing
|
||||
**Body**:
|
||||
- Write replaces section body content
|
||||
- Does NOT affect children (only text between headline and first child)
|
||||
|
||||
**Option C: Both**
|
||||
```
|
||||
sections/review-pr/
|
||||
├── data # Structured (all properties)
|
||||
├── title # Just title (convenience)
|
||||
├── todo # Just todo (convenience)
|
||||
└── raw # Original org text for this section
|
||||
```
|
||||
- Pro: Flexible — use what fits
|
||||
- Con: Redundancy, sync complexity
|
||||
**Properties**:
|
||||
- Write replaces entire drawer
|
||||
- Format: `KEY=value\n` lines
|
||||
|
||||
**Leaning toward:** Option A (one file per property). Matches Unix philosophy. Reads are cheap.
|
||||
## EDSL Example (Rust)
|
||||
|
||||
---
|
||||
Porting ollie's virtfs pattern:
|
||||
|
||||
### 3. Change Notification
|
||||
```rust
|
||||
fn build_namespace(state: &OrkState) -> FsNode {
|
||||
dir("/",
|
||||
file("ctl", 0o222, rdwr(|data| state.handle_ctl(data))),
|
||||
file("idx", 0o444, read(|| state.list_docs())),
|
||||
file("keywords", 0o444, read(|| state.keywords())),
|
||||
file("capture", 0o222, write(|data| state.capture(data))),
|
||||
|
||||
dir("agenda",
|
||||
file("today", 0o444, read(|| state.agenda_today())),
|
||||
file("week", 0o444, read(|| state.agenda_week())),
|
||||
file("todos", 0o444, read(|| state.all_todos())),
|
||||
),
|
||||
|
||||
each("{doc}", || state.doc_names(), |doc| {
|
||||
dir(doc,
|
||||
file("raw", 0o644,
|
||||
read(|| state.doc_raw(doc)),
|
||||
write(|data| state.write_doc_raw(doc, data))),
|
||||
file("meta", 0o444, read(|| state.doc_meta(doc))),
|
||||
|
||||
each("{section}", || state.section_ids(doc), |id| {
|
||||
section_node(state, doc, id)
|
||||
}),
|
||||
)
|
||||
}),
|
||||
)
|
||||
}
|
||||
|
||||
How does a GUI know when to refresh?
|
||||
|
||||
**Option A: No notification (poll)**
|
||||
- Client re-reads periodically or on user action
|
||||
- Simple, works everywhere
|
||||
- Latency for external changes
|
||||
|
||||
**Option B: Blocking read (statewait pattern)**
|
||||
- `cat sections/idx.wait` blocks until change, returns new content
|
||||
- Client loops: read → update UI → read again
|
||||
- Complex server state, client must handle reconnects
|
||||
|
||||
**Option C: External mechanism**
|
||||
- Server writes to a Unix socket / named pipe on change
|
||||
- Or: client uses inotify on .org files directly
|
||||
- Separates concerns
|
||||
|
||||
**Option D: Skip for v1**
|
||||
- CLI doesn't need it
|
||||
- GUI can poll or watch .org files with inotify
|
||||
- Add later if needed
|
||||
|
||||
**Leaning toward:** Option D for now. Don't over-engineer before we have a GUI.
|
||||
|
||||
---
|
||||
|
||||
### 4. Write Semantics
|
||||
|
||||
How do writes flow back to .org files?
|
||||
|
||||
**Option A: Direct property writes**
|
||||
```bash
|
||||
echo "DONE" > sections/review-pr/todo
|
||||
# Server rewrites work.org with updated TODO state
|
||||
```
|
||||
- Pro: Intuitive
|
||||
- Con: Partial writes tricky (tags append vs replace?)
|
||||
|
||||
**Option B: Command-based writes**
|
||||
```bash
|
||||
echo "todo DONE" > sections/review-pr/ctl
|
||||
echo "tag +urgent" > sections/review-pr/ctl
|
||||
echo "tag -work" > sections/review-pr/ctl
|
||||
```
|
||||
- Pro: Clear semantics, atomic operations
|
||||
- Con: Less intuitive than direct file writes
|
||||
|
||||
**Option C: Document-level raw write**
|
||||
```bash
|
||||
# Edit raw org content
|
||||
cat docs/work.org/raw > /tmp/edit.org
|
||||
vim /tmp/edit.org
|
||||
cat /tmp/edit.org > docs/work.org/raw
|
||||
# Server reparses and updates index
|
||||
```
|
||||
- Pro: Full control, no semantic mismatch
|
||||
- Con: Client does all the work
|
||||
|
||||
**Leaning toward:** Option A for simple cases (todo, priority), Option B (ctl) for complex ops (tag add/remove). Option C always available as escape hatch.
|
||||
|
||||
---
|
||||
|
||||
### 5. Agenda / Query Views
|
||||
|
||||
How to handle cross-document queries?
|
||||
|
||||
**Option A: Computed directories**
|
||||
```
|
||||
agenda/
|
||||
├── today # Sections with deadline/scheduled today
|
||||
├── week # This week
|
||||
├── todos # All open TODOs
|
||||
└── tags/
|
||||
└── {tag} # Sections with this tag
|
||||
```
|
||||
- Reads compute result on demand
|
||||
- Simple, no index maintenance
|
||||
|
||||
**Option B: Search file (rdwr)**
|
||||
```
|
||||
echo "todo:TODO tag:work" > search
|
||||
cat search
|
||||
# Returns matching section IDs
|
||||
```
|
||||
- More flexible queries
|
||||
- Single entry point
|
||||
|
||||
**Option C: Both**
|
||||
- Common queries as directories
|
||||
- `search` for custom queries
|
||||
|
||||
**Leaning toward:** Start with Option A (fixed views). Add search later.
|
||||
|
||||
---
|
||||
|
||||
## Proposed Structure (v1)
|
||||
|
||||
```
|
||||
/ork/
|
||||
├── ctl # Commands: reload, quit
|
||||
├── docs/
|
||||
│ ├── idx # doc\tpath\ttitle\ttodos\tdone (per line)
|
||||
│ └── {doc}/
|
||||
│ ├── raw # Full .org content (r/w)
|
||||
│ ├── meta # title, author, etc. (read-only)
|
||||
│ └── sections/
|
||||
│ ├── idx # id\tlevel\ttitle\ttodo\tpriority\ttags\tparent
|
||||
│ └── {id}/
|
||||
│ ├── title
|
||||
│ ├── todo
|
||||
│ ├── priority
|
||||
│ ├── tags
|
||||
│ ├── planning
|
||||
│ ├── properties
|
||||
│ ├── body
|
||||
│ ├── parent
|
||||
│ ├── children
|
||||
│ └── ctl # Commands: todo X, tag +x, tag -x
|
||||
├── agenda/
|
||||
│ ├── today
|
||||
│ ├── week
|
||||
│ └── todos
|
||||
└── capture # Write "* TODO text" to append to inbox
|
||||
fn section_node(state: &OrkState, doc: &str, id: &str) -> FsNode {
|
||||
dir(id,
|
||||
file("headline", 0o444, read(|| state.headline(doc, id))),
|
||||
file("title", 0o644,
|
||||
read(|| state.title(doc, id)),
|
||||
write(|data| state.set_title(doc, id, data))),
|
||||
file("keyword", 0o644,
|
||||
read(|| state.keyword(doc, id)),
|
||||
write(|data| state.set_keyword(doc, id, data))),
|
||||
// ... etc
|
||||
|
||||
// Nested children
|
||||
each("{child}", || state.child_ids(doc, id), |child_id| {
|
||||
section_node(state, doc, child_id)
|
||||
}),
|
||||
)
|
||||
}
|
||||
```
|
||||
|
||||
## Open Questions
|
||||
## Questions Resolved
|
||||
|
||||
1. **ID stability** — If CUSTOM_ID doesn't exist, generate from title hash? What if title changes?
|
||||
1. **Hierarchy**: Nested directories reflecting org structure ✓
|
||||
2. **IDs**: CUSTOM_ID > ID > slug+hash ✓
|
||||
3. **Operations**: Read/write per property, rdwr for ctl ✓
|
||||
4. **Tags**: Simple replace (add/remove via read-modify-write or future ctl)
|
||||
5. **Body**: Section content only, not children ✓
|
||||
|
||||
2. **Concurrency** — Multiple writers (emacs + orkmode)? Last-write-wins, or detect conflicts?
|
||||
## Not Needed (for v1)
|
||||
|
||||
3. **Large files** — Stream body content, or load all in memory?
|
||||
- `statewait` / blocking reads — GUI can poll or use inotify on .org files
|
||||
- `search` file — agenda views sufficient for now
|
||||
- Symlinks — just return data directly
|
||||
|
||||
4. **Symbolic links** — Should `agenda/today` entries be symlinks to actual section dirs?
|
||||
## Implementation Order
|
||||
|
||||
5. **Error handling** — Write fails (invalid todo keyword) — return error how? (9P has Rerror)
|
||||
|
||||
---
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
1. **ork-server crate** — 9P server using `nine` crate
|
||||
2. **VirtFS layer** — Map namespace to org-parser operations
|
||||
3. **File watcher** — Detect external .org changes, reparse
|
||||
4. **CLI update** — `ork` talks to server instead of parsing directly (optional)
|
||||
|
||||
## References
|
||||
|
||||
- [nine crate](https://crates.io/crates/nine) — Rust 9P2000
|
||||
- [9P protocol](http://9p.io/sys/man/5/INDEX.html) — Plan 9 manual
|
||||
- ~/src/ollie — Working 9P implementation (Go)
|
||||
1. Port virtfs EDSL to Rust
|
||||
2. Build 9P server using `nine` crate
|
||||
3. Implement read-only namespace first
|
||||
4. Add write operations
|
||||
5. File watching for external changes
|
||||
|
|
|
|||
Loading…
Reference in New Issue