sam: ^/$ are per-line (MultilineOption), matching plan9 regexp(7)
Verified against plan9port: regexp(7) defines ^ as 'beginning of a line' and $ as 'end of a line', and sam's regexp.c BOL (p==0 || prev=='\\n') / EOL (next char '\\n') confirm per-line anchoring. The engine was matching with buffer-wide anchors, so ,s/^/> /g only touched the first line and the sam idiom ,x/.+/ s/^/> / failed to prefix each line. Fix: compile every pattern with QRegularExpression::MultilineOption. '.' still does not cross newlines (sam-faithful; PCRE default). Add 3 regression tests (caretIsPerLine, dollarIsPerLine, samAnchorIdiom); 30/30 engine tests. Docs: correct docs/SAM.md — ^/$ per-line is faithful, not a deviation; reframe PCRE extras as an escape hatch (prefer structural composition), not 'free upgrades'. Fix the panel hint and PLAN.md accordingly.
This commit is contained in:
parent
997055f0b5
commit
4cdd762bb6
18
docs/PLAN.md
18
docs/PLAN.md
|
|
@ -386,14 +386,16 @@ program, click **Run** (or Ctrl+Return); each Run is one undo step.
|
|||
- **Deviations (documented)**: regexes use `QRegularExpression` (PCRE), not
|
||||
plan9 `regexp(7)`. Everyday patterns match identically; the one semantic
|
||||
divergence is leftmost-greedy vs sam's leftmost-longest (bites only on
|
||||
overlapping alternation like `a|ab`), and `^`/`$` anchor buffer-wide not
|
||||
per-line by default. katesam gains PCRE extras (`\d \w \b`, lookahead,
|
||||
non-greedy, inline flags). Full user-facing treatment with verified examples
|
||||
and workarounds in **`docs/SAM.md`**. Out of scope: multi-file menu
|
||||
(`b B n D`), external file I/O (`e r w f`), `"re"` file-addressing, and sam's
|
||||
own `u` (Kate's undo stack is the undo mechanism). The live GUI panel (tool
|
||||
view + transaction apply) is not exercisable headless and is unverified on a
|
||||
live display.
|
||||
overlapping alternation like `a|ab`). `^`/`$` are per-line (compiled with
|
||||
`MultilineOption`) and `.` does not cross newlines — both faithful to sam's
|
||||
`regexp(7)` (BOL/EOL in `regexp.c`). PCRE also accepts extras sam lacks
|
||||
(`\d \w \b`, lookahead, non-greedy, inline flags); the docs frame these as an
|
||||
escape hatch, not a feature — prefer structural composition. Full user-facing
|
||||
treatment with verified examples in **`docs/SAM.md`**. Out of scope: multi-file
|
||||
menu (`b B n D`), external file I/O (`e r w f`), `"re"` file-addressing, and
|
||||
sam's own `u` (Kate's undo stack is the undo mechanism). The live GUI panel
|
||||
(tool view + transaction apply) is not exercisable headless and is unverified
|
||||
on a live display.
|
||||
|
||||
## Build and test
|
||||
|
||||
|
|
|
|||
88
docs/SAM.md
88
docs/SAM.md
|
|
@ -15,10 +15,11 @@ Every example below is verified against the engine.
|
|||
- Everyday patterns behave **identically** to sam.
|
||||
- One real semantic difference: **PCRE is leftmost-greedy, sam is
|
||||
leftmost-longest.** It only bites on *overlapping alternation* (`a|ab`).
|
||||
- `^` and `$` anchor to the **whole buffer, not each line**, by default. This is
|
||||
the most common surprise. Use a structural loop or the `(?m)` flag.
|
||||
- You *gain* PCRE features sam never had: `\d \w \s \b`, lookahead/lookbehind,
|
||||
non-greedy `*?`, backreferences, inline flags `(?i)` `(?m)` `(?s)`.
|
||||
- `^` and `$` are **per-line** (match at every line boundary), the same as sam's
|
||||
`regexp(7)`. `.` does not cross newlines, also the same as sam.
|
||||
- PCRE *also* offers things sam lacks (`\d \w \b`, lookahead, non-greedy, inline
|
||||
flags). They work, but prefer sam-style structural composition (`x g v`) — see
|
||||
"Extras" below.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -39,46 +40,34 @@ silently produce a different edit in katesam. Non-overlapping alternation
|
|||
(`cat|dog`) and all non-alternation patterns are unaffected.
|
||||
|
||||
Quantifier greediness is the same in both (`a.*c` on `axxcxxc` → whole string in
|
||||
both). katesam additionally offers non-greedy `.*?`:
|
||||
|
||||
| program | input | result |
|
||||
|---|---|---|
|
||||
| `,s/a.*c/X/` | `axxcxxc` | `X` (greedy, same as sam) |
|
||||
| `,s/a.*?c/X/` | `axxcxxc` | `Xxxc` (non-greedy, PCRE bonus) |
|
||||
both). katesam additionally offers non-greedy `.*?` (a PCRE extra).
|
||||
|
||||
---
|
||||
|
||||
## Anchors `^` / `$` are buffer-wide, not per-line
|
||||
## Anchors `^` / `$` are per-line — same as sam
|
||||
|
||||
By default PCRE anchors `^`/`$` to the **start/end of the whole buffer**, so a
|
||||
global substitution touches only the first/last position — **not** every line:
|
||||
This is **not** a deviation. plan9 `regexp(7)` defines `^` as "the beginning of
|
||||
a line" and `$` as "the end of a line" (sam's `regexp.c`: `BOL` fires at offset
|
||||
0 or after a `\n`; `EOL` fires before a `\n`). katesam compiles every pattern
|
||||
with `QRegularExpression::MultilineOption`, so `^`/`$` match at every line
|
||||
boundary, exactly as sam does:
|
||||
|
||||
| program | input | result |
|
||||
|---|---|---|
|
||||
| `,s/^/> /g` | `a⏎b⏎c` | `> a⏎b⏎c` (only line 1!) |
|
||||
| `,s/$/;/g` | `a⏎b⏎c` | `a⏎b⏎c;` (only the end!) |
|
||||
| `,s/^/> /g` | `a⏎b⏎c` | `> a⏎> b⏎> c` (every line) |
|
||||
| `,s/$/;/g` | `a⏎b⏎c` | `a;⏎b;⏎c;` (every line) |
|
||||
|
||||
This differs from how people expect line-oriented edits to work. **Two correct
|
||||
ways** to act per line:
|
||||
|
||||
**1. Structural (sam-idiomatic, preferred).** Loop over lines with `x`, then act
|
||||
on each line's dot. `.+` matches the non-newline run of each line:
|
||||
The sam-idiomatic structural form works too, and is preferable because it
|
||||
composes with guards and other loops:
|
||||
|
||||
```
|
||||
,x/.+/ i/> / prefix every (non-empty) line with "> " → > a⏎> b⏎> c
|
||||
,x/.+/ s/^/> / prefix every (non-empty) line → > a⏎> b⏎> c
|
||||
,x/.+/ a/;/ append ";" to every line → a;⏎b;⏎c;
|
||||
```
|
||||
|
||||
**2. PCRE multiline flag `(?m)`** (a bonus sam lacks). Makes `^`/`$` match at
|
||||
every line boundary:
|
||||
|
||||
```
|
||||
,s/(?m)^/> /g → > a⏎> b⏎> c
|
||||
,s/(?m)$/;/g → a;⏎b;⏎c;
|
||||
```
|
||||
|
||||
Prefer the structural form — it is the real sam way and composes with other
|
||||
loop/guard steps. Reach for `(?m)` for quick one-liners.
|
||||
`.` does **not** cross newlines (sam: a "character" is "any character but
|
||||
newline"), which is also PCRE's default. Line-structured descent like
|
||||
`,x/.*\n/ …` therefore behaves as expected.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -95,36 +84,39 @@ loop/guard steps. Reach for `(?m)` for quick one-liners.
|
|||
|
||||
> `.` not crossing newlines matches sam, so line-structured descent
|
||||
> (`,x/.*\n/ ...`) behaves as expected. Use `(?s)` if you *want* `.` to span
|
||||
> newlines (PCRE bonus).
|
||||
> newlines.
|
||||
|
||||
---
|
||||
|
||||
## PCRE features sam never had (free upgrades)
|
||||
## Extras — available, but not the sam way
|
||||
|
||||
All verified working in katesam:
|
||||
PCRE accepts syntax plan9 `regexp(7)` never had. It all works, but treat it as
|
||||
an **escape hatch, not a headline**: sam's power comes from *composing* simple
|
||||
patterns with `x`/`y`/`g`/`v`/`{}`, not from clever single regexes. Prefer
|
||||
structural composition; reach for these only when it is genuinely simpler.
|
||||
|
||||
| feature | example | effect |
|
||||
| extra | example | note |
|
||||
|---|---|---|
|
||||
| shorthand classes | `,s/\d+/N/g` on `a12b345` | `aNbN` |
|
||||
| `\w+` in a loop | `,x/\w+/ c/W/` on `foo bar` | `W W` |
|
||||
| word boundary `\b` | `,s/\bfoo\b/X/g` on `foo foobar foo` | `X foobar X` |
|
||||
| backreference | `,s/(\w)\1/D/g` on `aabbc` | `DDc` |
|
||||
| lookahead | `,s/foo(?=bar)/X/g` on `foobar fooqux` | `Xbar fooqux` |
|
||||
| inline flags | `(?i)` ignore-case, `(?m)` multiline, `(?s)` dotall | per-pattern |
|
||||
| shorthand classes `\d \w \s` | `,s/\d+/N/g` | less typing than `[0-9]`; genuinely handy |
|
||||
| word boundary `\b` | `,s/\bfoo\b/X/g` | handy; sam would use `x/foo/` with context |
|
||||
| ignore-case `(?i)` | `,s/(?i)todo/DONE/g` | fills a real gap — sam has no case-insensitive match |
|
||||
| non-greedy `*?` | `,s/a.*?c/X/` | the structural `x/…/` loop is the sam alternative |
|
||||
| lookahead/behind | `,s/foo(?=bar)/X/` | un-sam; express context with `x`/`g`/`v` instead |
|
||||
| backreference in pattern | `,s/(\w)\1/D/g` | rarely the right tool |
|
||||
|
||||
These make many edits *easier* than in real sam. The tradeoff is that a pattern
|
||||
written for katesam may not be portable back to sam.
|
||||
A pattern using these is **not portable back to sam**. If portability or
|
||||
staying in the structural idiom matters, avoid them.
|
||||
|
||||
---
|
||||
|
||||
## Practical guidance
|
||||
|
||||
- **Order overlapping alternatives longest-first** (`\.tar\.gz|\.gz`, not the
|
||||
reverse). The only silent divergence.
|
||||
- **For per-line edits, prefer `,x/.+/ …`** over `^`/`$`. If you use anchors
|
||||
globally, remember they are buffer-wide, or add `(?m)`.
|
||||
- **Lean on PCRE extras** (`\b`, `\d`, lookahead) freely — they are reliable;
|
||||
just know they are not sam-portable.
|
||||
reverse). The only silent divergence from sam.
|
||||
- **`^`/`$` are per-line** — just like sam. For whole-line edits either anchor
|
||||
directly (`,s/^/> /g`) or, more sam-idiomatically, loop (`,x/.+/ …`).
|
||||
- **Prefer structural composition** (`x g v {}`) over PCRE extras. The extras
|
||||
work but pull you out of the sam idiom and are not sam-portable.
|
||||
- **Empty-match globals** (`s/x*/…/g`) advance one position per null match, as in
|
||||
sam — safe, but as always with `*`, double-check the result.
|
||||
|
||||
|
|
|
|||
|
|
@ -56,9 +56,8 @@ SamPanel::SamPanel(QWidget *parent)
|
|||
auto *hint = new QLabel(
|
||||
i18n("sam commands — e.g. <tt>,s/foo/bar/g</tt> or "
|
||||
"<tt>X/\\.cpp$/ ,s/old/new/g</tt> (project-wide). Ctrl+Return runs.<br/>"
|
||||
"Regex is PCRE: <tt>^</tt>/<tt>$</tt> are buffer-wide (use "
|
||||
"<tt>,x/.+/ …</tt> or <tt>(?m)</tt> per line); order overlapping "
|
||||
"alternatives longest-first."),
|
||||
"Regex is PCRE: <tt>^</tt>/<tt>$</tt> are per-line (like sam); order "
|
||||
"overlapping alternatives longest-first. See docs/SAM.md."),
|
||||
this);
|
||||
hint->setWordWrap(true);
|
||||
hint->setTextFormat(Qt::RichText);
|
||||
|
|
|
|||
|
|
@ -658,6 +658,13 @@ private:
|
|||
|
||||
// Build a QRegularExpression from a sam pattern. The empty pattern reuses
|
||||
// the last compiled pattern (sam behaviour).
|
||||
//
|
||||
// Multiline is ON: plan9 regexp(7) defines ^ as "beginning of a line" and $
|
||||
// as "end of a line" (sam regexp.c BOL = p==0 || prev=='\n'; EOL = next
|
||||
// char is '\n'). That is exactly QRegularExpression::MultilineOption, so it
|
||||
// is the faithful default, not an opt-in. '.' still does not cross newlines
|
||||
// (sam: "the word character means any character but newline"), which is
|
||||
// QRegularExpression's default (no DotMatchesEverythingOption).
|
||||
QRegularExpression compile(const QString &pat)
|
||||
{
|
||||
QString p = pat;
|
||||
|
|
@ -666,7 +673,7 @@ private:
|
|||
} else {
|
||||
m_lastRe = p;
|
||||
}
|
||||
QRegularExpression re(p);
|
||||
QRegularExpression re(p, QRegularExpression::MultilineOption);
|
||||
if (!re.isValid()) {
|
||||
fail(QStringLiteral("bad regexp: %1").arg(re.errorString()));
|
||||
}
|
||||
|
|
|
|||
|
|
@ -53,6 +53,9 @@ private Q_SLOTS:
|
|||
void peelFileLoopY();
|
||||
void peelFileLoopNone();
|
||||
void errorOnNoMatch();
|
||||
void caretIsPerLine();
|
||||
void dollarIsPerLine();
|
||||
void samAnchorIdiom();
|
||||
};
|
||||
|
||||
void TestSamEngine::substituteFirst()
|
||||
|
|
@ -256,5 +259,28 @@ void TestSamEngine::errorOnNoMatch()
|
|||
QVERIFY(!r.error.isEmpty());
|
||||
}
|
||||
|
||||
void TestSamEngine::caretIsPerLine()
|
||||
{
|
||||
// plan9 regexp(7): ^ matches the beginning of EACH line (regexp.c BOL =
|
||||
// p==0 || prev=='\n'). So a global ^ substitution prefixes every line.
|
||||
QCOMPARE(runAll(QStringLiteral(",s/^/> /g"), QStringLiteral("a\nb\nc")),
|
||||
QStringLiteral("> a\n> b\n> c"));
|
||||
}
|
||||
|
||||
void TestSamEngine::dollarIsPerLine()
|
||||
{
|
||||
// $ matches the end of EACH line (regexp.c EOL = next char is '\n').
|
||||
QCOMPARE(runAll(QStringLiteral(",s/$/;/g"), QStringLiteral("a\nb\nc")),
|
||||
QStringLiteral("a;\nb;\nc;"));
|
||||
}
|
||||
|
||||
void TestSamEngine::samAnchorIdiom()
|
||||
{
|
||||
// The canonical sam idiom: loop over lines and anchor within each line's
|
||||
// dot. With per-line anchors this prefixes every line.
|
||||
QCOMPARE(runAll(QStringLiteral(",x/.+/ s/^/> /"), QStringLiteral("a\nb\nc")),
|
||||
QStringLiteral("> a\n> b\n> c"));
|
||||
}
|
||||
|
||||
QTEST_MAIN(TestSamEngine)
|
||||
#include "test_samengine.moc"
|
||||
|
|
|
|||
Loading…
Reference in New Issue