Perf: measure switcher bottlenecks, add incremental re-rank
Add a benchmark harness (src/project/bench_switcher.cpp, manual target) that times the Go-to-File/Symbol pipeline against a real project and writes a metrics file. Measured on mobydick (12.2k files, 17.2k symbols); findings in docs/PERF.md. Bottleneck: PaletteModel::setQuery re-scored all N every keystroke (Debug: 270-600ms/keystroke; cost grew with query length). Fix: when the query is a pure append, re-score only the currently-visible subset — the match set is monotonic under appending, so this is exact. Later keystrokes drop from O(N) to O(matches) (symbols 'server' 30ms -> 9ms in Release). Guarded by test_palettemodel::incrementalMatchesFullScan. Also documented: shipped plugin is a Debug build (~5-8x slower than Release), and ctags/listFiles run ~0.1-1s synchronously on open (move off-thread = open follow-up).
This commit is contained in:
Levi Neely
parent
caa1e911b8
commit
8dd2995e36
|
|
@ -0,0 +1,71 @@
|
|||
# Switcher performance — measured bottlenecks
|
||||
|
||||
Benchmark harness: `src/project/bench_switcher.cpp` (manual target `bench_switcher`,
|
||||
not a ctest). Run:
|
||||
|
||||
```sh
|
||||
cmake -S . -B build-rel -DCMAKE_BUILD_TYPE=Release
|
||||
cmake --build build-rel --target bench_switcher
|
||||
./build-rel/bin/bench_switcher <project-root> /tmp/switcher_metrics.txt
|
||||
```
|
||||
|
||||
Reference project: `~/prj/r7.20/mobydick` — **12,234 files**, **17,184 ctags
|
||||
symbols**, 6.3 GB tree.
|
||||
|
||||
## One-time cost when the switcher opens (UI thread, synchronous)
|
||||
|
||||
| Stage | Release | Debug (shipped) |
|
||||
|----------------------------|--------:|----------------:|
|
||||
| `ProjectIndex::listFiles` | ~84 ms | ~78 ms |
|
||||
| `SymbolIndex::listSymbols` | ~974 ms | ~499 ms |
|
||||
| build items + `setItems` | ~5 ms | ~45-70 ms |
|
||||
|
||||
- `listFiles` is a `git ls-files` subprocess — acceptable but blocks the UI.
|
||||
- `listSymbols` is a **ctags subprocess (~0.5-1 s)** — the dominant open-cost,
|
||||
and it blocks the UI thread. (Cost is in the external binary, not our parse.)
|
||||
|
||||
## Per-keystroke cost — the interactive bottleneck
|
||||
|
||||
`PaletteModel::setQuery` → re-rank. Originally O(N) every keystroke; the cost
|
||||
*grew* with query length even as the result set shrank.
|
||||
|
||||
Full-scan (every keystroke re-scores all N), Release:
|
||||
|
||||
| Query | files | symbols |
|
||||
|------------------|---------:|---------:|
|
||||
| `server` | ~14 ms | ~30 ms |
|
||||
| `src server go` | ~17 ms | ~36 ms |
|
||||
|
||||
Debug (what ships): **270-342 ms (files)**, **459-598 ms (symbols)** per
|
||||
keystroke — unusable.
|
||||
|
||||
## Fix 1 — incremental narrowing (done)
|
||||
|
||||
When a query only **appends** to the previous one, re-score only the currently
|
||||
visible subset (the match set is monotonic under lengthening/adding a needle),
|
||||
turning later keystrokes from O(N) into O(previous matches). Exactness is
|
||||
guarded by `test_palettemodel::incrementalMatchesFullScan` (incremental ==
|
||||
full-scan, incl. the typo tier).
|
||||
|
||||
Effect once the set narrows (Release, type-forward):
|
||||
|
||||
| Query | full-scan | incremental |
|
||||
|----------|----------:|------------:|
|
||||
| files `server` | ~14 ms | **~6.6 ms** |
|
||||
| symbols `server` | ~30 ms | **~9.0 ms** |
|
||||
|
||||
Early keystrokes (`s`, `se`) remain full-N because almost everything matches —
|
||||
correct and unavoidable without a prefix index.
|
||||
|
||||
## Fix 2 — build the installed plugin optimized (recommended)
|
||||
|
||||
The shipped `olliepalette.so` is a **Debug** build (`-g`, no `-O`). Release is
|
||||
~5-8× faster on every hot path above. Install a `Release` or `RelWithDebInfo`
|
||||
build.
|
||||
|
||||
## Open follow-up — move indexing off the UI thread
|
||||
|
||||
`listFiles` (~85 ms) and especially `listSymbols` (~0.5-1 s) run synchronously
|
||||
when the switcher opens. For large projects they should run on a worker thread
|
||||
(or be cached + refreshed on `KateProjectBridge::projectChanged`) so the palette
|
||||
appears instantly and fills in as results arrive. Not yet done.
|
||||
|
|
@ -29,8 +29,18 @@ void PaletteModel::setQuery(const QString &query)
|
|||
return;
|
||||
}
|
||||
beginResetModel();
|
||||
m_query = query;
|
||||
rebuild();
|
||||
// Fast path: when the new query only appends to the old one, the match set
|
||||
// can only shrink (every FuzzyRanker tier is monotonic under lengthening a
|
||||
// needle or adding one), so we re-score just the currently-visible subset
|
||||
// instead of all N items. This turns the common type-forward case from
|
||||
// O(N) per keystroke into O(previous matches), which collapses quickly.
|
||||
const bool pureAppend = !m_query.isEmpty() && query.startsWith(m_query);
|
||||
if (pureAppend) {
|
||||
rebuildFromVisible(query);
|
||||
} else {
|
||||
m_query = query;
|
||||
rebuild();
|
||||
}
|
||||
endResetModel();
|
||||
}
|
||||
|
||||
|
|
@ -51,6 +61,33 @@ void PaletteModel::rebuild()
|
|||
m_visible.push_back({i, r.score + item.frecency, r.ranges});
|
||||
}
|
||||
|
||||
sortVisible();
|
||||
}
|
||||
|
||||
void PaletteModel::rebuildFromVisible(const QString &newQuery)
|
||||
{
|
||||
const QList<QString> needles = FuzzyRanker::tokenize(newQuery);
|
||||
m_query = newQuery;
|
||||
|
||||
// Re-score only the items that matched the shorter query. Rewrite m_visible
|
||||
// in place: survivors keep their slot, dropped items are compacted out.
|
||||
int write = 0;
|
||||
for (int r = 0; r < m_visible.size(); ++r) {
|
||||
const int src = m_visible.at(r).sourceIndex;
|
||||
const MatchResult res = FuzzyRanker::score(m_items.at(src).label, needles);
|
||||
if (!res.matched) {
|
||||
continue;
|
||||
}
|
||||
m_visible[write] = {src, res.score + m_items.at(src).frecency, res.ranges};
|
||||
++write;
|
||||
}
|
||||
m_visible.resize(write);
|
||||
|
||||
sortVisible();
|
||||
}
|
||||
|
||||
void PaletteModel::sortVisible()
|
||||
{
|
||||
// Highest score first; stable so equal scores keep input order.
|
||||
std::stable_sort(m_visible.begin(), m_visible.end(),
|
||||
[](const Visible &a, const Visible &b) { return a.score > b.score; });
|
||||
|
|
|
|||
|
|
@ -68,6 +68,11 @@ private:
|
|||
};
|
||||
|
||||
void rebuild();
|
||||
// Re-score only the currently-visible items against a longer (appended)
|
||||
// query; the match set is monotonic so narrowing is exact. Much cheaper
|
||||
// than a full rebuild for the common type-forward case.
|
||||
void rebuildFromVisible(const QString &newQuery);
|
||||
void sortVisible();
|
||||
|
||||
QList<PaletteItem> m_items;
|
||||
QList<Visible> m_visible;
|
||||
|
|
|
|||
|
|
@ -158,6 +158,47 @@ private Q_SLOTS:
|
|||
QCOMPARE(m.index(0, 0).data(PaletteModel::LabelRole).toString(),
|
||||
QStringLiteral("src/palette/palettemodel.cpp"));
|
||||
}
|
||||
|
||||
// Incremental narrowing (type-forward append) must produce exactly the same
|
||||
// visible rows, in the same order, as a single full-scan query — including
|
||||
// across the typo-tolerant tier. Guards the fast path in setQuery().
|
||||
void incrementalMatchesFullScan()
|
||||
{
|
||||
QList<PaletteItem> items;
|
||||
const QStringList labels = {
|
||||
QStringLiteral("src/server/main.cpp"),
|
||||
QStringLiteral("src/service/handler.cpp"),
|
||||
QStringLiteral("docs/SERVER.md"),
|
||||
QStringLiteral("README"),
|
||||
QStringLiteral("esrv/weird.txt"), // transposition bait
|
||||
QStringLiteral("server_test.go"),
|
||||
QStringLiteral("lib/observer.h"),
|
||||
};
|
||||
for (int i = 0; i < labels.size(); ++i) {
|
||||
items.push_back({QString::number(i), labels.at(i), QString(), 0});
|
||||
}
|
||||
|
||||
auto rowsFor = [&](const QStringList &keystrokes) {
|
||||
PaletteModel m;
|
||||
m.setItems(items);
|
||||
for (const QString &k : keystrokes) {
|
||||
m.setQuery(k); // incremental path when each extends the last
|
||||
}
|
||||
QStringList ids;
|
||||
for (int r = 0; r < m.rowCount(); ++r) {
|
||||
ids << m.index(r, 0).data(PaletteModel::IdRole).toString();
|
||||
}
|
||||
return ids;
|
||||
};
|
||||
|
||||
// Type-forward "server" one char at a time vs. one-shot full scan.
|
||||
const QStringList incremental =
|
||||
rowsFor({QStringLiteral("s"), QStringLiteral("se"), QStringLiteral("ser"),
|
||||
QStringLiteral("serv"), QStringLiteral("serve"),
|
||||
QStringLiteral("server")});
|
||||
const QStringList full = rowsFor({QStringLiteral("server")});
|
||||
QCOMPARE(incremental, full);
|
||||
}
|
||||
};
|
||||
|
||||
QTEST_MAIN(TestPaletteModel)
|
||||
|
|
|
|||
|
|
@ -24,3 +24,9 @@ if(Qt6Test_FOUND)
|
|||
target_link_libraries(test_kateprojectbridge PRIVATE project_lib Qt6::Core Qt6::Test)
|
||||
add_test(NAME kateprojectbridge COMMAND test_kateprojectbridge)
|
||||
endif()
|
||||
|
||||
# Manual benchmark (not a ctest): times the Go-to-File / Go-to-Symbol pipeline
|
||||
# against a real project root. Build with `cmake --build build --target
|
||||
# bench_switcher`, run `./build/bin/bench_switcher <root> [metrics-out]`.
|
||||
add_executable(bench_switcher bench_switcher.cpp)
|
||||
target_link_libraries(bench_switcher PRIVATE project_lib palette Qt6::Core)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,209 @@
|
|||
/*
|
||||
* SPDX-License-Identifier: LGPL-2.0-or-later
|
||||
*
|
||||
* Benchmark harness for the Go-to-File / Go-to-Symbol hot paths. NOT a unit
|
||||
* test — it is excluded from ctest and run manually against a real project to
|
||||
* measure where time goes at large N. It times each stage of the real pipeline
|
||||
* and appends a metrics report to a file.
|
||||
*
|
||||
* test_switcher_bench <project-root> [metrics-out]
|
||||
*
|
||||
* Stages timed:
|
||||
* - ProjectIndex::listFiles (git ls-files / walk)
|
||||
* - building file PaletteItems (label + frecency bonus)
|
||||
* - PaletteModel::setItems (files) (initial rebuild, empty query)
|
||||
* - PaletteModel::setQuery xN (per-keystroke re-rank at several query
|
||||
* lengths — this is the interactive cost)
|
||||
* - SymbolIndex::listSymbols (ctags over the file list)
|
||||
* - building symbol PaletteItems
|
||||
* - PaletteModel::setItems (symbols)
|
||||
* - PaletteModel::setQuery xN (symbols)
|
||||
*/
|
||||
#include "projectindex.h"
|
||||
#include "symbolindex.h"
|
||||
#include "palettemodel.h"
|
||||
#include "frecencystore.h"
|
||||
|
||||
#include <QCoreApplication>
|
||||
#include <QElapsedTimer>
|
||||
#include <QFile>
|
||||
#include <QString>
|
||||
#include <QStringList>
|
||||
#include <QTextStream>
|
||||
|
||||
#include <cstdio>
|
||||
|
||||
using namespace katecustom;
|
||||
|
||||
namespace
|
||||
{
|
||||
struct Timer {
|
||||
QElapsedTimer t;
|
||||
void start() { t.start(); }
|
||||
qint64 us() const { return t.nsecsElapsed() / 1000; }
|
||||
};
|
||||
|
||||
// Representative queries at growing lengths. Short queries are the worst case:
|
||||
// almost everything matches, so the result set and per-item work are largest.
|
||||
const QStringList kQueries = {
|
||||
QStringLiteral("s"),
|
||||
QStringLiteral("se"),
|
||||
QStringLiteral("ser"),
|
||||
QStringLiteral("serv"),
|
||||
QStringLiteral("server"),
|
||||
QStringLiteral("src server go"),
|
||||
};
|
||||
|
||||
QString buildFileLabel(const QString &rel)
|
||||
{
|
||||
return rel; // matches the plugin: full project-relative path as the label
|
||||
}
|
||||
|
||||
QString buildSymbolLabel(const Symbol &s)
|
||||
{
|
||||
QString label = s.name;
|
||||
if (!s.scope.isEmpty()) {
|
||||
label = s.scope + QStringLiteral("::") + s.name;
|
||||
}
|
||||
QString suffix = s.kind;
|
||||
if (!s.file.isEmpty()) {
|
||||
suffix = suffix.isEmpty() ? s.file : (s.kind + QStringLiteral(" ") + s.file);
|
||||
}
|
||||
if (!suffix.isEmpty()) {
|
||||
label += QStringLiteral(" — ") + suffix;
|
||||
}
|
||||
return label;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
int main(int argc, char **argv)
|
||||
{
|
||||
QCoreApplication app(argc, argv);
|
||||
if (argc < 2) {
|
||||
std::fprintf(stderr, "usage: %s <project-root> [metrics-out]\n", argv[0]);
|
||||
return 2;
|
||||
}
|
||||
const QString root = QString::fromLocal8Bit(argv[1]);
|
||||
const QString out = argc >= 3 ? QString::fromLocal8Bit(argv[2])
|
||||
: QStringLiteral("/tmp/switcher_metrics.txt");
|
||||
|
||||
QString report;
|
||||
QTextStream rs(&report);
|
||||
rs << "# switcher benchmark\n";
|
||||
rs << "root\t" << root << "\n";
|
||||
|
||||
FrecencyStore frecency; // empty store; bonus() is still called per item
|
||||
const qint64 now = 1700000000;
|
||||
|
||||
Timer t;
|
||||
|
||||
// --- Go to File -------------------------------------------------------
|
||||
t.start();
|
||||
const QStringList files = ProjectIndex::listFiles(root);
|
||||
const qint64 listFilesUs = t.us();
|
||||
rs << "files.count\t" << files.size() << "\n";
|
||||
rs << "files.listFiles_us\t" << listFilesUs << "\n";
|
||||
|
||||
t.start();
|
||||
QList<PaletteItem> fileItems;
|
||||
fileItems.reserve(files.size());
|
||||
for (const QString &rel : files) {
|
||||
PaletteItem item;
|
||||
item.id = QStringLiteral("ollie:file:") + rel;
|
||||
item.label = buildFileLabel(rel);
|
||||
item.frecency = frecency.bonus(
|
||||
QStringLiteral("ollie:file:") + root + QLatin1Char('\t') + rel, now);
|
||||
fileItems.push_back(item);
|
||||
}
|
||||
rs << "files.buildItems_us\t" << t.us() << "\n";
|
||||
|
||||
{
|
||||
PaletteModel model;
|
||||
t.start();
|
||||
model.setItems(fileItems);
|
||||
rs << "files.setItems_us\t" << t.us() << "\n";
|
||||
rs << "files.setItems_rows\t" << model.rowCount() << "\n";
|
||||
|
||||
// Independent (reset-before-each) full-scan cost per query:
|
||||
for (const QString &q : kQueries) {
|
||||
model.setQuery(QString()); // reset so each measurement is independent
|
||||
t.start();
|
||||
model.setQuery(q);
|
||||
rs << "files.fullQuery[" << q << "]_us\t" << t.us()
|
||||
<< "\trows\t" << model.rowCount() << "\n";
|
||||
}
|
||||
|
||||
// Realistic type-forward sequence (exercises incremental narrowing):
|
||||
// type "server" one character at a time without resetting.
|
||||
model.setQuery(QString());
|
||||
const QString word = QStringLiteral("server");
|
||||
for (int n = 1; n <= word.size(); ++n) {
|
||||
const QString q = word.left(n);
|
||||
t.start();
|
||||
model.setQuery(q);
|
||||
rs << "files.typeFwd[" << q << "]_us\t" << t.us()
|
||||
<< "\trows\t" << model.rowCount() << "\n";
|
||||
}
|
||||
}
|
||||
|
||||
// --- Go to Symbol -----------------------------------------------------
|
||||
if (SymbolIndex::ctagsAvailable()) {
|
||||
t.start();
|
||||
const QList<Symbol> symbols = SymbolIndex::listSymbols(root, files);
|
||||
const qint64 listSymbolsUs = t.us();
|
||||
rs << "symbols.count\t" << symbols.size() << "\n";
|
||||
rs << "symbols.listSymbols_us\t" << listSymbolsUs << "\n";
|
||||
|
||||
t.start();
|
||||
QList<PaletteItem> symItems;
|
||||
symItems.reserve(symbols.size());
|
||||
for (const Symbol &s : symbols) {
|
||||
PaletteItem item;
|
||||
const QString loc = s.file + QLatin1Char('\t') + QString::number(s.line);
|
||||
item.id = QStringLiteral("ollie:sym:") + loc;
|
||||
item.label = buildSymbolLabel(s);
|
||||
item.frecency = frecency.bonus(
|
||||
QStringLiteral("ollie:sym:") + root + QLatin1Char('\t') + loc, now);
|
||||
symItems.push_back(item);
|
||||
}
|
||||
rs << "symbols.buildItems_us\t" << t.us() << "\n";
|
||||
|
||||
PaletteModel model;
|
||||
t.start();
|
||||
model.setItems(symItems);
|
||||
rs << "symbols.setItems_us\t" << t.us() << "\n";
|
||||
rs << "symbols.setItems_rows\t" << model.rowCount() << "\n";
|
||||
|
||||
for (const QString &q : kQueries) {
|
||||
model.setQuery(QString());
|
||||
t.start();
|
||||
model.setQuery(q);
|
||||
rs << "symbols.fullQuery[" << q << "]_us\t" << t.us()
|
||||
<< "\trows\t" << model.rowCount() << "\n";
|
||||
}
|
||||
|
||||
model.setQuery(QString());
|
||||
const QString word = QStringLiteral("server");
|
||||
for (int n = 1; n <= word.size(); ++n) {
|
||||
const QString q = word.left(n);
|
||||
t.start();
|
||||
model.setQuery(q);
|
||||
rs << "symbols.typeFwd[" << q << "]_us\t" << t.us()
|
||||
<< "\trows\t" << model.rowCount() << "\n";
|
||||
}
|
||||
} else {
|
||||
rs << "symbols.ctags\tUNAVAILABLE\n";
|
||||
}
|
||||
|
||||
QFile f(out);
|
||||
if (!f.open(QIODevice::WriteOnly | QIODevice::Truncate | QIODevice::Text)) {
|
||||
std::fprintf(stderr, "cannot write %s\n", qPrintable(out));
|
||||
return 1;
|
||||
}
|
||||
f.write(report.toUtf8());
|
||||
f.close();
|
||||
|
||||
// Also echo to stdout for convenience.
|
||||
std::fputs(report.toUtf8().constData(), stdout);
|
||||
return 0;
|
||||
}
|
||||
Loading…
Reference in New Issue