Skip to content

Commit dc63e09

Browse files
yochcursoragent
andcommitted
Fix benchmark docs layout and Docs deploy concurrency.
Keep the hero summary table in README.md and generate a detailed VS_REFERENCE.md report for all scenarios and surfaces. Sync the detailed report to GitHub Pages media and avoid cancelling in-flight Pages deploys. Co-authored-by: Cursor <cursoragent@cursor.com>
1 parent f44fe3d commit dc63e09

6 files changed

Lines changed: 1394 additions & 45 deletions

File tree

.github/workflows/docs.yml

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ on:
1212
- 'CHANGELOG.md'
1313
- 'DESIGN_DOCUMENT.md'
1414
- 'benchmarks/README.md'
15+
- 'benchmarks/VS_REFERENCE.md'
1516
- 'benchmarks/baselines/reference.json'
1617
- 'typedoc.json'
1718
- 'examples/plain_js/**'
@@ -28,6 +29,7 @@ on:
2829
- 'CHANGELOG.md'
2930
- 'DESIGN_DOCUMENT.md'
3031
- 'benchmarks/README.md'
32+
- 'benchmarks/VS_REFERENCE.md'
3133
- 'benchmarks/baselines/reference.json'
3234
- 'typedoc.json'
3335
- 'examples/plain_js/**'
@@ -60,6 +62,7 @@ jobs:
6062
test -f docs/media/DESIGN_DOCUMENT.md
6163
test -f docs/media/CHANGELOG.md
6264
test -f docs/media/README.md
65+
test -f docs/media/VS_REFERENCE.md
6366
test -f docs/media/reference.json
6467
test -f docs/index.html
6568
test -f docs/demo/index.html
@@ -73,8 +76,8 @@ jobs:
7376
if: startsWith(github.ref, 'refs/tags/v') || github.event_name == 'workflow_dispatch'
7477
runs-on: ubuntu-latest
7578
concurrency:
76-
group: pages
77-
cancel-in-progress: true
79+
group: pages-deploy-${{ github.ref }}
80+
cancel-in-progress: false
7881
permissions:
7982
contents: read
8083
pages: write
@@ -105,6 +108,7 @@ jobs:
105108
test -f docs/media/DESIGN_DOCUMENT.md
106109
test -f docs/media/CHANGELOG.md
107110
test -f docs/media/README.md
111+
test -f docs/media/VS_REFERENCE.md
108112
test -f docs/media/reference.json
109113
test -f docs/index.html
110114
test -f docs/demo/index.html

README.md

Lines changed: 18 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -32,9 +32,25 @@ Internally it uses packed radix postings, typed arrays, and columnar stored fiel
3232
<!-- vs-reference:start — pnpm bench:readme -->
3333
### Measured vs MiniSearch
3434

35-
On the main benchmark set vs MiniSearch 7.2.0, frozen indexes use **~95% less index RAM** and win on **27/27 search cases**.
35+
Same corpora, same BM25-style queries, MiniSearch 7.2.0 as the reference.
3636

37-
Full comparison table, column definitions, and capture metadata: **[benchmarks/VS_REFERENCE.md](benchmarks/VS_REFERENCE.md)** (`benchmarks/baselines/reference.json` @ `1cdc405`, 2026-07-04).
37+
| Scenario | Docs | Index RAM | Binary size | Load JSON | Load binary | Freeze import | Search p50 |
38+
|----------|-----:|-----------|------------:|----------:|------------:|--------------:|-----------:|
39+
| Divina, with stored text | 14,097 | 0.82 vs 16.1 MB (~95% less) | ~71% less | 92 ms | 41 ms | 96 ms | ~30% faster |
40+
| Divina, index only | 14,097 | 0.71 vs 14.9 MB (~95% less) | ~75% less | 115 ms | 19 ms | 112 ms | ~30% faster |
41+
| Giant vocabulary (50k terms) | 50,000 | 1.82 vs 47.2 MB (~96% less) | ~81% less | 264 ms | 49 ms | 245 ms | ~46% faster |
42+
| Dense numeric ids | 100,000 | 4.91 vs 91.3 MB (~95% less) | ~73% less | 524 ms | 55 ms | 398 ms | ~30% faster |
43+
| Generic string ids | 100,000 | 4.90 vs 91.3 MB (~95% less) | ~74% less | 569 ms | 75 ms | 403 ms | ~27% faster |
44+
| Uint16 doc id boundary | 65,535 | 2.89 vs 58.6 MB (~95% less) | ~77% less | 356 ms | 45 ms | 269 ms | ~55% faster |
45+
| Uint32 doc id boundary | 65,536 | 3.51 vs 58.6 MB (~94% less) | ~74% less | 390 ms | 44 ms | 283 ms | ~54% faster |
46+
47+
Load JSON = `MiniSearch.loadJSON` on the same `toJSON` snapshot. Load binary = `loadBinarySync` after `saveBinarySync`. Freeze import = one-time `FrozenMiniSearch.fromJSON` (not the hot reload path).
48+
49+
Across this full run, frozen is faster on **27/27** search cases. Divina `inferno` (exact, paired p50): mutable 17.2 µs → frozen 13.1 µs (**-4 µs**, ratio 0.69).
50+
51+
Numbers are from `benchmarks/baselines/reference.json` @ `1cdc405`, captured 2026-07-04 on Node v24.16.0, 3 runs per scenario. Heap protocol v4 (isolated scenario processes, in-process trials, median+MAD; totalResident = heapUsed + external on both sides) — trend, not exact accounting. Index RAM column shows — for scenarios outside the heap allowlist.
52+
53+
Detailed tables for all 13 scenarios (search, load, migrate, heap, drift, …): **[benchmarks/VS_REFERENCE.md](benchmarks/VS_REFERENCE.md)**.
3854
<!-- vs-reference:end -->
3955

4056
---

benchmarks/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -115,6 +115,6 @@ pnpm bench:readme # regenerate benchmarks/VS_REFERENCE.md + README p
115115
node benchmarks/scripts/generate-readme-comparison.mjs --from=benchmarks/baselines/latest.json
116116
```
117117

118-
Public comparison output: [`benchmarks/VS_REFERENCE.md`](VS_REFERENCE.md) (full table). The root [`README.md`](../README.md) keeps a short pointer between `<!-- vs-reference:* -->` markers.
118+
Public comparison output: [`benchmarks/VS_REFERENCE.md`](VS_REFERENCE.md) (detailed tables for all scenarios). The root [`README.md`](../README.md) keeps the hero summary table between `<!-- vs-reference:* -->` markers.
119119

120120
Legacy `pnpm benchmark:baseline:update` was removed; use `pnpm bench:reference:update`.

0 commit comments

Comments
 (0)