Proposal: optional Rust-accelerated parser (drop-in, ~500× faster parse, with pure-Python fallback) — and a note on the per-call table rebuild

[tfoutrein]

Hi, and thanks for maintaining jsonpath-ng 🙏

TL;DR — We built pulse-jsonpath-ng (on PyPI:
pip install pulse-jsonpath-ng), a Rust/PyO3 parser that is iso-functional with jsonpath-ng and
~500× faster at parsing. It rebuilds jsonpath-ng's own AST classes and reuses your evaluator
(find), and delegates anything outside its fast path back to jsonpath_ng.parse. We'd love to know
whether you'd be open to jsonpath-ng optionally using it when present — fully opt-in, no impact on
users who don't install it, and no Rust added to this repository.

The root cause (might interest you regardless of Rust)

parse() is built on PLY with write_tables=0, so the LALR parse table (and the lexer table) are
rebuilt on every single call — ~2.5 ms to parse a small expression, ~300× the cost of the find it
precedes. A lot of code calls parse(...) inline per request, so this dominates. Even within pure
Python, caching the generated tables (or memoizing per expression) would help; a native parser removes
the cost entirely.

Performance

	median / parse()	speedup
jsonpath_ng.parse	~2.5 ms	—
pulse_jsonpath_ng.parse	~5 µs	~×500

import statistics, time, jsonpath_ng, pulse_jsonpath_ng
EXPR = "items[*].price"
def bench(parse, reps, rounds=15):
    out=[]
    for _ in range(rounds):
        t=time.perf_counter()
        for _ in range(reps): parse(EXPR)
        out.append((time.perf_counter()-t)/reps)
    return statistics.median(out)
r,c = bench(jsonpath_ng.parse,200), bench(pulse_jsonpath_ng.parse,20000)
print(f"jsonpath_ng {r*1e6:.0f} us -> pulse {c*1e6:.2f} us (x{r/c:.0f})")

What pulse-jsonpath-ng is

• A recursive-descent parser in Rust (PyO3/maturin, abi3 wheels for Linux x86_64/aarch64, macOS Apple
  Silicon, Windows; sdist elsewhere). It depends on jsonpath-ng.
• Iso by construction: it rebuilds the same jsonpath_ng.jsonpath classes (Root/Fields/Index/
  Slice/Child/This), so the parsed tree is structurally == to jsonpath_ng.parse(expr), and
  .find() is yours (reused) — same values, same paths.
• Covered natively: ./[...] chains — fields, *, numeric fields, $, `this`, indices,
  [a,b], slices [a:b:c]/[*], field brackets ['a','b'], quoted fields, a.[0] ≡ a[0].
• Always correct: anything outside that subset (.. descendants, |/&, where/wherenot,
  parent, parentheses, named operators, unicode IDs, and any invalid/ambiguous input) raises an
  internal NotImplementedError and is transparently delegated to jsonpath_ng.parse — which
  returns the right tree or raises the right JsonPathParserError/JsonPathLexerError. It's a strict
  subset: never a false accept.
• Proven by a differential oracle (tree ==, .find() parity, exception parity) over a curated corpus
  plus adversarial fuzzing.

The proposal (lightweight for you)

jsonpath-ng could, at import, optionally route parse through pulse_jsonpath_ng iff it's
installed, else use the current PLY path:

try:
    import pulse_jsonpath_ng as _accel   # optional, not a hard dependency
except ImportError:
    _accel = None
# ... use _accel.parse(...) when available, else the existing parser ...

• No Rust in this repo, no new mandatory dependency, no packaging change.
• Users on platforms without a wheel simply don't install the accelerator → unchanged behaviour.

What we're asking

Just whether this direction interests you. If yes, we're happy to:

1. open a small PR wiring the optional backend behind a feature flag / env var,
2. share the differential test-suite so iso-functionality is verifiable in your CI,
3. keep pulse-jsonpath-ng tracking jsonpath-ng's behaviour as the source of truth.

If you'd rather keep jsonpath-ng pure-Python, that's completely understandable — we'll maintain
pulse-jsonpath-ng as a standalone drop-in either way, and would still value your feedback (especially
on the table-rebuild point above).

Repo: https://github.com/AstekGroup/pulse-jsonpath-ng · PyPI: pulse-jsonpath-ng · License: Apache-2.0 (same as jsonpath-ng).

Thanks again!