Skip to content

[notify+outboxlog] Full-snapshot persistence on the hot path #144

Description

@bigdestiny2

Severity: HIGH (outboxlog) / MED (notify) (dmc + mafintosh lens)

Both services rewrite full state per mutation:

  • notify: every mutating RPC (send:359 and _finalizeRejectedIntentAndPersist:572 — even rate-limited sends) awaits _saveState(), which freezeClones all 11 maps (incl. up to maxEvents=10000 events), JSON.stringify + temp-write + rename, serialized on _persistChain → head-of-line blocking under concurrency.
  • outboxlog: every appendstatePersistence.saveSync(snapshot()) (outbox-log.js:491) does a synchronous writeFileSync+renameSync of up to 256MB on the event loop per O(1) append.

Fix: make the append-only journal the default durable path; demote the full snapshot to periodic/size-triggered compaction (flush on stop()). Min viable: dirty-flag + coalesced trailing write on a short timer. For notify: return the RPC independent of durable persistence (spec principle 7 permits best-effort) and don't synchronously persist the fast-decaying replay/dedupe/buckets maps. See the measurements issue for the bench that currently hides this.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions