Skip to content

001_authorizations.md

Latest commit

 

History

History
224 lines (139 loc) · 19 KB

001_authorizations.md

File metadata and controls

224 lines (139 loc) · 19 KB

Authorizations design

Motivation

Two distinct problems shape the allowance design. They map onto the two limits in Allowance Limits.

Block-throughput reference numbers:

Parameter Value
MaxTransactionSize 2 MiB
MaxBlockTransactions 512
MAX_BLOCK_LENGTH × NORMAL_DISPATCH_RATIO (90%) ~9 MiB / block (binding constraint)
Block time 6 s ⇒ 14 400 blocks/day
Sustained max throughput ~127 GiB/day, ~1.73 TiB / 2 weeks
RetentionPeriod 2 × 100 800 blocks = 14 days
AuthorizationPeriod 14 days (Westend / Paseo configs)

1. Wasted block space (soft)

The chain has ~9 MiB of body capacity per block. If store rejects every call the moment a user crosses their per-account allowance, blocks frequently sit empty even when authorized users have data ready to send — capacity is left on the table.

Accept over-allowance store calls at a lower priority instead. In-budget users still go first; over-budget calls fill whatever block space is left. ⇒ motivates a soft limit on temporary-storage allowance, enforced by priority rather than rejection.

2. Unbounded renewed storage on collators (hard)

A renew (force_renew / enable_auto_renew / auto-renewal cycle) re-anchors an existing stored item: when the original entry's RetentionPeriod is about to elapse, it lands a fresh Transactions[block] entry pointing at the same content, and the renewed entry's own RetentionPeriod clock starts from that block. Repeat indefinitely and a single piece of content can stay on chain forever.

Without bounds, at sustained-peak block usage one window of fresh store data alone is ~1.73 TiB, and re-renewals stack on top. ⇒ motivates a hard limit on renewed storage (per account and chain-wide).

Storage types

  • Temporary storage — happens through the store call. Lives on chain for one RetentionPeriod from its store block.
  • Renewed storage — happens through force_renew (sync, at dispatch time) or via the auto-renewal cycle (enable_auto_renew + do_process_auto_renewals). The renewed entry itself also lives one RetentionPeriod (from its renewal block); the original Transactions entry it pointed at ages out on its own clock.

The renew-family extrinsics:

  • force_renew(entry: TransactionRef) — synchronous renewal. TransactionRef::Position { block, index } or TransactionRef::ContentHash(_).
  • renew(entry: TransactionRef) — one-shot scheduler. Pre-pays the renewal at registration (same charge as force_renew); cycle delivers without re-charging.
  • enable_auto_renew(content_hash) — recurring scheduler. Pre-pays the first cycle at registration (same charge as force_renew); cycle 1 delivers without re-charging, cycle 2 onward charges per cycle.
  • disable_auto_renew(content_hash) — cancels the registration. Signed callers are rejected with CannotDisablePrepaidAutoRenewal while the registration is in its prepaid window (paid: true); they must wait for the first cycle to consume the prepayment. Root can disable regardless (governance/cleanup), but the prepayment is forfeit.

store, store_with_cid_config, force_renew, renew, enable_auto_renew, and disable_auto_renew are unconditionally feeless. Authorization is the sole economic gate. Wrapper calls (e.g. utility::batch) are rejected by ValidateStorageCalls.

Each TransactionInfo is stamped with kind: TransactionKind { Store, Renew }. The kind is what on_initialize's obsolete-block cleanup uses to tell which entries should decrement the chain-wide renewed-bytes counter when they age out — see Hard limit on renewed storage.

Allowance limits

PoP grants two numbers per account: bytes_allowance (size budget) and transactions_allowance (count budget). The same bytes_allowance is reused on the soft and hard sides, with different semantics.

  • Soft (temporary) — bytes_allowance and transactions_allowance are priority thresholds only. The boost stays on while in-budget on both axes (bytes <= bytes_allowance and transactions <= transactions_allowance) and drops to 0 once either is strictly over cap. A missing or 0-allowance grant also yields no boost. store calls are never rejected; they queue behind in-budget signers when over.
  • Hard (renewed) — bytes_allowance is a real cap on the per-window renew quota. force_renew, the renew one-shot at registration, and recurring auto-renewal cycles reject with PermanentAllowanceExceeded when bytes_permanent + size > bytes_allowance. The transaction-count axis does not gate renew. A separate chain-wide cap (MaxPermanentStorageSize) bounds the total renewed bytes on chain across all signers.

Authorization storage

  • One AuthorizationExtent per scope is kept in Authorizations, keyed by AuthorizationScope::{Account, Preimage}.
  • AuthorizationExtent { transactions, transactions_allowance, bytes, bytes_permanent, bytes_allowance } holds the soft-side counters (bytes, transactions), the per-window renew quota (bytes_permanent), and the caps.
  • bytes and transactions bump on store / store_with_cid_config. bytes_permanent bumps on force_renew, on the renew one-shot scheduler at registration, on enable_auto_renew's registration (prepayment for the first cycle), and on each cycle-2-onward recurring cycle in do_process_auto_renewals. The transactions axis bumps on all of those.

authorize_account semantics

Per existing entry state:

  • Unexpired: caps are additive (bytes_allowance += bytes, transactions_allowance += transactions). Matches PoP's claim_long_term_storage flow, which calls this once per claim and expects each to extend the caps. Consumed counters are preserved, expiry is left untouched.
  • Expired-but-present: caps are re-granted (bytes_allowance = bytes, transactions_allowance = transactions) and all consumed counters reset to 0, including bytes_permanent. The new window's renew quota is independent of the old window's renewals — the old data is still on chain and is tracked by the chain-wide PermanentStorageUsed counter, but it does not spend the new window's quota.
  • Missing: create a fresh entry with all counters at 0.

authorize_preimage follows the same shape, but transactions_allowance is fixed at 1 (a preimage grant is a single-shot store right) and the unexpired path replaces rather than adds.

refresh_account_authorization

Extends expiration by another AuthorizationPeriod and leaves all consumed counters (bytes, bytes_permanent, transactions) untouched. Refresh does not grant additional capacity. To start a fresh window, let the authorization expire and re-authorize. Origin is T::Authorizer (e.g. PoP).

Soft limit (priority boost)

Implemented by the AllowanceBasedPriority transaction extension via a runtime-selected BoostStrategy:

  • check_authorization saturates bytes and transactions upward and never rejects.
  • The boost only applies to signed account-scoped store / store_with_cid_config. All renew flavours and preimage-scoped stores get 0.
  • The strategy is fed the post-this-tx extent so the decision is "would this leave the holder in-budget on both axes?".
  • FlatBoost (default in both runtimes): ALLOWANCE_PRIORITY_BOOST while in-budget, 0 once over.
  • ProportionalBoost (alternative): scales with the tighter of the byte- and tx-budget remainders.

In-budget store txs sort strictly above over-budget ones. Pool nonce / arrival ordering breaks ties.

Hard limit on renewed storage

The hard cap is enforced at two levels, and a renewal that would breach either is rejected.

"Renew" here means any path that consumes the per-window renew quota: force_renew, the renew one-shot scheduler at registration, enable_auto_renew's registration (prepaying the first cycle), and each per-cycle charge in do_process_auto_renewals (cycle 2 onward for recurring registrations; the prepaid first cycle does not re-consume).

Per-account quota

A renew of size bytes for scope S is rejected with Error::PermanentAllowanceExceeded when

S.bytes_permanent + size > S.bytes_allowance

bytes_permanent is increment-only within a window and resets to 0 on (re-)authorize via the expired-but-present path. It measures "renew bytes consumed in the current authorization window", not lifetime on-chain footprint. The chain-wide cap is the source of truth for actual on-chain bytes.

Chain-wide cap

A renew is rejected with Error::ChainPermanentCapReached when

PermanentStorageUsed + size > T::MaxPermanentStorageSize::get()

PermanentStorageUsed is bumped on every successful renew. It is decremented in on_initialize (mandatory weight, bounded by MaxBlockTransactions) when an obsolete block is removed: each Transactions[obsolete][i] with kind == Renew contributes its size to a single saturating decrement, then Transactions[obsolete] is removed.

That obsolete-block cleanup is the only path that ever decrements PermanentStorageUsed. Transactions is the authoritative record of which renewed bytes are still on chain; the counter is just a precomputed sum maintained alongside it.

MaxPermanentStorageSize is a Config trait constant. The runtime picks the backing — parameter_types! { pub const … } (runtime-upgrade only) or parameter_types! { pub storage … } (storage-backed; mutable at runtime via system.set_storage).

Capacity planning signals

  • Event::PermanentStorageUsedUpdated { used } fires once per change to PermanentStorageUsed: once per successful renew (increment), and once per obsolete-block cleanup that ages out at least one kind == Renew entry (decrement).
  • Event::PermanentStorageNearCap { used, cap } fires on the rising edge across PERMANENT_STORAGE_NEAR_CAP_PERCENT (80%) of MaxPermanentStorageSize. Off-chain consumers can use this as a "raise the cap or coordinate another bulletin chain" trigger.

Why renewed bytes can't grow unboundedly

Stated up front: at any block n, total renewed bytes on chain are bounded by MaxPermanentStorageSize (chain-wide cap) and a single account's renewed bytes are bounded by (K + 1) × bytes_allowance where K = RetentionPeriod / AuthorizationPeriod (so 2 × bytes_allowance for the aligned Westend / Paseo configs where K = 1).

Why: every renewed byte ages out exactly RetentionPeriod blocks after its renew block (the obsolete-block cleanup in on_initialize). New renews are gated by the chain-wide cap, so the counter can only enter the in-bounds region. As old data ages out, the cap recovers.

The examples below trace the counters block-by-block to make the bound visible.

Example 1 — single user, single window

PoP authorizes Alice for bytes_allowance = 10 MiB. Alice does:

Block Action bytes_permanent PermanentStorageUsed Outcome
1 store 5 MiB; force_renew it 5 MiB 5 MiB OK (within quota)
2 store 5 MiB; force_renew it 10 MiB 10 MiB OK (at quota)
3 store 1 MiB; force_renew it — — PermanentAllowanceExceeded

The per-account cap holds: at most bytes_allowance bytes renewed per window.

Example 2 — single user, lifecycle across one AuthorizationPeriod

AuthorizationPeriod = RetentionPeriod = 14 days. PoP authorizes Alice with bytes_allowance = 10 MiB at block 0. Alice stores 10 MiB and force_renews it at block 1. The authorization is expired from block 14 days onward (now >= expiration); the renewed entry was indexed at block 1, so its RetentionPeriod clock fires at block 1 + 14 days + 1 (the on_initialize cleanup once obsolete reaches 1).

Block Authorization state bytes_permanent Alice's on-chain renewed bytes PermanentStorageUsed
0 unexpired (expires 14 days) 0 0 0
1 unexpired; Alice: store(10 MiB) + force_renew 10 MiB 10 MiB 10 MiB
1 → 14 days − 1 unexpired, idle 10 MiB 10 MiB 10 MiB
14 days expired-but-present; Alice's further store / force_renew reject with InvalidTransaction::Payment 10 MiB 10 MiB 10 MiB
14 days + 2 expired-but-present; on_initialize ages out the renew (obsolete = 1) 10 MiB (stale) 0 0

From here Alice's path branches:

  • PoP re-authorizes (authorize_account on the expired-but-present path) — the caps are re-granted and all consumed counters (bytes, bytes_permanent, transactions) reset to 0. Alice gets a fresh window and can store / force_renew again. Repeating the pattern every window gives steady-state on-chain footprint = bytes_allowance per account (= 10 MiB).
  • PoP does not re-authorize — the authorization sits expired-but-present until anyone calls remove_expired_account_authorization. Alice cannot store or force_renew. Her renewed data has already aged out.

Two things worth noting:

  1. bytes_permanent is not decremented when the renewed data ages out — that is the chain-wide PermanentStorageUsed's job. The per-account counter only resets on re-authorize (expired-but-present path). While the authorization is expired, check_authorization rejects on the expiration check before reading bytes_permanent, so the staleness is unobservable.
  2. Transactions is the source of truth for on-chain renewed bytes. The chain-wide counter mirrors that same total via increments at renew time and decrements at obsolete-block cleanup; the per-account counter does not need to.

Example 3 — single user, end-of-window renew (worst case)

Worst case for per-account on-chain footprint: force_renew right at the end of one window, re-claim immediately at the start of the next, force_renew again. Both renewals overlap on chain until the older one ages out.

Day Action bytes_permanent On-chain renewed bytes
13 force_renew 10 MiB 10 MiB 10 MiB
14 window 1 expired; re-claim → bytes_permanent = 0; force_renew 10 MiB 10 MiB 20 MiB
15–26 both batches on chain 10 MiB 20 MiB
27 day 13's batch ages out (cleanup decrements) 10 MiB 10 MiB
28 day 14's batch ages out; re-claim; new force_renew … …

Peak on-chain bytes per account: 2 × bytes_allowance. Generalising, with RetentionPeriod / AuthorizationPeriod = K, the bound is (K + 1) × bytes_allowance: at any moment up to K + 1 consecutive windows can overlap on chain (the current window's renew plus up to K earlier windows still inside their RetentionPeriod). Aligned periods (Westend / Paseo) give K = 1, so peak = 2 × bytes_allowance (during overlap windows).

Example 4 — chain-wide cap at scale

MaxPermanentStorageSize = 1.7 TiB. Many users renewing concurrently:

Block Action PermanentStorageUsed Outcome
n aggregate renews bring counter to 1.6 TiB 1.6 TiB PermanentStorageNearCap event fires (≥ 80% of cap)
n+k further renews would exceed 1.7 TiB 1.7 TiB ChainPermanentCapReached rejects new renews
n+k+RetentionPeriod obsolete cleanup decrements as old renewals age out < 1.7 TiB new renews accepted again

The chain-wide cap is a hard ceiling on PermanentStorageUsed; the on-chain renewed bytes equal the counter (modulo a transient lag inside on_initialize). The system self-corrects: as soon as the counter falls below the cap, renewals resume.

Example 5 — adversarial single-user renew spam

A user with maximum claim rate and full bytes_allowance every period contributes at most (K + 1) × bytes_allowance on-chain bytes simultaneously (Example 3). To exceed that, they would need to renew more in a single window than their bytes_allowance permits — exactly what Error::PermanentAllowanceExceeded rejects.

A user across many accounts (Sybil-like) is bounded by the chain-wide cap (Example 4), regardless of how many accounts they control.

Migration

STORAGE_VERSION = 4. Migrations are only relevant for the Paseo/Westend testnets carrying pre-existing on-chain state forward; see the pallet_bulletin_transaction_storage::migrations::{v1, v2, v3, v4} modules for the wiring. The v4 step re-encodes each AutoRenewals entry from { account } to { account, recurring, paid }. All pre-existing entries were written by the old non-prepaying enable_auto_renew, so they migrate as { recurring: true, paid: false } — their next do_process_auto_renewals cycle charges per-cycle, preserving their on-chain behaviour across the upgrade.

Capacity planning operational steps

When PermanentStorageNearCap fires governance can either:

  • Pass a referendum to upgrade collator disk capacity and raise MaxPermanentStorageSize (via runtime upgrade for ConstU64-backed configs, or system.set_storage for parameter_types! { pub storage }-backed configs).
  • Coordinate spawning another bulletin chain.

Auto-renewal

Auto-renewal reuses the manual renew code path so the Hard limit on renewed storage accounting fires consistently — per-account bytes_permanent increment, chain-wide PermanentStorageUsed cap check, kind = Renew stamp in Transactions, obsolete-cleanup decrement. Hard-cap checks live in check_authorization (called by the extension's check_signed for the manual flow and by do_process_auto_renewals for the auto flow); the unified renewal mechanics live in do_renew_in_memory (called by do_renew and by the auto-renewal drain loop).

Behaviour on auto-renewal failure (per-account quota or chain-wide cap rejected at cycle time, or MaxBlockTransactions reached): the registration is dropped from AutoRenewals, Event::AutoRenewalFailed is emitted, and the data expires normally. If the slot cap rejects a cycle after the charge has been applied, do_process_auto_renewals refunds the chain-wide PermanentStorageUsed bump so the cap does not leak; the per-account bytes_permanent / transactions increments are left in place — slot-cap rejection at inherent time is pathological (the inherent runs before user txs and len(pending) <= MaxBlockTransactions), so the simpler accounting is preferred over a per-auth refund that would silently apply across roll-overs.

The latest-entry guard in on_initialize skips an obsolete entry when TransactionByContentHash[hash] points to a later block — a manual force_renew may have moved the latest reference forward; the renewal cycle then fires from the new entry's expiry, not the original.

TODO

  • Reserve block-transaction slots for user txs. do_process_auto_renewals is mandatory and pushes into the same BlockTransactions slot as user store / force_renew. Cap auto-renewals to a fraction of MaxBlockTransactions or partition the slot budget.
  • Per-content dedup of re-renewals (nice-to-have). On a renew of X, look up the previous (block, idx) for X via TransactionByContentHash and cancel its pending decrement — drops the per-content double-count when the same content is renewed in multiple consecutive windows.