Expand description
Exit-side PIX supervision: the lifecycle state machine, the egress gate, and the per-session worker that drives them. PIX supervision for incoming PIX-enabled sessions.
This module implements the Exit-side supervision logic for sessions
using the Protocol for Incentivization of eXits (PIX). The Exit node
runs a deterministic supervisor that tracks each Secret Sharing Aggregate
(SSA) through a well-defined lifecycle, enforcing timeouts, deposit
sufficiency, recovery progress, and fault tolerances. Egress data
packets are gated behind a concurrent ServiceGate that follows the
front cycle: bounded predeposit service while it is unfunded and a
ceiling-limited path after it funds.
§Why supervision is needed
PIX sessions use SSAs: cryptographic aggregates that distribute the cost of a deposit across many packets sent from the Entry to the Exit. Each SSA requires:
- A commitment from the Entry (polynomial coefficients) that the Exit can verify.
- A deposit to an SSA-specific on-chain address before the Exit fully trusts the SSA.
- Recovery of the SSA shares from data packets as they arrive.
Without supervision, a misbehaving or stalled Entry can hold a session slot indefinitely without ever funding or completing recovery — a resource-exhaustion vector. The supervisor enforces hard per-SSA and per-session bounds so the Exit can reclaim resources deterministically.
§Architecture
The module is split into three components:
| Component | File | Role |
|---|---|---|
SessionPixSupervisor | supervisor | Pure state machine — no I/O, no async, no spawning. Driven by explicit Instant timestamps and service-gate snapshots. |
ServiceGate | gate | Concurrent, lock-free egress gate. Before the front funds: bounded predeposit budget. After it funds: ceiling on packets served without that cycle’s recovery progress. Callers park on a generation-counter waker. |
FillPlanner | fill | Pure rate law for the Exit’s own keep-alives. Answers, once a second, how fast this node must send in order for the funded cycle to complete inside its deadline — and emits nothing when the answer has not meaningfully changed. |
| Worker loop | worker | Per-session actor that bridges the pure supervisor to async reality. Receives commands via a backpressured channel, manages the deadline and fill timers, applies gate actions synchronously, and forwards actions to the caller. |
The SlotNotify multi-waker primitive is shared from utils and used
by ServiceGate to park and wake callers without a tokio dependency.
§The SessionPixSupervisor state machine
The supervisor tracks each SSA through these phases:
RequestSsa ──► SsaRequestSent ──► AwaitingCommitment
├── (CommitmentProgress arms/re-arms the re-request timer)
├── (its expiry → RequestCommitmentRetransmission, bounded)
│
CommitmentVerified
│
AwaitingDeposit
│
DepositConfirmed (≥ expected)
│
Recovering
├── (idle re-arms when no service)
├── hard deadline is immutable
└── progress resets idle timer
│
Recovered
├── bounded paid FIFO drain
├── successor progress → handoff
└── diagnostic tombstone expiry → RetireSsaKey deadlines (all configurable via SupervisorConfig):
- Commitment timeout — time from
SsaRequestSenttoCommitmentVerified. - Commitment re-request — time a partially delivered commitment may go quiet before the missing parts are
asked for again. The one deadline here that is not a deadline: it emits a repair and re-arms, never a close, and
the commitment timeout above remains the bound. It is armed only by
CommitmentProgress, so a commitment nothing was ever delivered for never asks — there is no scope for it to name. Bounded by count, atMAX_COMMITMENT_RETRANSMISSIONS, which is what the Entry will answer. - Deposit timeout — time from
CommitmentVerifiedto a sufficient deposit. - Recovery idle — time without any share arriving while service is being consumed. Service-gated: if no packets were served since the last progress snapshot, the timer re-arms instead of closing (prevents a slow-but-honest Entry from being disconnected). Deliberately not “useful progress”: a conforming Entry emits a whole window’s surplus in one run, none of it useful, and treating that as silence closes honest Sessions.
- Recovery hard deadline — absolute per-SSA backstop, never extended. This is a resource guard (session slot + reconstructor memory), not a liveliness mechanism.
Both recovery clocks start when the cycle reaches the paid transport front, not when its deposit confirms. A batch is served in index order, and the final SURBs buffered for a recovered predecessor still carry that predecessor’s shares. The predecessor keeps its original immutable hard clock while this bounded tail drains; the successor’s clocks start only when the negotiated remainder is exhausted or funded successor progress proves the FIFO boundary. A queued cycle is therefore not charged for either its batch wait or its predecessor’s pipeline delay.
Fault tracking — an UnverifiableShares event closes the Session on arrival, with no
tolerance to configure and no running totals to keep: the event means a polynomial’s share set
failed to open its commitment, which permanently dooms the cycle. Late fault reports against an
already terminal SSA are absorbed. Progress is the one narrow exception: the sole immediate
recovered predecessor may report its bounded, already-paid FIFO tail. See the configuration
section below for why this is not a dial.
Rolling SSAs — to maintain continuity, the supervisor requests the next batch when the
current one is “almost recovered” (early threshold reached) or fully recovered, so the commitment
and deposit work for the next cycle overlaps the tail of the current one. With a batch, only its
last cycle may ask: the request flags are per-cycle, so without that gate every member would
answer its own early signal with a batch of its own, compounding into ssas_per_request batches
per batch. The last rather than the first, because a batch is served in index order and gating on
the first would ask a whole batch too early. Retiring the cycle holding the gate hands it to the
newest survivor, or a batch whose last cycle lost its deposit would strand with no successor.
Allocation independently enforces the reservation: at most
MAX_OVERLAPPING_BATCHES generations and
MAX_OVERLAPPING_BATCHES × ssas_per_request live cycles may own reconstructor state. If the
early signal arrives while both generations are still live, the supervisor records the owed
request and retries it only after one generation releases its heavyweight cycle state. Recovered
supervisor tombstones do not count because full recovery removes their heavyweight reconstructor
state; one constant-bounded per-Session tail receipt is tracked separately.
§The ServiceGate — egress gating
Every egress data packet from the Exit back to the Entry must pass
through the ServiceGate via acquire:
§Pre-funding (predeposit)
While the current front cycle is unfunded, a provisional budget lets the Exit answer the Entry
for a bounded number of packets, so that the application is not held up for as long as an
on-chain deposit takes to confirm. The initial front starts with this allowance; a successor gets
a fresh allowance after its predecessor was funded and leaves the front, whether by recovery or a
tolerated failure. A failed funded cycle has already cost the Entry its full deposit, and
max_failed_cycles bounds how often that can happen. Losing an unfunded front does not mint
another grant. The budget is
min(target_useful_shares - 1, max_predeposit_packets); at production dimensions the first term
is in the hundreds of thousands, so the configured cap is what binds and the min only matters
for the small dimensions used in tests.
When the budget is exhausted, acquire parks the caller on a
SlotNotify future. A concurrent release_service, withhold_service,
notify_progress, or poison
wakes all parkers.
§Strict prepay (max_predeposit_packets = 0)
Zero is a supported setting, and it means the Exit serves nothing against any unfunded front: the first egress data packet parks, and the Entry has to commit and fund that cycle before a single payload byte flows back to it. The rule is restored on every paid rotation rather than applying only to the Session’s first SSA.
This does not deadlock, because nothing on the path to funding passes through the gate:
- The
SsaRequestgoes out on theSessionManager’s own message sender, not the Session’s gated sink. - The Entry’s commitment travels Entry→Exit, and the deposit is on-chain.
- The SURB-level keep-alive stream is deliberately left ungated, precisely so an exhausted budget cannot silence the
signal that keeps the Session fundable. Under strict prepay this carries more weight than it does at a non-zero
budget: with no egress at all, the keep-alives are the only traffic reaching the Entry, and therefore the only
thing resetting the idle timer on the Entry’s own session slot. An operator who both sets this budget to zero and
disables
surb_balance_notify_periodgives the Entry nothing to stay alive on, and it will evict the Session before a deposit can land.
So the Entry can always be asked for a deposit and can always answer, whatever the budget is.
What a zero budget costs is latency, not liveness: the application stalls until the deposit
confirms, instead of proceeding optimistically. If the deposit never arrives, the deposit
deadline closes the Session and poison fails the parked writer rather
than leaving it pending.
§Post-funding (ceiling)
Once the current front’s deposit is confirmed and the supervisor emits ReleaseService, the
gate flips to funded mode. It then enforces max_served_without_progress: a
ceiling on how many packets may be served between SSA recovery progress
events, as a defense-in-depth backstop even when the supervisor’s
service-gated idle timer is alive. Only progress from that funded front—or from its sole bounded
recovered predecessor while the FIFO drains—emits ProgressNotification and resets the
watermark; unfunded, off-front, older, or over-budget progress cannot buy service. A rotation to
an unfunded successor emits WithholdService and restores the bounded allowance. A funded
successor stays open, but another ReleaseService rebaselines its ceiling.
The gate is implemented with lock-free atomics and CAS loops. It uses
the generation-counter SlotNotify to avoid the two classic
race conditions of waker-vector approaches:
- Latent wake — notification between future creation and first
poll()is caught because the generation was captured at creation time and compared onpoll(). - Spurious
Ready— a secondpoll()of an already-registered future re-checks the generation; if unchanged it staysPending.
§Fill — keeping a funded cycle on course
Everything above is a rule for stopping. This is the one rule for going, and it exists because the rules for stopping cannot be met by an idle Session.
§What goes wrong without it
A PIX Exit is paid per return packet. Each Exit → Entry packet spends one SURB carrying one
share, and a funded cycle recovers — unlocks the deposit key it has already been paid for — only
once polys × (threshold + surplus) of them have gone back. That is 327 680 packets at the shipped
dimensions. Nothing in the protocol makes the application send them:
- an idle VPN client sends nothing at all;
- a tunnel between keystrokes sends a handful a minute;
- any client that reads more than it writes returns fewer packets than it receives, by definition.
Each of those leaves the Exit holding a cycle it was paid for and cannot complete.
max_recovery_time then closes the Session, and because
the deposit address derives from the client’s and the server’s commitments together, the money
is stranded rather than refunded. At production geometry the boundary is about 46 return packets
per second averaged over a whole cycle — which is not a light client, it is a saturated one.
So the Exit sends its own. The rate is planned once per SAMPLING_INTERVAL:
E = polys × (threshold + surplus) packets in one cycle
target = the recovered predecessor's paid FIFO tail if there is one, else the earliest
live cycle that is both funded and clocked, else none → rate 0
remaining = (E − shares seen on the target) × (1 + loss_margin)
finish_by = target.hard_deadline − (1 − finish_fraction) × max_recovery_time
required = remaining / max(finish_by − now, SAMPLING_INTERVAL) packets/s
organic = EMA over ORGANIC_WINDOW of the service gate's counter packets/s
fill = clamp(required − organic, heartbeat, max_rate)Both of the middle two lines change for a drain, which is the same law over a Session that has already been closed — see “Draining a closed Session” below.
The tail wins over the successor whenever it exists, for two reasons that point the same way: its SURBs sit ahead of the successor’s in the Exit’s own buffer, so packets sent now carry its shares; and it holds the binding deadline, since the successor’s clocks are deliberately not armed until the tail is exhausted.
§Why it is the keep-alive message
Fill reuses the Exit’s SURB-level keep-alive rather than introducing a message of its own, and the two share one stream and one rate controller running at the faster of the two. Every shipped Entry already parses that message, already counts it as a consumed SURB so its balancer refills, and already credits it as service against the successor gate — so an Exit that fills needs nothing new from its peer. A new flag bit would have been worse than useless: keep-alive flag decoding used to be strict, so a bit an older Entry did not recognise made the whole message fail to parse.
§What bounds it
A rule that sends is the one rule in this module that could be turned against the node running it, so it is bounded four times over:
| Bound | What it stops |
|---|---|
max_rate | The instantaneous cost of one Session. Validated upwards too — validate_incoming_session_pix_config refuses a ceiling below the rate the widest accepted quota needs, because a filler that cannot finish is worse than none. |
min_surb_reserve | Fill spending the SURBs the Session needs for its own replies. Not an optimisation: a return packet that finds no SURB holds every packet this node originates for surb_resolution_wait, so an over-eager filler stalls the Session it is saving. The SURB-level notification is exempt, being the message that asks for more. A ceiling rather than the figure used — the buffer it is measured against is the Entry’s choice, so the effective reserve is never more than a quarter of the target the Entry announced. |
| The stall rule | A doomed cycle, or a share-less SURB supply, burning max_rate × max_recovery_time. Once the target has not moved for max_recovery_idle, fill drops to its heartbeat until the cycle moves again. |
| The existing deadlines and the service gate | Unchanged. Fill is not a way around the backstop: an Exit with nothing to send strands exactly as it did before. |
The stall rule cannot be delegated to max_recovery_idle itself, and that is worth being explicit
about. The idle rule re-arms whenever no gated service was consumed since the last progress, and
fill bypasses the gate exactly as the keep-alive stream it reuses does — so a Session whose only
traffic is fill looks perfectly quiet to it and it never fires. The stall rule is the bound the
gate-based one cannot supply.
§What it costs the client
One cycle’s quota per deadline while idle, and nothing at all while the application is busier than the required rate. The client accepts that as the price of a Session whose cycles complete, and funds successors as it would have.
§Draining a closed Session
Fill answers a Session that has nothing to say. This answers one that is no longer there.
A PIX Session can end at the Exit long before its funded cycle does, by three routes that all
arrive at the same place: the session server closes the HoprSession (WriteClosed/EmptyRead),
the operator calls SessionManager::close_session, or the peer sends a SessionError. Until
now each of those tore the Session down at once, taking the action driver’s commitment guards and
the reconstructor state with it. If the Exit was already holding enough SURBs to finish the cycle,
the deposit both sides paid for was stranded — the address derives from the two commitments
together, so nothing refunds it.
So the Exit keeps draining, under fill.drain_after_close,
and only when the arithmetic says the drain can finish:
remaining = E − largest_shares_seen(target)
drain iff estimated_surbs − min_surb_reserve ≥ remainingThe target is the earliest live cycle, and only if it is both funded and clocked — the same cycle
fill would have been working for. A Session with no funded cycle, or one whose front has already
recovered and left a paid FIFO tail behind, has nothing a drain could buy and closes at once with
SessionPixCloseReason::SessionClosed.
Three things about that predicate are deliberate. The estimate is an upper bound — ring-buffer
overwrites, store eviction and invalidated relayers are all unobserved — so remaining carries no
loss margin to go with it; in-flight packets are already subtracted from the estimate, and full
recovery needs fewer than E shares, so both errors point the conservative way. The reserve stays
in force for the whole drain, because the failure it prevents is worse than a lost deposit: a
return-routed packet that finds no SURB holds every packet this node originates for
surb_resolution_wait. And the SURB-level notification — the one packet the reserve exempts — is
switched off for the duration, since the reason for the exemption is that it asks for more
SURBs on behalf of a Session that is gone: the Entry itself very likely still exists, but nothing
is left there to act on the request, so no refill ever answers it.
The rate law is the same one, with the two terms that assume a live Session taken out:
required = remaining × (1 + loss_margin) / SAMPLING_INTERVAL -- no aim point; nothing to pace for
fill = clamp(required, heartbeat, max_rate) -- no organic; nothing left to contributePacing a cycle to its aim point only makes sense while the Session is there to be paced, and
subtracting organic egress only makes sense while an application is there to provide it. After the
close neither is true — served_total is frozen for good — so a drain asks for its whole remainder
at once and max_rate is what actually sets the pace. Leaving the subtraction in would have been
worse than untidy: a client that hangs up on a cycle it has all but paid off leaves a small
remainder against a still-high average, which suppresses the drain to its heartbeat for the whole
of the averaging window — the one case the drain is most clearly worth doing. The stall rule is the
one bound that does not step aside: it costs a progressing drain nothing and it is all that
stands between a doomed one and max_rate sustained until a deadline fires.
What a drain does not do is carry the Session on in any other respect. It retires every sibling
cycle immediately (they are queued behind the target and can receive nothing while it drains) and
it never asks for a successor, on any route into the request — there is no Session left to serve a
successor’s quota, so ordering one would commit the Entry to a deposit against a cycle that could
never be served. It ends with SessionPixCloseReason::Drained when the cycle recovers, and with
RecoveryIdle or RecoveryDeadline when it does not: max_recovery_idle stops being
service-gated while draining, because a drained Session consumes no gated service ever again and
the re-arm would otherwise never fire. The Session slot survives on its time_to_idle, which
every progress event refreshes, so a productive drain keeps its slot and a stalled one is closed
by the idle deadline long before the 180 s eviction — which remains the backstop.
Two cases are deliberately excluded. Idle eviction does not drain — it reaches the cache’s own listener rather than any of the close paths above — and that is the right answer rather than an omission: a filling Session’s slot is refreshed by every progress event, so reaching the eviction at all means the cycle was already starved or stalled. A second explicit close of a draining Session forces the teardown for a different reason: it is the operator’s way to overrule the drain. And a funded successor that has not yet reached the front, or a recovered predecessor’s paid tail, is never drained for: a production SURB buffer is at most ten thousand deep and a cycle is 160 000–330 000 packets, so it could not cover a second one.
§The Worker — bridging pure logic to async
spawn_supervisor_worker creates the SessionPixSupervisor,
the ServiceGate, and a bounded async command channel. It returns
a SessionPixSupervisorHandle (cloneable, for sending events) and an
ActionRx receiver (for driving actions).
The worker loop:
- Reads the earlier of the next deadline and the next fill tick from the supervisor.
- If it has already passed, calls
handle_timersimmediately. - Otherwise, waits on the command channel with a timeout set to the remaining duration.
- On command received → calls
handle_eventoraction_result. - On timeout → calls
handle_timers, which runs the deadline sweep first and then the fill tick if one is due. The order matters: the sweep is what arms a promoted cycle’s clocks and consumes a spent FIFO tail, so a tick that ran first would plan against a lifecycle one interval out of date. - Applies gate-control actions immediately, then forwards all actions to the action channel (non-blocking
try_send).
A Session with no funded cycle to fill for schedules no tick at all, so the loop’s behaviour is byte-for-byte what it was before fill existed until the first deposit confirms.
Coalescing — ProgressNotification actions are coalescible: when
the action channel is transiently full, they are dropped rather than
blocking or failing the worker. The gate has already consumed the notification locally, and the
next forwarded notification replaces it for observers.
All other actions (RequestSsa, ReleaseService, WithholdService, RetireSsa, SetFillRate,
Close) are non-coalescible — if they cannot be delivered, the channel is
genuinely wedged and the worker fails the session. SetFillRate belongs on that side of the line
even though it arrives on a timer: the planner’s hysteresis means it will not re-emit a value it
believes is already applied, so a dropped change is not replaced by the next one — and the worst
case is a dropped zero, which leaves a Session filling for a cycle that no longer exists.
§Integration with SessionManager
§Exit side (incoming sessions)
When handle_incoming_session_initiation processes a session request
with Capability::UsePIX:
-
Validates the offered PIX parameters (polys, threshold, quota range).
-
Spawns the supervisor worker via
spawn_supervisor_worker(this emits the initialRequestSsaaction and creates the gate). -
Reads the initial action, calls
send_ssa_requeston the wire, and notifies the supervisor ofSsaRequestSent. -
Stores the supervisor handle and gate in the session slot (via
OnceLock). -
Constructs the
HoprSession— the egress adapter acquires the gate on every outgoing data packet. -
After session publication, spawns the action driver task that receives actions from
ActionRx. Gate control has already happened in the worker; the driver performs I/O and records the gate-mode telemetry:Action Behaviour RequestSsaCalls send_ssa_request, feeds back result to supervisor. Tracks each SSA inSsaCommitmentGuardfor Drop-safe cleanup.RequestCommitmentRetransmissionCalls send_commitment_retransmission_request, which reads the missing scope from the reconstructor and asks the Entry for it. Registers nothing and reports nothing back: it repeats on its own interval and the commitment deadline is the backstop.ReleaseServiceWorker calls gate.release_service(); driver records funded telemetry.WithholdServiceWorker calls gate.withhold_service(); driver records unfunded telemetry.ProgressNotificationWorker calls gate.notify_progress(); no driver I/O.RetireSsaCalls share_processor.retire_ssa, aborts the deposit observer task.SetFillRateApplies the rate to the Session’s shared keep-alive controller and records the gauge. CloseSilences fill, poisons gate, retires all SSAs, publishes close metric, removes session slot. A close that ends a drain is an ordinary outcome rather than a failure: it is logged as information and the Entry is sent no failure notice, since the notice is return-routed and the Session it would report on is already gone — sending it would only spend a SURB out of a buffer that may be empty. -
PIX protocol events from the packet pipeline arrive via
dispatch_pix_eventand are forwarded to the supervisor asSessionPixEvent::RecoveryProgress,UnverifiableShares,AlmostRecovered, orRecovered. -
When a commitment becomes verifiable, a
PixDepositObservertask loops on deposit confirmations, forwarding each asDepositConfirmedto the supervisor. -
SessionPixEvent::SessionClosedis the one event that does not come fromdispatch_pix_event. The manager’s own close paths raise it, frombegin_surb_drain, and the supervisor answers by either draining the funded cycle or closing at once — see “Draining a closed Session” above.
§Entry side (outgoing sessions)
The Entry does not run a supervisor — the Exit is authoritative for
lifecycle decisions. The Entry creates a session slot when
new_session() succeeds, but the slot’s pix_supervisor and
pix_egress_gate remain unpopulated. On receiving an SsaRequest
from the Exit, the Entry generates its client commitment via the
share generator, sends the commitment messages, and emits
ReadyToDeposit so the caller can fund the deposit address.
§Lifecycle sketch (Exit side)
Session Initiation (Entry→Exit, with UsePIX flag)
│
▼
handle_incoming_session_initiation
│ validate PIX params
│ spawn supervisor (emits initial RequestSsa)
│ send SsaRequest on the wire
│ install gate & handle in slot
│ construct HoprSession (egress adaptor acquires gate)
│ spawn action driver
▼
┌────────────────────────────────────────────────────┐
│ Ongoing lifecycle (concurrent) │
│ │
│ Entry → CommitmentVerified → supervisor │
│ Entry → DepositConfirmed → supervisor (via │
│ PixDepositObserver) │
│ Packets → share_processor → RecoveryProgress │
│ Action: ReleaseService → gate.release_service│
│ Action: WithholdService → gate.withhold_service│
│ Action: ProgressNotification → gate.notify_progress│
│ Action: RequestSsa (next) → send on wire │
│ Action: RetireSsa → reconstructor.retire│
│ Action: Close → poison + teardown │
└────────────────────────────────────────────────────┘§Configuration
§What each parameter prevents
Every field of SupervisorConfig, the failure it exists to stop, and what shipping default it
carries. Nothing here is a preference: each one is the only thing standing between the Exit and a
specific way of taking service without paying for it.
| Parameter | Default | What it prevents |
|---|---|---|
ssas_per_request | 1 | Nothing by itself — an exposure dial. With dynamic batching it is the largest batch the Exit may derive; otherwise it is the exact batch size. Raising it amortises the request round trip over several cycles, at the price of multiplying the unfunded exposure to that many quotas, and it scales the two deadlines below. |
allow_dynamic_ssa_batches | true | Lets a sub-range per-SSA offer satisfy quota_range by committing to the smallest batch whose total quota enters the range. Disable it to retain the fixed-batch, per-SSA admission rule. |
max_failed_cycles | 1 | An Entry losing one cycle per batch indefinitely while a single funded sibling holds the Session open. One loss is survivable, the second closes the Session. Only reachable above a batch of one, where the failing cycle is not always the last one standing. |
max_ssa_delivery_time | 20 s | An Entry that accepts a request and never delivers the commitment set, holding a session slot and a reconstructor cycle that can never be funded. |
commitment_recommit_interval | 3 s | A single lost packet costing the whole Session. A commitment set ships as hundreds of messages and is unusable until every one lands, so without this a drop stranded the cycle on the deadline above — after the Entry had already been told to fund it. The one parameter here that buys liveness rather than bounding an attack; it bounds nothing itself. |
max_deposit_wait | 60 s | An Entry that commits but never deposits — typically after it has already drawn the predeposit budget. |
max_recovery_idle | 60 s | An Entry, or a colluding first return relayer, consuming service while returning no shares. Service-gated, so a Session that is merely quiet is never punished. |
max_recovery_time | 4 h | A cycle that dribbles just enough progress to refresh the idle timer forever. A resource backstop for the slot and the reconstructor state, not the anti-drip rule. It must leave a whole cycle at the widest dimensions the node accepts — 655 360 packets of full emission at the defaults, the quantity quota_range prices — time to complete, or it closes honest Sessions instead. validate_incoming_session_pix_config judges that at fill.max_rate when fill is enabled (~61 min at the defaults) and at MAX_ASSUMED_SESSION_PACKET_RATE when it is not (~2 min). With fill it also sets the idle tariff; see “why four hours” on SupervisorConfig::max_recovery_time. |
max_off_front_share_fraction | 0.25 | An Entry spreading a batch’s shares across all of its cycles, taking ssas_per_request quotas of service while completing none of them — and a cycle short of completion pays nothing at all. |
min_share_order_sample | 16384 | Convicting on a thin sample: the shares that legitimately cross a cycle boundary out of order while in flight. |
max_predeposit_packets | 10000 | Bounds what an Entry can extract from an unfunded front. Restored only after a paid front handoff; 0 means strict prepay on every rotation. |
max_served_without_progress | 2048 | Packets served with no share of any kind coming back — in packets, so unlike the idle timer the bound does not move with the Session’s rate. Counts shares_seen, so a conforming Entry’s surplus resets it; see below. |
tombstone_retention_window | 30 s | Bounds how long recovered-cycle diagnostics and observer ownership remain; the separately bounded FIFO-tail receipt may outlive it. |
fill | on | The one rule here that sends rather than stops: a funded cycle only recovers once its whole emission has ridden back to the Entry, so an idle Session strands the deposit it has already been paid. See PixFillConfig. |
fill.drain_after_close | on | A Session closed at the Exit mid-cycle stranding a deposit both sides have already paid for, while the Exit still holds the SURBs that would have completed it. Entered only when the estimated buffer covers the whole remainder, and bounded by everything fill is bounded by. |
§What is not here: the price of a cycle
No parameter above says what a quota costs, and none should. The Exit’s deposit pool is handed the
DepositUpdated sender along with
AgreedSsaQuota when the deposit address arrives, and it is the component
that knows both the byte quota and the price it charges for one. It sends on that channel once the
deposit clears that price; the supervisor’s DepositConfirmed handler acts on the verdict rather
than recomputing it. The Entry side mirrors this — its own strategy prices the quota carried by
PixEvent::NewDepositAddress and decides what to pay.
There was briefly a min_deposit here, and a per-SSA expected_deposit on the commitment event to
go with it. Both are gone. The Session layer has no price for a byte to derive one from, and a
configured floor would not be a backstop but a second authority behind the first: set above the
pool’s price it kills a cycle the pool has already been paid for, on max_deposit_wait, with
nothing on either side able to break the tie.
§The one fault with no dial: unverifiable shares
An UnverifiableShares report closes the Session immediately, and there is no configuration for
it. This used to be a pair of tolerances, per-SSA and per-Session, both shipped at zero.
Shares are not checked on arrival — the non-constant coefficient commitments that made per-share
verification possible were dropped — so a report means a whole polynomial’s share set failed to
open its commitment. The reconstructor marks that part failed, releases its shares, and never
clears the flag; the SSA is the sum of every polynomial’s constant term, so from that moment the
cycle cannot be reconstructed by any means and will never pay. A tolerance would therefore not buy
recovery, only unpaid service — including in the case it looks written for, a false positive from
a verification bug, where the part is just as permanently failed. Closing on the first report also
caps the exposure at the threshold packets already served when the failure surfaces, rather than
a multiple of it.
What the tolerances bought instead was machinery: absolute cross-peer totals, per-SSA and per-Session running sums, and delta accounting against the maximum seen so far to keep concurrent ack batches from double-counting. None of it is observable at a limit of zero, since the first report ends the Session.
§Constraints between parameters
validate_pix_supervision enforces, at config-load time and against the reconstructor config
actually in use: max_recovery_idle >= max_ack_await_time; tombstone_retention_window >= max_ack_await_time; max_recovery_idle < unused_verifier_lifetime;
commitment_recommit_interval < max_ssa_delivery_time; ssas_per_request in
1..=MAX_SSA_BATCH_SIZE; both scaled deadlines under 24 h; non-zero durations; a share fraction in
0.0..=1.0; and non-zero max_served_without_progress, min_share_order_sample and
max_failed_cycles.
For fill it enforces a non-zero heartbeat under the same 24 h cap, a
finite finish_fraction in (0, 1], a finite loss_margin in [0, 1), a max_rate inside
1..=MAX_FILL_RATE, and a non-zero min_surb_reserve. One further constraint cannot be decided
here because it spans the quota: validate_incoming_session_pix_config requires max_rate to
clear the rate a cycle of the widest accepted quota needs to finish inside
finish_fraction × max_recovery_time, which is 64 packets/s at the shipped defaults.
§The surplus run, and why it no longer constrains anything
Emission is round-robin over a window of up to hopr_protocol_pix::SHARE_EMISSION_WINDOW (256)
polynomials advancing in lockstep. They reach threshold on the same pass and then take their
surplus shares together, so every block ends with surplus_shares × window consecutive packets
carrying no useful share — 4096 at the shipped dimensions, against a
max_served_without_progress of 2048.
That used to be a live constraint on two parameters, and an unmeetable one: the ceiling and the
idle timer both counted useful shares, so the run looked exactly like an Entry gone silent. The
gate would park the writer partway through, a parked writer spends no SURBs, and the idle rule
could not rescue it — its re-arm branch needs no service consumed since the last progress, and
2048 packets were. An honest Session closed with RecoveryIdle one emission block into a cycle.
Both now read SsaRecoveryProgress::shares_seen,
which counts every share the cycle accepts — surplus included, up to the negotiated budget per
polynomial. Full cryptographic recovery no longer cuts that signal off: the reconstructor drops
its heavy state but retains one shared per-polynomial remainder, and the supervisor keeps that
recovered predecessor as the paid front until the tail is exhausted or successor progress proves
the FIFO boundary. The run therefore resets the ceiling and refreshes the idle deadline exactly
like the useful shares around it, even across recovery, so neither parameter has to be sized
against the dimensions any more. What max_served_without_progress still bounds is genuine
silence: packets served with no share of any kind coming back.
The one dimension-dependent property that remains is not a supervisor parameter at all: a
polynomial’s whole emission, threshold + surplus, should fit one peer deferral bucket
(hopr_protocol_pix::MAX_DEFERRED_ACKS_PER_POLYNOMIAL, 128), or acknowledgements arriving before
the cycle’s commitments install can be dropped. The shipped 96 fits; both halves are a byte wide
on the wire, so the sum can reach 510. It is a sufficient condition rather than a required one —
only the pre-install prefix is deferred — which is why it is asserted on the defaults rather than
enforced.
§Worked example
A profile of ~602 packets/s sustained per direction and 5–6 s on-chain settlement for an
SSA deposit. At full SESSION_MTU segments that is ≈ 7 Mbps of Session
data each way, and every return packet carries one SURB and one share.
Dimensions first, because every supervisor value derives from them. 2048 × 64 with the
derived surplus of 16 gives 131 072 useful shares out of 163 840 emitted, and a cycle of 163 840
packets — about 4.5 min at this rate. The quota is priced on all of them, surplus included, so
it is 2048 × 80 × 1452 B = 226.9 MiB, which is exactly the bottom of the default
quota_range (a quarter of the 907.5 MiB the default dimensions imply), so no range change is
needed. The default 8192 × 64 would make that 18.1 min per cycle, which at 6 s settlement is a
long time to leave one cycle’s traffic unsettled for no benefit. Keep it a multiple of 256 so the
emission window never narrows.
| Parameter | Value | Why, at this profile |
|---|---|---|
PixGlobalConfig::num_ssa_parts (Entry) | 2048 | 226.9 MiB quota, 4.5 min cycle; multiple of the emission window |
PixGlobalConfig::ssa_part_size (Entry) | 64 | Shipped threshold; with the surplus below, it is what fixes the quota |
PixGlobalConfig::additional_shares (Entry) | unset | Derives to 16 — a quarter of the threshold, i.e. a 1.25× surplus factor and a fifth of the quota. Setting it explicitly buys loss tolerance and charges the Entry for it. Keep ssa_part_size + additional_shares inside one deferral bucket (128) |
ssas_per_request | 1 | The 85 % early signal leaves a 24 627-packet runway — 41 s — before the cycle drains, against 6 s of settlement plus a commitment round trip. There is nothing to amortise, and a batch of n would multiply the unfunded exposure to n × 226.9 MiB |
max_ssa_delivery_time | 20 s | 2048 commitments ship in ~71 forward packets, well under a second; the margin covers commitment generation |
max_deposit_wait | 30 s | 5× the 6 s settlement, leaving room for the Entry to notice ReadyToDeposit, submit, and the observer to see it |
max_predeposit_packets | 4096 | ~6.8 s of service, matching expected settlement rather than the deadline. This is the most exposed against an unfunded front after the initial grant or a paid handoff — 5.9 MB. Use 0 for strict prepay at every rotation; normal overlap usually funds the successor before it reaches the front |
max_served_without_progress | 2048 | Shipped value, and no longer dimension-dependent: the surplus run resets it like any other share, so this bounds genuine silence only |
max_recovery_idle | 60 s | Shipped value. Satisfies >= max_ack_await_time and < unused_verifier_lifetime. It no longer has to cover the surplus run — that resets it — so what it now implies is only that a Session returning nothing at all for a minute is closed |
max_recovery_time | 2 h | Resource backstop only. A cycle needs 272 s at full rate, so 2 h implies a floor of ~23 packets/s (~0.27 Mbps) — deliberately far below the idle rule, which is the instrument that should bind |
max_off_front_share_fraction | 0.25 | Shipped value. A conforming Entry sits near 0; two-way spreading is 0.5 |
min_share_order_sample | 16384 | Shipped value, and safe here: with emission clamped to one cycle the front cycle is essentially complete before any off-front progress is possible, so even a loss-doomed cycle peaks near 15 % against the 25 % ceiling |
tombstone_retention_window | 60 s | 2× the reconstructor’s 30 s ack window |
max_failed_cycles | 1 | Shipped value, and inert at this batch size of one — the failing cycle is always the last one standing, which closes the Session first |
fill.enabled | true | The cycle is 163 840 return packets and the hard deadline is 2 h, so without fill any Session averaging under 23 packets/s of return traffic over a cycle strands its deposit. Against the 602 packets/s this profile is sized for that sounds like a wide margin; it is not, because it is a return rate and any client that reads more than it writes falls under it |
fill.finish_fraction | 0.75 | Aims 30 min before the 2 h deadline, which is the margin for loss beyond loss_margin, for a mixnet delay spike, and for the successor’s commitment and deposit round trip. A cycle that misses is worth nothing at all rather than nearly everything, so the last quarter of the budget is not spent |
fill.loss_margin | 0.05 | Fill counts packets sent; the cycle advances on shares that arrive. The Exit cannot observe the difference, so it sends this much extra and lets the per-second re-plan absorb whatever the real figure turns out to be |
fill.max_rate | 250 | An idle cycle here needs 163 840 x 1.05 / 5400 s = 32 packets/s, so this is ~8x the requirement — headroom for a Session that fell behind, at a ceiling of ~2.9 Mbps. validate_incoming_session_pix_config enforces the floor against the widest accepted quota, which at the default quota_range is 128 packets/s |
fill.heartbeat | 60 s | The floor while organic egress covers the need. Not zero: the Entry’s own idle eviction is refreshed by this traffic, and its balancer has no SURB-level report without it |
fill.min_surb_reserve | 500 | SurbStoreConfig::distress_threshold. A ceiling: the effective reserve is min(500, announced_target / 4), floored at one, because the buffer it is measured against is sized by the Entry rather than here. At the 7 000-SURB balancer target below a quarter is 1 750, so 500 is what binds — leaving fill 93 % of the buffer and keeping the last 7 % for the application, which is the side that has something waiting on it. The derivation engages only on a Session whose Entry asked for under 2 000 SURBs, which without it could never be filled at all |
fill.drain_after_close | true | Shipped value, and inert at this profile’s buffer depth: 7 000 SURBs less the 500 reserve is 4 % of a 163 840-packet cycle, so a Session closed at any point before the last 6 500 shares simply closes. What it does buy is the endgame — a client that hangs up in the final seconds of a cycle it has all but paid off no longer strands the whole deposit, and the drain then finishes in about 26 s at max_rate |
What one cycle costs is not in this table, because it is not in this configuration: the 226.9 MiB quota is priced by the deposit pool, which is also what decides that a deposit has cleared it.
Related settings outside SupervisorConfig that this profile also pins:
| Parameter | Value | Why |
|---|---|---|
SurbStoreConfig::rb_capacity | 100 000 | An overwritten SURB is a permanently lost share, so the buffer must clear the balancer’s target with room for overshoot — 14× here |
SurbBalancerConfig::target_surb_buffer_size | 7 000 | ~11.6 s of return traffic at this rate; must cover the forward round trip or the Exit starves mid-cycle. It must also stay well under a cycle’s emission — a SURB minted while no cycle is committed carries no share, so a buffer deeper than a cycle parks a run of share-less SURBs at the head of the Exit’s FIFO that fill cannot get past. 7 000 against 163 840 is 4 %, so the run is a rounding error |
SessionManagerConfig::maximum_surb_buffer_size | 10 000 | Ceiling the balancer may be steered to |
SsaReconstructorConfig::max_ack_await_time | 30 s | Bounds how long an unacknowledged share is held; both max_recovery_idle and tombstone_retention_window must clear it |
SsaReconstructorConfig::unused_verifier_lifetime | 1800 s | Must exceed max_recovery_idle, so the supervisor gives up on a stalled cycle before the reconstructor reclaims what it would need to finish |
SsaReconstructorConfig::early_recovery_threshold | 0.85 | Sets the pipelining runway derived for ssas_per_request above — 24 627 packets, 41 s. Bounded below by MIN_EARLY_RECOVERY_THRESHOLD and equal to it today: the Entry’s successor gate is computed at that floor, so a lower value asks for the next batch before any conforming Entry admits the request. Raising it shortens the runway |
PixGlobalConfig::max_ssas_per_request (Entry) | 2 | Must be ≥ every peer Exit’s ssas_per_request; not negotiated, and an over-cap batch is refused in full |
IncomingSessionPixConfig::max_live_cycle_bytes | 3 GiB | Shipped value. At 2048 × 64 + 16 a Session reserves ≈20.5 MiB, so this profile admits ≈150 concurrent PIX Sessions rather than the ≈37 the default dimensions imply. It is the ceiling on live reconstructor state, and it — not maximum_managed_sessions — is what decides how many PIX Sessions this Exit accepts |
Re-exports§
pub use gate::GateBlockReason;pub use gate::GateClosed;pub use gate::GateVerdict;pub use gate::ServiceGate;pub use worker::ActionRx;pub use worker::SessionPixSupervisorHandle;pub use worker::spawn_supervisor_worker;
Modules§
- fill 🔒
- The PIX fill planner — the pure rate law behind the Exit’s own keep-alives.
- gate 🔒
ServiceGate— bounded predeposit egress gate for PIX sessions.- supervisor 🔒
- Deterministic
SessionPixSupervisor— the pure state machine for PIX session lifecycle. - telemetry 🔒
- Bounded node-level accounting for the PIX Exit.
- worker 🔒
- Per-session actor for the
SessionPixSupervisor.
Structs§
- Fill
Rate - A rate at which the Exit originates PIX fill keep-alives.
- PixFill
Config - How the Exit fills the return direction of a funded PIX cycle when the application will not.
- PixParams
- PIX dimensions agreed upon during session negotiation.
- Supervisor
Config - Configuration for the
SessionPixSupervisor.
Enums§
- Session
PixAction - Actions emitted by the
SessionPixSupervisorfor the caller to execute. - Session
PixClose Reason - Internal close reasons emitted by the supervisor.
- Session
PixEvent - Events consumed by the
SessionPixSupervisor.
Constants§
- MAX_
FILL_ 🔒RATE - Upper bound on
PixFillConfig::max_rate, in packets per second. - MAX_
SUPERVISOR_ 🔒DURATION - Upper bound on any supervisor duration, as it is actually armed.
- ORGANIC_
WINDOW 🔒 - Time constant of the exponential average the planner keeps of organic egress.
- SAMPLING_
INTERVAL 🔒 - How often the fill planner re-derives the required rate.
Functions§
- scaled_
deadline 🔒 - A per-cycle deadline duration as it is actually armed for a batch of
ssas_per_request. - validate_
fill_ 🔒fractions - Range-checks the two
PixFillConfigfractions, ahead of anything that computes with them. - validate_
pix_ supervision - Validates that
SupervisorConfigandSsaReconstructorConfigare mutually consistent.