Skip to main content

RelayerHealth

Struct RelayerHealth 

Source
pub struct RelayerHealth {
    relayers: DashMap<OffchainPublicKey, RaceHistory>,
    lost_neighbours: DashSet<OffchainPublicKey>,
    neighbours_changed: AtomicBool,
    dead: DashSet<OffchainPublicKey>,
    dead_since: DashMap<OffchainPublicKey, Instant>,
    confirmed: DashSet<OffchainPublicKey>,
    probation: DashSet<OffchainPublicKey>,
    known_dead_since: DashMap<OffchainPublicKey, Instant>,
    dead_over: DashMap<OffchainPublicKey, Hop>,
    suspected: DashMap<OffchainPublicKey, (Hop, Instant)>,
    published: DashMap<OffchainPublicKey, f64>,
    last_reply: LastReply,
}
Expand description

Per-relayer health from reply races: two SURBs of one packet share mint time, forward path and store slot, so the gap between their replies is down to the return paths alone.

Fields§

§relayers: DashMap<OffchainPublicKey, RaceHistory>§lost_neighbours: DashSet<OffchainPublicKey>

Neighbours we lost our own connection to, to re-plan when one comes or goes.

§neighbours_changed: AtomicBool§dead: DashSet<OffchainPublicKey>

Relayers judged dead: black holes as of the last flush, published at 0.

§dead_since: DashMap<OffchainPublicKey, Instant>

When each relayer in dead was judged, for VERDICT_CONFIRMATION. Removed with the relayer.

§confirmed: DashSet<OffchainPublicKey>

Relayers in dead whose verdict held for VERDICT_CONFIRMATION, and so is acted on. A verdict that lapses before it is confirmed is withdrawn rather than put on probation: it never held.

§probation: DashSet<OffchainPublicKey>

Relayers whose dead verdict has lapsed without a reply that is news about them coming home through them, published at PROBATION_HEALTH. Disjoint from dead; a relayer leaves both only by such a reply (see RelayerHealth::is_fresh_for).

§known_dead_since: DashMap<OffchainPublicKey, Instant>

When each relayer known dead was first judged dead in its current death, kept through probation and any later verdict until a reply revives it. A SURB minted through it after that is a probe the planner’s exploration sent, not stock the counterparty held when it died; see SurbRoundTripRegistry::write_off_dead.

§dead_over: DashMap<OffchainPublicKey, Hop>

For a relayer judged dead while it still delivered over its other hop, the hop its verdict is about: only a reply over that one lifts it (see Self::revive_by_reply). None for a relayer that delivered nothing at all, which any reply through it revives. Kept from the verdict through probation, removed with the death.

§suspected: DashMap<OffchainPublicKey, (Hop, Instant)>

Relayers that would be judged over one of their hops while delivering over the other, and since when: suspected, with nothing published or written off for them yet (see Self::has_stood). The hop is the one they are suspected over: a suspicion over the other hop is a new one.

§published: DashMap<OffchainPublicKey, f64>

Health last handed to the graph, below 1.0 only.

§last_reply: LastReply

When a reply last came home, through any relayer, for any session.

Implementations§

Source§

impl RelayerHealth

Source

pub(crate) fn record_race( &self, later: &[Option<OffchainPublicKey>; 5], later_surb: HoprSurbId, earlier: &[Option<OffchainPublicKey>; 5], winning_round_trip: Duration, by: Duration, )

Records that the reply over later came home by after the one over earlier, which took winning_round_trip, both on SURBs of one packet. Relayers on both legs cancel out.

Source

pub(crate) fn open_race( &self, waiting: &[Option<OffchainPublicKey>; 5], surb: HoprSurbId, home: &[Option<OffchainPublicKey>; 5], winning_round_trip: Duration, )

Records that the reply over home came home in winning_round_trip while surb, over waiting, is still out.

Source

fn discard_race( &self, waiting: &[Option<OffchainPublicKey>; 5], surb: &HoprSurbId, )

Drops the race surb, over waiting, opened without settling or booking it: its record is gone before its reply could close it, so the race says nothing about the relayers.

Source

fn discard_race_if_fresh( &self, waiting: &[Option<OffchainPublicKey>; 5], surb: &HoprSurbId, minted_at: Instant, )

discard_race for the relayers on waiting that a reply on a SURB minted at minted_at is news about (see Self::is_fresh_for), the filter note_home applies.

Source

fn record(&self, relayer: OffchainPublicKey, race: Race)

Source

pub fn set_neighbour_connected( &self, neighbour: OffchainPublicKey, connected: bool, )

Our own connection to a neighbour came or went.

Source

pub fn take_neighbour_change(&self) -> bool

Whether a neighbour came or went since the last call.

Source

pub(crate) fn record_reply(&self)

Records that a reply came home now, through any relayer, for any session.

Source

fn record_reply_at(&self, at: Instant)

Source

fn last_reply(&self) -> Option<Instant>

When a reply last came home, to millisecond resolution, or None if none has.

Source

pub fn any_known_dead(&self) -> bool

Whether any relayer is judged dead or on probation: when none is, there is nothing to write off.

Source

fn is_known_dead(&self, relayer: &OffchainPublicKey) -> bool

Whether the relayer is judged dead or on probation after such a verdict: its SURBs are written off either way.

Source

fn is_confirmed_dead(&self, relayer: &OffchainPublicKey) -> bool

Known dead and old enough to act on: on probation, or dead with a verdict that a flush at least VERDICT_CONFIRMATION after it found still standing (refresh marks that, before any re-plan of the same flush, so a slow re-plan cannot stand in for the second look).

Source

fn note_home(&self, leg: &[Option<OffchainPublicKey>; 5], minted_at: Instant)

A reply on a SURB minted at minted_at came home over leg: every relayer on it that the reply is news about (see Self::is_fresh_for) delivered, over the hop it has on this leg. This is what “home since” means for a black hole (see RaceHistory::unanswered), whether or not the reply settled a race.

Source

fn mark_dead(&self, relayer: OffchainPublicKey)

Records a relayer entering dead now, which starts its death.

Source

fn has_stood(&self, relayer: &OffchainPublicKey, hop: Hop) -> bool

Whether a relayer that is a black hole over hop only has been one for VERDICT_CONFIRMATION, with nothing home over it in between. It is suspected from the first flush that finds it so, and judged at the first one that finds this.

Over one hop a few late replies are all the evidence there is: only what travels that hop answers them, not every reply through the relayer as for one judged as a whole. And a path is late when it is first used, at the start of a session or after its stock was wiped, by up to a second on a cluster. Measured there with a verdict at the first flush: 136 verdicts in 84 runs on relayers that had nothing wrong with them, each steering the planner off the relayer, 129 of them gone by the next flush and 6 lifted before it by a reply over the hop; of the 90 on the relayer that was cut off, stalled or killed, 5 were gone. So the first flush only suspects, and the planner is not told. The stock is written off no later for it: that waited for the second flush anyway. A relayer suspected over the other hop is suspected anew over this one, and has a flush of its own to stand.

Source

fn known_dead_since(&self, relayer: &OffchainPublicKey) -> Option<Instant>

When the relayer’s current death began, while it is known dead.

Source

fn is_fresh_for(&self, relayer: &OffchainPublicKey, minted_at: Instant) -> bool

Whether a reply on a SURB minted at minted_at is news about relayer. It is, unless the relayer’s verdict was acted on (confirmed, or the relayer is on probation) and the SURB is older than the verdict. A relayer that is slow rather than dead delivers such replies late, after the verdict that judged them overdue: they are what the verdict was about, not proof that it ended, and taking them as proof put a stalling relayer back into routing after every verdict. Before the verdict is confirmed they still count, so that a relayer that is merely slow is revived by its own late replies without costing a wipe, and a probe minted after the verdict always counts.

Source

fn revive_by_reply( &self, leg: &[Option<OffchainPublicKey>; 5], minted_at: Instant, )

A reply on a SURB minted at minted_at came home over leg, so every relayer on it is delivering: any that was judged dead or on probation is alive again, unless the reply is not news about it (see Self::is_fresh_for), or it was judged while delivering over its other hop and this is that one (dead_over). Its health is then what its races say, which still holds the ones it lost, so it recovers gradually as they are forgotten.

Source

pub fn refresh(&self, memory: u32) -> Vec<(OffchainPublicKey, f64, f64)>

Re-judges every relayer once per flush; returns (relayer, previous, current) health changes for the graph, dead being 0.0 and probation PROBATION_HEALTH. A race open longer than a SURB is remembered counts as lost by that long. memory: margins a race is remembered for, 0 for dead, on probation or healthy.

A relayer is alive until it is a black hole, which makes it dead. Its verdict does not end with the condition: once no overdue reply is out any more it goes on probation instead, and stays there until a reply on a SURB minted after the verdict comes home through it (revive_by_reply, is_fresh_for), a probe’s above all. One on probation that is a black hole again is dead again.

Source

fn judge(&self, relayer: &OffchainPublicKey, memory: u32) -> Option<Judgement>

Margin, its uncertainty and fastest round-trip. Races are remembered for memory times margin plus spread, so the window scales with the relayer.

Trait Implementations§

Source§

impl Debug for RelayerHealth

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
Source§

impl Default for RelayerHealth

Source§

fn default() -> RelayerHealth

Returns the “default value” for a type. Read more

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
§

impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
where ST: ?Sized, DT: ?Sized,

§

impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
where ST: ?Sized, DT: ?Sized,

§

impl<T> Conv for T

§

fn conv<T>(self) -> T
where Self: Into<T>,

Converts self into T using Into<T>. Read more
§

impl<T> FmtForward for T

§

fn fmt_binary(self) -> FmtBinary<Self>
where Self: Binary,

Causes self to use its Binary implementation when Debug-formatted.
§

fn fmt_display(self) -> FmtDisplay<Self>
where Self: Display,

Causes self to use its Display implementation when Debug-formatted.
§

fn fmt_lower_exp(self) -> FmtLowerExp<Self>
where Self: LowerExp,

Causes self to use its LowerExp implementation when Debug-formatted.
§

fn fmt_lower_hex(self) -> FmtLowerHex<Self>
where Self: LowerHex,

Causes self to use its LowerHex implementation when Debug-formatted.
§

fn fmt_octal(self) -> FmtOctal<Self>
where Self: Octal,

Causes self to use its Octal implementation when Debug-formatted.
§

fn fmt_pointer(self) -> FmtPointer<Self>
where Self: Pointer,

Causes self to use its Pointer implementation when Debug-formatted.
§

fn fmt_upper_exp(self) -> FmtUpperExp<Self>
where Self: UpperExp,

Causes self to use its UpperExp implementation when Debug-formatted.
§

fn fmt_upper_hex(self) -> FmtUpperHex<Self>
where Self: UpperHex,

Causes self to use its UpperHex implementation when Debug-formatted.
§

fn fmt_list(self) -> FmtList<Self>
where &'a Self: for<'a> IntoIterator,

Formats each item in a sequence. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

§

impl<T> FutureExt for T

§

fn with_context(self, otel_cx: Context) -> WithContext<Self>

Attaches the provided Context to this type, returning a WithContext wrapper. Read more
§

fn with_current_context(self) -> WithContext<Self>

Attaches the current Context to this type, returning a WithContext wrapper. Read more
§

impl<T> Instrument for T

§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided [Span], returning an Instrumented wrapper. Read more
§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> IntoEither for T

Source§

fn into_either(self, into_left: bool) -> Either<Self, Self>

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
§

impl<T> Pipe for T
where T: ?Sized,

§

fn pipe<R>(self, func: impl FnOnce(Self) -> R) -> R
where Self: Sized,

Pipes by value. This is generally the method you want to use. Read more
§

fn pipe_ref<'a, R>(&'a self, func: impl FnOnce(&'a Self) -> R) -> R
where R: 'a,

Borrows self and passes that borrow into the pipe function. Read more
§

fn pipe_ref_mut<'a, R>(&'a mut self, func: impl FnOnce(&'a mut Self) -> R) -> R
where R: 'a,

Mutably borrows self and passes that borrow into the pipe function. Read more
§

fn pipe_borrow<'a, B, R>(&'a self, func: impl FnOnce(&'a B) -> R) -> R
where Self: Borrow<B>, B: 'a + ?Sized, R: 'a,

Borrows self, then passes self.borrow() into the pipe function. Read more
§

fn pipe_borrow_mut<'a, B, R>( &'a mut self, func: impl FnOnce(&'a mut B) -> R, ) -> R
where Self: BorrowMut<B>, B: 'a + ?Sized, R: 'a,

Mutably borrows self, then passes self.borrow_mut() into the pipe function. Read more
§

fn pipe_as_ref<'a, U, R>(&'a self, func: impl FnOnce(&'a U) -> R) -> R
where Self: AsRef<U>, U: 'a + ?Sized, R: 'a,

Borrows self, then passes self.as_ref() into the pipe function.
§

fn pipe_as_mut<'a, U, R>(&'a mut self, func: impl FnOnce(&'a mut U) -> R) -> R
where Self: AsMut<U>, U: 'a + ?Sized, R: 'a,

Mutably borrows self, then passes self.as_mut() into the pipe function.
§

fn pipe_deref<'a, T, R>(&'a self, func: impl FnOnce(&'a T) -> R) -> R
where Self: Deref<Target = T>, T: 'a + ?Sized, R: 'a,

Borrows self, then passes self.deref() into the pipe function.
§

fn pipe_deref_mut<'a, T, R>( &'a mut self, func: impl FnOnce(&'a mut T) -> R, ) -> R
where Self: DerefMut<Target = T> + Deref, T: 'a + ?Sized, R: 'a,

Mutably borrows self, then passes self.deref_mut() into the pipe function.
§

impl<T> Pointable for T

§

const ALIGN: usize

The alignment of pointer.
§

type Init = T

The type for initializers.
§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
§

impl<T> PolicyExt for T
where T: ?Sized,

§

fn and<P, B, E>(self, other: P) -> And<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns [Action::Follow] only if self and other return Action::Follow. Read more
§

fn or<P, B, E>(self, other: P) -> Or<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns [Action::Follow] if either self or other returns Action::Follow. Read more
§

impl<T> Read<Exclusive, BecauseExclusive> for T
where T: ?Sized,

Source§

impl<T> Same for T

Source§

type Output = T

Should always be Self
§

impl<T> Tap for T

§

fn tap(self, func: impl FnOnce(&Self)) -> Self

Immutable access to a value. Read more
§

fn tap_mut(self, func: impl FnOnce(&mut Self)) -> Self

Mutable access to a value. Read more
§

fn tap_borrow<B>(self, func: impl FnOnce(&B)) -> Self
where Self: Borrow<B>, B: ?Sized,

Immutable access to the Borrow<B> of a value. Read more
§

fn tap_borrow_mut<B>(self, func: impl FnOnce(&mut B)) -> Self
where Self: BorrowMut<B>, B: ?Sized,

Mutable access to the BorrowMut<B> of a value. Read more
§

fn tap_ref<R>(self, func: impl FnOnce(&R)) -> Self
where Self: AsRef<R>, R: ?Sized,

Immutable access to the AsRef<R> view of a value. Read more
§

fn tap_ref_mut<R>(self, func: impl FnOnce(&mut R)) -> Self
where Self: AsMut<R>, R: ?Sized,

Mutable access to the AsMut<R> view of a value. Read more
§

fn tap_deref<T>(self, func: impl FnOnce(&T)) -> Self
where Self: Deref<Target = T>, T: ?Sized,

Immutable access to the Deref::Target of a value. Read more
§

fn tap_deref_mut<T>(self, func: impl FnOnce(&mut T)) -> Self
where Self: DerefMut<Target = T> + Deref, T: ?Sized,

Mutable access to the Deref::Target of a value. Read more
§

fn tap_dbg(self, func: impl FnOnce(&Self)) -> Self

Calls .tap() only in debug builds, and is erased in release builds.
§

fn tap_mut_dbg(self, func: impl FnOnce(&mut Self)) -> Self

Calls .tap_mut() only in debug builds, and is erased in release builds.
§

fn tap_borrow_dbg<B>(self, func: impl FnOnce(&B)) -> Self
where Self: Borrow<B>, B: ?Sized,

Calls .tap_borrow() only in debug builds, and is erased in release builds.
§

fn tap_borrow_mut_dbg<B>(self, func: impl FnOnce(&mut B)) -> Self
where Self: BorrowMut<B>, B: ?Sized,

Calls .tap_borrow_mut() only in debug builds, and is erased in release builds.
§

fn tap_ref_dbg<R>(self, func: impl FnOnce(&R)) -> Self
where Self: AsRef<R>, R: ?Sized,

Calls .tap_ref() only in debug builds, and is erased in release builds.
§

fn tap_ref_mut_dbg<R>(self, func: impl FnOnce(&mut R)) -> Self
where Self: AsMut<R>, R: ?Sized,

Calls .tap_ref_mut() only in debug builds, and is erased in release builds.
§

fn tap_deref_dbg<T>(self, func: impl FnOnce(&T)) -> Self
where Self: Deref<Target = T>, T: ?Sized,

Calls .tap_deref() only in debug builds, and is erased in release builds.
§

fn tap_deref_mut_dbg<T>(self, func: impl FnOnce(&mut T)) -> Self
where Self: DerefMut<Target = T> + Deref, T: ?Sized,

Calls .tap_deref_mut() only in debug builds, and is erased in release builds.
§

impl<T> TryConv for T

§

fn try_conv<T>(self) -> Result<T, Self::Error>
where Self: TryInto<T>,

Attempts to convert self into T using TryInto<T>. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

§

fn vzip(self) -> V

§

impl<T> WithSubscriber for T

§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a [WithDispatch] wrapper. Read more
§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a [WithDispatch] wrapper. Read more