Cardano Updates

  Bug 1 — CommitFinalized leaves stale decommitTx in state:
  After a snapshot confirming a decommit is confirmed on-chain via
  IncrementTx, the `decommitTx` field was not cleared. This caused the
  next ReqSn to re-include the already-posted decommit, leading to an
  invalid snapshot request. Fix: clear `decommitTx` in the CommitFinalized
  aggregate case when the confirmed snapshot carried a decommit.

  Bug 2 — Leader stuck in RequestedSnapshot after stale ReqSn echo fails:
  When a leader's own ReqSn echo failed validation (e.g. stale decommit or
  expired deposit), the Error outcome left seenSnapshot as RequestedSnapshot
  forever — the timer was noop in that state so the head could never make
  progress. Fix: add `abortOwnEchoOnFail` wrapper in onOpenNetworkReqSn
  that converts RequireFailed on own echo to a new SnapshotRequestAborted
  state change, resetting seenSnapshot to LastSeenSnapshot so the timer can
  retry with fresh content.

  Bug 3 — onChainTick fires DepositExpired/DepositActivated on every tick:
  Deposit status was recomputed from scratch on each tick, causing
  DepositExpired and DepositActivated events to fire repeatedly for already-
  transitioned deposits. Fix: filter to only emit events for deposits whose
  status actually changed in this tick.

  Also fixes restartedNodeCanObserveCommitTx cluster test: the restarted
  node now uses withHydraNodeCatchingUp so the Committed message is not
  consumed by the NodeSynced wait before the assertion can see it.

Signed-off-by: Sasha Bogicevic <[email protected]>

  withHydraNode waits for NodeSynced before yielding the client, consuming
  all messages (including Committed) in the process. Switch the restarted
  node to withHydraNodeCatchingUp so the Committed message is observed
  directly during chain replay. Also scale all wait timeouts by blockTime.

Signed-off-by: Sasha Bogicevic <[email protected]>

Signed-off-by: Sasha Bogicevic <[email protected]>

  Restore WaitOnDepositActivation (was regressed to noop on this branch) so
  non-leader nodes retry a ReqSn that includes a deposit they haven't yet
  activated, matching master behaviour.

  Update four BehaviorSpec tests to reflect the immediate-ReqSn model introduced
  in 7e570004b:
  - "snapshots are created": tx 40 lands in sn=1 alone; txs 41+42 batch into sn=2
  - "depending transactions confirmed in order": firstTx → sn=1, secondTx → sn=2
  - "conflicting transactions": SnapshotConfirmed fires before TxInvalid
  - "commit snapshot only approved when deposit settled": relax positive assertion
    to [n1] — n2 dropped n1's AckSn while waiting for deposit activation and
    never independently confirms; n2 still observes CommitFinalized on-chain

Signed-off-by: Sasha Bogicevic <[email protected]>

Signed-off-by: Sasha Bogicevic <[email protected]>

  isEmpty returned False while async threads were running but the queue
  was empty, causing runToCompletion to block forever in dequeue waiting
  for items that would never arrive. Use STM retry to wait until all
  tracked threads complete before declaring the queue empty.

Signed-off-by: Sasha Bogicevic <[email protected]>

  The versionNeedsSnapshot timer fires an empty version-bump snapshot
  after each IncrementTx. When Close is sent before that snapshot
  confirms at the closing node, the node correctly contests with the
  better snapshot, extending the contestation deadline by one
  contestation period (10 * blockTime).

  The previous buffer of 3 * blockTime was too short in that case,
  causing a timeout before ReadyToFanout arrived. Extended to
  13 * blockTime to cover the contest round plus block-latency buffer.

Signed-off-by: Sasha Bogicevic <[email protected]>

Signed-off-by: Sasha Bogicevic <[email protected]>

  When two nodes race to post the same on-chain transaction (e.g.
  IncrementTx), the losing node's submission fails after a 1-second delay
  in txSubmissionClient. During this blocking wait, the timer (200 Hz)
  fills the bounded input queue (100 items), causing the subsequent
  enqueue of PostTxError to deadlock permanently. The main loop never
  recovers and chain events (OnIncrementTx → CommitFinalized) are never
  processed.

  Fix by running postTx in a background thread via asyncTracked, which is
  tracked by isEmpty so runToCompletion in tests correctly waits for the
  background work to finish. This also makes PostTxError delivery
  guaranteed for all tx types (CollectComTx, CloseTx, etc.), not just
  the racing ones.

  With the main loop no longer blocking on postTx, the 1-second
  threadDelay in txSubmissionClient is no longer needed — the chain
  observer naturally processes the winning transaction before PostTxError
  is enqueued.

Signed-off-by: Sasha Bogicevic <[email protected]>

Signed-off-by: Sasha Bogicevic <[email protected]>

It was green for the wrong reason.

Signed-off-by: Sasha Bogicevic <[email protected]>

Signed-off-by: Sasha Bogicevic <[email protected]>

Signed-off-by: Sasha Bogicevic <[email protected]>

Signed-off-by: Sasha Bogicevic <[email protected]>

  withHydraNode waits for NodeSynced before yielding the client, consuming
  all messages (including Committed) in the process. Switch the restarted
  node to withHydraNodeCatchingUp so the Committed message is observed
  directly during chain replay. Also scale all wait timeouts by blockTime.

Signed-off-by: Sasha Bogicevic <[email protected]>

This is going to be an issue for most of these tests:

The contestation period is used for the upper bound of increment txs
(and others): so it must not be too big for a deposit with a small
deadline / depositPeriod to be able to be picked up.

  Introduce tryEnqueueClient on InputQueue — a non-blocking enqueue that
  returns False immediately when the TBQueue is full, without the timer-
  coalescing side-effect of tryEnqueue. Wire it through as tryWireClientInput
  on DraftHydraNode.

  HTTP POST /transaction now returns 503 immediately if the queue is full
  rather than blocking. WebSocket NewTx sends an InvalidInput error message
  back to the client instead of silently dropping or blocking.

  Adds WaitOnDepositActivation to WaitReason (restoring parity with master
  where it was regressed to noop on this branch).

  Wait — WaitOnDepositActivation is already in the staged commit. Let me check:

Signed-off-by: Sasha Bogicevic <[email protected]>

  Restore WaitOnDepositActivation (was regressed to noop on this branch) so
  non-leader nodes retry a ReqSn that includes a deposit they haven't yet
  activated, matching master behaviour.

  Update four BehaviorSpec tests to reflect the immediate-ReqSn model introduced
  in 7e570004b:
  - "snapshots are created": tx 40 lands in sn=1 alone; txs 41+42 batch into sn=2
  - "depending transactions confirmed in order": firstTx → sn=1, secondTx → sn=2
  - "conflicting transactions": SnapshotConfirmed fires before TxInvalid
  - "commit snapshot only approved when deposit settled": relax positive assertion
    to [n1] — n2 dropped n1's AckSn while waiting for deposit activation and
    never independently confirms; n2 still observes CommitFinalized on-chain

Signed-off-by: Sasha Bogicevic <[email protected]>

Signed-off-by: Sasha Bogicevic <[email protected]>

This does not work for > 1 cluster size yet though