4.8.11 Connection And Stream Renewal

Requests And Policy

Public Network.connect! reuses the same live connection without starting renewal when policy/lifetime overrides are omitted or an explicit finite requirement is already covered. An uncovered ttl:/expire: requirement uses connection-protect-renewal! and connection-request-renewal! after releasing the Network mutex. The target is resolved once; a ready-reuse caller’s renewal deadline comes from its original call start, while an establishment joiner is additionally capped by its inherited establishment deadline. Satisfied callers need not wait for a larger unrelated request. These internal helpers accept resolved operation inputs, not proof of a valid grant; only the authenticated engine installs authority.

Every pending caller is checked against its own cutoff before operation, wire-result, or general-coverage settlement. A longer shared round cannot grant late success to a shorter-budget caller. Remote RESULT settlement applies the remote request cutoff too. A result settled in time remains valid when its waiter reacquires the parent mutex after its deadline; a still-pending waiter records timeout before unwinding.

lease: 'renewable enables sticky local automatic interest through connection-enable-renewable!. A pending session transfers accepted policy/seed to the elected parent before activation; later attachments check that parent’s actual capacity. The latest accepted nonfalse seed replaces the one retained connection seed. Omitted auth and later fixed/default callers do not clear it or disable policy. Accepted policy survives caller detachment, not physical connection closure. Serving a peer’s renewal admits protection but does not enable local automatic interest.

Connection.open-stream! with lease: 'connection captures the installed connection expiration as the initial strict protocol-authority target. It does not renew the connection or enable its automatic policy. The original optional parent becomes the stream’s seed. Linked scope creation occurs at accepted IO registration: validated ACCEPT receipt for the opener, writer ACCEPT selection for the recipient. Thus the recipient can process a peer renewal before local ACCEPT release/bookkeeping, and the opener can participate before its local open callback finishes. Registered linked IO, not completed pending?/success? bookkeeping, establishes renewal eligibility; the separate opening deadline and callback/accounting obligations still apply. Registration, connection installation and successful OPEN completion refresh the desired connection target. An already-covering stream needs no exchange; otherwise the original opener schedules reauthorization, coalescing newer desired targets without changing an active round.

The parent-serialized connection-opening-expiration helper uses the installed StreamIO lease when IO exists, otherwise the selected positive opening expiration, otherwise the current parent expiration for linked OPEN or the captured requirement for fixed OPEN. The caller holds ConnectionParent.mx. A linked OPEN’s captured requirement remains its initial credential-coverage target, not an immutable pre-selection liveness cutoff. Parent renewal neither restarts the separate opening deadline nor makes an invalid or expired selected stream grant acceptable.

Both policies exclude nonfalse ttl:/expire: before admission or policy mutation. Omitted policy preserves the independent fixed connection/stream TTL defaults of 3600 seconds. No automatic reconnect, infinite expiration or expired-authority grace period is supplied. Automatic operations query current context output authority plus the retained seed, so an expired seed need not pin future renewal. Adaptive direct grants use the finite configured TTL window, extended to the strict target when necessary; delegated grants respect the entire parent-chain horizon. Selection preserves longest-valid-expiration ordering and original bundle indices.

Ownership

ConnectionParent.mx serializes renewal admission, protocol transitions and shared publication; credential preparation belongs to the admitted worker. These records are implementation exports, not public mutation APIs:

ConnectionParent.expire is the actual installed connection lease after attachment; Connection.expire reads it under the parent mutex, falling back to the candidate only before attachment. StreamIO.expire is the actual installed stream grant, independent of the connection minimum and the opening’s historical expiration. Renewal does not mutate the handshake candidate, reconstruct public objects or reinterpret historical opening metadata as installed authority. Last installed values remain observable after failure/close and never roll backward.

Protocol Transitions

HELLO uses version 1, with no version-0 fallback. The connection capability remains /network/connect/v0; it is not the transport version. Mode bytes distinguish fixed and adaptive HELLO/connection operations, and fixed versus linked OPEN. The 13-byte envelope and private preaccepted startup API remain unchanged. That trusted private path still has no renewal dispatcher or authority-validation shortcut. Full payload schemas are in wire.md.

Round/request IDs start at one, never wrap or reuse, and require no lifetime tombstone list. Connection frames use envelope stream ID zero; stream renewal uses the already committed OPEN ID plus its own round ID. The reader validates the bounded envelope before allocation, then extracts correlation before interpreting live payloads. Zero IDs and live-scope future responses, wrong roles, fixed/unregistered stream renewal and impossible active phases/indices are connection-fatal. Correlated late responses for retired rounds/streams are discarded without deserializing stale credentials; ordinary lifecycle expiry is handled before live-payload interpretation.

Namespace exhaustion is independent of available worker/control slots. After the last permitted active/deferred exchange and any pending wire result settle, new uncovered interests fail with Closed (remote requests receive reason-limit) without waiting for their operation cutoffs. Retired workers and terminal wire controls may remain owned; failure of new work cannot release them. Service settles newly exhausted scopes in the same pass. Covered callers and installed authority remain valid, and automatic targets never overflow a maximum-u64 installed lease.

Under the parent mutex, after lock acquisition, renewal dispatch observes both the active round’s deadline and the deferred OFFER’s deadline before classifying any ID against that scope’s state. An expired deferred descriptor retires before a higher OFFER’s overlap check; the still-live old ACK owner keeps its original deadline and actual output borrow. This does not permit a second still-live overlapping deferred OFFER.

The deferred slot remains an overlap owner after ACK release and until admission or retirement, independently of scope.active. Matching live OFFERs and pending live REQUESTs are fatal duplicates, not retired traffic just because their IDs equal a high-water mark. Matching deferred ABORTs decode and validate the entire live payload, including a nonzero reason, before changing state. Retired payload garbage remains ignored after bounded envelope/correlation validation.

Credential generation, token decoding/verification and bounded token-bundle serialization run on the admitted asynchronous worker, outside parent/stream locks, never on the reader, writer or service thread. Final bounded payload/header fill and queue publication use parent serialization. Worker handles are published before external work starts; workers recheck lifecycle after asynchronous results and unwind via the throwing worker gate. Reader/installation paths use the boolean lifecycle gate: old authorization/drain expiry or aborted StreamIO returns false, not an invalid protocol exception. Invalid live installation lifetime or live role/index/phase still raises; no blanket Closed/Timeout catch converts bad wire into cancellation. Renewal headroom is captured as round-admission time plus renewal-timeout, even when the inherited operation deadline is earlier. Old scope/connection leases and selected credentials must meet it, and installation must strictly advance authority.

The coordinator becomes committed at actual COMMIT writer selection, not queueing. The recipient becomes committed on valid COMMIT receipt and remains accountable through actual ACK output release. Its earlier local installation does not retire the round or cancel its original cutoff. Failure/cutoff during that interval closes the connection or resets only the affected stream, retaining installed metadata and any actual output borrow. The coordinator may receive ACK, install, and send the next OFFER before the recipient releases the previous ACK. In ack-sent the recipient retains at most one bounded deferred OFFER descriptor, with original ID/target/deadline and no preparation worker or staged output. It admits that descriptor only after the old round has retired and released all worker/ control ownership, rechecking remaining deadline, headroom and capacity. ABORT or expiry retires the deferred slot independently of the ACK owner; a second overlapping deferred OFFER or premature non-ABORT response is not another queue entry.

Capacity And Ordering

Protection is sticky and admitted atomically against configured and currently available control capacity. With C = ConnectionLimits.control-payload, it partitions four frame slots and C + 13 + 114 bytes (114 = 3 * 38, three header-inclusive REQUEST-sized controls) inside the existing aggregate limits. It neither raises those limits nor eagerly allocates buffers. Once enabled, connection renewal uses this partition; OPEN, stream renewal and ordinary controls use the remainder. Protected capacity is not automatic interest, and ordinary work cannot borrow the unused protected allowance. No existing borrow is revoked to enable protection.

The connection round has an independent worker slot. renewing-streams (default 16) separately caps locally coordinated and peer-coordinated stream worker ownership, at most twice that many stream scopes plus the connection scope. It is independent of pending/total OPEN admission and caller count. Each scope owns at most one worker; a retired but still-running encoder or unreleased control keeps its slot charged. Service considers connection scope first, then linked streams by installed expiration; within a scope explicit requests precede automatic interest. All output still shares the existing FIFO, control-burst/DATA fairness and single selected transport borrow. Worker priority is not permission to bypass selected or queued wire ownership.

Before starting credential work, an encoding round reserves one unqueued control and the full per-frame payload allowance plus 13 bytes, with the payload ceiling the minimum of local and peer control limits. It does not serialize into a tiny currently free remainder and misclassify transient contention as credential failure. Automatic work waits for its slot and full allowance without consuming an attempt or creating a worker. The stream recipient has no bundle to encode and checks fixed AUTH capacity instead. Encoding uses the existing bounded encode-auth-bundle; unused allowance is refunded only after a live publication gate measures the final payload. The actual header-inclusive charge survives queueing, selection and output until release. Mandatory replies that cannot fit remain connection-fatal, not dropped or queued elsewhere. Close/cancellation cannot reclaim active staging; its worker finalizer does so after the provider/encoder returns. This bounds serialized output and owned workers, not arbitrary context work, decoded-object heap use or shutdown latency.

Automatic connection timing leads expiry by (2 + ceil(linked / renewing-streams)) * T + 1 seconds, where T is renewal-timeout; the trigger never moves later within one installed generation. Starts are spaced by at least T. Attempts are remembered per installed connection expiration, or per stream expiration/desired target, to avoid retry spinning after refusal. New targets coalesce; old work must actually release before a replacement worker starts. Only the specified precommit peer authentication/lifetime refusals permit credential failover under the same operation budget; exceptions and other failures are not unlimited retry permission.

Stream IO And Cleanup

Parent stream installation holds parent -> input -> output, rechecks parent liveness and the round cutoff after the directional lock waits, then calls stream-io-install-lease-locked!. That helper first applies old stream/drain expiry and refuses aborted or actually expired state. By default it also refuses gracefully retirable IO. The internal committed? argument permits an authenticated transaction already in progress to finish installation on gracefully retirable IO, without changing FIN, rings, credits, drain state or callback eligibility. It validates a positive u64 strictly forward proposal, but does not validate credentials, connection authority or protocol state; the engine supplies that authenticated context. The lock-taking wrapper is only an internal standalone owner/test seam, not a sufficient parent installation boundary. See stream.md for the locked helper contract.

New-work eligibility is separate from existing-round liveness. A final read after FIN, or completion of the normal close callback (entry.done?), cannot turn a live owned exchange into timeout or discard its COMMIT/ACK. Existing rounds retain the actual installed lease and their original operation cutoff; only their actual workers/controls release ownership. Finishing committed metadata installation is not resurrection and must not RESET a peer that still has unread buffered data.

Successful installation wakes both directional CVs and the existing passive leaf, preserving object/handle identity, rings, credit, real DATA borrows and FIN/drain state. A half-closed stream can renew; an aborted or fully retirable stream cannot be resurrected. Application IO captures only its user budget once; waits combine that budget with the current installed authority and reacquire the mutex even after a timed wait before deciding expiry. Renewal and timeout setters never restart a Reader, whole-slice Writer or initial Writer.close drain budget. !NoTimeout does not mean unlimited authorization. Writer.write still consumes the complete supplied slice or raises, not a short successful prefix.

Renewal repeats no allow/open callbacks and creates no second open/close pair. Original callback eligibility and stream-before-connection close ordering remain. Cancellation is cooperative only: publish state and notify under the associated mutex, close native IO before potentially blocking stream cleanup, and join workers outside locks after their external calls finish. Protocol retirement/close never releases a still-borrowed frame or a still-running worker’s staging charge. Reaping joins completed workers, then checks retirement and all control releases before returning slots; Network shutdown includes these workers before connection-close.

After a complete shutdown join barrier, a final renewal retirement pass handles workers and controls that outlived service. It removes released round ownership, obsolete operations and terminal released wire requests before completed openings are pruned. It uses the same worker/control-release checks as live reaping without rejoining and changing an already-recorded error. Seeds, installed leases, public metadata and unfinished caller-owned request finalizers are not indiscriminately cleared. See acceptance.md for retained-view evidence and limits.

Startup cleanup covers reservation and owner-list publication as well as worker creation. Failure before a worker starts cancels its staging and finishes the round with the original error. Reply publication retains local ownership through attachment, fill and notification, cancelling that exact reservation if publication fails. If cancellation itself fails, the scheduler records the cleanup error as fatal under parent mx, leaving queued ownership for socket-first close rather than blocking FIFO. Fill owns its own cancellation attempt. Send tracks entry/normal return from fill: if fill raises with work still owned, send latches the failure as fatal without retrying cancellation; a released reservation needs no further cleanup. Attachment and post-fill notification failures retain send’s separate cleanup responsibility.

Actual output release returns accounting and marks work released before notifying opening/renewal owners. References clear even if a notification raises; the late failure is published under parent mx before unwinding. Ungated cleanup cannot replay completed accounting or notifications, and a competing close cannot replace the already-recorded failure, including a raised #f. Reference cleanup uses the outer catch established before release. No additional callback protection is allocated after irreversible bookkeeping, including for ordinary DATA frames without opening or renewal callbacks.

Historical Verification

The preceding hardening increment added eight protocol cases containing eighteen scenarios. The review-fix focused command with renewal-protocol-test.ss, connection-transport-test.ss and connection-framed-test.ss passes all 56 cases (20, 20 and 16 respectively). Revision-scoped build/full-regression evidence is recorded in implementation-notes.md. The older review results below are historical checkpoints, not the current coverage count.

Main reports that the final formatted production source built successfully with:

export GERBIL_BUILD_CORES=8 && make stdlib

The final review-focused command passed all twelve protocol cases and twenty-five StreamIO cases, with both MODULE-OK markers, HARNESS-OK and final OK:

./build.sh test -v 5 std/ensemble/network/renewal-protocol-test.ss std/ensemble/network/stream-io-test.ss

All nine original renewal-test.ss cases, containing fifteen real-network scenarios, also passed on Unix/TLS. The :std/io import and minimum-zero EOF-read fixes remain. All fourteen network-api-test.ss public API cases passed after the native SSL error-queue repair; temporary diagnostics have since been removed.

Together with the original renewal suite, this is twenty-one named renewal cases, with the proof levels distinguished below. The fourteen opening cases are a separate count: their two new source-loaded unit cases cover four linked/fixed pre-IO expiry crossings and four invalid/expired selected-grant scenarios. See connection.md for their unit-clock boundary, not a real connection-renewal claim.

The final formatted revision passed the full 36-module network/UCAN/supporting-IO regression, including both renewal suites, the existing single and ten-concurrent 4 MiB transfers, and native SSL/Reader coverage. Every module reported MODULE-OK, followed by HARNESS-OK and final OK, without errors; the latest test-only run took 551.305 seconds with a 1200000 ms timeout. Logs are /tmp/opencode/renewal-review-focused-20260913.log and /tmp/opencode/renewal-review-full-20260913.log. Exact commands, earlier failures and remaining review belong in implementation-notes.md. Passing that command does not close the dedicated acceptance-matrix gaps listed below.

For combined renewal-suite runs, allow a 300-second harness budget. The original longer scenarios use 32-second observation windows, and credential-refusal scenarios use 24 seconds. The protocol suite uses TTL 60 and renewal timeout four seconds, with twelve-second emergency gate bounds. Gates are released before joined fixture cleanup begins.