Skip to content

4.8.1 Ensemble Network Interfaces

Import :std/ensemble/network/interface for the abstractions shared by ensemble network implementations. This module defines contracts only, not a transport. DIRECTION-IN and DIRECTION-OUT are re-exported from :std/os/device. The lifecycle, lease resolution, authorization, and callback rules below are requirements on implementations, not networking behavior implemented by this module. The concrete implementation available through the network facade supports fixed leases, explicit connection renewal, opt-in renewable connections, and connection-linked stream reauthorization. renewal-plan.md preserves the design handoff; see Implementation Status for the current verification scope.

4.8.1.1 Ownership And Configuration

A network owns its TLS context, listeners, connections, streams, and network workers. It borrows the supplied CapabilityContext and NetworkMonitor; closing the network does not close either borrowed object. The caller must keep them usable while the network needs them. Cached keys borrowed from the context must not be explicitly released by network code.

Defaults and resource limits live together in config.ss, documented in config.md: NetworkConfig, NetworkLimits, ConnectionLimits, and StreamLimits. There is no separate limits module. Configuration, limits, and returned metadata objects are immutable by convention while in use; no defensive copying or live configuration update is promised. Per-call lease overrides do not mutate configuration.

4.8.1.2 Network

Network extends Closer and owns connections and listening addresses.

Method Arguments Result
host None Local host DID
connect! Peer DID, list of Address, optional positional Token; ttl:, expire:, lease: Connection
listen! Address Actual bound Address
peers None List of connected host DIDs
connections None List of open connections
listening None List of listening addresses

connect! opens, reuses, or joins the one shared connection per canonical peer DID. Implementations normalize public DIDs and reject self-connections. For new establishment, Unix addresses are tried first, then other addresses in preference order, sequentially within the original establishment budget. TCP connections use mutual TLS and verify the actual certificate key against the expected DID. Unix connections use signed identity proofs over plaintext under the agreed trusted local-OS/path threat model. UCAN authorization does not replace identity checks.

An empty address list means reuse-or-join: reuse a usable established connection or join pending establishment for that peer, including explicit renewal when required. If neither exists, fail without dialing, discovery, or reconnecting to old address metadata. Joining neither replaces the attempt’s address list nor restarts its deadline, and shared failure does not start a new per-waiter attempt.

The optional positional auth defaults to #f. A supplied token must be a usable DELEGATE parent for local invocation evidence; without one, credential generation uses the borrowed context’s output trust policy. This is not a per-caller credential-provenance constraint on an already shared connection. Supplied parents from joined callers may participate in credential failover under the original budget. Peer identity and mutual UCAN authorization remain required.

listen! completes bind/listen setup before returning and publishing a listener. For TCP port zero it returns the assigned port in the actual Address, and records that same address in listening. Another port-zero request creates another listener. Failed or duplicate/address-in-use binds are errors, not silent reuse or replacement. Unix listeners first acquire the stable sidecar lock, then remove a stale socket before binding and clean up the socket before releasing the lock. Non-socket paths are never removed. This is cooperative pathname ownership, not inode-based replacement protection: callers must not externally rename or replace the socket or sidecar while owned. See listener.md.

host remains available after close. peers, connections, and listening return observational snapshots while open, not liveness guarantees. These registry queries, connect!, and listen! raise Closed after network closure rather than returning empty snapshots.

4.8.1.3 Lease Arguments

Both opening methods retain the optional positional auth and accept independent optional keywords, so no auth placeholder is needed to supply lease arguments:

Argument Default Checked argument contract
ttl: #f #f or a positive exact integer duration in seconds
expire: #f #f or an exact integer in [0, 2^64 - 1], absolute Unix seconds
Network.connect! lease: #f Exactly #f or the symbol 'renewable
Connection.open-stream! lease: #f Exactly #f or the symbol 'connection

Explicit #f means omitted. Both supplied values must satisfy their argument contracts even when expire: takes precedence over ttl:. Inexact integer-valued numbers, fractions, strings, and #t are rejected. TTL zero/negative values and negative/out-of-u64 expirations are rejected. TTL is a duration, not a wire field: there is no arbitrary duration ceiling in the abstract interface.

Lease policy uses the exact method-specific symbol, not strings, #t, numeric wire modes, or the other method’s policy. #f means fixed/default behavior, not an indefinite lease. A nonfalse policy is mutually exclusive with nonfalse ttl: or expire:. The concrete implementation rejects that combination before reserving work or changing an existing connection’s policy. The signatures check each argument independently; they do not perform cross-argument validation, reserve resources, or attach policy through hidden contract side effects. Both lifetime arguments still require validation, even when expire: wins. Fixed mode continues to allow ttl: and expire: together, and the positional auth order is unchanged.

In fixed mode the implementation resolves the required expiration once at operation start: use non-#f expire: unchanged, otherwise start plus non-#f ttl:, otherwise the configured TTL for new establishment or stream opening. The connection and stream defaults are independently 3600 seconds. Negotiation, joining, and retries must not move this target; credentials must cover it in full, not silently shorten it. Resolved expiration range checks, overflow prevention before encoding, current time validity, and operation-budget/headroom checks belong to operation code. Accepting expire: 0 at the argument boundary is not permission to establish an expired lease. The interface does not read the clock, select a precedence branch, or fall back from an unusable explicit expiration to TTL.

For an existing connection, omitted policy and lifetime overrides reuse the valid lease without renewal. A resolved deadline already covered also reuses it; a later one requires explicit mutual renewal. Each joined caller retains its own requirement, and satisfied callers need not wait for longer requests. Ordinary pre-commit renewal failure preserves the still-valid old lease and reports failure; expiry or unresolved post-commit uncertainty closes the connection. Default/fixed callers do not enable automatic renewal. No policy enables implicit reconnect, infinite timestamps, or an expired-authorization grace period.

connect!(..., lease: 'renewable) requests adaptive finite establishment with no application-requested final expiration. Credentials still bound every installed lease; directly issued grants use a finite configured TTL window, not maximum-u64 as infinity. On a live connection, it enables sticky local automatic interest and reuses the same object without gratuitous renegotiation unless its timer is due. Later default/fixed callers cannot disable that interest. Either physical endpoint may enable it; the original physical initiator coordinates coalesced renewal rounds. Serving a peer’s round does not enable local automatic interest.

A renewable pending joiner records interest for the elected connection without changing the active handshake target, mode, or deadline. Acceptance requires serialized admission of the protected connection-renewal capacity in config.md. Before activation, policy ownership transfers to the parent; later joiners must attach there without bypassing real capacity checks. Once accepted, policy survives caller timeout/detachment, but not physical connection closure or failed establishment. The latest accepted renewable attachment carrying auth replaces the one retained seed parent; each new automatic operation also queries current context credentials. Omitted auth does not accumulate choices.

Authorization leases are independent of I/O timeouts and the configured handshake, stream-open, and renewal operation budgets. Timeout changes never extend authority.

4.8.1.4 Connection

Connection extends NetworkTimeout and Closer. It is a point-to-point peer connection that may multiplex multiple logical streams.

Method Arguments Result
network None Owning Network
address None Local Address
peer None Peer host DID
peer-address None Remote Address
direction None DIRECTION-IN or DIRECTION-OUT
expire None Last installed negotiated expiration, integer Unix seconds
open-stream! Protocol string, optional positional Token; ttl:, expire:, lease: Stream

Inbound connections were initiated by the peer; outbound connections were initiated locally. @Connection is the forward type alias used by Network.

expire is the minimum expiration of the two accepted connection credentials, not merely the requested target. Authorized renewal advances it at the local commit transition: the responder upon valid COMMIT, the coordinator upon ACK. These transitions are not globally simultaneous. After closure it retains the last installed value, not the close time.

open-stream! requires opener-to-recipient protocol-specific invocation authority, using the supplied DELEGATE parent or context policy. No reciprocal stream token is required merely to send responses on the accepted bidirectional stream. The stream’s accepted credential must cover its independent requested expiration even when that outlives the connection’s current lease. Opening a stream never renews the connection. A higher-level caller can pass the same absolute expire: to connect! and open-stream! to coordinate both leases without relative-TTL drift.

With lease: 'connection, OPEN instead captures the currently installed connection expiration as its initial strict requirement. Protocol authority must cover that value in full, though the selected grant may last longer. Linked streams are legal on fixed connections, following successful explicit extensions without enabling automatic connection renewal. OPEN completion must recheck for a raced connection extension and schedule any needed stream reauthorization rather than freeze the earlier target. The original optional parent is the stream’s seed; later operations also refresh context-provided protocol authority.

The owning network, addresses, peer DID, direction, and last installed expiration remain accessible after close. Implementations capture transport metadata while available instead of querying a closed socket. New streams and timeout changes on a normally closed connection raise Closed; a connection closed by another failure preserves and rethrows that stored failure, including a raised #f.

4.8.1.5 Stream

Stream extends NetworkTimeout and Closer. Its reader and writer carry the data of a single logical protocol stream over a connection.

Method Arguments Result
connection None Underlying Connection
id None Integer stream identifier within the connection
direction None DIRECTION-IN or DIRECTION-OUT
protocol None Protocol string
expire None Last installed stream credential expiration, integer Unix seconds
reader None Reader
writer None Writer

An inbound stream was opened by the peer. @Stream is the forward type alias used by Connection. Its lease is independent: connection renewal alone does not change Stream.expire. A fixed stream’s installed expiration never changes. A linked stream advances only through authenticated protocol-specific reauthorization: the original recipient installs on valid stream COMMIT, the original opener on ACK. It reports the actual credential expiration, never the minimum of the connection and stream expirations, a policy symbol, or infinity. Effective usability is bounded by both expirations; connection closure terminates every stream even if its own token remains valid.

After a connection extension, already-covering linked-stream authority needs no exchange; otherwise the original stream opener coordinates a one-way stream round. New connection targets coalesce without retargeting an active round. Successful installation preserves the Stream, Reader, Writer, connection reference, ID, direction, protocol, buffers, credit and actual DATA borrows, FIN/drain state, and callback eligibility. It repeats neither allow nor open callbacks. A half-closed stream may still renew for its remaining direction; aborted or fully retirable streams cannot be resurrected.

Ordinary precommit stream reauthorization failure leaves the old authorization usable until expiration and suppresses busy-loop retries. Postcommit confirmation failure resets only that stream by its operation cutoff. Malformed shared framing, wrong roles, impossible active transitions, and bad active selected indices remain connection-fatal protocol violations. Renewal never resets an application’s captured I/O or writer-drain deadline; waits must reevaluate current installed authority without mistaking an extended old expiration for an I/O timeout.

The owning connection, ID, direction, protocol, and accepted expiration remain available after close. Reader/writer getters return the existing handles, not fresh usable I/O objects. Expiration getters do not imply continued usability. Connection and stream return signatures declare :integer expiration values; implementations are responsible for installing validated absolute leases.

Connection and stream inherit set-input-timeout! and set-output-timeout! from NetworkTimeout, with IOTimeout arguments. Configured defaults apply to new objects. Connection setters affect shared transport IO, not individual stream application budgets; Stream setters affect the next read or write/drain. Setters on normally closed directions raise Closed; an existing abort rethrows its stored failure. Neither setter makes a closed direction usable again.

Closing the whole stream or its reader aborts both directions, discards unread and unsent data, and wakes blocked operations with Closed unless an earlier failure was already recorded. Authorization expiry also aborts without draining unauthorized traffic. Closing the writer instead prohibits new writes and drains accepted DATA followed by FIN under the stream’s write timeout, without waiting for peer consumption. Drain timeout/cancellation aborts that stream, not an otherwise healthy shared connection. Orderly EOF after buffered data is consumed does not abort the opposite direction. Repeated close is idempotent.

Byte IO

The existing Reader and Writer interfaces take u8vector buffers, not strings or character ports. Their slice offsets, lengths and read minimum are byte counts:

Method Positional arguments Result
Reader.read Buffer, optional start (0), end (buffer length), need (0) Bytes read; wait for data on a nonempty, non-EOF slice even with need zero. EOF before the minimum raises PrematureEndOfInput; EOF with no remaining minimum returns the available count, including zero.
Writer.write Buffer, optional start (0), end (buffer length) Exactly end minus start, or an exception; no successful short write.

Offsets and need are fixnums with 0 <= start <= end <= buffer length and 0 <= need <= end - start; end is exclusive. Reader additionally requires start < buffer length; Writer permits an empty write. Application code serializes operations within each direction; reading and writing may proceed concurrently. Do not mutate a write’s source slice or concurrently use a read’s destination while the operation is running. A successful write has copied every requested byte for transmission and permits source reuse, but does not acknowledge transport delivery or peer consumption. Failure after partial progress is not permission to replay the whole write.

Each read, whole-slice write and initial writer-close drain captures one IO deadline. Partial progress, wakeups, later timeout settings and renewal never restart it. Waits recheck the currently installed authority; !NoTimeout removes the IO deadline, not either authorization bound. The stream’s first recorded abort is rethrown; otherwise authorization expiry is checked before the captured IO timeout. Orderly EOF and expected half-close errors do not abort the reverse direction. See stream.md for the implementation boundary.

4.8.1.6 NetworkMonitor

NetworkMonitor supplies admission decisions and lifecycle notifications:

  • allow-connection? takes a peer DID and direction and returns a boolean.
  • on-open-connection and on-close-connection receive a Connection.
  • allow-stream? takes a peer DID, protocol, and direction and returns a boolean.
  • on-open-stream and on-close-stream receive a Stream.

Admission callbacks are advisory resource gates, not authentication or reservations. Outgoing connection admission uses the expected canonical peer DID before dialing; incoming admission follows identity and UCAN validation. Stream admission runs locally before outgoing OPEN and remotely after credential validation before acceptance. False or an exception fails the new admission. Unchanged connection reuse and renewal invoke neither allow nor open callbacks; fresh establishment after closure is a new admission. Internal configured capacity bounds remain hard bounds independently of advisory monitor results.

All callbacks run inline, outside network/connection locks, and must return promptly. They must not perform blocking I/O or wait for work requiring the invoking network path to progress. Inbound stream delivery dispatches application I/O to an application worker. Different objects’ callbacks may run concurrently; the monitor owns synchronization for host-wide accounting.

on-open-connection runs before activation, not after public readiness. Its Connection exposes metadata, callback-safe close and timeout settings, but cannot yet open streams. An application worker must use Network.connect! to await ready publication before opening a stream; an empty address list joins the pending establishment without dialing. Merely handing the callback’s object to another worker does not establish readiness. Never wait for that call inside the callback.

The open callback is final admission and completes before success is reported to initiating callers or joined waiters. It may reject by closing the affected object and raising Closed. This is expected rejection, not an unexpected worker error. Any open-callback exception, including Closed, aborts the object without its corresponding close callback. The failing callback must roll back any registration it performed. Local callers receive the original exception after cleanup; inbound stream failure is handled locally rather than escaping the shared reader and destroying unrelated streams. Remote acceptance/notification cannot be undone.

Each object’s open and close notifications occur at most once and never overlap. Only a normally returning open callback qualifies for a later close notification. Closure during open marks the object closed and wakes operations immediately, but defers notification until open finishes: deliver close on normal return, even if open closed the object; suppress it on any exception. Connection-open finishes before stream-open notifications. On connection closure, eligible stream-close notifications finish before connection-close, with no ordering requirement between different streams. Outstanding stream opens first finish under the same rule.

Unexpected callback errors are logged through the shared network logger without leaking tokens/private material or local exception details to peers. Close-callback exceptions count as notification completion and cannot prevent resource cleanup, other notifications, or cause recursive notification.

4.8.1.7 Network Shutdown

Network.close is idempotent and blocking: it finishes owned work, resource cleanup, and eligible notifications before returning. Calls from any network-owned worker, including monitor callbacks and finalizers, raise contextual ContractViolation before network locking, state mutation or joins. The rule includes cross-network calls and already closing/closed networks. Dispatch global shutdown to an application worker instead; callback-spawned threads do not inherit the worker marker. Return from the callback rather than waiting there for the closer. Connection.close and Stream.close remain callback-safe; they must not wait for their invoking callback to finish. Notification ordering may defer notifications, not resource shutdown. The borrowed context and monitor remain caller-owned after shutdown.

Cooperative Cancellation

Network close, stream abort and pending cancellation use explicit state under the associated mutex and CV notification while holding that mutex. Waiters recheck state and existing deadlines. Closing owned sockets interrupts blocked IO structurally before joins; production cancellation does not invoke exception-raising thread-interrupt!. There is no early public waiter-cancel API: public callers use existing deadlines and global close. Internal waiter withdrawal must not cancel shared work solely because one caller leaves.

Credential/admission workers finish their current context, encoder or callback call without unsafe preemption, then observe cancellation before further publication. Their quotas, resources and worker ownership remain held until work, cleanup and eligible notifications finish. A waiter returning does not release a still-running worker’s reservation or transport borrow. Network shutdown joins that work and completes callbacks; it cannot promise bounded completion of an arbitrary external call that never returns.

Constructor failures, callback exceptions and failures after normal callback return still require cleanup and the callback-pairing rules above. Use synchronous fault injection at source-private test boundaries to check those obligations without adding production hooks. Arbitrary external thread interruption, forced termination and arbitrary continuation escapes carry no network correctness guarantee. The Interrupt API and asynchronous raising-recovery machinery were explicitly withdrawn by vyzo, not deprecated or retained as a fallback. The remaining network catches and abort/release/wake/native-close retries are removed; normal ownership cleanup, SSL lifetime cleanup and the shared Reader minimum fix remain. See the later agreed decision.

Implementation Status

The runtime implements explicit connection renewal from either physical role, automatic renewable policy, and linked-stream reauthorization on the existing objects and I/O handles. Concrete methods validate policy/lifetime exclusion before admission. Renewal uses bounded workers, protected control staging, and the existing connection reader/writer and service timer; it is not a contract-only or codec-only placeholder.

See the facade integration status and main-owned current checkpoint for revision-scoped verification and final acceptance. The design handoff and historical checkpoints below do not describe current missing functionality. This docs-only audit does not run builds or tests.

Historical Checkpoints

Before the API withdrawal, main verified the concrete original-lease implementation: the 8-core stdlib build, all 11 public network API and 12 OPEN cases, and the broad 33-module network/UCAN/shared-IO regression passed. The latter excluded std/sync/threads-test and std/sync/rwlock-test, unchanged at that checkpoint. A formatting-only rebuild and all 23 public API/OPEN cases passed again. Network test source contains neither thread-interrupt! nor thread-terminate!; production never calls thread-interrupt!. No remaining Interrupt catches provide a synchronous fallback. These earlier results did not verify later revisions, prove the native crash fixed or establish full API completion: renewal was still required at that checkpoint. See the main-owned handoff for exact commands and revision scope; this interface module still defines contracts only.

At the subsequent API-removal checkpoint, main reported a successful 8-core make stdlib full transitive stdlib rebuild after std/error changed, not a core/full Gambit build. All 54 focused cases passed (RWLock 3, framed 16, transport 20, native Reader 15). The revised 33-module network/UCAN/supporting-IO command passed, replacing deleted std/error-test with cooperative std/sync/rwlock-test and excluding the shared std/sync/threads-test intentional-termination suite. No Interrupt references were found in src/**/*.ss; build/lib introspection confirmed no export. Static review found no normal-cleanup regressions or missed asynchronous-only overhead. CV ownership guards and release/notification flags remain necessary for ordinary timeouts/errors; native SSL lifetime cleanup and Reader minimum/EOF fixes remain. These results are reported by main, whose handoff is authoritative for their revision scope. A possible final formatting-only rebuild was not claimed complete at that checkpoint.

Those checkpoints preceded the Stage 1 renewal contract/codec revision, which had source-level checks only. Concrete method validation, handshake version/role handling, and renewal installation, scheduling, resource, and I/O integration were still pending at that stage. Those implementation limitations do not describe the current runtime above.