Relay Broker WebSocket

Endpoint: wss://api.thuum.dev/api/{domain}/relay/connect?token=…. The connection needs a key linked to a Relay Broker claim. Framing, refs and general errors are on WebSocket protocol.

One connection may be a client (it sends requests: needs P on the topic), a server (it serves topics and answers their requests: needs S), or both. Thuum reads topic, id, from / to, done and the size of data; it never reads data, and stores nothing. The terms are on Relay Broker.

Operations#

A head or body with to is a response op; without it, a request op. Thuum pushes these ops, which get no reply:

Fields#

from and to#

An address is opaque: it can't be guessed or forged, it is valid only for the topic it was issued on, and a client that reconnects gets a new one. A server tells requests apart by from and id together, since two clients may both use id: "1", and sends every op of the response with to set to the request's from. A to that isn't an address issued for the op's topic is a bad_request.

done#

A request or response is one head, then any number of body parts; the part with done: true ends it. Without a body, the head itself carries done: true. When the length of a body isn't known in advance, end it with a body that has done: true and no data: an end marker, not a part. @thuum/client always ends a body this way.

serve#

Serve a topic. The reply carries the connection's role on it: active if nobody else serves the topic, standby otherwise, queued behind the servers that asked first. Serving a topic the connection already serves answers its current role.

→ [{ "op": "serve", "topic": "echo", "ref": "1" }]
← [{ "op": "serve:ok", "topic": "echo", "ref": "1", "role": "active" }]

A connection serves at most 100 topics.

unserve#

Stop serving a topic; the first standby becomes active. Idempotent, and needs no flag. Requests the connection already received can still be answered.

→ [{ "op": "unserve", "topic": "echo", "ref": "2" }]
← [{ "op": "unserve:ok", "topic": "echo", "ref": "2" }]

role#

Pushed when a server's role on a topic changes: active (it receives new requests), standby (it receives none, but may still answer the requests it has) or none (it no longer serves the topic). A standby is told when it becomes active:

← [{ "op": "role", "topic": "echo", "role": "active" }]

Requests: head and body#

A client sends a head, then its body parts. Each op goes to the server that is active when it arrives, with the client's address as from:

→ [{ "op": "head", "topic": "echo", "id": "r1", "data": { "hello": "world" }, "ref": "1" }]
→ [{ "op": "body", "topic": "echo", "id": "r1", "data": "a", "ref": "2" }]
→ [{ "op": "body", "topic": "echo", "id": "r1", "done": true, "ref": "3" }]

The server receives (addresses are shortened on this page):

← [{ "op": "head", "topic": "echo", "id": "r1", "from": "g1gA….TqZ8…", "data": { "hello": "world" } }]
← [{ "op": "body", "topic": "echo", "id": "r1", "from": "g1gA….TqZ8…", "data": "a" }]
← [{ "op": "body", "topic": "echo", "id": "r1", "from": "g1gA….TqZ8…", "done": true }]

And the client gets each op's :ok once the server's connection has taken it:

← [{ "op": "head:ok", "topic": "echo", "id": "r1", "ref": "1" }]
← [{ "op": "body:ok", "topic": "echo", "id": "r1", "ref": "2" }]
← [{ "op": "body:ok", "topic": "echo", "id": "r1", "ref": "3" }]

Responses: head and body with to#

The server answers with its own head and body parts, each with to. It may start before the request's last part:

→ [{ "op": "head", "topic": "echo", "id": "r1", "to": "g1gA….TqZ8…", "data": { "echoed": { "hello": "world" } }, "ref": "7" }]
→ [{ "op": "body", "topic": "echo", "id": "r1", "to": "g1gA….TqZ8…", "data": "a", "ref": "8" }]
→ [{ "op": "body", "topic": "echo", "id": "r1", "to": "g1gA….TqZ8…", "done": true, "ref": "9" }]
← [{ "op": "head:ok", "topic": "echo", "id": "r1", "ref": "7" }]
← [{ "op": "body:ok", "topic": "echo", "id": "r1", "ref": "8" }]
← [{ "op": "body:ok", "topic": "echo", "id": "r1", "ref": "9" }]

The client receives the response by topic and id, without to:

← [{ "op": "head", "topic": "echo", "id": "r1", "data": { "echoed": { "hello": "world" } } }]
← [{ "op": "body", "topic": "echo", "id": "r1", "data": "a" }]
← [{ "op": "body", "topic": "echo", "id": "r1", "done": true }]

A server answers with its S on the topic, whatever its role: a server that became standby finishes the requests it has.

cancel#

A client that gives up on a request sends cancel. Thuum passes it to the topic's active server, if there is one, and answers at once:

→ [{ "op": "cancel", "topic": "echo", "id": "r1", "ref": "4" }]
← [{ "op": "cancel:ok", "topic": "echo", "id": "r1", "ref": "4" }]

← [{ "op": "cancel", "topic": "echo", "id": "r1", "from": "g1gA….TqZ8…" }]

The server should stop working on the request. The client ignores parts that still arrive for it.

Replies#

• serve, unserve, cancel, and every op refused for its own content (bad_request, forbidden, unavailable), are answered at once, in the reply array of the message that carried them, in op order.
• head:ok and body:ok come later, in a message of their own, when the destination connection has taken the op. They echo topic, id and ref. So do server_gone, client_gone and busy.
• Pushed ops and later replies may share a message.

:ok means Thuum handed the op to the other side's connection, not that the app there processed it: only the response says that.

Errors#

→ [{ "op": "head", "topic": "echo", "id": "r2", "data": {}, "done": true, "ref": "5" }]
← [{ "op": "head:error", "topic": "echo", "id": "r2", "ref": "5", "code": "unavailable", "message": "Nobody is serving \"echo\"" }]

→ [{ "op": "serve", "topic": "payroll", "ref": "6" }]
← [{ "op": "serve:error", "topic": "payroll", "ref": "6", "code": "forbidden", "message": "Missing S claim on topic" }]

← [{ "op": "body:error", "topic": "echo", "id": "r1", "ref": "8", "code": "client_gone", "message": "The client's connection ended" }]

Treat any head:error or body:error as the end of that request. Problems with a whole message (invalid JSON, a binary frame, more than 100 ops) get a general { "op": "error", "code": "bad_request" }, and an unknown op { "op": "error", "code": "unknown_op" }.

A server that dies after taking a request's ops is not reported to the client: Thuum doesn't know who was waiting. Every client must run its own timeout. An unexpected server error closes the connection with code 1011 instead of replying: its details are never logged, because they could hold your data. Reconnect, and treat every unfinished request as failed.

Flow control#

The replies are the flow control: there is no separate op.

• Window. A sender keeps at most a few parts of each request (or response) without their :ok, and waits for one before sending more. @thuum/client uses 8. The wait is a round trip to Thuum, not to the far end, so a stream isn't slowed by the distance between client and server.
• max_pending_ops. A connection has at most 256 ops awaiting their :ok. At the limit, Thuum stops reading its frames until one is taken, and TCP slows the sender down. Meanwhile it still takes the ops sent to that connection, so two connections waiting on each other can't lock.
• busy. An op the destination hasn't taken within 30 seconds fails with busy.

Nothing is queued but the ops in flight between two connections, in memory.

With the client#

const socket = acme.relayBroker.connect();
socket.onRole(({ topic, role }) => console.log(topic, role));
socket.onError(({ error, topic, id }) => console.warn(topic, id, error));
await socket.ready;

// As a server
await socket.serve('echo', async (request) => ({ head: request.head, body: request.body }));

// As a client
const response = await socket.request('echo', { hello: 'world' }, ['a', 'b'], {
	timeoutMs: 10_000
});
for await (const part of response.body) console.log(part);

await socket.unserve('echo');
socket.close();

The socket sends each part in its own message with a ref of its own, keeps the window, ends bodies with an end marker, and matches responses by topic and id. Failed requests reject with RelayBrokerSocketError, whose code is one of the codes above, or the socket's own timeout, cancelled or closed. See Sockets.