Verbs: Other machines
Verbs: Other machines: part of the Control Protocol of TUIOS.
This page is one part of docs/protocol.md of the tuios repository. The Control Protocol page lists every part.
stream-pane
Stream one pane as bytes. A client that is not tuios uses it to show a pane in its own terminal emulator, such as a phone app.
Params:
session(optional).window(required): the window id or name of the pane.from_seq(optional int): theseqthat the client got to on an earlier stream of this pane.boot_id(optional): theboot_idof the stream thatfrom_seqcame from.lease_cols,lease_rows(optional ints): hold a size lease from the start. See The size lease.session_id(optional): the session by its id, which a rename does not change. It wins oversession. The reply carries it.fit(optional bool): make the lease a viewer's. See The size lease.viewer(optional object): the terminal the stream shows the pane in:bg,fg(#rrggbb),palette(sixteen comma-separated#rrggbb),kittyandsixel(bool). While no TUI client shows the session, the daemon answers the pane's OSC 10, 11 and 4, DA1 and kitty graphics queries from the newest viewer's report. A client that attaches pushes its own.
Request:
{"id": 1, "verb": "stream-pane", "params": {"session": "work", "window": "3f2a9c1e", "from_seq": 48211, "boot_id": "9f2c41d07a3e8b65"}}Response, one line:
{"id": 1, "result": {"type": "pane_stream", "mode": "resume", "seq": 48211, "cols": 80, "rows": 24, "boot_id": "9f2c41d07a3e8b65", "session": "work", "window": "3f2a9c1e-...", "title": "zsh"}}modeisresumeorsnapshot. It isresumeonly whenboot_idis this daemon's and the daemon still holds the output afterfrom_seq. Then the first frames are output fromfrom_seq. Otherwise it issnapshot, and the first frame isS.seqis the stream position where the first frame starts.colsandrowsare the size of the pane atseq. Inresumemode, resize your emulator to them when they are different.windowis the window id.input_offsetsis true when the daemon takesJframes. An older daemon leaves it out.
After the response line, the connection carries binary frames in both directions. A frame is one type byte, a 4-byte payload length (big-endian), and the payload. All numbers are big-endian. The connection does not take JSON again.
Skip a frame type that you do not know. Read its length, then read and drop that many bytes. The daemon can add a frame type in a later release, and a client that stops on an unknown type breaks then. The daemon sends a new frame type only when the client can ignore it safely.
The daemon charges each snapshot to its memory budget for client reads, until
the S frame is sent. The charge is 160 bytes for each cell of the screen and
of 500 rows of history, and at most the whole budget. When the budget has no
room for 2 seconds, stream-pane fails with busy. When the stream needs a
new snapshot and the budget has no room, the stream ends with an X frame
that starts with busy:. Connect again with from_seq.
Frames from the daemon:
| Type | Payload | Meaning |
|---|---|---|
S (0x53) | u64 seq, u16 cols, u16 rows, bytes | A snapshot. Reset the emulator to cols by rows and write the bytes into it. |
O (0x4F) | u64 seq, bytes | Output. Write the bytes into the emulator. seq is the stream position after the last byte. |
R (0x52) | u64 seq, u16 cols, u16 rows | The pane changed size at seq. Resize the emulator. |
E (0x45) | JSON {"code", "message"}, and accepted on busy | The daemon refused an I, J or L frame, or released a lease that the client did not renew (code lease_expired). The other codes are the verb error codes. The stream continues. |
X (0x58) | UTF-8 text | The stream ended: exited N when the pane's program exited with status N, killed N when signal N ended it, closed when its window closed first, the daemon stopped, or refused: ... when the caller may no longer read the pane. The daemon then closes the connection. |
Frames from the client:
| Type | Payload | Meaning |
|---|---|---|
I (0x49) | bytes | Input for the pane, as typed. The client encodes the keys. |
J (0x4A) | u64 offset, bytes | Input, as I, that starts at this stream's input byte offset. Typed only when offset is the number of input bytes the stream took. |
L (0x4C) | u16 cols, u16 rows | Hold a size lease. 0 0 releases it. |
The snapshot bytes paint the pane on a fresh xterm-compatible emulator: up to 500 rows of history above the screen, every cell with its colours and attributes, soft wraps, and the cursor with its position, shape and visibility. A palette colour stays a palette index, and a truecolour stays a truecolour. When a program uses the alternate screen, the bytes paint the shell's screen first and then switch to the alternate screen. The bytes also set the scroll region, the character sets, the pen, the kitty keyboard flags, modifyOtherKeys, and the DEC modes that change the input a client must send: application cursor keys, the keypad mode, bracketed paste, focus reports, mouse tracking and its encodings, and alternate scroll. They do not set left and right margins, origin mode, insert mode (IRM), newline mode (LNM), the saved cursor or protected cells.
The daemon takes the snapshot from its own emulator under the lock that reads
the emulator's position. So the snapshot shows exactly the stream up to its
seq, and the output frames continue from that seq.
Every O frame starts where the one before it ended. A client keeps the last
seq it wrote into its emulator and sends it as from_seq when it connects
again. The daemon keeps the last 64 KiB of each pane's output. When the client
is too slow and the daemon cannot keep its output, the daemon continues from
those 64 KiB, or sends a new S frame when they do not reach back far enough.
The seq also moves past some bytes that the daemon does not send: a kitty
graphics frame that a newer frame of the same image replaced while the client
was behind.
An I frame passes the checks that send-text passes: the link policy needs
write, the pane grants of the caller, and a restricted connection. An L
frame passes the checks that resize passes. A refused frame gets an E
frame, and the stream continues. A frame from the client larger than 64 KiB
ends the stream. Send a longer paste as more than one I frame.
The daemon writes I frames into the pane in order, on its own goroutine. A
pane that does not read its input does not delay L frames. Up to 256 KiB of
input, in up to 4096 frames, waits for the pane, so a client may send one
frame per key and a burst is never refused while the pane reads. When the
pane has not read that much, the next I frame gets an E frame with code
busy and accepted, the number of input bytes this stream took so far, all
typed or waiting, in order. Every I frame after it is refused the same way
until the pane has read what waits, so nothing is typed out of order. Send
again from byte accepted. The daemon logs each run of refusals.
A frame the client sent before it saw the refusal can reach the daemon after
the pane has read what waited, and an I frame then is typed, after the bytes
that were refused. A client that resends uses J frames instead, when the
reply has input_offsets: true: each one names the input byte it starts at,
and the daemon types it only at the offset it expects, accepted. A J
frame at any other offset gets E with busy and accepted, nothing of it
is typed, and the refusals of a full queue are not changed by it. So a resend
from accepted is typed once, and a frame from before the resend never is.
I and J frames count toward the same offset, and a stream may send both.
A daemon without input_offsets refuses a J frame as an unknown type: send
I frames to it.
The daemon checks the caller again each second, as it checks a new
stream-pane. When a pane loses its read grant, or the link policy no longer
allows list, the stream ends with an X frame.
Link capability: list for the verb. write for I and L frames.
Errors: invalid_params when window is missing or a lease size is not
between 1 and 4096. window_not_found and session_not_found as for every
verb.
The size lease
The stream does not change the size of the pane or of the session. The client shows the pane at the size that the pane has.
A lease holds one pane at a size, while the stream is open. Each dimension is
the smaller of the lease and the size that the session's clients ask for. So
a lease never makes a pane larger. While the lease is held, a resize from any
client is limited to the lease. When the stream releases the lease with
L 0 0, or the connection closes, the pane goes back to the size that the
clients asked for last. Other panes and the session size do not change. An
attached tuios client shows the smaller pane in the same rectangle.
A stream that came over a link is the exception. When the link drops, its
lease stays for 15 seconds after the connection closes. A viewer that opens
the stream again in that time holds the pane at the same size, so the program
in the pane does not redraw twice. When the far side closes the stream on
purpose, tuios stdio-proxy tells the daemon first (see closes under
What a linked machine may do here), and the lease ends at once.
When two streams hold a lease on one pane, the smaller lease applies in each dimension.
A stream opened with fit holds a viewer's lease. While no TUI client shows
the session, the pane takes the size of the largest viewer's lease, which can
make it larger than the size its clients asked for. When the last viewer's
lease ends, the pane goes back to the size its clients asked for. While a
client shows the session, a viewer's lease only holds the pane smaller, as any
lease does. The daemon decides under the pane's lock, and decides again on
every attach, detach and disconnect.
Leases resize a pane at most once every 100 ms. A lease that comes sooner applies when the 100 ms end, and the pane takes the newest lease.
A lease lasts 30 seconds. To keep it, send the same L frame again every 10
seconds. When the client sends no L frame for 30 seconds, the daemon
releases the lease within one more second. It then sends an E frame with
code lease_expired, and the pane goes back to the size that the clients
asked for. A new L frame sets the lease again.
The renewal bounds a lease that a lost client holds. A phone that drops off the network sends nothing, and ssh and TCP can keep its connection open for hours. A ping from the daemon does not find that sooner: its write goes into the buffers of the proxy, ssh and TCP, and does not fail until they are full. The stream itself ends when the connection closes, or when a write to the client blocks for 10 seconds. Until then, the stream holds only its place in the pane output. It never holds back the pane.
attach-presence
Give this connection the person's human_nonce without an attach. A phone
that only answers the Inbox uses it.
Params: session (optional). Without session, the nonce is for all
sessions.
Request:
{"id": 1, "verb": "attach-presence"}Response:
{"id": 1, "result": {"type": "presence", "human_nonce": "294e0a278581acd0f07ff40561c52bf7", "client_id": "client-1791531791506586672"}}respond, reply-approval, answer-ask, dismiss-attention and every other
verb that takes human_nonce accept this nonce. A presence made with
session acts only in that session, and a presence made with no session acts
in every session. See Nonce scope. The other rules are the
same as for the nonce of an attach:
- The caller must be outside every pane of this daemon. Over a link, the
stream must come on the link-human socket, so the hub must open it with
{"human":true}. Otherwise the call fails withforbidden. - A verb that uses the nonce must come over the same kind of connection: a local socket, a plain link or a link-human stream.
- When the kernel gives the process ids, the verb must come from the same
process. All streams of one
tuios stdio-proxycome from the same process, so a phone can send the nonce on any stream of that link.
The presence is not an attach. It does not change the session size, it does
not count as a client that shows or focuses a pane, and it gets no
broadcasts. So request-approval holds a prompt for the Inbox and does not
answer viewed. The connection continues to take verbs. The presence ends
when the connection closes, or when the connection calls
restrict-connection. A second call replaces the nonce.
Link capability: list. The verbs that use the nonce need their own
capability, for example respond for reply-approval.
register-push
Register a phone's Web Push subscription. From then on, the daemon pushes Inbox items to the phone. The phone needs no connection to tuios. Its push service, or its UnifiedPush distributor, wakes it.
Params:
endpoint(required): the push service's address for the phone. It must behttps. It can behttponly when its host islocalhostor a loopback or private IP address, and[notify.webpush] allow_insecureistrue. A loopback or private host, given aslocalhostor an IP address, needsallow_insecurealso withhttps. A link-local, multicast or unspecified IP address is refused. The daemon checks a host name when it connects: it does not connect to an address that the endpoint could not name. A push does not follow a redirect.p256dh(required): the phone's public key, the uncompressed P-256 point (65 bytes, first byte0x04), base64url. Padding is optional.auth(required): the phone's authentication secret, 16 bytes, base64url.device(required): a name for the phone, 1 to 64 bytes of printable text. A second call with the same name replaces the phone. Over a link, a call with the name of a registered phone fails withforbidden.kinds(optional): the Inbox kinds to push. The default isapproval,plan,askandquestion.human_nonce(required): the nonce of an attach, or of an attach-presence made with no session, from the same process.
Request:
{"id": 1, "verb": "register-push", "params": {"endpoint": "https://push.example.net/s/abc", "p256dh": "BKZ91bcH6n7KanE3PpyzUohdcLmK8Nze7jWdSbKRc9I4ZnFUdH6u_p5rNRnpzq_6CCKbzDwOprt1CjQPXjNbY04", "auth": "GT7vKYlZkqz3Qw7cTswShg", "device": "pixel", "human_nonce": "294e0a278581acd0f07ff40561c52bf7"}}Response:
{"id": 1, "result": {"type": "push_registered", "device": "pixel", "kinds": ["approval", "plan", "ask", "question"], "replaced": false, "vapid_public_key": "BCM4cML1lw6X1BHn3-nOS6-QtjYUql5FDx-etpdO8RHCTTipUMWpGY46YBB4bLwbc31yGx_O9LQVABj-B0Dqvp0", "machine": "studio"}}vapid_public_key is the daemon's VAPID public key (RFC 8292), the
uncompressed P-256 point, base64url. A phone that subscribes through a push
service that takes an application server key gives it this key. Get it first
with list-push. The daemon makes the key once and keeps it in
the state directory, at push/vapid.pem, with mode 600. The phones are in
push/subscriptions.json, with mode 600.
Who may call it:
- Only the person. A missing nonce, a nonce that does not verify, and any
call from a process inside a pane fail with
not_human. - A phone gets the Inbox of every session, so it is an act in no one
session (see Nonce scope). A presence made for one
session covers only that session. With such a nonce,
register-push,list-pushandremove-pushfail withnot_human, and the message says the nonce is for another session. - Over a link, the link needs the
respondcapability. Without it the call fails withforbidden. A link can not replace a registered phone. - A restricted connection cannot call it.
Each registration opens an Inbox item of kind errored, with no session,
named phone NAME. Its summary says who registered the phone (this machine,
or the link from a named machine) and whether it replaced a phone of that
name. A remove-push over a link opens the same item. A remove-push on
this machine closes it.
At most 16 phones. A bad endpoint or key fails with invalid_params.
What the phone gets
Each push is one HTTP POST to endpoint with these headers:
| Header | Value |
|---|---|
Content-Encoding | aes128gcm |
Content-Type | application/octet-stream |
TTL | 3600 for the open and the close of approval, plan, ask and question, else 120 |
Urgency | high for an open of approval, plan, ask or question, else normal |
Topic | 32 base64url characters, the same for the open and the close of one item. A push service that holds an open replaces it with the close. |
Authorization | vapid t=JWT, k=VAPID_PUBLIC_KEY |
The JWT is ES256, with the claims aud (the endpoint's origin), exp (12
hours from the send) and sub ([notify.webpush] subject, by default
https://tuios.dev/push). The push service reads the JWT, so the default
sub names no machine.
The TTL of an item that waits for the person is one hour. The item stays
answerable while it is open, often after the hook's hold ends, and a phone
that sleeps can get a push some minutes late. A stale open does not outlive
its item: the close has the same Topic and the same TTL, and a push
service that still holds the open replaces it with the close.
The body is encrypted for the phone by RFC 8291, as one record of the
aes128gcm content coding (RFC 8188). The body is:
salt (16 bytes) | rs (4 bytes, big endian, 4096) | idlen (1 byte, 65) | keyid (65 bytes) | ciphertextkeyid is the sender's one-time public key, as_public, an uncompressed
P-256 point. The phone keeps ua_private (its private key), ua_public (the
p256dh it registered) and auth_secret. To decrypt, with HMAC-SHA-256 as
HMAC(key, data):
ecdh_secret = ECDH(ua_private, as_public), the 32-byte x coordinate.PRK_key = HMAC(auth_secret, ecdh_secret).IKM = HMAC(PRK_key, "WebPush: info" || 0x00 || ua_public || as_public || 0x01).PRK = HMAC(salt, IKM).CEK = HMAC(PRK, "Content-Encoding: aes128gcm" || 0x00 || 0x01), first 16 bytes.NONCE = HMAC(PRK, "Content-Encoding: nonce" || 0x00 || 0x01), first 12 bytes.- Decrypt
ciphertextwith AES-128-GCM, keyCEK, nonceNONCE, no associated data. The 16-byte tag is at the end. - Remove the zero bytes at the end, then the
0x02before them, which marks the last record. tuios pads the JSON with zero bytes up to 256, 512, 1024, 2048 or 3072 bytes, the smallest that holds it, so the length of a push tells little about its text.
Steps 2 and 3 are HKDF (RFC 5869) with auth_secret as the salt. Steps 4 to
6 are HKDF with salt as the salt. The ntfy UnifiedPush distributor gives
the body to the app base64-encoded.
The plain text is JSON, at most 3 KiB, with no HTML escapes:
{"v":1,"type":"open","id":"1","kind":"approval","session":"demo","window":"f9ab67c1-6226-4335-8c7e-ea784574a3ef","harness":"claude-code","name":"Terminal f9ab67c1","summary":"approve Bash: npm test","request_id":"69a569a6ab65115e","options":["once","deny"],"machine":"studio"}| Field | What it is |
|---|---|
v | 1. A change that breaks a reader gets a new number. |
type | open or close. |
id | The Inbox item's id. A close has the id of its open. |
kind | The Inbox kind. |
session, window | The pane the item is about. |
host | The linked machine the item is on, empty for this one. |
harness, name, summary | What the Inbox row shows. |
risk | The risk rules the call matched, as the Inbox item has them. |
request_id, options | A held prompt's request and its answers, for reply-approval. |
machine | The name of the machine that sent the push. |
truncated | true when text was cut to fit 3 KiB. |
An empty field is left out. A payload that does not fit 3 KiB is cut, not
dropped: first the summary, then the risk, then each text field. The
id, request_id and options are never cut, because an answer gives them
back. A close carries only v, type, id, kind,
session, window, host and machine.
The daemon sends an open when an item opens. It sends the open again,
with the same id, when the item gets a held prompt, or loses one: then
request_id, options or risk change. A phone replaces the notification
with the same id. It sends a close when the item closes, also when you
answer it at the desk. A phone that does not know the id ignores the close.
Delivery never blocks the daemon. Each phone has its own queue. A push that
gets no answer, 429 or a 5xx is tried again after 1, 4 and 15 seconds.
When the push service answers 404 or 410, the daemon removes the phone,
logs it, and opens the Inbox item phone NAME to say so.
[notify] enabled = false stops the opens. A close still goes, so a phone
can remove what it shows. max_per_hour limits the opens to each phone. An
item from another machine is not pushed: that machine pushes to its own
phones.
Link capability: respond.
Verbs: Files and worktrees
Verbs: Files and worktrees: part of the Control Protocol of TUIOS.
Verbs: Other verbs
Verbs: Other verbs: part of the Control Protocol of TUIOS.
tuios lives only at tuios.dev and github.com/Gaurav-Gosain/tuios. tuios has no X or Twitter account. An account that offers a tuios token or presale is not this project.