# Wire Protocol

> HTML: https://remote.ivjn.us/docs/en/protocol/

This document specifies the complete wire format between agent, exchange, and cli. Definitions live in `crates/proto`, and all three share the same code: fields are defined in one place, so inconsistencies surface at compile time rather than during a production session.

It serves operators inspecting network traffic to diagnose problems, and developers changing the protocol or writing third-party clients.

---

## Contents

- [Overview](#overview)
- [Transport and Endpoints](#transport-and-endpoints)
- [Protocol Version](#protocol-version)
- [Control Messages](#control-messages)
  - [DeviceUp (agent → exchange)](#deviceup-agent--exchange)
  - [DeviceDown (exchange → agent)](#devicedown-exchange--agent)
  - [OperatorUp (cli → exchange)](#operatorup-cli--exchange)
  - [OperatorDown (exchange → cli)](#operatordown-exchange--cli)
  - [Shared Data Structures](#shared-data-structures)
- [Binary Data Frames](#binary-data-frames)
- [Session ID Translation Between Two Namespaces](#session-id-translation-between-two-namespaces)
- [Handshake Sequence](#handshake-sequence)
- [Error Handling and Failure Categories](#error-handling-and-failure-categories)
- [Keepalive and Timeouts](#keepalive-and-timeouts)
- [Backpressure](#backpressure)
- [Android Screen Control](#android-screen-control)
- [End-to-End Sealed Sessions](#end-to-end-sealed-sessions)

---

## Overview

The protocol has two layers sharing one WebSocket connection:

| Layer | Carrier | Encoding | Contents |
|---|---|---|---|
| Control messages | WebSocket **text** frames | JSON | Registration, heartbeats, opening/closing sessions, result notifications |
| Data frames | WebSocket **binary** frames | 9-byte fixed header + raw bytes | PTY byte streams, command stdout/stderr |

Embedding streaming data in JSON as base64 increases its size by approximately 33%. PTY traffic consists of frequent small packets echoing each keystroke, so this overhead directly affects interaction latency. Control messages are infrequent, making JSON readability more valuable: packet captures are immediately understandable.

By convention, each WebSocket binary frame **carries exactly one data frame**. Extra or missing bytes are errors.

---

## Transport and Endpoints

| Endpoint | Client | Authentication |
|---|---|---|
| `GET /ws/device` | agent | The **first frame** after connection establishment must be a `register` control message containing `device_id` and `token` |
| `GET /ws/operator` | remote-cli | HTTP request header `Authorization: Bearer <运维令牌>` in the WebSocket handshake |

The two paths use separate credential types. Device tokens cannot access the operator endpoint, and vice versa.

Operator credentials use a request header rather than a URL query string because URLs appear in reverse proxy logs, browser history, and Referer headers. Device tokens use the first frame rather than a request header because some embedded clients cannot easily customize WebSocket handshake headers. The first-frame approach also allows protocol version mismatches and invalid tokens to use the same response message.

Production deployments must use `wss://`: both credential types appear in plaintext during the handshake.

---

## Protocol Version

```rust
pub const PROTOCOL_VERSION: u16 = 1;
```

Versions are compared in both directions during the handshake. A mismatch causes immediate rejection with an explicit explanation. This prevents version mismatches from appearing as an inexplicable parsing failure in a later message, which would be difficult to diagnose.

- Device side: agent includes its version in `register`. On a mismatch, the server returns `register_ack` with `ok: false`, describes both versions in `error`, and closes the connection.
- Operator side: the server proactively sends `welcome` after connection establishment. If the CLI detects a mismatch, it reports an error and instructs the user to upgrade to a version matching `server_version`.

**Unknown message `type` values are always rejected**, never silently interpreted as a known type.

---

## Control Messages

All control messages are JSON objects tagged by the `type` field (serde internally tagged enums). Tag values and field names use `snake_case`. `session_id` is transparently serialized as a plain number.

### DeviceUp (agent → exchange)

| `type` | Fields | Purpose |
|---|---|---|
| `register` | `protocol_version: u16`<br>`device_id: string`<br>`token: string`<br>`info: DeviceInfo` | Must be the **first** message after connection. The server accepts no other messages before it, with a 15-second deadline |
| `heartbeat` | `metrics: Metrics` | Periodic operational metrics. Connection liveness is handled separately by native WebSocket ping/pong: a missing heartbeat does not imply a broken connection, or vice versa |
| `exec_started` | `session_id`<br>`job_id`<br>`hold`<br>`timeout_secs` | Command started; `job_id` is used in the hold list and for reattachment after reconnecting |
| `exec_exit` | `session_id`<br>`code: i32 \| null`<br>`timed_out: bool` | Command finished. `code` is `null` when a signal terminates the process; `timed_out` indicates forced termination due to timeout |
| `exec_list_resp` | `session_id`<br>`jobs: ExecHoldInfo[]` | List of hold jobs |
| `ubus_resp` | `session_id`<br>`code: i32`<br>`data: any` | ubus call result. `code` is the ubus return code; 0 means success |
| `android_resp` | `session_id`<br>`result: AndroidResult` | Android screen operation completed; this message ends the session |
| `pty_closed` | `session_id`<br>`code: i32 \| null` | PTY session ended; `code` is the shell exit code |
| `session_error` | `session_id`<br>`message: string` | Session-level error. The device sends this when it cannot complete a request, rather than silently dropping it |

A second `register` on an already registered connection is treated as protocol misuse and logged as a warning.

Example:

```json
{"type":"register","protocol_version":1,"device_id":"lab-router","token":"a1b2….c2Vj…","info":{"hostname":"OpenWrt","model":"ath79/generic","openwrt_version":"OpenWrt 23.05.5","kernel":"5.15.137","arch":"mips","agent_version":"0.1.0"}}
{"type":"exec_exit","session_id":31,"code":0,"timed_out":false}
```

### DeviceDown (exchange → agent)

| `type` | Fields | Purpose |
|---|---|---|
| `register_ack` | `ok: bool`<br>`error: string \| null`<br>`heartbeat_interval_secs: u64` | Registration result. With `ok: false`, `error` provides the reason, then the server closes the connection. The server specifies the heartbeat interval so it can reduce frequency centrally as the fleet grows without reconfiguring each device. agent clamps 0 to at least 1 second to prevent a busy timer from saturating the CPU |
| `device_access` | `managers: DeviceManager[]`<br>`updated_at: u64` (milliseconds) | Who has permission on this device. `DeviceManager` = `operator_id`, `username`, `email?`, `role`, `level` (`manage`/`view`), `source` (`role_default`/`grant`). Sent after registration and permission changes only to agents advertising the `device_access` capability; see repository `docs/permissions.md` |
| `exec_req` | `session_id`<br>`command: string`<br>`timeout_secs: u64`<br>`hold: bool` (default `true`)<br>`hold_timeout_secs?: u64` | Execute a command. With `hold=true` (default), the process can continue after the operator disconnects. Timeout uses `hold_timeout_secs` (absent/0 → 3600, hard maximum 86400). `hold=false` cancels the command when the operator disconnects |
| `exec_cancel` | `session_id` | Terminate (kill) command execution early |
| `exec_detach` | `session_id` | Stop streaming output to the current operator while retaining the hold process |
| `exec_list_req` | `session_id` | Query current hold jobs |
| `exec_attach_req` | `session_id`<br>`job_id` | Reattach a hold job to this session and replay buffered output |
| `exec_hold_kill` | `session_id`<br>`job_id` | Terminate a hold job by job_id |
| `exec_hold_tail` | `session_id`<br>`job_id` | Follow new output only, without replaying the buffer |
| `exec_hold_cat` | `session_id`<br>`job_id` | Dump the current buffer and end with `exec_hold_cat_done` |
| `ubus_req` | `session_id`<br>`object: string`<br>`method: string`<br>`params: any` | Make a ubus call. When `params` is `null`, no third positional argument is passed to `ubus` |
| `pty_open` | `session_id`<br>`rows: u16`<br>`cols: u16` | Open a PTY session. Zero dimensions fall back to 24×80 |
| `pty_resize` | `session_id`<br>`rows: u16`<br>`cols: u16` | Terminal window dimensions changed |
| `pty_close` | `session_id` | Close a PTY session |
| `android_req` | `session_id`<br>`request: AndroidAction` | One Android screen operation |
| `android_cancel` | `session_id` | Cancel an unfinished Android request |

### OperatorUp (cli → exchange)

Authentication happens during the WebSocket handshake, so there are no login messages.

| `type` | Fields | Purpose |
|---|---|---|
| `open_exec` | `session_id`<br>`device_id`<br>`command`<br>`timeout_secs`<br>`hold` (default true)<br>`hold_timeout_secs?` | Execute a command on the target device. hold is enabled by default; `hold=true` with timeout 0 → 3600s. `hold=false` with timeout 0 → Exchange default of 60s |
| `list_exec_holds` | `session_id`<br>`device_id` | Query exec jobs and active downloads, uploads, and directory queries |
| `list_device_sessions` | `session_id`<br>`device_id` | Query the Exchange routing table directly without consuming a device session slot |
| `kill_device_session` | `session_id`<br>`device_id`<br>`target_session_id` | Cancel by device-side `target_session_id` and release the route locally; includes hold exec and consumes no extra slot |
| `attach_exec` | `session_id`<br>`device_id`<br>`job_id` | Attach to a hold job and resume receiving output |
| `exec_hold_kill` / `exec_hold_tail` / `exec_hold_cat` | `session_id`<br>`device_id`<br>`job_id` | kill / tail / cat for hold jobs |
| `detach_session` | `session_id` | Stop waiting in the foreground and detach without killing the hold process |
| `open_ubus` | `session_id`<br>`device_id: string`<br>`object: string`<br>`method: string`<br>`params: any` | Make a ubus call |
| `open_pty` | `session_id`<br>`device_id: string`<br>`rows: u16`<br>`cols: u16` | Open an interactive PTY session |
| `open_android` | `session_id`<br>`device_id: string`<br>`request: AndroidAction` | Initiate an Android screen operation; read-only accounts are prohibited |
| `pty_resize` | `session_id`<br>`rows: u16`<br>`cols: u16` | Terminal window dimensions changed. No `device_id`: the session already exists, so the server can look up the target device |
| `close_session` | `session_id` | End a session proactively: normal PTY exit or user interruption of command execution |

The `session_id` in all four `open_*` messages is **allocated by the CLI itself** (see [Session ID Translation Between Two Namespaces](#session-id-translation-between-two-namespaces)), allowing subsequent frames immediately after the request without waiting for an ID acknowledgment.

### OperatorDown (exchange → cli)

| `type` | Fields | Purpose |
|---|---|---|
| `welcome` | `protocol_version: u16`<br>`server_version: string` | Sent proactively by the server after connection establishment to verify matching protocol versions |
| `device_sessions` | `session_id`<br>`sessions: DeviceSessionInfo[]`<br>`limit`<br>`count` | Current device-side sessions in Exchange |
| `device_session_killed` | `session_id`<br>`target_session_id`<br>`released: bool` | Released locally; false if it did not exist |
| `exec_exit` | `session_id`<br>`code: i32 \| null`<br>`timed_out: bool` | Command ended, translated from the device-side message of the same name |
| `ubus_resp` | `session_id`<br>`code: i32`<br>`data: any` | ubus call result |
| `android_resp` | `session_id`<br>`result: AndroidResult` | Android result with the session ID translated to the operator namespace |
| `pty_closed` | `session_id`<br>`code: i32 \| null` | PTY session ended |
| `session_failed` | `session_id`<br>`reason: SessionFailure`<br>`message: string` | Session failure. `reason` distinguishes causes so the CLI can decide whether retrying is worthwhile |

### Shared Data Structures
`ExecHoldInfo`: `job_id`, `kind` (`exec` / `download` / `upload` / `file`), `age_secs`, `session_id?`, `command_preview`, `status`, `timeout_secs`, `started_at_unix_ms`, `exit_code?`, `timed_out`. For compatibility, missing `kind` defaults to `exec`, and missing `age_secs` defaults to 0. List responses calculate age from the current clock. Command/file path summaries are limited to approximately 120 characters. Non-exec jobs support kill only, not tail/cat/attach.

`DeviceSessionInfo`: `session_id: u32` (device-side ID), `operator_session_id: u32`, `kind: string`, `hold: bool`, `age_secs: u64`, `deadline_unix_ms: u64`. Exchange answers session-list and forced-cancellation requests directly without opening another device session; read-only accounts receive `forbidden`.

#### `DeviceInfo`

Static device information, reported once in `register` and stored in `devices.info`. Everything except `agent_version` is optional: most sources are unavailable when agent runs outside OpenWrt, such as on a development machine or in a container. Registration should still succeed rather than startup failing. Missing fields are omitted during serialization.

| Field | Type | Source |
|---|---|---|
| `hostname` | `string?` | `/proc/sys/kernel/hostname` |
| `model` | `string?` | `DISTRIB_TARGET` in `/etc/openwrt_release`, falling back to `model` from `ubus call system board` if unavailable |
| `openwrt_version` | `string?` | `DISTRIB_DESCRIPTION`, or ubus `release.description` |
| `kernel` | `string?` | `/proc/sys/kernel/osrelease` |
| `arch` | `string?` | Compile-time constant, more reliable than runtime detection |
| `agent_version` | `string` | The agent version, used to diagnose version mismatches |

#### `Metrics`

Collected again for each heartbeat, using only values available directly from `/proc` without forking subprocesses. Failure to read one item does not fail the whole collection: losing one metric is preferable to failing the whole heartbeat and incorrectly reporting the device offline.

| Field | Type | Description |
|---|---|---|
| `uptime_secs` | `u64` | `/proc/uptime` |
| `load` | `[f32; 3]` | 1/5/15-minute load averages from `/proc/loadavg` |
| `mem_total_kb` | `u64` | `/proc/meminfo` |
| `mem_available_kb` | `u64` | Uses `MemAvailable` rather than `MemFree`: the latter excludes reclaimable caches and significantly underestimates available memory on routers, producing false alerts |
| `cpu_usage_pct` | `f32?` | Calculated from the difference between two `/proc/stat` samples; omitted on the first heartbeat because there is no reference sample |
| `interfaces` | `InterfaceStats[]` | `/proc/net/dev`. Reports **cumulative** receive/transmit bytes rather than rates, leaving the server free to calculate rates over any time window. Omitted when empty |

`InterfaceStats`: `{ name: string, rx_bytes: u64, tx_bytes: u64 }`

---

## Binary Data Frames

```text
 0        1                    5                    9
 +--------+--------------------+--------------------+
 |  kind  |     session_id     |    payload_len     |  payload (payload_len 字节)
 |   u8   |    u32 (大端)      |    u32 (大端)      |
 +--------+--------------------+--------------------+
 └──────────── 9 字节定长头 ────────────┘
```

| Offset | Length | Field | Description |
|---|---|---|---|
| 0 | 1 | `kind` | Stream type; see below |
| 1 | 4 | `session_id` | Big-endian `u32`. 0 is reserved as a sentinel meaning no session |
| 5 | 4 | `payload_len` | Big-endian `u32`, payload byte count, maximum 1 MiB |
| 9 | `payload_len` | `payload` | Raw bytes without encoding or escaping |

### Stream Types

| Value | Name | Direction | Description |
|---|---|---|---|
| 1 | `ExecStdout` | Device → operator | Remote command standard output |
| 2 | `ExecStderr` | Device → operator | Standard error, transmitted separately from stdout so the CLI can render it differently, such as in red |
| 3 | `PtyData` | **Bidirectional** | PTY byte stream. Upstream carries keyboard input, downstream carries screen output; the same type serves both directions |

If agent receives `ExecStdout` / `ExecStderr` downstream, it logs a warning and discards the frame: this indicates an implementation error at the peer.

### Design Notes

**`payload_len` is indeed redundant with the WebSocket length field.** It remains for two reasons: decoding can detect truncation (a declaration of 11 bytes with only 5 supplied must be rejected rather than silently accepted as a shorter packet), and the frame format can remain unchanged if raw TCP is needed later.

The **1 MiB payload limit** is not intended for normal traffic: PTY and command output are incremental streams far smaller than that. It prevents a peer from exhausting server memory with a header claiming a 2 GiB payload.

**Empty payloads are valid**: zero-byte reads occasionally occur in PTY sessions. Such a frame consists of the 9-byte header alone.

**Payloads contain arbitrary bytes**, including non-UTF-8 sequences, and must be preserved exactly.

### Decoding Errors

| Error | Trigger |
|---|---|
| `FrameTooShort { len }` | Total length below 9 bytes |
| `UnknownStreamKind(v)` | First byte is not 1/2/3 |
| `PayloadTooLarge { len }` | Declared `payload_len` exceeds 1 MiB |
| `LengthMismatch { declared, actual }` | Declared length differs from the actual remaining byte count |

A data-frame decoding failure affects only that frame: exchange logs a warning and discards it without disconnecting.

---

## Session ID Translation Between Two Namespaces

**This is the part of the protocol most easily misunderstood.**

### In One Sentence

The **initiator** allocates `session_id`, which is **unique only within its own connection**. Operator and device connections each maintain independent ID namespaces, and exchange translates between them. One logical session has two different IDs.

```text
  remote-cli                    exchange                      remote-agent
                                                        （设备 lab-router）

  ┌────────────┐        ┌────────────────────┐            ┌────────────┐
  │ 运维编号   │        │      路由表        │            │ 设备编号   │
  │  空间      │        │                    │            │  空间      │
  │            │        │  lab-router:       │            │            │
  │  session 7 │───────▶│    31 → (op-A, 7)  │───────────▶│ session 31 │
  │            │◀───────│                    │◀───────────│            │
  └────────────┘        │  op-A:             │            └────────────┘
                        │    7 → (lab-router,│
   由 CLI 自己的        │         31)        │             由 exchange
   SessionIdGenerator   └────────────────────┘             的 DeviceHandle
   分配（1,2,3…）                                          分配（1,2,3…）
```

### Why This Design

**Avoid a round trip.** If the server allocated all IDs, the CLI would have to wait for an acknowledgment saying its session ID is 31 before sending keyboard input after opening a PTY. CLI allocation allows data frames immediately after `open_pty`.

**Avoid ID collisions between operators.** Two operators may simultaneously use local ID 1 on the same device. The device must distinguish them to avoid sending one person's output to another. exchange allocates device-side IDs separately for each device connection, mapping the two `session 1` instances to device-side `31` and `32`.

**Keep operator behavior from influencing device-side IDs.** exchange controls device-side IDs entirely; no operator can specify or guess them.

### Two Tables

exchange maintains one table per connection, with both tables always updated together:

| Location | Table | Key → value |
|---|---|---|
| `DeviceHandle.sessions` | Device-side forward table | Device session ID → (operator send handle, operator session ID, operator account ID, session type, audit row ID) |
| `OperatorHandle.sessions` | Operator-side reverse index | Operator session ID → (device ID, device session ID) |

The reverse index enables cleanup: when an operator disconnects, the server must find and terminate its sessions on **every device**, otherwise processes and PTYs with no output recipient would remain.

### Where Translation Happens

| Direction | Message | Translation |
|---|---|---|
| Operator → device (`open_*`) | Control message | **Allocate** a new ID in the device namespace, register it in both tables, and use the new ID in the message sent to the device |
| Operator → device (`pty_resize` / `close_session`) | Control message | Look up the device session ID through the reverse index, replace it, and forward |
| Operator → device | Data frame | Look up the device session ID through the reverse index and overwrite the 4 header bytes in place, leaving the payload unchanged |
| Device → operator | Control message | Look up the operator session ID and target connection in the forward table, replace the ID, and deliver |
| Device → operator | Data frame | Look up the operator session ID in the forward table and overwrite the header in place, leaving the payload unchanged |

Data-frame translation is the **only** rewrite: exchange does not parse, buffer, or change any payload byte. `DataFrame::with_session_id` replaces the ID in place without reallocating the payload.

### Allocation Rules

Both allocators increment from 1 and **skip 0**, the no-session sentinel used, for example, in connection-level error messages.

- CLI (`SessionIdGenerator`): wraps on overflow and skips 0. Using four billion sessions in one connection is unrealistic, but wrapping is better suited to a long-running process than a panic.
- Device (`DeviceHandle::open_session`): after wrapping, if an ID belongs to a live session, continues to the next free slot.

exchange rejects operator ID 0 and reuse of an active session ID. Device connection validity, concurrency limits, and registration in both indexes are checked within the same critical section, preventing simultaneous session opens from multiple operators from exceeding the limit.

### When No Route Exists

**Silently discard without an error.** A peer still sending trailing data just after session closure is a normal race. This applies in both directions:

- Device replies for a session that no longer exists → debug log; the operator may already have disconnected.
- Operator sends data for an unowned session → trace log and discard.

### Session Cleanup Paths

| Trigger | Cleanup |
|---|---|
| Device ends a session normally (`exec_exit` / `ubus_resp` / `android_resp` / `pty_closed` / `session_error`) | Remove the forward route and owning connection's reverse index together, update the audit record, and deliver the translated message to the operator |
| Operator `close_session` | Remove both indexes, tell the original device connection to terminate, immediately acknowledge closure to the operator, and update the cancellation audit record. Repeated closes do nothing |
| Device disconnects | Take all sessions for that connection, send `session_failed` / `device_disconnected` to each operator, and update the audit record with `{"outcome":"device_disconnected"}` |
| Operator disconnects | Traverse its reverse index to find its sessions on each device. Send termination instructions for non-hold sessions; for hold exec send only `exec_detach`, leaving the process running. **Clean up only its own sessions**; other operators on the same device are unaffected |
| Device reconnects and replaces the old connection | Invalidate all sessions on the old connection and notify each operator with `device_disconnected` |

The reverse index binds to a specific device connection; it does not redirect by device ID to a new connection after reconnection. Multiple operator connections for one account each own an index, so disconnecting one does not cancel another connection's sessions.

Acknowledgments for explicit cancellation: exec returns `exec_exit` (`code: null`, `timed_out: false`), PTY returns `pty_closed` (`code: null`), and ubus returns `session_failed` (`reason: device_error`). This confirms exchange has closed the session, not that the remote process has exited; subsequent trailing data and exit messages from the device are discarded.

At the concurrency limit, Exchange first reclaims routes whose operators are gone or whose deadlines have passed under the same session lock, then checks the limit again. The exec deadline is the actual dispatched time limit plus a 15-second grace period; files use 3600 seconds plus 15 seconds; other types use 24 hours. Hold exec within its deadline is not killed when the operator leaves. Reclamation/explicit `kill_device_session` sends the corresponding cancellation to the device and immediately removes both indexes without waiting for a device response. Available operators receive `exec_exit` (`timed_out:true`) for exec or `session_failed` (`device_error`, `session reclaimed`) for other types. Audit updates are asynchronous and best effort.

### Device Reconnection and epoch

Each device connection receives a random `epoch` (UUID) during registration. When the same device reconnects, **the new connection replaces the old one**, rather than being rejected. It is impossible to reliably distinguish a duplicate connection from a dead connection the server has not detected yet (a half-open connection); rejecting the new one would prevent reconnection until the old one timed out.

Cleanup for the old connection may execute **after** registration of the new one. Deregistration must therefore check epoch: remove the connection table entry only if its current epoch matches the connection being cleaned up. Without this check, old-connection cleanup would immediately mark the newly connected device offline.

---

## Handshake Sequence

### Device Side

```text
  remote-agent                                        exchange
      │                                                 │
      │  ①  WebSocket 握手（wss://host/ws/device）      │
      │────────────────────────────────────────────────▶│
      │◀────────────────────────────────────────────────│
      │            101 Switching Protocols              │
      │                                                 │
      │                                          ┌──────┴──────┐
      │                                          │ 启动 15 秒  │
      │                                          │ 注册计时器  │
      │                                          └──────┬──────┘
      │  ②  {"type":"register","protocol_version":1,    │
      │      "device_id":"…","token":"…","info":{…}}    │
      │────────────────────────────────────────────────▶│
      │                                                 │
      │                              ┌──────────────────┴──────────────────┐
      │                              │ a. 比对 protocol_version            │
      │                              │ b. split_token → 前缀查 devices 表  │
      │                              │ c. 校验 enabled                     │
      │                              │ d. argon2 verify（前缀不符也照做，  │
      │                              │    避免用耗时区分前缀是否存在）     │
      │                              └──────────────────┬──────────────────┘
      │                                                 │
      │  ③  {"type":"register_ack","ok":true,           │
      │      "error":null,"heartbeat_interval_secs":30} │
      │◀────────────────────────────────────────────────│
      │                                                 │
      │                                          ┌──────┴──────┐
      │                                          │ 登记连接表  │
      │                                          │ 顶替旧连接  │
      │                                          │ 回写 info 与│
      │                                          │ last_seen_at│
      │                                          └──────┬──────┘
      │                                                 │
      │  ④  退避曲线复位，写状态文件 state=connected     │
      │                                                 │
      │══════════════ 转发循环 ═══════════════════════════│
      │                                                 │
      │  heartbeat（每 30 秒，携带 Metrics）             │
      │────────────────────────────────────────────────▶│
      │  WebSocket Ping（随每次 heartbeat 发出）         │
      │────────────────────────────────────────────────▶│
      │                                     WebSocket Pong
      │◀────────────────────────────────────────────────│
      │                                     WebSocket Ping（每 30 秒）
      │◀────────────────────────────────────────────────│
      │  Pong（tungstenite 排队，agent 显式 flush）       │
      │────────────────────────────────────────────────▶│
```

**Failure paths:**

- First frame is not text, is invalid JSON, or is not `register` → log a warning and close immediately without an ack.
- No `register` within 15 seconds → close.
- Version mismatch → return `register_ack{ok:false, error:"协议版本不匹配：设备为 X，服务端为 Y"}` and close.
- Invalid token / nonexistent device / disabled device → always return the same `"设备 ID 或令牌无效"` and close. **Do not distinguish causes**: doing so would make the device inventory enumerable.

agent handles these two failure categories separately:

| agent category | Trigger | Reconnection strategy |
|---|---|---|
| `Rejected` | Receives an ack with `ok: false` | Fixed **300-second** wait. Retrying cannot fix an invalid token; frequent retries waste resources on both sides and flood server logs. A longer wait gives someone time to update configuration |
| `Transport` | Connection failure, handshake timeout (45 seconds), malformed response, interrupted connection | Exponential backoff: starts at 1 second, doubles each time up to 300 seconds, multiplied by random jitter in [0.5, 1.5) |

**Backoff resets only after successful registration**, not merely TCP establishment. Otherwise, immediate rejection after connection would degrade into dense retries at millisecond intervals.

**Jitter is essential.** When a server restarts, hundreds or thousands of devices detect the disconnect at once. With an identical deterministic retry curve, they remain synchronized and hit the newly started server in waves, turning a restart into a sustained cascading failure.

### Operator Side

```text
  remote-cli                                          exchange
      │                                                 │
      │  ①  WebSocket 握手（wss://host/ws/operator）    │
      │      Authorization: Bearer <prefix>.<secret>    │
      │────────────────────────────────────────────────▶│
      │                                                 │
      │                              ┌──────────────────┴──────────────────┐
      │                              │ 在 upgrade 之前完成鉴权：           │
      │                              │ split_token → 前缀查 operators 表   │
      │                              │ → argon2 verify → 校验 enabled      │
      │                              └──────────────────┬──────────────────┘
      │                                                 │
      │      鉴权失败 → HTTP 401「凭证无效」（不升级）   │
      │◀────────────────────────────────────────────────│
      │                                                 │
      │◀────────────────────────────────────────────────│
      │            101 Switching Protocols              │
      │                                                 │
      │  ②  {"type":"welcome","protocol_version":1,     │
      │      "server_version":"0.1.0"}                  │
      │◀────────────────────────────────────────────────│
      │                                                 │
      │  ③  比对 protocol_version（15 秒超时）          │
      │      不一致 → 报错退出，提示升级 CLI            │
      │                                                 │
      │══════════════ 可以发起会话 ═══════════════════════│
      │                                                 │
      │  {"type":"open_pty","session_id":1,             │
      │   "device_id":"lab-router","rows":40,"cols":120}│
      │────────────────────────────────────────────────▶│
      │                                                 │──▶ 查设备在线
      │                                                 │──▶ 查并发上限
      │                                                 │──▶ 查 readonly 角色
      │                                                 │──▶ 写审计（先于下发）
      │                                                 │──▶ 分配设备侧编号
      │  二进制帧（PtyData, session 1, 键盘输入）        │
      │────────────────────────────────────────────────▶│──▶ 改写为设备侧编号
      │                                                 │
      │  二进制帧（PtyData, session 1, 屏幕输出）        │
      │◀────────────────────────────────────────────────│◀── 改写为运维侧编号
```

The CLI can send the next data frame **without waiting for any acknowledgment**. This is precisely why the initiator allocates session IDs.

---

## Error Handling and Failure Categories

### `SessionFailure`

Used in the `reason` field of `session_failed`; JSON values use snake_case.

| Value | Meaning | Producer | Retryable |
|---|---|---|---|
| `device_offline` | Device does not exist or is currently offline | exchange finds no device in the connection table during `open_*` | ✅ |
| `device_disconnected` | Device disconnected during the session | Device disconnected / replaced by a new connection / send queue already closed during dispatch | ✅ |
| `device_error` | Device rejected the request or execution failed | agent returns `session_error` (PTY creation, command startup, or ubus call failure), or the peer consumes too slowly during forwarding | ❌ |
| `rate_limited` | Concurrent session limit exceeded | exchange finds the device's active sessions have reached `--max-sessions-per-device` (default 32) during `open_*` | ✅ |
| `forbidden` | Operator credentials cannot access the device | A read-only account (`role = readonly`) attempts to initiate any session | ❌ |

`is_retryable()` deliberately excludes `forbidden` and `device_error`: retrying a permission issue or internal device error is pointless and only causes unnecessary CLI activity.

The CLI appends an actionable hint after the original `message`. For example, `device_offline` suggests checking device status with `remote-cli device list`. `device_error` is the exception: the device's own error is the most accurate explanation and should not be diluted by boilerplate.

### Failure Semantics by Layer

| Layer | Behavior | Scope |
|---|---|---|
| Frame decoding failure | Log and discard the frame | One frame |
| Control message parsing failure | Warn and ignore the message | One message; connection continues |
| Session-level failure | `session_failed` / `session_error` | One session; connection continues |
| Registration failure | Close after `register_ack{ok:false}` | Entire connection |
| Version mismatch | Close after an explicit error message | Entire connection |
| Queue lag | Proactively close connection | Entire connection |

The principle: **keep failures local whenever possible**, since a device connection may carry simultaneous sessions for multiple operators.

---

## Keepalive and Timeouts

| Mechanism | Value | Description |
|---|---|---|
| exchange WebSocket Ping | Every `--heartbeat-secs` (default 30 seconds) | exchange sends to both devices and operators. Client libraries automatically queue Pong; agent must explicitly flush the sink after receiving Ping to send Pong promptly |
| agent WebSocket Ping | Every `heartbeat_interval_secs` (provided in the server ack) | agent sends native Ping in addition to each JSON `heartbeat`, avoiding Cloudflare's approximately 100-second idle-disconnect boundary if exchange Ping is delayed |
| Connection silence timeout | `heartbeat × --missed-pings` (default 30 × 3 = 90 seconds) | A device or operator connection with no inbound messages during this interval is considered lost and disconnected. Pong counts as activity; `--missed-pings` set to 0 falls back to at least one heartbeat interval |
| exchange write timeout | 10 seconds | Any single WebSocket data or Ping write timeout ends the connection, preventing slow connections from blocking cleanup |
| Business heartbeat | Every `heartbeat_interval_secs` (provided in the server ack) | agent sends JSON `heartbeat` with `Metrics` and a separate WebSocket Ping. Metrics reporting and native ping/pong serve separate purposes; business heartbeats alone cannot determine connection status |
| Online/offline events | `tick_count` for notification triggers `online` / `offline` (default 3, range 1–100); broadcast/audit fallback `REMOTE_MISSED_THRESHOLD` / `REMOTE_ONLINE_THRESHOLD` | Operators configure two triggers, rather than two counts on one trigger. `offline` notifies after `tick_count` consecutive misses; `online` notifies after `tick_count` consecutive successes. Each trigger's `device_ids` can be one device ID (string), a set of IDs (array), or omitted / `null` for all workspace devices. A trigger does not repeat notifications while already in its reported state; it must first rearm through `tick_count` consecutive opposite results. In-process presence broadcasts and audits do not read these triggers and fall back to `REMOTE_MISSED_THRESHOLD` / `REMOTE_ONLINE_THRESHOLD` (also defaulting to 3). Whether the socket remains in the connection table is determined separately by `--missed-pings` |
| Device registration timeout | 15 seconds | `register` must be sent within this interval after connection establishment |
| agent handshake timeout | 45 seconds per phase (`HANDSHAKE_TIMEOUT`) | `connect_device_ws` (TCP, optional HTTP CONNECT, TLS, WebSocket handshake) and waiting for `register_ack` after sending `register` each get this limit; 45 seconds is not their combined limit |
| agent write / Pong flush timeout | 30 seconds (`WRITE_TIMEOUT`) | Each send (registration, JSON heartbeat, Ping, session output) and each post-registration Pong flush has its own deadline. Write failure or timeout ends the connection and triggers reconnection. Pong flush while waiting for registration is governed by the 45-second ack wait above |
| agent TCP keepalive | Probe after 30 seconds idle | Enables `TCP_NODELAY` and TCP keepalive after direct connection or HTTP CONNECT; TCP settings do not replace WebSocket Ping/Pong |
| CLI welcome timeout | 15 seconds | Time the CLI waits for `welcome` after connection |
| Command execution timeout | Specified by `timeout_secs` in `open_exec` | agent clamps to at least 1 second. On timeout, SIGTERM the entire process group, then SIGKILL after a 3-second grace period |
| ubus call timeout | 15 seconds (fixed within agent) | ubus is local RPC; exceeding this interval generally indicates the corresponding service is stuck |

An idle shell client still replies with Pong. agent explicitly flushes the response automatically queued by the library without waiting for the next business heartbeat, so lack of keyboard input does not cause a false disconnect (#42). `--heartbeat-secs` must be greater than 0; invalid values are rejected during startup argument parsing.

---

## Backpressure

exchange maintains a bounded send queue of **512** messages for each connection.

A full queue means the peer cannot keep up, usually because a PTY connection has stalled or the client is suspended. The response is to **disconnect that connection proactively** rather than keep buffering: unbounded buffering would turn one stalled client into a memory leak affecting the whole service. 512 messages provide ample headroom for interactive PTY traffic (normally queue length is in single digits); hitting the limit generally means the peer has stopped consuming.

Delivery always uses nonblocking `try_send`. The routing path holds a connection-table lock; any `await` would tie lock duration to peer consumption speed, allowing a slow client to stall the entire table.

The two delivery failures are handled differently:

| Result | Meaning | Handling |
|---|---|---|
| `Lagging` | Queue full | Interrupt the connection using a separate close signal, clean up its sessions, and make a best effort to notify the other endpoint of failure |
| `Closed` | Receiver released | Normal connection termination; debug log only |

Connection close signals bypass the send queue, allowing cleanup even when the queue is full or writes are blocked. Device replacement after reconnect and token rotation use the same mechanism.

agent also has an outbound queue of capacity 256. A full queue means the network cannot send data at all, so further accumulation is pointless; disconnect and reconnect. When the PTY input queue (64) is full, keystrokes are simply discarded: accumulating small interactive inputs serves no purpose.
## Android Screen Control

Defined in `crates/proto/src/android.rs`, using the existing two-namespace session routing, authentication, concurrency limits, and auditing. The `request` in `open_android` / `android_req` is a JSON object discriminated by `action`:

| `action` | Other fields | Behavior |
|---|---|---|
| `status` | None | Query local authorization and SDK version; the screen control switch need not be enabled, but the remote service must be running |
| `tree` | None | Accessibility node tree of the currently active app |
| `screenshot` | None | One JPEG screenshot |
| `tap` | `x, y: u32` | Tap original screen pixel coordinates |
| `swipe` | `x1, y1, x2, y2, duration_ms: u32` | Swipe, duration 50–5000 ms |
| `swipe_path` | `points: [{x, y}]` (2–64 points), `duration_ms: u32` | Multi-segment swipe: one press continuously traverses all points, interpolated uniformly by length, duration 50–5000 ms |
| `global` | `key: string` | `back` / `home` / `recents` / `notifications` / `quick_settings` |
| `input_text` | `text: string` | Replace the focused editable field, maximum 8192 UTF-8 bytes, no NUL |

`AndroidResult` is discriminated by `kind`; all fields are required:

| `kind` | Fields |
|---|---|
| `status` | `controls_enabled, accessibility_enabled, screen_capture_enabled: bool`; `sdk: u32` |
| `tree` | `nodes: AndroidNode[]`; `screen_width, screen_height: u32`; `truncated: bool` |
| `screenshot` | `mime_type: "image/jpeg"`; `data_base64: string`; `width, height, screen_width, screen_height: u32` |
| `ok` | None; indicates completion of input, gesture, or system action |

Nodes contain strings `class_name, package, view_id, text, description`, `bounds: [left, top, right, bottom]` (signed screen pixel coordinates), and booleans `clickable, editable, focused, password`. At most 200 nodes, with each string limited to 256 UTF-16 units. `truncated` indicates only the node limit. Password values and descriptions are empty, and the agent's own configuration subtree is omitted.

Screenshot longest edge is limited to 1600 pixels. `width/height` are image dimensions; `screen_width/screen_height` are dimensions before scaling. Gesture coordinates use the original dimensions. JSON returned by the Java bridge is limited to 2 MiB, with an additional 1024 bytes allowed for the WebSocket envelope. No new binary frame type is added.

```json
{"type":"open_android","session_id":1,"device_id":"phone","request":{"action":"input_text","text":"Hello\nWorld"}}
{"type":"android_req","session_id":31,"request":{"action":"input_text","text":"Hello\nWorld"}}
{"type":"android_resp","session_id":31,"result":{"kind":"ok"}}
{"type":"android_resp","session_id":1,"result":{"kind":"ok"}}
```

`screen_capture_enabled` indicates currently available screenshot authorization (MediaProjection or Android 11+ accessibility); all actual screen operations still require `controls_enabled`. Insufficient permission, inaccessible windows, and failed operations return agent `session_error`, translated by Exchange to `session_failed`. Non-Android agents return an explicit unsupported error.

Agent handles at most 4 Android requests concurrently and encodes only one screenshot at a time. Java operations wait at most 10 seconds (up to 5 seconds for a screen frame); agent request timeout is 15 seconds, CLI timeout 20 seconds. On operator cancellation or disconnection, Exchange sends `android_cancel`; agent closes the local socket and the APK cancels queued work that has not run yet. Completed gestures or text changes are not rolled back.

The audit action is `android`; detail stores the action and coordinates. `input_text` stores only `characters` (Unicode character count), and successful results record only `outcome: "completed"`. Text content, UI trees, and images are not written to audit records. `readonly` accounts cannot initiate even `status`, `tree`, or screenshots.

## File Session Extension

Agents supporting file operations advertise the `files` capability. Exchange translates `open_file {session_id, device_id, request}` to `file_req {session_id, request}`. `request.action` is `list`, `download`, or `upload`; field definitions are in `crates/proto/src/files.rs`.

Bidirectional control messages use JSON: the controller sends `file_input {session_id, input}`, and the device returns `file_event {session_id, event}`. `input.kind` is `chunk {offset, data}`, `ack {offset}`, or `finish`; `event.kind` is `ready {size}`, `chunk {offset, data}`, `progress {received}`, `complete {size}`, or `list {path, entries, truncated}`. Chunk contents use standard Base64, each chunk is at most 64 KiB, and the sender waits for acknowledgment before continuing.

A download's initial `ready` requires `ack {offset: 0}`; subsequent chunks acknowledge the cumulative byte count. Upload starts with `ready`, and each `progress` drives the next chunk; after sending everything, `finish` commits it. `complete` and `list` are terminal messages; errors use existing `session_error` / `session_failed`. `close_session` sends `file_cancel`; cancellation and disconnection both remove session routes. File messages are forwarded only on file sessions, preserving isolation between operator and device session IDs.

Protocol version remains 1; the new messages require matching Exchange and Agent upgrades. For operation limits and LAN interfaces, see [File Transfer and Direct LAN Connections](/docs/en/file-transfer-and-lan.md).

## Multi-Platform Extensions

Protocol version remains 1. Registration `info` adds optional `platform` (`openwrt` / `linux` / `freebsd` / `netbsd` / `windows` / `android` / `macos`), `os_version`, and `capabilities` (`exec` / `pty` / `ubus` / `android_control` / `files` / `sensors` / `net`). OpenWrt reports `openwrt_version`.

Heartbeat `metrics.unavailable` is an optional string array identifying currently unavailable `uptime`, `load`, `memory`, `cpu`, and `interfaces` metrics. If absent, parse using the old format. Values corresponding to listed metrics are only compatibility placeholders; clients should show them as unavailable. Windows does not provide Unix load averages, and Android may restrict some `/proc` sources through system permissions.

Commands are interpreted by the target device's shell: `/bin/sh` on Linux/OpenWrt/macOS, PowerShell on Windows, and `/system/bin/sh` on Android. PTY frame format and control messages are identical across the three systems; Windows uses ConPTY. Non-OpenWrt devices return `session_error` for ubus requests.

## End-to-End Sealed Sessions

Protocol version remains 1. New messages: OperatorUp `open_sealed {session_id, device_id, hello}`, `sealed_input {session_id, packet}`; DeviceDown `sealed_open {session_id, hello}`, `sealed_input`, `sealed_close {session_id, detach}`; DeviceUp / OperatorDown `sealed_accept {session_id, accept}`, `sealed_event {session_id, packet, last}`. New binary frame type `4 = Sealed` (payload `counter ‖ AEAD(inner_kind ‖ bytes)`), `DeviceInfo.seal_key`, and capability `sealed`.

For sealed sessions, Exchange performs only ID translation, concurrency limiting, and routing. It neither parses nor audits payloads (audit records contain only public keys, authorization level and expiry, and final outcome), and does not decide permissions beyond checking whether the account can see the device. Routes are released on `last:true`. Sealed routes accept only `Sealed` frames; ordinary routes reject `Sealed` frames. agent discards any plaintext sent to sealed session IDs. For authorization, handshake, key derivation, threat model, see [End-to-End Sealed Sessions](/docs/en/sealed-sessions.md).