3 Commits
Author SHA1 Message Date
yuanyuanxianganddeepseek-flash 1922f87dd2 Feature: Multi-device screen wall on the Go server
Bring the Go web backend up to parity with the MFC server's screen wall.
A browser tab can now stream several devices over one WebSocket: the
single `watching` field becomes a per-connection watch set guarded by the
hub mutex, capped at 32 devices per socket. The challenge advertises
`caps.multiWatch` and `caps.keyframeRequest`, binary frames stay keyed by
the low 32 bits of the device ID, and connect/disconnect replies echo the
device id and request id so the wall can route concurrent responses.

Input commands (mouse, key, clipboard, rdp_reset) resolve their target
from the watch set: an explicit id must be a current subscription, and an
absent id is accepted only while exactly one device is watched. Legacy
frontends that do not send `multiWatch` keep replace-all semantics, so a
repeated device switch cannot grow the set without bound. Offline
devices are removed from every watch set, and teardown is computed from a
snapshot taken outside the hub lock to avoid re-entrant locking.

Add the `keyframe` command: it replays the cached IDR, then asks the
device to emit a fresh one, rate-limited to one request per device per
second. This exposed a Go-only corruption bug: the client answers the
request with TOKEN_KEYFRAME (134), which the dispatcher treated as a
no-op. That token only appears on forced IDRs because the client's GOP is
effectively infinite, so the dropped IDR left the browser decoding
P-frames against a reference chain it never received, producing
persistent garbling. Relay it as a keyframe instead. Also drop the
cached IDR when the resolution changes, mirroring the C++ server, so a
late joiner cannot receive an old-geometry keyframe.

Add tests for the watch-set transitions and for relaying TOKEN_KEYFRAME.

Co-Authored-By: deepseek-flash
2026-10-18 11:51:28 +02:00
yuanyuanxianganddeepseek-flash 8eab26c1f6 Feature: Web screen wall for multi-device live monitoring
Add a third view to the web UI, the screen wall, which shows several
devices at once in one tab with 1x1 / 2x2 / 3x3 / 4x4 / 5x5 layouts and a
device sidebar on the left. Tiles are monitoring-only; clicking one
enlarges that device into the existing single-screen page for control, and
Back returns to the wall. The same device can be added several times: all
tiles of one device share a single decoder and a single stream.

The wall decodes at most 9 streams on desktop and 4 on touch devices.
Tiles beyond the budget keep receiving frames but have their decoder
closed and show the last frame plus a Paused badge; they are promoted
automatically when a live tile goes away. Subscriptions are reference
counted on the client (streams map with owner sets), so a stream is only
connected on the 0 -> 1 transition and disconnected on 1 -> 0, and the
wall and the single-screen page can share one device without tearing the
stream down.

Server side, WebClient now holds a set of watched devices instead of a
single id. A new multiWatch command flag selects set semantics, while
clients without it keep the old replace-all behaviour, so an old frontend
against the new server is unchanged. connect and disconnect responses echo
id and req so the frontend can route them, input commands carry an id and
are resolved against the watch set (dropped when the set has several
devices and no id), and the per-client watch set is capped at 32.

For fast recovery after a pause or when joining late, the server caches
each device's raw IDR payload and replays it on connect and on a new
keyframe command, then asks the client to force a new IDR. The client gets
COMMAND_SCREEN_KEYFRAME_REQ = 169 and an atomic request flag that is only
cleared after an IDR actually came out of the encoder, so a failed capture
or EAGAIN does not lose the request. The frontend retries the keyframe
request up to three times; an old client that ignores the command simply
recovers on the next natural GOP.

Also fix the keyframe cache double wrapping: BroadcastH264Frame used to
cache the full packet header and HandleConnect wrapped it again. The cache
now stores data + 9, len - 9 and every sender wraps it through
BuildFramePacket, and the cache is cleared on cold start and on resolution
change. StopRemoteDesktop is always called outside m_ClientsMutex, with
the ids collected under the lock first.

Co-Authored-By: deepseek-flash
2026-10-18 11:03:17 +02:00
yuanyuanxianganddeepseek-v4-pro 04f4eb9947 Fix: share web login session across browser tabs
The login token and role were stored in sessionStorage, which is
isolated per tab, so opening a second tab forced a fresh password
login. Switch every token/role read and write to localStorage so all
tabs on the same origin share the session and a new tab auto-resumes
it on load. Expiry (device_list ok:false) and explicit logout still
clear it as before.

Co-Authored-By: deepseek-v4-pro
2026-10-17 22:44:50 +02:00
13 changed files with 2301 additions and 269 deletions

No files matched your search

+17 -1
View File
@@ -14,6 +14,7 @@
#include <functional>
#include <future>
#include <memory>
#include <atomic>
#include <emmintrin.h> // SSE2
#include "common/config.h"
#include "VideoEncoderBase.h"
@@ -149,6 +150,9 @@ public:
int m_FrameID; // 帧序号
int m_GOP; // 关键帧间隔
bool m_SendKeyFrame; // 发送关键帧
// Web 端请求下一帧强制 IDR(COMMAND_SCREEN_KEYFRAME_REQ)。只对 H.264 生效;
// 必须在成功编码出 IDR 后清除,否则 ScanNextScreen 失败/硬编 EAGAIN 时请求会丢。
std::atomic<bool> m_bForceKeyFrame{false};
std::unique_ptr<VideoEncoderBase> m_encoder; // 编码器,ensureEncoder() lazy 创建,走 EncoderFactory 探测
bool m_bEncoderPrimed = false; // encoder 是否已成功产出过一个包;
// false 时禁止 skip——避免单显示器路径
@@ -257,6 +261,11 @@ public:
m_BlockBuffers[blockY] = new BYTE[m_ulFullWidth * m_ulFullHeight * 4 * 2 / m_BlockNum + 12];
}
}
// 请求下一帧强制 IDR(可跨线程调用)。只影响 H.264 编码路径;
// 实际清除发生在成功编码出 IDR 之后。
void RequestKeyFrame() { m_bForceKeyFrame.store(true, std::memory_order_relaxed); }
virtual ~ScreenCapture()
{
if (m_BitmapInfor_Full != NULL) {
@@ -876,7 +885,12 @@ public:
{
BYTE algo = m_bAlgorithm;
int frameID = m_FrameID + 1;
bool keyFrame = (frameID % m_GOP == 0);
// Web 端可请求强制 IDR(COMMAND_SCREEN_KEYFRAME_REQ),只对 H.264 有意义。
// 标志在成功编码出 IDR 后才清除(见下方 H.264 keyframe 分支),
// 保证 ScanNextScreen 失败或硬编 EAGAIN 时请求不会丢失。
bool forceKeyFrame = (algo == ALGORITHM_H264) &&
m_bForceKeyFrame.load(std::memory_order_relaxed);
bool keyFrame = forceKeyFrame || (frameID % m_GOP == 0);
m_RectBuffer[0] = keyFrame ? TOKEN_KEYFRAME : TOKEN_NEXTSCREEN;
LPBYTE data = m_RectBuffer + 1;
// 写入使用了哪种算法
@@ -967,6 +981,8 @@ public:
*ulNextSendLength = 1 + offset + encoded_size;
memcpy(data + offset, encoded_data, encoded_size);
m_bEncoderPrimed = true; // 与下方 FirstBuffer 同步:自此 skip 安全
// IDR 已成功产出,满足(可能的)强制关键帧请求。
m_bForceKeyFrame.store(false, std::memory_order_relaxed);
break;
}
default:
+7
View File
@@ -930,6 +930,13 @@ VOID CScreenManager::OnReceive(PBYTE szBuffer, ULONG ulLength)
SwitchScreen();
break;
}
case COMMAND_SCREEN_KEYFRAME_REQ: {
// Web 屏幕墙恢复/新加入者请求:下一帧强制 IDR,立即重建参考链。
if (m_ScreenSpyObject) {
m_ScreenSpyObject->RequestKeyFrame();
}
break;
}
case CMD_FULL_SCREEN: {
int fullScreen = szBuffer[1];
iniFile cfg(CLIENT_PATH);
+1
View File
@@ -328,6 +328,7 @@ enum {
COMMAND_TLS_ENABLE = 166, // 服务端 → 客户端:启用 TLS 传输(写注册表,下次启动生效)
COMMAND_TLS_DISABLE = 167, // 服务端 → 客户端:停用 TLS 传输(写注册表,下次启动生效)
COMMAND_REGEN_CLIENT_ID = 168, // 服务端 → 客户端:撞 ID 的克隆机重生成 client_id(掺随机 UUID)后重连
COMMAND_SCREEN_KEYFRAME_REQ = 169, // 服务端 → 客户端:下一次屏幕编码强制输出 IDR(Web 屏幕墙恢复用)
TOKEN_DECRYPT = 199,
TOKEN_REGEDIT = 200, // 注册表
+310 -132
View File
@@ -45,6 +45,27 @@ static void ReplaceAll(std::string& s, const std::string& from, const std::strin
}
}
// 设备 ID 在 JSON 里以字符串传输(避免 64 位精度丢失),但兼容数字写法。
static uint64_t ParseDeviceIdValue(const Json::Value& v) {
if (v.isString()) return strtoull(v.asString().c_str(), nullptr, 10);
if (v.isUInt64()) return v.asUInt64();
if (v.isInt64()) return (uint64_t)v.asInt64();
return 0;
}
static uint64_t ParseDeviceIdField(const Json::Value& root, const char* key) {
return ParseDeviceIdValue(root[key]);
}
// req 是前端生成的关联号;字符串/数字都接受,原样回显。
static std::string ParseReqField(const Json::Value& root) {
const Json::Value& v = root["req"];
if (v.isString()) return v.asString();
if (v.isUInt64()) return std::to_string(v.asUInt64());
if (v.isInt64()) return std::to_string(v.asInt64());
return "";
}
// Challenge-response nonce storage (prevents replay attacks)
static std::map<void*, std::string> s_ClientNonces;
static std::mutex s_NonceMutex;
@@ -337,6 +358,10 @@ void CWebService::ServerThread(int port) {
Json::Value challenge;
challenge["cmd"] = "challenge";
challenge["nonce"] = nonce;
// 能力协商:新前端据此启用屏幕墙(多设备订阅)与 keyframe 请求。
// 缺少这些字段的服务端(Go / 旧 C++ 二进制)会让新前端自动降级。
challenge["caps"]["multiWatch"] = true;
challenge["caps"]["keyframeRequest"] = true;
Json::StreamWriterBuilder builder;
builder["indentation"] = "";
SendText(ws_ptr, Json::writeString(builder, challenge));
@@ -379,14 +404,13 @@ void CWebService::ServerThread(int port) {
} else if (cmd == "get_devices") {
HandleGetDevices(ws_ptr, token);
} else if (cmd == "connect") {
// Support both string and number ID formats
std::string id_str = root.get("id", "").asString();
uint64_t device_id = id_str.empty() ? 0 : strtoull(id_str.c_str(), nullptr, 10);
HandleConnect(ws_ptr, token, device_id);
HandleConnect(ws_ptr, token, ParseDeviceIdField(root, "id"),
ParseReqField(root), root.get("multiWatch", false).asBool());
} else if (cmd == "disconnect") {
std::string disc_id_str = root.get("id", "").asString();
uint64_t disc_device_id = disc_id_str.empty() ? 0 : strtoull(disc_id_str.c_str(), nullptr, 10);
HandleDisconnect(ws_ptr, token, disc_device_id);
HandleDisconnect(ws_ptr, token, ParseDeviceIdField(root, "id"),
ParseReqField(root), root.get("multiWatch", false).asBool());
} else if (cmd == "keyframe") {
HandleKeyframe(ws_ptr, token, ParseDeviceIdField(root, "id"), ParseReqField(root));
} else if (cmd == "ping") {
HandlePing(ws_ptr, token);
} else if (cmd == "mouse") {
@@ -396,11 +420,11 @@ void CWebService::ServerThread(int port) {
} else if (cmd == "clipboard_set") {
HandleClipboardSet(ws_ptr, msg);
} else if (cmd == "clipboard_get") {
HandleClipboardGet(ws_ptr, token);
HandleClipboardGet(ws_ptr, token, ParseDeviceIdField(root, "id"));
} else if (cmd == "rdp_reset") {
HandleRdpReset(ws_ptr, token);
HandleRdpReset(ws_ptr, token, ParseDeviceIdField(root, "id"));
} else if (cmd == "audio_toggle") {
HandleAudioToggle(ws_ptr, token);
HandleAudioToggle(ws_ptr, token, ParseDeviceIdField(root, "id"));
} else if (cmd == "get_salt") {
HandleGetSalt(ws_ptr, msg);
} else if (cmd == "create_user") {
@@ -464,7 +488,7 @@ void CWebService::HeartbeatThread() {
std::lock_guard<std::mutex> lock(m_ClientsMutex);
for (const auto& [ws_ptr, client] : m_Clients) {
// Only check clients that are watching a device (streaming video)
if (client.watch_device_id > 0 && client.last_activity > 0) {
if (!client.watch_device_ids.empty() && client.last_activity > 0) {
uint64_t elapsed = now - client.last_activity;
if (elapsed > HEARTBEAT_TIMEOUT_SEC) {
timedOutClients.push_back(ws_ptr);
@@ -625,22 +649,36 @@ void CWebService::HandleGetDevices(void* ws_ptr, const std::string& token) {
SendText(ws_ptr, BuildDeviceListJson(username));
}
void CWebService::HandleConnect(void* ws_ptr, const std::string& token, uint64_t device_id) {
void CWebService::HandleConnect(void* ws_ptr, const std::string& token, uint64_t device_id,
const std::string& req, bool multi_watch) {
// 所有响应都带 id/req:屏幕墙有多个并发 connect,前端必须能按设备/请求号路由。
auto send_error = [&](const std::string& err) {
Json::Value res;
res["cmd"] = "connect_result";
res["ok"] = false;
res["msg"] = err;
res["id"] = std::to_string(device_id);
if (!req.empty()) res["req"] = req;
Json::StreamWriterBuilder builder;
builder["indentation"] = "";
SendText(ws_ptr, Json::writeString(builder, res));
};
std::string username, role;
if (!ValidateToken(token, username, role)) {
SendText(ws_ptr, BuildJsonResponse("connect_result", false, "Invalid token"));
send_error("Invalid token");
return;
}
// Check device exists and is online
if (!m_pParentDlg) {
SendText(ws_ptr, BuildJsonResponse("connect_result", false, "Service not ready"));
send_error("Service not ready");
return;
}
context* ctx = m_pParentDlg->FindHost(device_id);
if (!ctx) {
SendText(ws_ptr, BuildJsonResponse("connect_result", false, "Device not online"));
send_error("Device not online");
return;
}
@@ -665,7 +703,7 @@ void CWebService::HandleConnect(void* ws_ptr, const std::string& token, uint64_t
}
}
if (!hasAccess) {
SendText(ws_ptr, BuildJsonResponse("connect_result", false, "Permission denied"));
send_error("Permission denied");
return;
}
}
@@ -675,30 +713,76 @@ void CWebService::HandleConnect(void* ws_ptr, const std::string& token, uint64_t
// remote_open, cleared on session teardown), so AI input cannot be injected into a
// picture a human is watching/controlling.
if (IsMcpTriggered(device_id)) {
SendText(ws_ptr, BuildJsonResponse("connect_result", false, "Device busy: remote control session active"));
send_error("Device busy: remote control session active");
return;
}
// Check max clients per device
int current_count = GetWebClientCount(device_id);
if (current_count >= m_nMaxClientsPerDevice) {
SendText(ws_ptr, BuildJsonResponse("connect_result", false, "Too many viewers"));
return;
}
// Update client state
// 该 ws 是否已经订阅这台设备(屏幕墙点击 tile 放大时会重复 connect 以拿
// 分辨率/关键帧)。重复 connect 必须跳过每设备 viewer 上限检查。
bool already = false;
{
std::lock_guard<std::mutex> lock(m_ClientsMutex);
auto it = m_Clients.find(ws_ptr);
if (it != m_Clients.end()) {
it->second.watch_device_id = device_id;
already = it->second.watch_device_ids.count(device_id) > 0;
}
}
// Start remote desktop session if this is the first web viewer
if (current_count == 0) {
// Check max clients per device
int current_count = GetWebClientCount(device_id);
if (!already && current_count >= m_nMaxClientsPerDevice) {
send_error("Too many viewers");
return;
}
// Update client state. 新前端(multiWatch=true)向集合里插入;旧前端保持
// 单设备语义:connect 替换整个集合并停掉其它设备,避免集合无限增长。
std::vector<uint64_t> to_stop;
bool too_many_watch = false;
{
std::lock_guard<std::mutex> lock(m_ClientsMutex);
auto it = m_Clients.find(ws_ptr);
if (it != m_Clients.end()) {
auto& watch = it->second.watch_device_ids;
if (!multi_watch) {
for (uint64_t id : watch) {
if (id != device_id) to_stop.push_back(id);
}
watch.clear();
}
if (already) {
watch.insert(device_id); // already true 时必然在集合内,这里只是兜底
} else if (watch.size() >= kMaxWatchPerClient) {
too_many_watch = true;
} else {
watch.insert(device_id);
}
}
}
if (too_many_watch) {
send_error("Too many devices on one connection");
return;
}
// StopRemoteDesktop 会再次锁 m_ClientsMutex,必须在锁外调用。
for (uint64_t id : to_stop) {
StopRemoteDesktop(id);
}
// Start remote desktop session if this is the first web viewer. 若之前已订阅
// 但捕获已意外结束(例如 MFC 对话框关闭),补一次冷启动。
bool need_start = (current_count == 0) ||
(already && GetScreenContext(device_id) == nullptr);
if (need_start) {
if (!StartRemoteDesktop(device_id)) {
SendText(ws_ptr, BuildJsonResponse("connect_result", false, "Failed to start remote desktop"));
// 回滚本次插入,避免一个没有屏幕会话的设备长期占着订阅/计数。
{
std::lock_guard<std::mutex> lock(m_ClientsMutex);
auto it = m_Clients.find(ws_ptr);
if (it != m_Clients.end()) {
it->second.watch_device_ids.erase(device_id);
}
}
send_error("Failed to start remote desktop");
return;
}
}
@@ -724,6 +808,8 @@ void CWebService::HandleConnect(void* ws_ptr, const std::string& token, uint64_t
Json::Value res;
res["cmd"] = "connect_result";
res["ok"] = true;
res["id"] = std::to_string(device_id);
if (!req.empty()) res["req"] = req;
// Only include dimensions if we have valid cached values
// Otherwise, client will wait for resolution_changed message
if (width > 0 && height > 0) {
@@ -741,48 +827,72 @@ void CWebService::HandleConnect(void* ws_ptr, const std::string& token, uint64_t
builder["indentation"] = "";
SendText(ws_ptr, Json::writeString(builder, res));
// Send cached keyframe if available
// Send cached keyframe if available. 缓存里是 raw H.264 payload(见
// BroadcastH264Frame),这里统一用 BuildFramePacket 封装。
std::vector<uint8_t> cached;
{
std::lock_guard<std::mutex> lock(m_DeviceCacheMutex);
auto it = m_DeviceCache.find(device_id);
if (it != m_DeviceCache.end() && !it->second->keyframe_cache.empty()) {
if (it != m_DeviceCache.end()) {
std::lock_guard<std::mutex> cache_lock(it->second->cache_mutex);
auto packet = BuildFramePacket(device_id, true,
it->second->keyframe_cache.data(),
it->second->keyframe_cache.size());
SendBinary(ws_ptr, packet.data(), packet.size());
cached = it->second->keyframe_cache;
}
}
if (!cached.empty()) {
auto packet = BuildFramePacket(device_id, true, cached.data(), cached.size());
SendBinary(ws_ptr, packet.data(), packet.size());
}
}
void CWebService::HandleDisconnect(void* ws_ptr, const std::string& token, uint64_t requested_device_id) {
void CWebService::HandleDisconnect(void* ws_ptr, const std::string& token, uint64_t requested_device_id,
const std::string& req, bool multi_watch) {
auto send_reply = [&](bool ok, const std::string& err) {
Json::Value res;
res["cmd"] = "disconnect_result";
res["ok"] = ok;
if (!err.empty()) res["msg"] = err;
if (requested_device_id > 0) res["id"] = std::to_string(requested_device_id);
if (!req.empty()) res["req"] = req;
Json::StreamWriterBuilder builder;
builder["indentation"] = "";
SendText(ws_ptr, Json::writeString(builder, res));
};
std::string username, role;
if (!ValidateToken(token, username, role)) {
SendText(ws_ptr, BuildJsonResponse("disconnect_result", false, "Invalid token"));
send_reply(false, "Invalid token");
return;
}
// Get the device_id before clearing watch state
uint64_t device_id = 0;
std::vector<uint64_t> to_stop;
{
std::lock_guard<std::mutex> lock(m_ClientsMutex);
auto it = m_Clients.find(ws_ptr);
if (it != m_Clients.end()) {
// Only disconnect if no specific device requested, or if it matches current device
// This prevents race condition when quickly switching devices
if (requested_device_id == 0 || it->second.watch_device_id == requested_device_id) {
device_id = it->second.watch_device_id;
it->second.watch_device_id = 0;
auto& watch = it->second.watch_device_ids;
if (multi_watch && requested_device_id != 0) {
// 屏幕墙:只取消这一路订阅,其它设备继续。
if (watch.erase(requested_device_id) > 0) {
it->second.keyframe_req_at.erase(requested_device_id);
to_stop.push_back(requested_device_id);
}
} else if (requested_device_id == 0 || watch.count(requested_device_id) > 0) {
// 旧前端 / 全量断开:保持原来的“请求 id 必须匹配”的竞态保护,
// 快速切换设备时不会误清刚建立的新订阅。
to_stop.assign(watch.begin(), watch.end());
watch.clear();
it->second.keyframe_req_at.clear();
}
}
}
// Close remote desktop if this was the last viewer
if (device_id > 0) {
StopRemoteDesktop(device_id);
// Close remote desktop if this was the last viewer. StopRemoteDesktop 会再次
// 锁 m_ClientsMutex,必须在锁外调用。
for (uint64_t id : to_stop) {
StopRemoteDesktop(id);
}
SendText(ws_ptr, BuildJsonResponse("disconnect_result", true));
send_reply(true, "");
}
void CWebService::HandlePing(void* ws_ptr, const std::string& token) {
@@ -795,6 +905,22 @@ void CWebService::HandlePing(void* ws_ptr, const std::string& token) {
SendText(ws_ptr, Json::writeString(builder, res));
}
uint64_t CWebService::ResolveWatchDevice(void* ws_ptr, uint64_t requested_device_id) {
std::lock_guard<std::mutex> lock(m_ClientsMutex);
auto it = m_Clients.find(ws_ptr);
if (it == m_Clients.end()) return 0;
const auto& watch = it->second.watch_device_ids;
if (requested_device_id != 0) {
// 显式 id 必须属于本连接的订阅集合,防止越权向别的设备注入输入。
return watch.count(requested_device_id) ? requested_device_id : 0;
}
// 旧前端不带 id:只有集合恰好一台设备时才回退。多设备时绝不猜,
// 否则会变成跨设备输入注入。
if (watch.size() == 1) return *watch.begin();
return 0;
}
void CWebService::HandleMouse(void* ws_ptr, const std::string& msg) {
// Parse JSON
Json::Value root;
@@ -817,16 +943,7 @@ void CWebService::HandleMouse(void* ws_ptr, const std::string& msg) {
}
// Get device being watched
uint64_t device_id = 0;
{
std::lock_guard<std::mutex> lock(m_ClientsMutex);
auto it = m_Clients.find(ws_ptr);
if (it != m_Clients.end()) {
device_id = it->second.watch_device_id;
it->second.last_activity = (uint64_t)time(nullptr);
}
}
uint64_t device_id = ResolveWatchDevice(ws_ptr, ParseDeviceIdField(root, "id"));
if (device_id == 0) return;
// Get screen context (not main context!)
@@ -904,16 +1021,7 @@ void CWebService::HandleKey(void* ws_ptr, const std::string& msg) {
}
// Get device being watched
uint64_t device_id = 0;
{
std::lock_guard<std::mutex> lock(m_ClientsMutex);
auto it = m_Clients.find(ws_ptr);
if (it != m_Clients.end()) {
device_id = it->second.watch_device_id;
it->second.last_activity = (uint64_t)time(nullptr);
}
}
uint64_t device_id = ResolveWatchDevice(ws_ptr, ParseDeviceIdField(root, "id"));
if (device_id == 0) return;
// Get screen context
@@ -953,16 +1061,7 @@ void CWebService::HandleClipboardSet(void* ws_ptr, const std::string& msg) {
}
// Get device being watched
uint64_t device_id = 0;
{
std::lock_guard<std::mutex> lock(m_ClientsMutex);
auto it = m_Clients.find(ws_ptr);
if (it != m_Clients.end()) {
device_id = it->second.watch_device_id;
it->second.last_activity = (uint64_t)time(nullptr);
}
}
uint64_t device_id = ResolveWatchDevice(ws_ptr, ParseDeviceIdField(root, "id"));
if (device_id == 0) return;
// Get screen context
@@ -989,24 +1088,15 @@ void CWebService::HandleClipboardSet(void* ws_ptr, const std::string& msg) {
ctx->Send2Client(szData.data(), (ULONG)szData.size());
}
void CWebService::HandleClipboardGet(void* ws_ptr, const std::string& token) {
void CWebService::HandleClipboardGet(void* ws_ptr, const std::string& token, uint64_t requested_device_id) {
// Validate token
std::string username, role;
if (!ValidateToken(token, username, role)) {
return;
}
// Get device being watched
uint64_t device_id = 0;
{
std::lock_guard<std::mutex> lock(m_ClientsMutex);
auto it = m_Clients.find(ws_ptr);
if (it != m_Clients.end()) {
device_id = it->second.watch_device_id;
it->second.last_activity = (uint64_t)time(nullptr);
}
}
// Get the device being watched
uint64_t device_id = ResolveWatchDevice(ws_ptr, requested_device_id);
if (device_id == 0) return;
// Get screen context
@@ -1020,7 +1110,7 @@ void CWebService::HandleClipboardGet(void* ws_ptr, const std::string& token) {
ctx->Send2Client(&bToken, sizeof(BYTE));
}
void CWebService::HandleRdpReset(void* ws_ptr, const std::string& token) {
void CWebService::HandleRdpReset(void* ws_ptr, const std::string& token, uint64_t requested_device_id) {
std::string username, role;
if (!ValidateToken(token, username, role)) {
SendText(ws_ptr, BuildJsonResponse("rdp_reset_result", false, "Invalid token"));
@@ -1028,15 +1118,7 @@ void CWebService::HandleRdpReset(void* ws_ptr, const std::string& token) {
}
// Get the device being watched by this client
uint64_t device_id = 0;
{
std::lock_guard<std::mutex> lock(m_ClientsMutex);
auto it = m_Clients.find(ws_ptr);
if (it != m_Clients.end()) {
device_id = it->second.watch_device_id;
}
}
uint64_t device_id = ResolveWatchDevice(ws_ptr, requested_device_id);
if (device_id == 0) {
SendText(ws_ptr, BuildJsonResponse("rdp_reset_result", false, "No device connected"));
return;
@@ -1060,19 +1142,14 @@ void CWebService::HandleRdpReset(void* ws_ptr, const std::string& token) {
}
}
void CWebService::HandleAudioToggle(void* ws_ptr, const std::string& token) {
void CWebService::HandleAudioToggle(void* ws_ptr, const std::string& token, uint64_t requested_device_id) {
std::string username, role;
if (!ValidateToken(token, username, role)) {
SendText(ws_ptr, BuildJsonResponse("audio_toggle_result", false, "Invalid token"));
return;
}
uint64_t device_id = 0;
{
std::lock_guard<std::mutex> lock(m_ClientsMutex);
auto it = m_Clients.find(ws_ptr);
if (it != m_Clients.end()) device_id = it->second.watch_device_id;
}
uint64_t device_id = ResolveWatchDevice(ws_ptr, requested_device_id);
if (device_id == 0) {
SendText(ws_ptr, BuildJsonResponse("audio_toggle_result", false, "No device connected"));
return;
@@ -1087,6 +1164,78 @@ void CWebService::HandleAudioToggle(void* ws_ptr, const std::string& token) {
SendText(ws_ptr, BuildJsonResponse("audio_toggle_result", true));
}
void CWebService::HandleKeyframe(void* ws_ptr, const std::string& token, uint64_t device_id,
const std::string& req) {
Json::Value res;
res["cmd"] = "keyframe_result";
res["id"] = std::to_string(device_id);
if (!req.empty()) res["req"] = req;
auto send = [&](bool ok, const std::string& err) {
res["ok"] = ok;
if (!err.empty()) res["msg"] = err;
Json::StreamWriterBuilder builder;
builder["indentation"] = "";
SendText(ws_ptr, Json::writeString(builder, res));
};
std::string username, role;
if (!ValidateToken(token, username, role)) {
send(false, "Invalid token");
return;
}
if (device_id == 0) {
send(false, "No device");
return;
}
// 校验该 ws 确实订阅了这台设备;同一设备最多 1 次/秒,前端重试不会打爆编码器。
bool allowed = false;
{
std::lock_guard<std::mutex> lock(m_ClientsMutex);
auto it = m_Clients.find(ws_ptr);
if (it != m_Clients.end() && it->second.watch_device_ids.count(device_id)) {
uint64_t now = (uint64_t)time(nullptr);
uint64_t& last = it->second.keyframe_req_at[device_id];
if (last == 0 || now > last) {
last = now;
allowed = true;
}
}
}
if (!allowed) {
send(false, "Not watching device or rate limited");
return;
}
CONTEXT_OBJECT* ctx = GetScreenContext(device_id);
if (!ctx) {
send(false, "No active screen session");
return;
}
// 先把缓存里的 raw IDR 单独发给请求方,让它立刻有帧可画;随后再让客户端
// 强制生成新的 IDR 重置参考链(缓存 IDR 无法保证后续 P 帧的引用正确)。
std::vector<uint8_t> cached;
{
std::lock_guard<std::mutex> lock(m_DeviceCacheMutex);
auto it = m_DeviceCache.find(device_id);
if (it != m_DeviceCache.end()) {
std::lock_guard<std::mutex> cache_lock(it->second->cache_mutex);
cached = it->second->keyframe_cache;
}
}
if (!cached.empty()) {
auto packet = BuildFramePacket(device_id, true, cached.data(), cached.size());
SendBinary(ws_ptr, packet.data(), packet.size());
}
BYTE bToken = COMMAND_SCREEN_KEYFRAME_REQ;
ctx->Send2Client(&bToken, sizeof(BYTE));
send(true, "");
}
//////////////////////////////////////////////////////////////////////////
// User Management Handlers
//////////////////////////////////////////////////////////////////////////
@@ -1280,20 +1429,21 @@ void CWebService::RegisterClient(void* ws_ptr, const std::string& client_ip) {
}
void CWebService::UnregisterClient(void* ws_ptr) {
uint64_t device_id = 0;
std::set<uint64_t> watched;
// Get device_id and remove client
// Copy the watch set and remove the client under the lock.
{
std::lock_guard<std::mutex> lock(m_ClientsMutex);
auto it = m_Clients.find(ws_ptr);
if (it != m_Clients.end()) {
device_id = it->second.watch_device_id;
watched = std::move(it->second.watch_device_ids);
m_Clients.erase(it);
}
}
// Close remote desktop if this was the last viewer
if (device_id > 0) {
// Close remote desktops whose last viewer just disconnected.
// StopRemoteDesktop re-locks m_ClientsMutex, so this must run outside the lock.
for (uint64_t device_id : watched) {
StopRemoteDesktop(device_id);
}
@@ -1317,7 +1467,7 @@ int CWebService::GetWebClientCount(uint64_t device_id) {
std::lock_guard<std::mutex> lock(m_ClientsMutex);
int count = 0;
for (const auto& [ptr, client] : m_Clients) {
if (client.watch_device_id == device_id) {
if (client.watch_device_ids.count(device_id)) {
count++;
}
}
@@ -1664,7 +1814,7 @@ void CWebService::BroadcastFrame(uint64_t device_id, const uint8_t* data, size_t
// Broadcast to all watching clients
std::lock_guard<std::mutex> lock(m_ClientsMutex);
for (auto& [ws_ptr, client] : m_Clients) {
if (client.watch_device_id == device_id) {
if (client.watch_device_ids.count(device_id)) {
SendBinary(ws_ptr, packet.data(), packet.size());
}
}
@@ -1689,15 +1839,19 @@ void CWebService::BroadcastH264Frame(uint64_t device_id, const uint8_t* data, si
if (!data || len < 9 || m_bStopping) return;
// Broadcast to all watching clients
std::lock_guard<std::mutex> lock(m_ClientsMutex);
for (auto& [ws_ptr, client] : m_Clients) {
if (client.watch_device_id == device_id) {
SendBinary(ws_ptr, data, len);
{
std::lock_guard<std::mutex> lock(m_ClientsMutex);
for (auto& [ws_ptr, client] : m_Clients) {
if (client.watch_device_ids.count(device_id)) {
SendBinary(ws_ptr, data, len);
}
}
}
// Cache keyframe (check FrameType byte at offset 4)
// Cache keyframe (check FrameType byte at offset 4). 只缓存 raw payload:
// HandleConnect/HandleKeyframe 发送时会统一用 BuildFramePacket 封装,
// 缓存完整包会导致双重封装(历史 bug)。
if (data[4] == 1) { // IDR frame
CacheKeyframe(device_id, data, len);
CacheKeyframe(device_id, data + 9, len - 9);
}
}
@@ -1716,12 +1870,17 @@ void CWebService::NotifyResolutionChange(uint64_t device_id, int width, int heig
it->second->screen_height = height;
it->second->top_down = top_down;
if (!client_type.empty()) it->second->client_type = client_type;
// 旧 IDR 对应旧分辨率/编码器,不能再发给新加入者。
{
std::lock_guard<std::mutex> cache_lock(it->second->cache_mutex);
it->second->keyframe_cache.clear();
}
}
// Notify watching clients
Json::Value res;
res["cmd"] = "resolution_changed";
res["id"] = device_id;
res["id"] = std::to_string(device_id);
res["width"] = width;
res["height"] = height;
res["top_down"] = top_down;
@@ -1733,7 +1892,7 @@ void CWebService::NotifyResolutionChange(uint64_t device_id, int width, int heig
std::lock_guard<std::mutex> lock(m_ClientsMutex);
for (auto& [ws_ptr, client] : m_Clients) {
if (client.watch_device_id == device_id) {
if (client.watch_device_ids.count(device_id)) {
SendText(ws_ptr, json);
}
}
@@ -1766,7 +1925,7 @@ void CWebService::NotifyAudioState(uint64_t device_id, bool enabled) {
Json::Value res;
res["cmd"] = "audio_state";
res["id"] = device_id;
res["id"] = std::to_string(device_id);
res["enabled"] = enabled;
Json::StreamWriterBuilder builder;
@@ -1775,7 +1934,7 @@ void CWebService::NotifyAudioState(uint64_t device_id, bool enabled) {
std::lock_guard<std::mutex> lock(m_ClientsMutex);
for (auto& [ws_ptr, client] : m_Clients) {
if (client.watch_device_id == device_id) {
if (client.watch_device_ids.count(device_id)) {
SendText(ws_ptr, json);
}
}
@@ -1787,6 +1946,7 @@ void CWebService::BroadcastCursor(uint64_t device_id, uint8_t cursor_index) {
// Build JSON message
Json::Value res;
res["cmd"] = "cursor";
res["id"] = std::to_string(device_id);
res["index"] = cursor_index;
Json::StreamWriterBuilder builder;
@@ -1796,7 +1956,7 @@ void CWebService::BroadcastCursor(uint64_t device_id, uint8_t cursor_index) {
// Send to all watching clients
std::lock_guard<std::mutex> lock(m_ClientsMutex);
for (auto& [ws_ptr, client] : m_Clients) {
if (client.watch_device_id == device_id) {
if (client.watch_device_ids.count(device_id)) {
SendText(ws_ptr, json);
}
}
@@ -1807,7 +1967,7 @@ void CWebService::NotifyClipboardText(uint64_t device_id, const std::string& tex
Json::Value res;
res["cmd"] = "clipboard_text";
res["id"] = device_id;
res["id"] = std::to_string(device_id);
res["text"] = text;
Json::StreamWriterBuilder builder;
@@ -1816,7 +1976,7 @@ void CWebService::NotifyClipboardText(uint64_t device_id, const std::string& tex
std::lock_guard<std::mutex> lock(m_ClientsMutex);
for (auto& [ws_ptr, client] : m_Clients) {
if (client.watch_device_id == device_id) {
if (client.watch_device_ids.count(device_id)) {
SendText(ws_ptr, json);
}
}
@@ -1855,6 +2015,18 @@ bool CWebService::StartRemoteDesktop(uint64_t device_id) {
// Send COMMAND_SCREEN_SPY with H264 algorithm
// If client is already capturing (MFC opened first), it will re-send TOKEN_BITMAPINFO
// This creates a new hidden Web dialog while MFC dialog remains visible
// Cold start: drop any stale IDR cached from a previous capture session, it
// may have a different resolution / encoder instance.
{
std::lock_guard<std::mutex> lock(m_DeviceCacheMutex);
auto it = m_DeviceCache.find(device_id);
if (it != m_DeviceCache.end()) {
std::lock_guard<std::mutex> cache_lock(it->second->cache_mutex);
it->second->keyframe_cache.clear();
}
}
BYTE bToken[32] = { 0 };
bToken[0] = COMMAND_SCREEN_SPY;
bToken[1] = 0; // DXGI mode: 0=GDI
@@ -1872,7 +2044,7 @@ void CWebService::StopRemoteDesktop(uint64_t device_id) {
{
std::lock_guard<std::mutex> lock(m_ClientsMutex);
for (const auto& [ws_ptr, client] : m_Clients) {
if (client.watch_device_id == device_id) {
if (client.watch_device_ids.count(device_id)) {
watchingCount++;
}
}
@@ -1901,7 +2073,7 @@ void CWebService::StopRemoteDesktop(uint64_t device_id) {
{
std::lock_guard<std::mutex> lock(m_ClientsMutex);
for (const auto& [ws, client] : m_Clients)
if (client.watch_device_id == device_id) count++;
if (client.watch_device_ids.count(device_id)) count++;
}
if (count == 0 && m_pParentDlg) {
ClearWebTriggered(device_id);
@@ -2223,14 +2395,20 @@ void CWebService::FlushDeviceChanges() {
if (!offline.empty()) {
std::lock_guard<std::mutex> lock(m_ClientsMutex);
for (auto& [ws_ptr, client] : m_Clients) {
if (offline.count(client.watch_device_id)) {
// 一个 ws 可能同时看多台设备:只移除真正离线的那些,保留其它。
std::vector<uint64_t> gone;
for (uint64_t id : offline) {
if (client.watch_device_ids.count(id)) gone.push_back(id);
}
for (uint64_t id : gone) {
Json::Value res;
res["cmd"] = "device_offline";
res["id"] = std::to_string(client.watch_device_id);
res["id"] = std::to_string(id);
Json::StreamWriterBuilder builder;
builder["indentation"] = "";
SendText(ws_ptr, Json::writeString(builder, res));
client.watch_device_id = 0;
client.watch_device_ids.erase(id);
client.keyframe_req_at.erase(id);
}
}
}
+24 -7
View File
@@ -68,12 +68,16 @@ struct WebClient {
std::string token;
std::string username;
std::string role; // "admin" | "viewer"
uint64_t watch_device_id; // 0 = not watching any device
// 当前这个 ws 订阅的设备集合。屏幕墙允许多路复用;旧前端仍按单设备语义使用
// (connect 替换整个集合)。
std::set<uint64_t> watch_device_ids;
// 每台设备的 keyframe 请求节流时间戳(秒),防止前端重试打爆服务端/客户端。
std::map<uint64_t, uint64_t> keyframe_req_at;
uint64_t connected_at;
uint64_t last_activity; // Last message/pong received (for heartbeat timeout)
std::string client_ip;
WebClient() : watch_device_id(0), connected_at(0), last_activity(0) {}
WebClient() : connected_at(0), last_activity(0) {}
};
// Login attempt tracking for rate limiting
@@ -185,15 +189,24 @@ private:
void HandleLogin(void* ws_ptr, const std::string& msg, const std::string& client_ip);
void HandleGetSalt(void* ws_ptr, const std::string& msg);
void HandleGetDevices(void* ws_ptr, const std::string& token);
void HandleConnect(void* ws_ptr, const std::string& token, uint64_t device_id);
void HandleDisconnect(void* ws_ptr, const std::string& token, uint64_t requested_device_id = 0);
void HandleConnect(void* ws_ptr, const std::string& token, uint64_t device_id,
const std::string& req = "", bool multi_watch = false);
void HandleDisconnect(void* ws_ptr, const std::string& token, uint64_t requested_device_id = 0,
const std::string& req = "", bool multi_watch = false);
void HandleKeyframe(void* ws_ptr, const std::string& token, uint64_t device_id,
const std::string& req);
void HandlePing(void* ws_ptr, const std::string& token);
void HandleMouse(void* ws_ptr, const std::string& msg);
void HandleKey(void* ws_ptr, const std::string& msg);
void HandleClipboardSet(void* ws_ptr, const std::string& msg);
void HandleClipboardGet(void* ws_ptr, const std::string& token);
void HandleRdpReset(void* ws_ptr, const std::string& token);
void HandleAudioToggle(void* ws_ptr, const std::string& token);
void HandleClipboardGet(void* ws_ptr, const std::string& token, uint64_t device_id = 0);
void HandleRdpReset(void* ws_ptr, const std::string& token, uint64_t device_id = 0);
void HandleAudioToggle(void* ws_ptr, const std::string& token, uint64_t device_id = 0);
// 输入类命令的设备解析:优先用显式 id(必须在 watch 集合内);否则当集合恰好
// 只有一台设备时回退到它(兼容不带 id 的旧前端)。多设备且无 id 时返回 0(丢弃),
// 绝不能回退到任意集合成员,否则会变成跨设备输入注入。
uint64_t ResolveWatchDevice(void* ws_ptr, uint64_t requested_device_id);
// Token management
std::string GenerateToken(const std::string& username, const std::string& role);
@@ -246,6 +259,10 @@ private:
static const int HEARTBEAT_INTERVAL_SEC = 30; // Send ping every 30 seconds
static const int HEARTBEAT_TIMEOUT_SEC = 90; // Disconnect if no activity for 90 seconds
// 每个 ws 最多订阅的设备数(屏幕墙最多 25 个 tile,留余量)。防止异常客户端
// 订阅数百台设备拖慢每一帧的广播循环。
static constexpr size_t kMaxWatchPerClient = 32;
// Heartbeat thread function
void HeartbeatThread();
+10 -5
View File
@@ -138,11 +138,16 @@ func (h *MyHandler) OnReceive(ctx *connection.Context, data []byte) {
case protocol.TokenClipboardText:
h.handleClipboardText(ctx, data)
case protocol.TokenKeyframe:
// Sent by the client only when frameID % m_GOP == 0; the client's
// DEFAULT_GOP is 0x7FFFFFFF (effectively infinite), so this token
// is essentially unused in practice. Treat as a no-op for now —
// IDRs always arrive in-band via TOKEN_NEXTSCREEN and we catch
// them via the H264 NAL scan in handleScreenFrame.
// The client switches to this token when it emits an IDR on request
// (COMMAND_SCREEN_KEYFRAME_REQ) or on a GOP boundary. The header
// layout is identical to TOKEN_NEXTSCREEN, but the payload is always
// an IDR, so relay it as frameType=1 without sniffing. Dropping it
// (the old behaviour, harmless while the client's DEFAULT_GOP of
// 0x7FFFFFFF meant this token never appeared) breaks keyframe
// recovery: the browser keeps decoding P-frames against a reference
// chain it never received, so the picture stays corrupt until the
// screen session restarts.
h.handleScreenFrame(ctx, data, true)
case protocol.CmdCursorImage:
// Custom cursor bitmaps — relayed in Phase 5+ when the web cursor
// overlay learns to render arbitrary BGRA images. Drop silently for
+72
View File
@@ -0,0 +1,72 @@
package main
import (
"testing"
"github.com/yuanyuanxiang/SimpleRemoter/server/go/connection"
"github.com/yuanyuanxiang/SimpleRemoter/server/go/hub"
"github.com/yuanyuanxiang/SimpleRemoter/server/go/protocol"
)
type frameRecorder struct {
frames []recordedFrame
}
type recordedFrame struct {
deviceID string
packet []byte
isKey bool
}
func (r *frameRecorder) OnScreenFrame(deviceID string, packet []byte, isKeyframe bool) {
r.frames = append(r.frames, recordedFrame{deviceID, packet, isKeyframe})
}
func (r *frameRecorder) OnDeviceOnline(hub.DeviceInfo) {}
func (r *frameRecorder) OnDeviceOffline(string) {}
func (r *frameRecorder) OnDeviceUpdate(string, int, string) {}
func (r *frameRecorder) OnResolutionChange(string, int, int) {}
func (r *frameRecorder) OnCursorChange(string, byte) {}
func (r *frameRecorder) OnClipboardText(string, string) {}
func (r *frameRecorder) OnTerminalReady(string, bool) {}
func (r *frameRecorder) OnTerminalData(string, []byte) {}
func (r *frameRecorder) OnTerminalClosed(string, string) {}
// TestOnReceiveRelaysKeyframeToken guards the forced-IDR recovery path: when
// the browser asks for a keyframe, the client replies with TOKEN_KEYFRAME.
// Dropping that token (the old no-op) leaves the browser decoding P-frames
// against a reference chain it never received, i.e. persistent corruption.
func TestOnReceiveRelaysKeyframeToken(t *testing.T) {
h := hub.New()
rec := &frameRecorder{}
h.Subscribe(rec)
h.Register(&hub.Device{ID: "dev", PeerIP: "10.0.0.1"},
&connection.Context{RemoteAddr: "10.0.0.1:1000"})
screenCtx := &connection.Context{RemoteAddr: "10.0.0.1:1001"}
if !h.BindScreenConn("dev", screenCtx) {
t.Fatal("screen bind failed")
}
// [TOKEN_KEYFRAME][algo=H264][cursorPos:8][cursorIdx:1][IDR NAL]
data := []byte{
protocol.TokenKeyframe, 2,
0, 0, 0, 0, 0, 0, 0, 0,
0,
0x00, 0x00, 0x00, 0x01, 0x65, 0x88, 0x84,
}
(&MyHandler{hub: h}).OnReceive(screenCtx, data)
if len(rec.frames) != 1 {
t.Fatalf("frames relayed = %d, want 1 (TOKEN_KEYFRAME must not be dropped)", len(rec.frames))
}
f := rec.frames[0]
if f.deviceID != "dev" {
t.Fatalf("deviceID = %q, want dev", f.deviceID)
}
if !f.isKey || f.packet[4] != 1 {
t.Fatalf("isKeyframe=%v frameType=%d, want true/1", f.isKey, f.packet[4])
}
if want := 9 + len(data) - (1 + protocol.ScreenFrameHeaderLen); len(f.packet) != want {
t.Fatalf("packet len = %d, want %d", len(f.packet), want)
}
}
+7
View File
@@ -581,6 +581,13 @@ func (h *Hub) PublishResolution(deviceID string, width, height int) {
if d, ok := h.devices[deviceID]; ok {
d.screenWidth = width
d.screenHeight = height
// The cached IDR belongs to the previous capture setup. The client
// re-sends TOKEN_BITMAPINFO on the same sub-conn after a window
// resize / capture restart, so this is not only the
// new-sub-conn path (BindScreenConn already clears there). Serving
// the old-resolution IDR alongside the new dimensions leaves the
// decoder misconfigured and garbles the picture until the next IDR.
d.lastKeyframe = nil
}
h.mu.Unlock()
for _, s := range h.snapshotSubscribers() {
+1
View File
@@ -112,6 +112,7 @@ const (
CommandShell byte = 40 // COMMAND_SHELL - ask device to open a shell sub-connection
CommandTerminalRsize byte = 81 // CMD_TERMINAL_RESIZE - [cmd:1][cols:2 LE][rows:2 LE]
CmdRestoreConsole byte = 82 // CMD_RESTORE_CONSOLE - RDP session "归位": switch back to the console session and restart capture
CommandScreenKeyframeReq byte = 169 // COMMAND_SCREEN_KEYFRAME_REQ - force the client to emit an IDR immediately
CommandBye byte = 204 // COMMAND_BYE - disconnect
CommandHeartbeat byte = 216 // CMD_HEARTBEAT_ACK
+96
View File
@@ -0,0 +1,96 @@
package web
import (
"fmt"
"reflect"
"sort"
"testing"
)
func sortedWatchIDs(c *wsClient) []string {
out := make([]string, 0, len(c.watchIDs))
for id := range c.watchIDs {
out = append(out, id)
}
sort.Strings(out)
return out
}
func TestApplyConnectWatchMulti(t *testing.T) {
c := &wsClient{watchIDs: make(map[string]struct{})}
if stopped, tooMany := applyConnectWatch(c, "a", true); len(stopped) != 0 || tooMany {
t.Fatalf("first connect = (%v, %v), want ([], false)", stopped, tooMany)
}
if stopped, tooMany := applyConnectWatch(c, "b", true); len(stopped) != 0 || tooMany {
t.Fatalf("second connect = (%v, %v), want ([], false)", stopped, tooMany)
}
if got, want := sortedWatchIDs(c), []string{"a", "b"}; !reflect.DeepEqual(got, want) {
t.Fatalf("watch set = %v, want %v", got, want)
}
// Repeated connect (focus switch) must be a no-op.
if stopped, tooMany := applyConnectWatch(c, "a", true); len(stopped) != 0 || tooMany {
t.Fatalf("repeat connect = (%v, %v), want ([], false)", stopped, tooMany)
}
if got, want := sortedWatchIDs(c), []string{"a", "b"}; !reflect.DeepEqual(got, want) {
t.Fatalf("watch set after repeat = %v, want %v", got, want)
}
}
func TestApplyConnectWatchLegacyReplaces(t *testing.T) {
c := &wsClient{watchIDs: map[string]struct{}{"a": {}, "b": {}}}
stopped, tooMany := applyConnectWatch(c, "c", false)
if tooMany {
t.Fatal("legacy connect hit the cap, want not")
}
sort.Strings(stopped)
if want := []string{"a", "b"}; !reflect.DeepEqual(stopped, want) {
t.Fatalf("stopped = %v, want %v", stopped, want)
}
if got, want := sortedWatchIDs(c), []string{"c"}; !reflect.DeepEqual(got, want) {
t.Fatalf("watch set = %v, want %v", got, want)
}
}
func TestApplyConnectWatchCap(t *testing.T) {
c := &wsClient{watchIDs: make(map[string]struct{})}
for i := 0; i < maxWatchPerClient; i++ {
applyConnectWatch(c, fmt.Sprintf("d%02d", i), true)
}
if _, tooMany := applyConnectWatch(c, "overflow", true); !tooMany {
t.Fatal("expected the cap to reject a new device")
}
if len(c.watchIDs) != maxWatchPerClient {
t.Fatalf("watch set size = %d, want %d", len(c.watchIDs), maxWatchPerClient)
}
// An already watched device must still be accepted at the cap.
if _, tooMany := applyConnectWatch(c, "d00", true); tooMany {
t.Fatal("repeat connect at the cap was rejected")
}
}
func TestResolveWatchDevice(t *testing.T) {
h := &wsHub{}
c := &wsClient{watchIDs: map[string]struct{}{"a": {}}}
if got := h.resolveWatchDevice(c, []byte(`{"id":"a"}`)); got != "a" {
t.Fatalf("explicit watched id = %q, want a", got)
}
if got := h.resolveWatchDevice(c, []byte(`{"id":"nope"}`)); got != "" {
t.Fatalf("explicit unwatched id = %q, want empty", got)
}
// Old frontend: no id, exactly one watched device.
if got := h.resolveWatchDevice(c, []byte(`{}`)); got != "a" {
t.Fatalf("no id, single device = %q, want a", got)
}
c.watchIDs["b"] = struct{}{}
if got := h.resolveWatchDevice(c, []byte(`{}`)); got != "" {
t.Fatalf("no id, two devices = %q, want empty", got)
}
if got := h.resolveWatchDevice(c, []byte(`{"id":"a"}`)); got != "a" {
t.Fatalf("explicit id, two devices = %q, want a", got)
}
}
+66 -18
View File
@@ -52,8 +52,16 @@ type wsClient struct {
token string // set once authenticated
role string // mirrors session role after login
addr string // client address for logs
watching string // device ID this browser is currently streaming, "" when on the list
termWatching string // device ID for an open web terminal session, "" otherwise
// watchIDs is the set of devices this browser is streaming (the screen
// wall watches several at once). Empty while on the device list.
// Guarded by wsHub.mu.
watchIDs map[string]struct{}
// keyframeAt throttles keyframe requests to one per device per second.
// Unix seconds, guarded by wsHub.mu.
keyframeAt map[string]int64
}
// queue writes a JSON text frame onto the send buffer. Drops silently if the
@@ -85,6 +93,24 @@ func (c *wsClient) close() {
})
}
// challengeCaps advertises optional protocol features. It is appended to every
// challenge, including the rotation after a failed login, because the browser
// overwrites its capability state on each challenge. Old frontends ignore it.
const challengeCaps = `,"caps":{"multiWatch":true,"keyframeRequest":true}`
// maxWatchPerClient bounds the number of devices one WebSocket may stream at
// once. The wall tops out at 25 tiles; the slack covers a focus switch racing
// a teardown. This keeps the per-frame broadcast loop cheap for a misbehaving
// or buggy client.
const maxWatchPerClient = 32
// isWatching reports whether this connection subscribed to deviceID.
// Callers must hold wsHub.mu (read or write).
func (c *wsClient) isWatching(deviceID string) bool {
_, ok := c.watchIDs[deviceID]
return ok
}
// ----- ws hub: registry of all connected browsers -------------------------
type wsHub struct {
@@ -253,10 +279,19 @@ func (h *wsHub) OnDeviceOffline(id string) {
"cmd": "device_offline",
"id": id,
})
h.mu.RLock()
defer h.mu.RUnlock()
// Drop the id from every watch set as well: the device is gone, so this
// connection no longer holds a subscription for it. The browser does the
// same with its local refcount. Without this, a stale entry would keep
// countWatchers non-zero and later block the screen teardown.
h.mu.Lock()
defer h.mu.Unlock()
for c := range h.clients {
if c.watching == id && c.token != "" {
if !c.isWatching(id) {
continue
}
delete(c.watchIDs, id)
delete(c.keyframeAt, id)
if c.token != "" {
c.queue(msg)
}
}
@@ -269,12 +304,13 @@ func (h *wsHub) OnDeviceOffline(id string) {
func (h *wsHub) OnCursorChange(deviceID string, index byte) {
msg := mustJSON(map[string]any{
"cmd": "cursor",
"id": deviceID,
"index": index,
})
h.mu.RLock()
defer h.mu.RUnlock()
for c := range h.clients {
if c.watching == deviceID && c.token != "" {
if c.isWatching(deviceID) && c.token != "" {
c.queue(msg)
}
}
@@ -294,7 +330,7 @@ func (h *wsHub) OnResolutionChange(deviceID string, width, height int) {
h.mu.RLock()
defer h.mu.RUnlock()
for c := range h.clients {
if c.watching == deviceID && c.token != "" {
if c.isWatching(deviceID) && c.token != "" {
c.queue(msg)
}
}
@@ -311,7 +347,7 @@ func (h *wsHub) OnClipboardText(deviceID string, text string) {
h.mu.RLock()
defer h.mu.RUnlock()
for c := range h.clients {
if c.watching == deviceID && c.token != "" {
if c.isWatching(deviceID) && c.token != "" {
c.queue(msg)
}
}
@@ -325,7 +361,7 @@ func (h *wsHub) OnScreenFrame(deviceID string, packet []byte, _ bool) {
h.mu.RLock()
defer h.mu.RUnlock()
for c := range h.clients {
if c.watching == deviceID && c.token != "" {
if c.isWatching(deviceID) && c.token != "" {
c.queueBinary(packet)
}
}
@@ -424,12 +460,22 @@ func (h *wsHub) register(c *wsClient) {
func (h *wsHub) unregister(c *wsClient) {
h.mu.Lock()
delete(h.clients, c)
// Snapshot the watch set and clear it under the same lock. The teardown
// below must run OUTSIDE the lock: countWatchers takes h.mu.RLock and
// deferredCloseScreen takes h.mu.Lock.
watched := make([]string, 0, len(c.watchIDs))
for id := range c.watchIDs {
watched = append(watched, id)
}
c.watchIDs = nil
c.keyframeAt = nil
h.mu.Unlock()
// If this client was the last viewer of a device, tear down the screen
// session so the device stops encoding. Done OUTSIDE the lock so the
// hub's mutators can take their own locks without risk of recursion.
if c.watching != "" && h.countWatchers(c.watching) == 0 {
h.deferredCloseScreen(c.watching)
// session so the device stops encoding.
for _, id := range watched {
if h.countWatchers(id) == 0 {
h.deferredCloseScreen(id)
}
}
// Terminal sessions are single-viewer by design, so any open session
// belongs to this client. Tear it down so the next viewer doesn't
@@ -492,11 +538,13 @@ func (h *wsHub) serve(w http.ResponseWriter, r *http.Request) {
}
client := &wsClient{
conn: conn,
send: make(chan wsMsg, wsSendBuffer),
closed: make(chan struct{}),
nonce: nonce,
addr: clientIP(r, h.trustForwardedFor),
conn: conn,
send: make(chan wsMsg, wsSendBuffer),
closed: make(chan struct{}),
nonce: nonce,
addr: clientIP(r, h.trustForwardedFor),
watchIDs: make(map[string]struct{}),
keyframeAt: make(map[string]int64),
}
h.register(client)
defer h.unregister(client)
@@ -504,7 +552,7 @@ func (h *wsHub) serve(w http.ResponseWriter, r *http.Request) {
go h.writeLoop(client)
// Greet with a challenge nonce so the browser can compute the login response.
client.queue([]byte(`{"cmd":"challenge","nonce":"` + nonce + `"}`))
client.queue([]byte(`{"cmd":"challenge","nonce":"` + nonce + `"` + challengeCaps + `}`))
h.readLoop(client)
}
+250 -57
View File
@@ -24,7 +24,8 @@ func (h *wsHub) rotateChallenge(c *wsClient) {
return
}
c.nonce = n
c.queue([]byte(`{"cmd":"challenge","nonce":"` + n + `"}`))
// Must carry caps too: the browser replaces serverCaps on every challenge.
c.queue([]byte(`{"cmd":"challenge","nonce":"` + n + `"` + challengeCaps + `}`))
}
// dispatch routes one inbound message to its handler. The `raw` payload is
@@ -50,6 +51,8 @@ func (h *wsHub) dispatch(c *wsClient, cmd string, raw []byte) {
case "connect":
h.handleConnect(c, raw)
case "keyframe":
h.handleKeyframe(c, raw)
case "rdp_reset":
h.handleRdpReset(c, raw)
case "audio_toggle":
@@ -240,66 +243,196 @@ func (h *wsHub) resetLoginThrottle(c *wsClient, username string) {
// screen sub-connection. Closing the TCP sub-conn is the signal the C++
// device firmware uses to stop screen capture, so this is how we ask the
// device to free its encoder.
func (h *wsHub) handleDisconnect(c *wsClient, _ []byte) {
// Mirror handleConnect: take h.mu so event-handler readers
// (OnResolutionChange/OnScreenFrame) get a consistent view of c.watching.
func (h *wsHub) handleDisconnect(c *wsClient, raw []byte) {
var in struct {
ID string `json:"id"`
MultiWatch bool `json:"multiWatch"`
Req string `json:"req"`
}
_ = json.Unmarshal(raw, &in)
// Take h.mu so event-handler readers (OnResolutionChange/OnScreenFrame)
// get a consistent view of the watch set.
var closed []string
h.mu.Lock()
prev := c.watching
c.watching = ""
if in.MultiWatch && in.ID != "" {
// New frontend: drop only this device; the other wall tiles keep
// streaming on the same connection.
if c.isWatching(in.ID) {
delete(c.watchIDs, in.ID)
delete(c.keyframeAt, in.ID)
closed = append(closed, in.ID)
}
} else {
// Old frontend, or a disconnect without an id: it only ever watched
// one device, so clear everything.
for id := range c.watchIDs {
closed = append(closed, id)
}
c.watchIDs = make(map[string]struct{})
c.keyframeAt = make(map[string]int64)
}
h.mu.Unlock()
c.queue([]byte(`{"cmd":"disconnect_result","ok":true}`))
if prev != "" && h.countWatchers(prev) == 0 {
h.deferredCloseScreen(prev)
res := map[string]any{"cmd": "disconnect_result", "ok": true}
if in.ID != "" {
res["id"] = in.ID
}
if in.Req != "" {
res["req"] = in.Req
}
c.queue(mustJSON(res))
// deferredCloseScreen takes h.mu, so it must run outside the lock above.
for _, id := range closed {
if h.countWatchers(id) == 0 {
h.deferredCloseScreen(id)
}
}
}
// countWatchers returns how many authenticated clients still have their
// `watching` field pointing at deviceID. Called from disconnect paths.
// countWatchers returns how many clients still have deviceID in their watch
// set. Called from disconnect paths.
func (h *wsHub) countWatchers(deviceID string) int {
h.mu.RLock()
defer h.mu.RUnlock()
n := 0
for c := range h.clients {
if c.watching == deviceID {
if c.isWatching(deviceID) {
n++
}
}
return n
}
// applyConnectWatch mutates c.watchIDs for a connect request. The caller must
// hold h.mu.
//
// It returns the ids dropped by the legacy replace-all semantics; the caller
// must stop their screen sessions outside the lock (only if nobody else still
// watches them). tooMany reports that the per-connection cap blocked the id.
func applyConnectWatch(c *wsClient, id string, multiWatch bool) (stopped []string, tooMany bool) {
if c.watchIDs == nil {
c.watchIDs = make(map[string]struct{})
}
if !multiWatch {
// Old frontend: one device per connection. Replace the whole set so
// switching devices repeatedly cannot grow it without bound.
for old := range c.watchIDs {
if old != id {
stopped = append(stopped, old)
}
}
c.watchIDs = make(map[string]struct{})
}
if _, ok := c.watchIDs[id]; ok {
return stopped, false // repeated connect (focus switch) is a no-op
}
if len(c.watchIDs) >= maxWatchPerClient {
return stopped, true
}
c.watchIDs[id] = struct{}{}
return stopped, false
}
// resolveWatchDevice maps an input command to the device it targets. The new
// frontend sends an explicit id, which must be in this connection's watch set
// so a stale or forged id can never inject input into a device the browser is
// not viewing. Old frontends omit id — accepted only while exactly one device
// is watched. With several devices and no id the command is dropped.
func (h *wsHub) resolveWatchDevice(c *wsClient, raw []byte) string {
var in struct {
ID string `json:"id"`
}
_ = json.Unmarshal(raw, &in)
h.mu.RLock()
defer h.mu.RUnlock()
if in.ID != "" {
if c.isWatching(in.ID) {
return in.ID
}
return ""
}
if len(c.watchIDs) == 1 {
for id := range c.watchIDs {
return id
}
}
return ""
}
func (h *wsHub) handleConnect(c *wsClient, raw []byte) {
if !h.requireAuth(c, raw, "connect_result") {
return
}
var in struct {
ID string `json:"id"`
ID string `json:"id"`
MultiWatch bool `json:"multiWatch"`
Req string `json:"req"`
}
if err := json.Unmarshal(raw, &in); err != nil || in.ID == "" {
c.queue(mustJSON(map[string]any{"cmd": "connect_result", "ok": false, "msg": "Bad request"}))
return
}
// If a screen session is already live for this device (another browser
// is already watching), reuse it: hand the new viewer the current
// resolution and the most recent IDR keyframe so its decoder can start
// rendering immediately, without waiting for the next IDR (~15 s).
cache := h.devices.ScreenState(in.ID)
if cache.Active {
c.queue(mustJSON(map[string]any{
"cmd": "connect_result", "ok": true,
"width": cache.Width, "height": cache.Height,
}))
if len(cache.Keyframe) > 0 {
c.queueBinary(cache.Keyframe)
// Every response echoes id and req: one socket may now serve several
// devices (screen wall), so the browser routes by id (and req when it
// needs to pair a reply with a specific request).
reply := func(ok bool, msg string) map[string]any {
m := map[string]any{"cmd": "connect_result", "ok": ok, "id": in.ID}
if in.Req != "" {
m["req"] = in.Req
}
h.mu.Lock()
// Cancel any pending deferred close so the relay stays alive.
if msg != "" {
m["msg"] = msg
}
return m
}
var stopped []string
var tooMany bool
h.mu.Lock()
stopped, tooMany = applyConnectWatch(c, in.ID, in.MultiWatch)
// A pending deferred close means the previous session is still alive for
// a few seconds; cancel it so the cached resolution + IDR hot path wins.
// Only when the subscription was accepted: a rejected connect must not
// cancel the teardown of a device nobody is watching any more, or that
// session would keep streaming forever.
if !tooMany {
if t, ok := h.screenCloseTimers[in.ID]; ok {
t.Stop()
delete(h.screenCloseTimers, in.ID)
}
c.watching = in.ID
h.mu.Unlock()
}
h.mu.Unlock()
if tooMany {
c.queue(mustJSON(reply(false, "Too many devices on one connection")))
return
}
// Only the legacy replace-all semantics can produce stopped ids. Their
// screen sessions are torn down outside the lock; deferredCloseScreen is
// a no-op while another browser still watches the device.
for _, id := range stopped {
if h.countWatchers(id) == 0 {
h.deferredCloseScreen(id)
}
}
// If a screen session is already live for this device (another browser
// is already watching, or this connection reconnected), reuse it: hand
// the viewer the cached resolution and the most recent IDR keyframe so
// its decoder can start rendering immediately, without waiting for the
// next IDR (~15 s).
cache := h.devices.ScreenState(in.ID)
if cache.Active {
m := reply(true, "")
m["width"] = cache.Width
m["height"] = cache.Height
c.queue(mustJSON(m))
if len(cache.Keyframe) > 0 {
c.queueBinary(cache.Keyframe)
}
return
}
@@ -312,41 +445,101 @@ func (h *wsHub) handleConnect(c *wsClient, raw []byte) {
cmd[2] = protocol.AlgorithmH264
cmd[3] = 1
// CRITICAL: bind c.watching BEFORE asking the device to start capturing.
// On fast reconnects the device's screen sub-conn handshake completes in
// <100 ms, so TOKEN_BITMAPINFO and even the first H264 frame can arrive
// before this handler finishes — and the resolution_changed / frame
// broadcasts in wsHub filter on c.watching. With the assignment after
// SendToDevice the new viewer silently misses the very first IDR and
// resolution_changed, leaving the page stuck on "Waiting for video".
//
// The write needs to share the lock event handlers use to read c.watching
// (they iterate h.clients under h.mu.RLock). Without that the write is a
// data race; on a fast reconnect the reader goroutine can keep observing
// the previous value ("") long enough to drop the first resolution_changed
// and the first IDR, which produces the exact "every other quick reconnect
// goes black" symptom — the C++ server avoids it because it does the same
// state mutation under std::mutex and reaps the memory-barrier as a bonus.
h.mu.Lock()
c.watching = in.ID
h.mu.Unlock()
// CRITICAL: the watch entry is inserted above (applyConnectWatch) BEFORE
// asking the device to start capturing. On fast reconnects the device's
// screen sub-conn handshake completes in <100 ms, so TOKEN_BITMAPINFO and
// even the first H264 frame can arrive before this handler finishes — and
// the resolution_changed / frame broadcasts filter on the watch set. If
// the entry were added afterwards the new viewer would silently miss the
// very first IDR and resolution_changed, leaving the page stuck on
// "Waiting for video".
if err := h.devices.SendToDevice(in.ID, cmd); err != nil {
// Roll back the watching flag if we never managed to kick capture.
// Roll back the subscription: no screen session was started.
h.mu.Lock()
c.watching = ""
delete(c.watchIDs, in.ID)
delete(c.keyframeAt, in.ID)
h.mu.Unlock()
msg := "Device offline"
if err != hub.ErrDeviceOffline {
msg = "Failed to start screen capture"
h.log.Error("SendToDevice(%s): %v", in.ID, err)
}
c.queue(mustJSON(map[string]any{"cmd": "connect_result", "ok": false, "msg": msg}))
c.queue(mustJSON(reply(false, msg)))
return
}
h.log.Debug("[timing] COMMAND_SCREEN_SPY sent to device=%s (cold start)", in.ID)
c.queue(mustJSON(map[string]any{"cmd": "connect_result", "ok": true}))
c.queue(mustJSON(reply(true, "")))
}
// handleKeyframe asks the device to emit an IDR immediately. The browser calls
// it when it resumes decoding a wall tile, or right after connecting to reset
// the reference chain. The server first replays its cached IDR so the decoder
// has something to paint at once, then sends COMMAND_SCREEN_KEYFRAME_REQ to
// the client, whose fresh IDR replaces the cache (the cached one cannot
// guarantee the following P-frames reference it).
//
// Mirrors CWebService::HandleKeyframe. Requests are limited to one per device
// per second, so a retrying frontend cannot spin the encoder.
func (h *wsHub) handleKeyframe(c *wsClient, raw []byte) {
if !h.requireAuth(c, raw, "keyframe_result") {
return
}
var in struct {
ID string `json:"id"`
Req string `json:"req"`
}
_ = json.Unmarshal(raw, &in)
reply := func(ok bool, msg string) map[string]any {
m := map[string]any{"cmd": "keyframe_result", "ok": ok}
if in.ID != "" {
m["id"] = in.ID
}
if in.Req != "" {
m["req"] = in.Req
}
if msg != "" {
m["msg"] = msg
}
return m
}
if in.ID == "" {
c.queue(mustJSON(reply(false, "Bad request")))
return
}
allowed := false
h.mu.Lock()
if c.isWatching(in.ID) {
now := time.Now().Unix()
if last, ok := c.keyframeAt[in.ID]; !ok || now > last {
if c.keyframeAt == nil {
c.keyframeAt = make(map[string]int64)
}
c.keyframeAt[in.ID] = now
allowed = true
}
}
h.mu.Unlock()
if !allowed {
c.queue(mustJSON(reply(false, "Not watching device or rate limited")))
return
}
cache := h.devices.ScreenState(in.ID)
if !cache.Active {
c.queue(mustJSON(reply(false, "No active screen session")))
return
}
if len(cache.Keyframe) > 0 {
c.queueBinary(cache.Keyframe)
}
if err := h.devices.SendToScreen(in.ID, []byte{protocol.CommandScreenKeyframeReq}); err != nil {
c.queue(mustJSON(reply(false, "Failed to request keyframe")))
return
}
c.queue(mustJSON(reply(true, "")))
}
// handleRdpReset asks the device to switch its screen capture back to the
@@ -363,7 +556,7 @@ func (h *wsHub) handleRdpReset(c *wsClient, raw []byte) {
if !h.requireAuth(c, raw, "rdp_reset_result") {
return
}
deviceID := c.watching
deviceID := h.resolveWatchDevice(c, raw)
if deviceID == "" {
h.log.Warn("rdp_reset: no device watched (addr=%s role=%s)", c.addr, c.role)
return
@@ -440,7 +633,7 @@ func (h *wsHub) handleMouse(c *wsClient, raw []byte) {
if err := json.Unmarshal(raw, &in); err != nil {
return
}
deviceID := c.watching
deviceID := h.resolveWatchDevice(c, raw)
if deviceID == "" {
return // browser hasn't picked a device yet
}
@@ -525,7 +718,7 @@ func (h *wsHub) handleKey(c *wsClient, raw []byte) {
if in.KeyCode == protocol.VKLWin || in.KeyCode == protocol.VKRWin {
return // Windows keys stay local; matches both C++ server and client
}
deviceID := c.watching
deviceID := h.resolveWatchDevice(c, raw)
if deviceID == "" {
return
}
@@ -583,7 +776,7 @@ func (h *wsHub) handleClipboardSet(c *wsClient, raw []byte) {
if in.Text == "" {
return
}
deviceID := c.watching
deviceID := h.resolveWatchDevice(c, raw)
if deviceID == "" {
return
}
@@ -607,7 +800,7 @@ func (h *wsHub) handleClipboardGet(c *wsClient, raw []byte) {
if !h.requireAuth(c, raw, "clipboard_get_result") {
return
}
deviceID := c.watching
deviceID := h.resolveWatchDevice(c, raw)
if deviceID == "" {
return
}
+1440 -49
View File
File diff suppressed because it is too large. Load diff