Feature: Web screen wall for multi-device live monitoring

Add a third view to the web UI, the screen wall, which shows several
devices at once in one tab with 1x1 / 2x2 / 3x3 / 4x4 / 5x5 layouts and a
device sidebar on the left. Tiles are monitoring-only; clicking one
enlarges that device into the existing single-screen page for control, and
Back returns to the wall. The same device can be added several times: all
tiles of one device share a single decoder and a single stream.

The wall decodes at most 9 streams on desktop and 4 on touch devices.
Tiles beyond the budget keep receiving frames but have their decoder
closed and show the last frame plus a Paused badge; they are promoted
automatically when a live tile goes away. Subscriptions are reference
counted on the client (streams map with owner sets), so a stream is only
connected on the 0 -> 1 transition and disconnected on 1 -> 0, and the
wall and the single-screen page can share one device without tearing the
stream down.

Server side, WebClient now holds a set of watched devices instead of a
single id. A new multiWatch command flag selects set semantics, while
clients without it keep the old replace-all behaviour, so an old frontend
against the new server is unchanged. connect and disconnect responses echo
id and req so the frontend can route them, input commands carry an id and
are resolved against the watch set (dropped when the set has several
devices and no id), and the per-client watch set is capped at 32.

For fast recovery after a pause or when joining late, the server caches
each device's raw IDR payload and replays it on connect and on a new
keyframe command, then asks the client to force a new IDR. The client gets
COMMAND_SCREEN_KEYFRAME_REQ = 169 and an atomic request flag that is only
cleared after an IDR actually came out of the encoder, so a failed capture
or EAGAIN does not lose the request. The frontend retries the keyframe
request up to three times; an old client that ignores the command simply
recovers on the next natural GOP.

Also fix the keyframe cache double wrapping: BroadcastH264Frame used to
cache the full packet header and HandleConnect wrapped it again. The cache
now stores data + 9, len - 9 and every sender wraps it through
BuildFramePacket, and the cache is cleared on cold start and on resolution
change. StopRemoteDesktop is always called outside m_ClientsMutex, with
the ids collected under the lock first.

Co-Authored-By: deepseek-flash
This commit is contained in:
yuanyuanxianganddeepseek-flash committed 2026-10-18 11:03:17 +02:00
1 parent 04f4eb9947
commit 8eab26c1f6
6 files changed
+1790 -180

No files matched your search

+17 -1
View File
@@ -14,6 +14,7 @@
#include <functional>
#include <future>
#include <memory>
#include <atomic>
#include <emmintrin.h> // SSE2
#include "common/config.h"
#include "VideoEncoderBase.h"
@@ -149,6 +150,9 @@ public:
int m_FrameID; // 帧序号
int m_GOP; // 关键帧间隔
bool m_SendKeyFrame; // 发送关键帧
// Web 端请求下一帧强制 IDR(COMMAND_SCREEN_KEYFRAME_REQ)。只对 H.264 生效;
// 必须在成功编码出 IDR 后清除,否则 ScanNextScreen 失败/硬编 EAGAIN 时请求会丢。
std::atomic<bool> m_bForceKeyFrame{false};
std::unique_ptr<VideoEncoderBase> m_encoder; // 编码器,ensureEncoder() lazy 创建,走 EncoderFactory 探测
bool m_bEncoderPrimed = false; // encoder 是否已成功产出过一个包;
// false 时禁止 skip——避免单显示器路径
@@ -257,6 +261,11 @@ public:
m_BlockBuffers[blockY] = new BYTE[m_ulFullWidth * m_ulFullHeight * 4 * 2 / m_BlockNum + 12];
}
}
// 请求下一帧强制 IDR(可跨线程调用)。只影响 H.264 编码路径;
// 实际清除发生在成功编码出 IDR 之后。
void RequestKeyFrame() { m_bForceKeyFrame.store(true, std::memory_order_relaxed); }
virtual ~ScreenCapture()
{
if (m_BitmapInfor_Full != NULL) {
@@ -876,7 +885,12 @@ public:
{
BYTE algo = m_bAlgorithm;
int frameID = m_FrameID + 1;
bool keyFrame = (frameID % m_GOP == 0);
// Web 端可请求强制 IDR(COMMAND_SCREEN_KEYFRAME_REQ),只对 H.264 有意义。
// 标志在成功编码出 IDR 后才清除(见下方 H.264 keyframe 分支),
// 保证 ScanNextScreen 失败或硬编 EAGAIN 时请求不会丢失。
bool forceKeyFrame = (algo == ALGORITHM_H264) &&
m_bForceKeyFrame.load(std::memory_order_relaxed);
bool keyFrame = forceKeyFrame || (frameID % m_GOP == 0);
m_RectBuffer[0] = keyFrame ? TOKEN_KEYFRAME : TOKEN_NEXTSCREEN;
LPBYTE data = m_RectBuffer + 1;
// 写入使用了哪种算法
@@ -967,6 +981,8 @@ public:
*ulNextSendLength = 1 + offset + encoded_size;
memcpy(data + offset, encoded_data, encoded_size);
m_bEncoderPrimed = true; // 与下方 FirstBuffer 同步:自此 skip 安全
// IDR 已成功产出,满足(可能的)强制关键帧请求。
m_bForceKeyFrame.store(false, std::memory_order_relaxed);
break;
}
default: