Feature: Add window-relative remote_mouse coords and PTY pipes in terminal_exec

New Windows clients append an optional 20-byte WINDOWINFO_MCP_DESKTOP block
(virtual desktop rect, magic 'VDCP') after the window list, so the server can
map window-relative coordinates into the capture frame. remote_mouse accepts
an optional window_hwnd; x/y and drag x2/y2 are then normalized against the
intersection of the window and the virtual desktop, which is exactly the
source rect of get_screenshot mode=window. The target window must be foreground
by default, and window_hwnd/expect_hwnd must agree when both are given.
structuredContent gains coordinateWindow and resolved (frame coords plus the
basis rect), so callers can verify the mapping.

terminal_exec now allows pipes, && and || in PTY sessions (ConPTY on Windows,
POSIX PTY on Linux) because the sentinel runs on its own line; legacy non-PTY
sessions still reject & and |. A read-only PeekTermSession lets the server
reject unsupported syntax before BeginTermCommand. POSIX still rejects bare
background & (&&, &>, 2>&1 stay allowed).

Docs: Mcp_Design.md sections 12.1/20.2/22 and Mcp_Terminal_Design.md 4.4/4.6
are synchronized with the protocol, the new validation order, and the basis
rect semantics.

Co-Authored-By: deepseek-flash
This commit is contained in:
yuanyuanxianganddeepseek-flash committed 2026-11-10 18:03:25 +01:00
1 parent 235a8a622c
commit 5501889d33
10 files changed
+383 -60

No files matched your search

+11 -1
View File
@@ -911,7 +911,7 @@ static void ReplyMcpWindowInfo(IOCPClient* client, uint8_t mode)
// 字节最高位置 1,服务端据此报告"列表不完整",而不是把第 97 个以后的窗口当成不存在。
size_t count = windows.size() > 96 ? 96 : windows.size();
const bool truncated = windows.size() > count;
std::vector<BYTE> pkt(4 + count * sizeof(WINDOWINFO_MCP), 0);
std::vector<BYTE> pkt(4 + count * sizeof(WINDOWINFO_MCP) + sizeof(WINDOWINFO_MCP_DESKTOP), 0);
pkt[0] = TOKEN_MCP_WINDOW_INFO;
pkt[1] = (BYTE)(mode | (truncated ? 0x80 : 0));
pkt[2] = (BYTE)(count & 0xFF);
@@ -919,6 +919,16 @@ static void ReplyMcpWindowInfo(IOCPClient* client, uint8_t mode)
if (count)
memcpy(pkt.data() + 4, windows.data(), count * sizeof(WINDOWINFO_MCP));
// 尾块:虚拟桌面矩形,供服务端把窗口相对坐标换算成抓屏帧归一化坐标。
// 老服务端只读前 count 条记录,尾部字节被忽略(非破坏性扩展)。
WINDOWINFO_MCP_DESKTOP desk = {};
desk.magic = WINDOWINFO_DESKTOP_MAGIC;
desk.left = ::GetSystemMetrics(SM_XVIRTUALSCREEN);
desk.top = ::GetSystemMetrics(SM_YVIRTUALSCREEN);
desk.width = ::GetSystemMetrics(SM_CXVIRTUALSCREEN);
desk.height = ::GetSystemMetrics(SM_CYVIRTUALSCREEN);
memcpy(pkt.data() + 4 + count * sizeof(WINDOWINFO_MCP), &desk, sizeof(desk));
if (client && client->IsConnected())
client->Send2Server((char*)pkt.data(), (int)pkt.size());
}