Feature: Support remote clipboard and input on Android hosts

The MCP remote-control family (remote_open / remote_close / remote_mouse /
remote_keyboard / remote_clipboard) previously stopped at Windows and Linux,
even though the Android client already translated injected events into touch
gestures and global actions. This wires up the missing piece and opens the five
tools to APK hosts.

Client side, the screen sub-connection now handles COMMAND_SCREEN_SET_CLIPBOARD:
the native handler forwards the raw GBK bytes to CaptureService.onClipboardText,
which decodes them, strips the trailing NUL that the human clipboard-sync
senders append, and writes the system clipboard from a background app (allowed
on Android 10+, only reads are restricted). Pasting cannot be a Ctrl+V combo,
so VK_V is mapped to ACTION_PASTE on the focused node, falling back to the
accessibility focus and then to the first editable node, since WebView and
self-drawn inputs often do not flag FOCUS_INPUT or isEditable.

A new capability bit CLIENT_CAP_SCREEN_CLIPBOARD (0x0800) tells the server the
APK understands both paths, so builds without the handler get an explicit
-32005 instead of a package that is silently dropped. action=type is rejected
outright for APK hosts - physical key events would only turn 'v' into a paste -
and the error points at remote_clipboard + key=V.

The platform table, server instructions and the five tool descriptions now
describe Android support, which keys actually act as global actions (ESC, HOME,
APPS, PRINTSCREEN), and that injection needs the accessibility service while
capture needs an authorized MediaProjection session, both without receipts.

Verified on the API 35 emulator against a Debug server: ASCII and GBK Chinese
text round-tripped through the clipboard into a focused Settings search field,
type / expect_hwnd / window_hwnd returned -32005, and PRINTSCREEN produced a
saved screenshot. check_platform_table.py and verify_tools.py --offline pass.

Co-Authored-By: deepseek-flash
This commit is contained in:
yuanyuanxianganddeepseek-flash committed 2026-11-11 20:59:10 +01:00
1 parent ac43ad92ef
commit 0a0b709e8a
9 files changed
+351 -179

No files matched your search

+6
View File
@@ -22,6 +22,7 @@
extern uint64_t g_myClientID;
// 定义在 main.cpp,供子连接的 OnReceive 调用
extern void DispatchControlEvent(uint32_t msgVal, uint64_t wParam, int32_t ptX, int32_t ptY);
extern void DispatchClipboardText(const uint8_t* data, size_t size);
// Linux/macOS 共用的 BITMAPINFOHEADER 布局(与 Windows 完全一致)
#pragma pack(push, 1)
@@ -123,6 +124,11 @@ public:
case CMD_QUALITY_LEVEL:
if (size >= 2) LOGI_SH("QualityLevel=%d", (int)(int8_t)data[1]);
break;
case COMMAND_SCREEN_SET_CLIPBOARD:
// 服务端写剪贴板: [cmd:1][GBK text:N](与 Windows CF_TEXT 同编码;人类剪贴板
// 同步会带 '\0' 结尾,MCP 路径不带,Kotlin 侧统一去掉尾随 NUL)
if (size > 1) DispatchClipboardText(data + 1, (size_t)size - 1);
break;
case COMMAND_SCREEN_CONTROL: {
// 服务端通过子连接下发鼠标/键盘控制包(MSG64 固定 48 字节)
// 坐标从 lParam(offset 24)读取,与 Windows/Linux/macOS 客户端一致: