Feature: Support remote clipboard and input on Android hosts
The MCP remote-control family (remote_open / remote_close / remote_mouse / remote_keyboard / remote_clipboard) previously stopped at Windows and Linux, even though the Android client already translated injected events into touch gestures and global actions. This wires up the missing piece and opens the five tools to APK hosts. Client side, the screen sub-connection now handles COMMAND_SCREEN_SET_CLIPBOARD: the native handler forwards the raw GBK bytes to CaptureService.onClipboardText, which decodes them, strips the trailing NUL that the human clipboard-sync senders append, and writes the system clipboard from a background app (allowed on Android 10+, only reads are restricted). Pasting cannot be a Ctrl+V combo, so VK_V is mapped to ACTION_PASTE on the focused node, falling back to the accessibility focus and then to the first editable node, since WebView and self-drawn inputs often do not flag FOCUS_INPUT or isEditable. A new capability bit CLIENT_CAP_SCREEN_CLIPBOARD (0x0800) tells the server the APK understands both paths, so builds without the handler get an explicit -32005 instead of a package that is silently dropped. action=type is rejected outright for APK hosts - physical key events would only turn 'v' into a paste - and the error points at remote_clipboard + key=V. The platform table, server instructions and the five tool descriptions now describe Android support, which keys actually act as global actions (ESC, HOME, APPS, PRINTSCREEN), and that injection needs the accessibility service while capture needs an authorized MediaProjection session, both without receipts. Verified on the API 35 emulator against a Debug server: ASCII and GBK Chinese text round-tripped through the clipboard into a focused Settings search field, type / expect_hwnd / window_hwnd returned -32005, and PRINTSCREEN produced a saved screenshot. check_platform_table.py and verify_tools.py --offline pass. Co-Authored-By: deepseek-flash
This commit is contained in:
1 parent
ac43ad92ef
commit
0a0b709e8a
9 files changed
+351
-179
No files matched your search
@@ -436,9 +436,48 @@ class ControlService : AccessibilityService() {
|
||||
0x24 -> performGlobalAction(GLOBAL_ACTION_HOME) // VK_HOME
|
||||
0x5D -> performGlobalAction(GLOBAL_ACTION_RECENTS) // VK_APPS
|
||||
0x2C -> performGlobalAction(GLOBAL_ACTION_TAKE_SCREENSHOT) // PrtSc
|
||||
0x56 -> pasteIntoFocused() // VK_V → 粘贴
|
||||
}
|
||||
}
|
||||
|
||||
// 远程剪贴板的粘贴动作。Android 没有 Ctrl+V 组合键,而且本客户端不注入文本,
|
||||
// 所以把 VK_V 映射成输入框的 ACTION_PASTE:由目标应用自己读剪贴板,
|
||||
// 不受 Android 10+「后台应用不能读剪贴板」的限制。
|
||||
@Suppress("DEPRECATION")
|
||||
private fun pasteIntoFocused() {
|
||||
val root = rootInActiveWindow ?: run {
|
||||
Log.w(TAG, "paste: no active window")
|
||||
return
|
||||
}
|
||||
// 焦点链:输入焦点 → 无障碍焦点 → 整棵树里第一个可编辑节点。部分应用只置
|
||||
// FOCUS_ACCESSIBILITY;也不拿 isEditable 当门槛(WebView / 自绘输入框常常不置位
|
||||
// 但照样响应 ACTION_PASTE),真正的判据是 performAction 的返回值。
|
||||
val target = root.findFocus(AccessibilityNodeInfo.FOCUS_INPUT)
|
||||
?: root.findFocus(AccessibilityNodeInfo.FOCUS_ACCESSIBILITY)
|
||||
?: findFirstEditable(root)
|
||||
if (target == null) {
|
||||
Log.w(TAG, "paste: no focused or editable node")
|
||||
} else {
|
||||
if (!target.performAction(AccessibilityNodeInfo.ACTION_PASTE))
|
||||
Log.w(TAG, "paste: ACTION_PASTE rejected by ${target.packageName}")
|
||||
target.recycle()
|
||||
}
|
||||
root.recycle()
|
||||
}
|
||||
|
||||
// 找不到焦点时的兜底:返回整棵树里第一个可见的可编辑节点(调用方负责 recycle)
|
||||
@Suppress("DEPRECATION")
|
||||
private fun findFirstEditable(node: AccessibilityNodeInfo): AccessibilityNodeInfo? {
|
||||
if (node.isEditable && node.isVisibleToUser) return AccessibilityNodeInfo.obtain(node)
|
||||
for (i in 0 until node.childCount) {
|
||||
val child = node.getChild(i) ?: continue
|
||||
val hit = findFirstEditable(child)
|
||||
child.recycle()
|
||||
if (hit != null) return hit
|
||||
}
|
||||
return null
|
||||
}
|
||||
|
||||
private fun dispatchTap(x: Float, y: Float) {
|
||||
if (Build.VERSION.SDK_INT < Build.VERSION_CODES.N) return
|
||||
val path = Path().apply { moveTo(x, y); lineTo(x + 1f, y) }
|
||||
|
||||
Reference in new issue
Block a user