Commit Graph
14 Commits
Author SHA1 Message Date
yuanyuanxianganddeepseek-flash 4aedca8ee0 Feature: Support voice monitoring on Android hosts
The master's voice monitoring menu sends COMMAND_AUDIO; the client opens a
ConnAuth sub-connection with TOKEN_AUDIO_START, the server opens CAudioDlg
and answers COMMAND_NEXT, then audio flows both ways. The protocol matches
the Windows client, so the server needs no changes.

The server's ACM format was read from Audio.cpp and AudioDlg.cpp first:
WAVE_FORMAT_GSM610, 8000 Hz mono, 65-byte blocks of 320 samples (40 ms),
about 1625 B/s. libgsm 1.0.13 (AOSP external/libgsm mirror) is vendored
under cpp/gsm and built with -DSASR -DWAV49 so its MS-GSM packing is
byte-identical to ACM GSM610. Encode and decode use separate gsm states
because frame_index/frame_chain live in the state.

Uplink packets are always exactly 1000 bytes: CAudio::PlayBuffer copies
1000 bytes from the buffer start regardless of packet length, so a short
packet would over-read. Since 1000 is not a multiple of 65, blocks
straddle packets; the stream is continuous and codec state carries over.

AudioMonitor.kt owns AudioRecord (8 kHz/16-bit/mono, 320 samples a read)
and AudioTrack playback of decoded downlink PCM. RECORD_AUDIO is requested
with POST_NOTIFICATIONS before the projection dialog, and CaptureService
adds the microphone FGS type only when held, falling back on failure. An
8-second dialog wait or missing microphone reports a tip to the master
instead of failing silently.

Verified on an Android 15 emulator against the Debug server: dialog
opened, uplink started, server receive counter grew at ~1.6 KB/s.
Downlink playback was not exercised (the test machine has no recording
device); that path is code-reviewed only.

Co-Authored-By: deepseek-flash
2026-11-11 21:58:54 +01:00
yuanyuanxianganddeepseek-flash 3ab5154db1 Feature: Honor remote desktop tuning commands on Android hosts
The Android client dropped CMD_QUALITY_LEVEL, CMD_FPS, CMD_QUALITY_PROFILES
and CMD_SCREEN_TUNING as log lines, and hardcoded ScreenSettings.QualityLevel
to Good. Because the server initializes adaptive quality from the client's
reported level, that hardcoded value pinned every Android session to a fixed
profile: adaptive quality could never start.

ScreenTuning.kt now owns the state in SharedPreferences (level, max fps and
the master's six-row profile table), and C++ only parses the wire format.
The level is persisted when the master sets it explicitly, including -1
adaptive and -2 disabled, and is reported back in TOKEN_BITMAPINFO on the
next screen session. A concrete level switch applies the profile bitrate at
runtime via MediaCodec.setParameters(PARAMETER_KEY_VIDEO_BITRATE), which is
smooth and needs no encoder rebuild.

Frame rate is stored as-is (255 = unlimited, matching Windows) and only
reaches KEY_FRAME_RATE at the next capture start, since surface encoders
have no runtime fps cap. Ultra and High default to a zero bitrate because
their Windows algorithms are DIFF/RGB565, so Android falls back to the
previous 4000 kbps. CMD_SCREEN_TUNING stays log-only: all four flags belong
to the Windows capture/x264 path and Android does not claim
SCREEN_CAP_TUNING_FLAGS. CMD_SCREEN_SIZE is ignored; the server's pending
resolution-change mark self-heals after one second.

Verified on the emulator: a seeded Level 5 starts capture at 800 kbps/8 fps
and the server title shows Minimal; a master-side switch to Good changes the
live bitrate to 3000, and a restart picks up fps 30 from CMD_FPS.

Co-Authored-By: deepseek-flash
2026-11-11 21:58:54 +01:00
yuanyuanxianganddeepseek-flash a801a2999a Feature: Support input blocking and privacy overlay on Android hosts
remote_block_input rejected anything but Windows because the Android
client had no COMMAND_SCREEN_BLOCK_INPUT handler. This adds the overlay
on the client and the server gating, and wires up COMMAND_SCREEN_BLANK.

Client: the accessibility service hangs a full-screen
TYPE_ACCESSIBILITY_OVERLAY on itself. block=true makes it transparent
and touchable so it swallows physical taps; black=true paints it black.
Both are absolute states, so repeated commands are idempotent, and the
handler clears the overlay when the screen sub-connection dies (Android
has no "released when the calling thread exits" fallback). During a
remote gesture the overlay is flipped to FLAG_NOT_TOUCHABLE, mirroring
Windows' BlockInput(false) wrap; the flag change and the injection must
be separated by a short delay or the gesture is swallowed.

Server: CLIENT_CAP_PRIVACY_OVERLAY distinguishes new APKs from old ones;
Linux/macOS and old APKs still get -32005 up front. Tool masks,
instructions and mcp/tools.json follow the change. COMMAND_SCREEN_BLANK
is implemented but not exposed as a tool: MediaProjection captures the
overlay, so blacking out the screen blinds the operator too and cannot
emulate Windows' WDA_EXCLUDEFROMCAPTURE.

Verified on the Android 15 emulator: blocked taps are swallowed while
remote clicks still land, the block re-arms after each gesture, the
watchdog auto-unblocks at the deadline, a dropped sub-connection clears
the overlay, and a black overlay shows up in both screencap and
get_screenshot. The committed ghost.apk is stale; rebuild before release.

Co-Authored-By: deepseek-flash
2026-11-11 21:58:54 +01:00
yuanyuanxianganddeepseek-flash 915c587964 Feature: Support remote chat on Android hosts
The chat tools were Windows-only because the Android client had no
COMMAND_CHAT handler: the server asked for a chat sub-connection and the
command was silently dropped, leaving chat_open to time out. This adds
the full path on the client and the matching server gating.

Client: COMMAND_CHAT on the main connection spawns a chat thread that
opens an authenticated sub-connection, sends TOKEN_CHAT_START and waits
for the server to bind it. Incoming GBK lines are decoded and dispatched
to ChatSession, which drives an in-app chat activity; when accessibility
is available the activity is pulled to the front, otherwise a tappable
notification is posted. Outgoing text is strictly GBK encoded so emoji
and unmappable characters are rejected instead of becoming '?'. When the
server ends the session the activity closes itself, matching the Windows
client's DestroyWindow, and a new session starts from an empty log.

Server: a new CLIENT_CAP_REMOTE_CHAT bit lets the server tell new APKs
from old ones. ResolveHost now accepts Windows and Android hosts and
rejects older APKs up front with -32005 rather than waiting for the
command to expire. Tool platform masks, instructions, the docs matrix
and mcp/tools.json follow the same change.

Verified end to end on the Android 15 emulator: chat_open pops the
activity, Chinese text from the server renders, a device reply reaches
chat_wait, and ending the session from the master closes the activity
instead of leaving a dead window. The committed ghost.apk is stale and
must be rebuilt before release.

Co-Authored-By: deepseek-flash
2026-11-11 21:58:44 +01:00
yuanyuanxianganddeepseek-flash 29d7ee1a18 Feature: Support host uninstall on Android and expose uninstall_host
The server's "uninstall host" action is the single-byte TOKEN_UNINSTALL on
the main connection. Windows clients have handled it for a long time; the
Android client ignored it, so the menu entry silently did nothing there.

Handle it in the Android client the only way Android allows: ask the
system to uninstall the package. The client first replies with COMMAND_BYE
and drops the connection, then opens the package installer's confirmation
dialog through the accessibility service, or posts a notification that
resumes the dialog from MainActivity when accessibility is off. The user
still has to confirm on the device; if they cancel, the client has already
left the server and must be reopened by hand. The manifest gains
REQUEST_DELETE_PACKAGES, without which the installer activity refuses the
request outright.

New APKs advertise CLIENT_CAP_UNINSTALL so the server can reject old
clients instead of sending a command nobody handles. The new MCP tool
uninstall_host is gated on McpManage and McpReadonly=0, rejects Linux and
macOS hosts, and offers the same wait=true verification as power_action,
reporting went_offline once the host disappears from the online list.

Verified on the emulator end to end: the tool returned went_offline after
200 ms, the system dialog appeared, confirming it removed the package.

Co-Authored-By: deepseek-flash
2026-11-11 20:59:10 +01:00
yuanyuanxianganddeepseek-flash fd1b085279 Feature: Add remote lock screen for Android hosts
Implement the Ctrl+Alt+Del path on Android as a system lock screen.
Android has no secure attention sequence, so COMMAND_SCREEN_CTRL_ALT_DEL
is mapped in the client to GLOBAL_ACTION_LOCK_SCREEN and the server
exposes it as the new MCP tool remote_lock_screen.

The Android client resolves ControlService.lockScreen()/canLockScreen()
in nativeInit and dispatches the command from the screen sub-connection
through the usual attach/detach JNI path. The lock action is posted to
the accessibility service handler because performGlobalAction must not
run on the receiver thread.

Login advertises the new CLIENT_CAP_LOCK_SCREEN bit only when the
accessibility service is bound and the API level is 28+, so the server
can reject old APKs and Windows/Linux hosts up front instead of
reporting a lock that never happened. There is no receipt: a successful
call only means the command was delivered, and the tool description
tells the caller to confirm with get_screenshot.

Server side adds the schema and handler next to remote_clipboard, audits
the call, and gates it on client type plus the capability bit. Since it
is the only APK-only tool, check_platform_table.py gains a documented
exemption from the Windows baseline invariant, and make_tools_json.py
learns an APK-only platform set. tools.json is regenerated from the live
server and the plan document now counts 62 tools.

Co-Authored-By: deepseek-flash
2026-11-11 20:59:10 +01:00
yuanyuanxianganddeepseek-flash 0a0b709e8a Feature: Support remote clipboard and input on Android hosts
The MCP remote-control family (remote_open / remote_close / remote_mouse /
remote_keyboard / remote_clipboard) previously stopped at Windows and Linux,
even though the Android client already translated injected events into touch
gestures and global actions. This wires up the missing piece and opens the five
tools to APK hosts.

Client side, the screen sub-connection now handles COMMAND_SCREEN_SET_CLIPBOARD:
the native handler forwards the raw GBK bytes to CaptureService.onClipboardText,
which decodes them, strips the trailing NUL that the human clipboard-sync
senders append, and writes the system clipboard from a background app (allowed
on Android 10+, only reads are restricted). Pasting cannot be a Ctrl+V combo,
so VK_V is mapped to ACTION_PASTE on the focused node, falling back to the
accessibility focus and then to the first editable node, since WebView and
self-drawn inputs often do not flag FOCUS_INPUT or isEditable.

A new capability bit CLIENT_CAP_SCREEN_CLIPBOARD (0x0800) tells the server the
APK understands both paths, so builds without the handler get an explicit
-32005 instead of a package that is silently dropped. action=type is rejected
outright for APK hosts - physical key events would only turn 'v' into a paste -
and the error points at remote_clipboard + key=V.

The platform table, server instructions and the five tool descriptions now
describe Android support, which keys actually act as global actions (ESC, HOME,
APPS, PRINTSCREEN), and that injection needs the accessibility service while
capture needs an authorized MediaProjection session, both without receipts.

Verified on the API 35 emulator against a Debug server: ASCII and GBK Chinese
text round-tripped through the clipboard into a focused Settings search field,
type / expect_hwnd / window_hwnd returned -32005, and PRINTSCREEN produced a
saved screenshot. check_platform_table.py and verify_tools.py --offline pass.

Co-Authored-By: deepseek-flash
2026-11-11 20:59:10 +01:00
yuanyuanxianganddeepseek-flash f29150d20e Feature: Add screen preview to the Android client
get_screenshot and the server's online-list and hover thumbnails were
gated to Windows and X11 Linux. The Android client now answers
COMMAND_SCREEN_PREVIEW_REQ on the main connection with
TOKEN_SCREEN_PREVIEW_RSP plus a JPEG, and advertises
CLIENT_CAP_SCREEN_PREVIEW at login only when it can really capture.

Capture goes through AccessibilityService.takeScreenshot (API 30+),
enabled by canTakeScreenshot in the service config. MediaProjection is
deliberately not reused: its VirtualDisplay already feeds the H.264
encoder and Android 14+ refuses a second one from the same projection,
while accessibility capture runs alongside the live stream. The JPEG is
scaled to maxWidth and encoded in Kotlin; native only copies the bytes
and drops a request while one is in flight. Only mode=screen is
supported, and capabilities are re-evaluated on every login so turning
the service off withdraws the capability.

Also fix two review findings: a screenshot callback that lands after
the 3 s timeout now recycles its bitmap instead of leaking it, and the
server's failure hint lists the possible Android causes instead of
asserting disabled accessibility.

Verified on the emulator against a local debug server: get_screenshot
returns 640x1422 and 720x1600 JPEGs, the 120-wide list thumbnail works,
and both MCP tool-table checkers pass with apk added for get_screenshot,
get_activity_history and get_client_log.

Co-Authored-By: deepseek-flash
2026-11-11 20:58:40 +01:00
yuanyuanxianganddeepseek-flash 5ed1a07dc4 Feature: Add activity history to the Android client
get_activity_history was gated to Windows and Linux. The Android client
now answers COMMAND_QUERY_ACTIVITY with a TOKEN_REPORT_ACTIVITY snapshot
on the main connection, the same RPC shape as the other platforms; the
server takes the first packet without CancelIO, so no sub-connection or
request correlation is involved.

The sampler reuses linux/LinuxActivityHistory.h, whose ActivityHistory
class and formatters are platform-neutral, and is fed by a probe that
asks the new ControlService.currentActivityTitle() once per second over
JNI. The accessibility service already tracks the foreground package
from TYPE_WINDOW_STATE_CHANGED; the title is "label (package)" with the
label resolved once and cached, "Home" for the launcher, and idle for
lock or screen-off, so no permission beyond the service users already
enable is needed. Without the service the report says so instead of
recording placeholder "Android" entries.

Also scope the connection teardown to its own generation: a
ConnectionThread that exits late (CaptureService restart) no longer
clears the new run's g_running or g_pMainClient, and the report thread
takes one local snapshot of the main client pointer.

Verified on the emulator against a local debug server: five records
(Settings, Chrome, launcher, Home) came back newest-first with dwell
times, and disabling accessibility produced the permission hint.

Co-Authored-By: deepseek-flash
2026-11-11 13:14:46 +01:00
yuanyuanxianganddeepseek-flash 36618b0ad0 Feature: Install Android APK updates pushed from the master
The master now sends COMMAND_UPDATE to Android hosts: it picks a
"Build Client -> Android" APK, checks the ZIP magic and that the package
carries this master's fingerprint, then sends [cmd][size][APK][xxh64].
The client verifies length and hash, writes filesDir/update.apk and
commits a PackageInstaller session, so the system installer still asks
the user to confirm - an app cannot replace itself silently. Write and
hash failures come back over TOKEN_CLIENT_MSG, so the master sees why
nothing happened instead of a silently dropped packet.

Sending the wrong file is what the fingerprint check guards against: a
template without the master id installs fine and then cannot connect
back. "Send anyway" is offered only after spelling that out, and the
client advertises CLIENT_CAP_APK_UPDATE (0x0400) so an old APK gets
"install a new build once by hand" instead of a packet it would discard.

Also report the real versionName/versionCode at login instead of the
hardcoded v1.0.0, add android/tools/verify_update_apk.py (pre-send check
of size, ZIP_STORED libs, ABI, package, versionCode and the decrypted
CONNECT_ADDRESS, which it still locates after BuildDlg rewrote szFlag),
and translate the nine new strings in en_US, zh_TW and de_DE.

Verified on the emulator and against the live release server: generate,
sign, send, install, and the device reconnects and re-grants screen
capture. Language packs are GBK and were appended as bytes.

Co-Authored-By: deepseek-flash
2026-11-11 00:53:29 +01:00
yuanyuanxiangandClaude Sonnet 4.6 5296e534ed Fix: Android client stops service when server sends BYE (delete client)
ConnectionThread now calls CaptureService.onNativeExit() via JNI on exit
when g_bExit == S_CLIENT_EXIT (server-initiated). The Java side posts
stopSelf() on the service handler. Guard: instance is null in the
user-initiated teardown path (onDestroy nulls it before nativeStop),
so onNativeExit is a no-op in that case.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-25 17:38:01 +02:00
yuanyuanxiangandClaude Sonnet 4.6 5a1430e904 Feat: Android client device grouping with persistent group name
- szPCName sent as "model/groupName" format matching Linux client
- CMD_SET_GROUP handler updates g_SETTINGS.szGroupName in memory
- Group name persisted to filesDir/yama_group; survives process restart
- LoadGroupName() at startup overrides build-time patch value if file exists

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-25 17:38:01 +02:00
yuanyuanxiang 218ee4f43d Feat: TV remote control via D-pad focus navigation using AccessibilityService 2026-06-24 22:31:14 +02:00
yuanyuanxiang 45553ec5b6 Feat: Android client Phase 0-3 full implementation 2026-06-21 23:00:05 +02:00