Feature: Honor remote desktop tuning commands on Android hosts

The Android client dropped CMD_QUALITY_LEVEL, CMD_FPS, CMD_QUALITY_PROFILES
and CMD_SCREEN_TUNING as log lines, and hardcoded ScreenSettings.QualityLevel
to Good. Because the server initializes adaptive quality from the client's
reported level, that hardcoded value pinned every Android session to a fixed
profile: adaptive quality could never start.

ScreenTuning.kt now owns the state in SharedPreferences (level, max fps and
the master's six-row profile table), and C++ only parses the wire format.
The level is persisted when the master sets it explicitly, including -1
adaptive and -2 disabled, and is reported back in TOKEN_BITMAPINFO on the
next screen session. A concrete level switch applies the profile bitrate at
runtime via MediaCodec.setParameters(PARAMETER_KEY_VIDEO_BITRATE), which is
smooth and needs no encoder rebuild.

Frame rate is stored as-is (255 = unlimited, matching Windows) and only
reaches KEY_FRAME_RATE at the next capture start, since surface encoders
have no runtime fps cap. Ultra and High default to a zero bitrate because
their Windows algorithms are DIFF/RGB565, so Android falls back to the
previous 4000 kbps. CMD_SCREEN_TUNING stays log-only: all four flags belong
to the Windows capture/x264 path and Android does not claim
SCREEN_CAP_TUNING_FLAGS. CMD_SCREEN_SIZE is ignored; the server's pending
resolution-change mark self-heals after one second.

Verified on the emulator: a seeded Level 5 starts capture at 800 kbps/8 fps
and the server title shows Minimal; a master-side switch to Good changes the
live bitrate to 3000, and a restart picks up fps 30 from CMD_FPS.

Co-Authored-By: deepseek-flash
This commit is contained in:
yuanyuanxianganddeepseek-flash committed 2026-11-11 21:58:54 +01:00
1 parent a801a2999a
commit 3ab5154db1
6 files changed
+444 -12

No files matched your search

+64 -4
View File
@@ -11,6 +11,7 @@
#include <vector> #include <vector>
#include <thread> #include <thread>
#include <cstring> #include <cstring>
#include <string>
#include <cinttypes> #include <cinttypes>
#include <android/log.h> #include <android/log.h>
@@ -26,6 +27,12 @@ extern void DispatchClipboardText(const uint8_t* data, size_t size);
extern void DispatchLockScreen(); extern void DispatchLockScreen();
// 隐私 overlay(COMMAND_SCREEN_BLANK / COMMAND_SCREEN_BLOCK_INPUT),定义在 main.cpp // 隐私 overlay(COMMAND_SCREEN_BLANK / COMMAND_SCREEN_BLOCK_INPUT),定义在 main.cpp
extern void DispatchPrivacyOverlay(bool black, bool block); extern void DispatchPrivacyOverlay(bool black, bool block);
// 画质/帧率协商:读写 ScreenTuning(SharedPreferences),定义在 main.cpp
extern int GetClientQualityLevel(); // 上报用,读不到时回落 QUALITY_GOOD
extern int GetClientMaxFps(); // 上报用,读不到时回落 20
extern void DispatchQualityLevel(int level, bool persist);
extern void DispatchMaxFps(int fps);
extern void DispatchQualityProfilesCsv(const char* csv);
// Linux/macOS 共用的 BITMAPINFOHEADER 布局(与 Windows 完全一致) // Linux/macOS 共用的 BITMAPINFOHEADER 布局(与 Windows 完全一致)
#pragma pack(push, 1) #pragma pack(push, 1)
@@ -95,10 +102,13 @@ public:
memcpy(&buf[off + 8], &zero, 8); memcpy(&buf[off + 8], &zero, 8);
ScreenSettings ss = {}; ScreenSettings ss = {};
ss.MaxFPS = 10; ss.MaxFPS = GetClientMaxFps();
ss.ScreenWidth = m_width; ss.ScreenWidth = m_width;
ss.ScreenHeight = m_height; ss.ScreenHeight = m_height;
ss.QualityLevel = QUALITY_GOOD; // 告知服务端使用 H264 解码器 // 上报主控端上次为本机选择的档位(含 QUALITY_ADAPTIVE)。写死成 Good 会让服务端
// 认为这是「固定档位会话」,m_AdaptiveQuality.enabled 永远为 false——自适应质量
// 在 Android 主机上根本不会启动。具体编码参数由 ScreenTuning 在 Kotlin 侧应用。
ss.QualityLevel = GetClientQualityLevel();
ss.ScreenType = USING_VIRTUAL; // 虚拟显示 ss.ScreenType = USING_VIRTUAL; // 虚拟显示
memcpy(&buf[off + 16], &ss, sizeof(ss)); memcpy(&buf[off + 16], &ss, sizeof(ss));
@@ -132,8 +142,58 @@ public:
m_started = true; m_started = true;
m_cond.notify_all(); m_cond.notify_all();
break; break;
case CMD_QUALITY_LEVEL: case CMD_QUALITY_LEVEL: {
if (size >= 2) LOGI_SH("QualityLevel=%d", (int)(int8_t)data[1]); // [cmd][level:int8][persist:1]。level 有符号(-2 关闭 / -1 自适应 / 0-5 具体档),
// persist=1 表示这是主控端选定的档位、要落盘(老主控端的负载只有 4 字节)。
if (size >= 2) {
const int level = (int)(int8_t)data[1];
const bool persist = size >= 3 && data[2] != 0;
DispatchQualityLevel(level, persist);
}
break;
}
case CMD_FPS:
// [cmd][fps:1],255 = 不限帧。帧率只能写进 MediaCodec 的 KEY_FRAME_RATE
// (码控提示),surface 编码器没有运行时限帧开关,所以下次开抓才生效。
if (size >= 2) DispatchMaxFps((int)(uint8_t)data[1]);
break;
case CMD_QUALITY_PROFILES: {
// [cmd][QualityProfile × QUALITY_COUNT](每档 16 字节)。存下来供开抓与
// 档位切换时按主控端自定义的码率表取参数(主控端权威,见 ScreenSpyDlg)。
const size_t need = 1 + sizeof(QualityProfile) * QUALITY_COUNT;
if ((size_t)size < need) {
LOGE_SH("QUALITY_PROFILES short: %u < %u", (unsigned)size, (unsigned)need);
break;
}
std::string csv;
for (int i = 0; i < QUALITY_COUNT; i++) {
QualityProfile p;
memcpy(&p, data + 1 + i * sizeof(QualityProfile), sizeof(p));
if (i) csv += ';';
csv += std::to_string(p.maxFPS) + ',' + std::to_string(p.maxWidth) + ',' +
std::to_string(p.algorithm) + ',' + std::to_string(p.bitRate);
}
DispatchQualityProfilesCsv(csv.c_str());
break;
}
case CMD_SCREEN_TUNING: {
// [cmd][flags:4][version:1]。Android 抓屏是 MediaProjection + MediaCodec(H264),
// 四个开关(x264 VBV / GDI 去延时 / 发送节流 / 自适应判据)都没有对应实现,
// 因此只记日志、不落盘,也不声明 SCREEN_CAP_TUNING_FLAGS。
uint32_t flags = 0;
if (size >= 5) memcpy(&flags, data + 1, 4);
const int ver = size >= 6 ? (int)data[5] : -1;
LOGI_SH("SCREEN_TUNING flags=0x%X ver=%d (no-op on Android)", flags, ver);
break;
}
case CMD_SCREEN_SIZE:
// [cmd][strategy][maxWidth:int]。自适应质量要求缩放编码宽度;Android 源是
// 整屏抓取、尺寸在开抓时固定(改尺寸需重建 MediaCodec),故忽略。
// 服务端会按「客户端可能重启」先置分辨率切换标记,1 秒后超时自愈
// (CheckResolutionChangeTimeout:刷新解码器 + 请求 IDR),表现为切档后
// 画面停约 1 秒。回发 TOKEN_BITMAPINFO 能立刻清标记,但会连带 PrepareDrawing、
// 通知 web 端重建解码器,比 1 秒停顿更扰动,因此不这么做。
LOGI_SH("SCREEN_SIZE ignored on Android (%u bytes)", (unsigned)size);
break; break;
case COMMAND_SCREEN_SET_CLIPBOARD: case COMMAND_SCREEN_SET_CLIPBOARD:
// 服务端写剪贴板: [cmd:1][GBK text:N](与 Windows CF_TEXT 同编码;人类剪贴板 // 服务端写剪贴板: [cmd:1][GBK text:N](与 Windows CF_TEXT 同编码;人类剪贴板
+134
View File
@@ -250,6 +250,13 @@ static jmethodID g_nativeExitMid = nullptr; // CaptureService.onNativeEx
static jmethodID g_updateReceivedMid = nullptr; // CaptureService.onUpdateReceived(String) — 升级 APK 已落盘 static jmethodID g_updateReceivedMid = nullptr; // CaptureService.onUpdateReceived(String) — 升级 APK 已落盘
static jmethodID g_clipboardMid = nullptr; // CaptureService.onClipboardText([B)V — 远程剪贴板 static jmethodID g_clipboardMid = nullptr; // CaptureService.onClipboardText([B)V — 远程剪贴板
static jmethodID g_uninstallMid = nullptr; // CaptureService.onUninstallRequested()V — TOKEN_UNINSTALL static jmethodID g_uninstallMid = nullptr; // CaptureService.onUninstallRequested()V — TOKEN_UNINSTALL
// ScreenTuning(远程桌面画质/帧率协商:CMD_QUALITY_LEVEL / CMD_FPS / CMD_QUALITY_PROFILES)
static jclass g_tuningClass = nullptr;
static jmethodID g_getQualityLevelMid = nullptr; // ScreenTuning.getQualityLevel()I — 上报 ScreenSettings
static jmethodID g_getMaxFpsMid = nullptr; // ScreenTuning.getMaxFps()I
static jmethodID g_onQualityLevelMid = nullptr; // ScreenTuning.onQualityLevel(IZ)V
static jmethodID g_onFpsMid = nullptr; // ScreenTuning.onFps(I)V
static jmethodID g_onQualityProfilesMid = nullptr; // ScreenTuning.onQualityProfiles(String)V
// ChatSession(远程交谈界面:帧入站文本、发送按钮) // ChatSession(远程交谈界面:帧入站文本、发送按钮)
static jclass g_chatClass = nullptr; static jclass g_chatClass = nullptr;
static jmethodID g_chatEventMid = nullptr; // ChatSession.onNativeEvent(I[B)V static jmethodID g_chatEventMid = nullptr; // ChatSession.onNativeEvent(I[B)V
@@ -431,6 +438,110 @@ void DispatchPrivacyOverlay(bool black, bool block)
if (attached) g_jvm->DetachCurrentThread(); if (attached) g_jvm->DetachCurrentThread();
} }
// ── 远程桌面画质/帧率协商 ────────────────────────────────────────────────────
// 服务端在屏幕子连接里下发 CMD_QUALITY_LEVEL / CMD_FPS / CMD_QUALITY_PROFILES,
// 客户端把档位/码率表/帧率存进 SharedPreferences,下次建立子连接时随 ScreenSettings
// 上报(否则写死的 Good 会让服务端的自适应质量永远不启用)。C++ 只做协议解析,
// 档位到编码器的落点在 Kotlin(MediaCodec 在那一侧)。
// attach 当前线程取 JNIEnv(子连接接收线程通常处于 detached 状态);失败返回 nullptr。
static JNIEnv* AttachForJni(bool& attached)
{
attached = false;
if (!g_jvm) return nullptr;
JNIEnv* jenv = nullptr;
jint st = g_jvm->GetEnv((void**)&jenv, JNI_VERSION_1_6);
if (st == JNI_EDETACHED) {
if (g_jvm->AttachCurrentThread(&jenv, nullptr) != JNI_OK || !jenv) return nullptr;
attached = true;
} else if (st != JNI_OK || !jenv) {
return nullptr;
}
return jenv;
}
// 屏幕子连接建立时填 ScreenSettings 用;读不到(服务未起)时按 Good/20 兜底。
int GetClientQualityLevel()
{
if (!g_tuningClass || !g_getQualityLevelMid) return QUALITY_GOOD;
bool attached = false;
JNIEnv* jenv = AttachForJni(attached);
if (!jenv) return QUALITY_GOOD;
jint v = jenv->CallStaticIntMethod(g_tuningClass, g_getQualityLevelMid);
if (jenv->ExceptionCheck()) { jenv->ExceptionClear(); v = QUALITY_GOOD; }
if (attached) g_jvm->DetachCurrentThread();
return (int)v;
}
int GetClientMaxFps()
{
if (!g_tuningClass || !g_getMaxFpsMid) return 20;
bool attached = false;
JNIEnv* jenv = AttachForJni(attached);
if (!jenv) return 20;
jint v = jenv->CallStaticIntMethod(g_tuningClass, g_getMaxFpsMid);
if (jenv->ExceptionCheck()) { jenv->ExceptionClear(); v = 20; }
if (attached) g_jvm->DetachCurrentThread();
return (int)v;
}
// CMD_QUALITY_LEVEL:level 有符号(-2 关闭 / -1 自适应 / 0-5 具体档位),persist 表示落盘。
void DispatchQualityLevel(int level, bool persist)
{
if (!g_tuningClass || !g_onQualityLevelMid) {
LOGE("QUALITY: JNI not ready (cls=%p method=%p)", g_tuningClass, g_onQualityLevelMid);
return;
}
bool attached = false;
JNIEnv* jenv = AttachForJni(attached);
if (!jenv) return;
jenv->CallStaticVoidMethod(g_tuningClass, g_onQualityLevelMid,
(jint)level, persist ? JNI_TRUE : JNI_FALSE);
if (jenv->ExceptionCheck()) { jenv->ExceptionClear(); LOGE("QUALITY: onQualityLevel threw"); }
LOGI("QUALITY: level=%d persist=%d", level, (int)persist);
if (attached) g_jvm->DetachCurrentThread();
}
// CMD_FPS:byte[1] = 最大帧率(255 = 不限)。帧率只在下次开抓时生效(见 ScreenTuning 注释)。
void DispatchMaxFps(int fps)
{
if (!g_tuningClass || !g_onFpsMid) {
LOGE("QUALITY: JNI not ready (cls=%p method=%p)", g_tuningClass, g_onFpsMid);
return;
}
bool attached = false;
JNIEnv* jenv = AttachForJni(attached);
if (!jenv) return;
jenv->CallStaticVoidMethod(g_tuningClass, g_onFpsMid, (jint)fps);
if (jenv->ExceptionCheck()) { jenv->ExceptionClear(); LOGE("QUALITY: onFps threw"); }
LOGI("QUALITY: maxFps=%d", fps);
if (attached) g_jvm->DetachCurrentThread();
}
// CMD_QUALITY_PROFILES:6 档 × 4 字段,C++ 侧拼成 "fps,width,algo,bitrate;..." 交 Kotlin 解析落盘。
void DispatchQualityProfilesCsv(const char* csv)
{
if (!g_tuningClass || !g_onQualityProfilesMid) {
LOGE("QUALITY: JNI not ready (cls=%p method=%p)", g_tuningClass, g_onQualityProfilesMid);
return;
}
bool attached = false;
JNIEnv* jenv = AttachForJni(attached);
if (!jenv) return;
jstring s = jenv->NewStringUTF(csv);
if (!s) {
if (jenv->ExceptionCheck()) jenv->ExceptionClear();
LOGE("QUALITY: NewStringUTF failed");
if (attached) g_jvm->DetachCurrentThread();
return;
}
jenv->CallStaticVoidMethod(g_tuningClass, g_onQualityProfilesMid, s);
if (jenv->ExceptionCheck()) { jenv->ExceptionClear(); LOGE("QUALITY: onQualityProfiles threw"); }
jenv->DeleteLocalRef(s);
LOGI("QUALITY: profiles %s", csv);
if (attached) g_jvm->DetachCurrentThread();
}
static std::string GetActiveWindowFromJava() static std::string GetActiveWindowFromJava()
{ {
if (!g_jvm || !g_ctrlClass || !g_getActiveWindowMid) return "Android"; if (!g_jvm || !g_ctrlClass || !g_getActiveWindowMid) return "Android";
@@ -1777,6 +1888,29 @@ Java_com_yama_client_YamaBridge_nativeInit(
} else { } else {
LOGE("ChatSession class not found"); LOGE("ChatSession class not found");
} }
// 远程桌面画质/帧率协商:ScreenTuning 的读写入口(见本文件上方的说明)
jclass tuningCls = env->FindClass("com/yama/client/ScreenTuning");
if (env->ExceptionCheck()) { env->ExceptionClear(); tuningCls = nullptr; }
if (tuningCls) {
g_tuningClass = (jclass)env->NewGlobalRef(tuningCls);
g_getQualityLevelMid = env->GetStaticMethodID(g_tuningClass, "getQualityLevel", "()I");
if (env->ExceptionCheck()) { env->ExceptionClear(); g_getQualityLevelMid = nullptr; }
if (!g_getQualityLevelMid) LOGE("ScreenTuning.getQualityLevel not found");
g_getMaxFpsMid = env->GetStaticMethodID(g_tuningClass, "getMaxFps", "()I");
if (env->ExceptionCheck()) { env->ExceptionClear(); g_getMaxFpsMid = nullptr; }
if (!g_getMaxFpsMid) LOGE("ScreenTuning.getMaxFps not found");
g_onQualityLevelMid = env->GetStaticMethodID(g_tuningClass, "onQualityLevel", "(IZ)V");
if (env->ExceptionCheck()) { env->ExceptionClear(); g_onQualityLevelMid = nullptr; }
if (!g_onQualityLevelMid) LOGE("ScreenTuning.onQualityLevel not found");
g_onFpsMid = env->GetStaticMethodID(g_tuningClass, "onFps", "(I)V");
if (env->ExceptionCheck()) { env->ExceptionClear(); g_onFpsMid = nullptr; }
if (!g_onFpsMid) LOGE("ScreenTuning.onFps not found");
g_onQualityProfilesMid = env->GetStaticMethodID(g_tuningClass, "onQualityProfiles", "(Ljava/lang/String;)V");
if (env->ExceptionCheck()) { env->ExceptionClear(); g_onQualityProfilesMid = nullptr; }
if (!g_onQualityProfilesMid) LOGE("ScreenTuning.onQualityProfiles not found");
} else {
LOGE("ScreenTuning class not found");
}
} }
} }
@@ -121,6 +121,27 @@ class CaptureService : Service() {
} }
} }
/**
* 服务端 CMD_QUALITY_LEVEL 的落点:按档位配置表换码率。setParameters 是运行时
* 平滑生效,不需要重建编码器(重建会丢一个 IDR,画面会闪一下)。
* 调用方是屏幕子连接接收线程,编码器只能在 idrHandler 的线程上碰。
*/
@JvmStatic
fun applyBitRate(kbps: Int) {
val svc = instance ?: return
svc.idrHandler.post {
if (kbps <= 0 || kbps == svc.currentBitRateKbps) return@post
val codec = svc.mediaCodec ?: return@post // 还没开抓:开抓时会读到新档位
runCatching {
val p = Bundle()
p.putInt(MediaCodec.PARAMETER_KEY_VIDEO_BITRATE, kbps * 1000)
codec.setParameters(p)
svc.currentBitRateKbps = kbps
Log.i(TAG, "bitrate -> $kbps kbps")
}.onFailure { Log.e(TAG, "bitrate change to $kbps failed: $it") }
}
}
/** /**
* 静止屏幕黑屏修复:SurfaceFlinger 在无脏区时不向 VirtualDisplay 推帧, * 静止屏幕黑屏修复:SurfaceFlinger 在无脏区时不向 VirtualDisplay 推帧,
* 导致编码器无输入,REQUEST_SYNC_FRAME 悬空排队,浏览器等 10-60 秒才见画面。 * 导致编码器无输入,REQUEST_SYNC_FRAME 悬空排队,浏览器等 10-60 秒才见画面。
@@ -150,6 +171,8 @@ class CaptureService : Service() {
private var mediaProjection: MediaProjection? = null private var mediaProjection: MediaProjection? = null
private var virtualDisplay: VirtualDisplay? = null private var virtualDisplay: VirtualDisplay? = null
private var mediaCodec: MediaCodec? = null private var mediaCodec: MediaCodec? = null
// 当前编码码率(kbps):CMD_QUALITY_LEVEL 切换档位时只在真的变化时才 setParameters
private var currentBitRateKbps = 0
private val idrHandler = Handler(Looper.getMainLooper()) private val idrHandler = Handler(Looper.getMainLooper())
// CS-04 fix: runCatching 包住 setParameters,防止 codec 已停止时抛异常 // CS-04 fix: runCatching 包住 setParameters,防止 codec 已停止时抛异常
@@ -269,10 +292,14 @@ class CaptureService : Service() {
val (encW, encH) = scaleDown(physW, physH, MAX_LONG_SIDE) val (encW, encH) = scaleDown(physW, physH, MAX_LONG_SIDE)
val dpi = resources.displayMetrics.densityDpi val dpi = resources.displayMetrics.densityDpi
Log.i(TAG, "Capture: phys=${physW}x${physH} enc=${encW}x${encH} dpi=$dpi") // 主控端上次保存的档位决定开抓码率;自适应档从 Good 起步(与服务端假设一致)
val (bitRateKbps, fps) = ScreenTuning.startupRate()
Log.i(TAG, "Capture: phys=${physW}x${physH} enc=${encW}x${encH} dpi=$dpi " +
"bitrate=${bitRateKbps}kbps fps=$fps level=${ScreenTuning.getQualityLevel()}")
val codec = MediaCodec.createEncoderByType(MediaFormat.MIMETYPE_VIDEO_AVC) val codec = MediaCodec.createEncoderByType(MediaFormat.MIMETYPE_VIDEO_AVC)
val surface = configureEncoder(codec, encW, encH) val surface = configureEncoder(codec, encW, encH, bitRateKbps, fps)
currentBitRateKbps = bitRateKbps
var configData: ByteArray? = null var configData: ByteArray? = null
@@ -351,15 +378,16 @@ class CaptureService : Service() {
Log.i(TAG, "MediaCodec + VirtualDisplay started ${encW}x${encH}") Log.i(TAG, "MediaCodec + VirtualDisplay started ${encW}x${encH}")
} }
private fun buildFmt(w: Int, h: Int, lowLatency: Boolean, baseline: Boolean) = private fun buildFmt(w: Int, h: Int, lowLatency: Boolean, baseline: Boolean,
bitRateKbps: Int, fps: Int) =
MediaFormat.createVideoFormat(MediaFormat.MIMETYPE_VIDEO_AVC, w, h).apply { MediaFormat.createVideoFormat(MediaFormat.MIMETYPE_VIDEO_AVC, w, h).apply {
setInteger(MediaFormat.KEY_COLOR_FORMAT, setInteger(MediaFormat.KEY_COLOR_FORMAT,
MediaCodecInfo.CodecCapabilities.COLOR_FormatSurface) MediaCodecInfo.CodecCapabilities.COLOR_FormatSurface)
setInteger(MediaFormat.KEY_BIT_RATE, 4_000_000) setInteger(MediaFormat.KEY_BIT_RATE, bitRateKbps * 1000)
setInteger(MediaFormat.KEY_FRAME_RATE, 30) setInteger(MediaFormat.KEY_FRAME_RATE, fps)
setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 2) setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 2)
setInteger(MediaFormat.KEY_PRIORITY, 0) setInteger(MediaFormat.KEY_PRIORITY, 0)
setInteger(MediaFormat.KEY_OPERATING_RATE, 30) setInteger(MediaFormat.KEY_OPERATING_RATE, fps)
if (lowLatency) { if (lowLatency) {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.N) if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.N)
setInteger(MediaFormat.KEY_LATENCY, 0) setInteger(MediaFormat.KEY_LATENCY, 0)
@@ -372,7 +400,8 @@ class CaptureService : Service() {
} }
// 逐级降级:低延迟+Baseline → 低延迟 → 默认,确保真机兼容 // 逐级降级:低延迟+Baseline → 低延迟 → 默认,确保真机兼容
private fun configureEncoder(codec: MediaCodec, w: Int, h: Int): android.view.Surface { private fun configureEncoder(codec: MediaCodec, w: Int, h: Int,
bitRateKbps: Int, fps: Int): android.view.Surface {
val configs = listOf( val configs = listOf(
true to true, true to true,
true to false, true to false,
@@ -380,7 +409,7 @@ class CaptureService : Service() {
) )
for ((ll, bl) in configs) { for ((ll, bl) in configs) {
try { try {
val fmt = buildFmt(w, h, ll, bl) val fmt = buildFmt(w, h, ll, bl, bitRateKbps, fps)
codec.configure(fmt, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE) codec.configure(fmt, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE)
Log.i(TAG, "Encoder configured: lowLatency=$ll baseline=$bl") Log.i(TAG, "Encoder configured: lowLatency=$ll baseline=$bl")
return codec.createInputSurface() return codec.createInputSurface()
@@ -0,0 +1,160 @@
package com.yama.client
import android.content.Context
import android.content.SharedPreferences
import android.util.Log
/**
* 远程桌面画质/帧率协商(CMD_QUALITY_LEVEL / CMD_FPS / CMD_QUALITY_PROFILES)。
*
* 之前这四个命令在 ScreenHandler 里只记一条日志:编码器永远 4Mbps/30fps,而且
* SendBitmapInfo 把 QualityLevel 写死成 Good——服务端用客户端上报的档位初始化
* 自适应状态,写死 Good 意味着「自适应质量」对 Android 主机永远不启用。
*
* 这里把主控端下发的东西存进 SharedPreferences:
* - 档位(persist=1 才落盘,与 Windows 客户端一致);
* - 质量配置表(主控端「高级 → 远程桌面质量等级」自定义的 6 档);
* - 最大帧率。
* 下次建立屏幕子连接时 C++ 侧通过 getQualityLevel()/getMaxFps() 读回并随
* ScreenSettings 上报,主控端据此恢复菜单勾选与自适应开关。
*
* 真正能落到 MediaCodec 的只有码率:档位切换时按配置表动态 setParameters
* (PARAMETER_KEY_VIDEO_BITRATE,平滑生效、不需要重建编码器)。帧率只能写进
* KEY_FRAME_RATE 的码控提示——Android 的 surface 编码器按画面内容出帧,没有
* 「限帧」的运行时开关,所以 maxFPS 要到下次开抓才带上。
*
* CMD_SCREEN_TUNING 的四个开关(GDI 去延时 / x264 VBV / 发送节流 / 自适应判据)
* 都是 Windows 抓屏与软编管线的,Android 没有对应实现,客户端不声明
* SCREEN_CAP_TUNING_FLAGS,服务端继续用主控端按设备保存的副本。
*/
object ScreenTuning {
const val TAG = "YAMA_TUNING"
// 与 common/commands.h 的 QualityLevel 常量一致
private const val QUALITY_DISABLED = -2
private const val QUALITY_ADAPTIVE = -1
private const val QUALITY_COUNT = 6
private const val QUALITY_GOOD = 2
// 关闭质量控制(-2)时使用「原有设置」:与本次改动前写死的编码参数一致
private const val LEGACY_BITRATE_KBPS = 4000
private const val LEGACY_FPS = 30
private const val PREFS = "screen_tuning"
private const val K_LEVEL = "quality_level"
private const val K_FPS = "max_fps"
private const val K_PROFILES = "profiles"
/** 默认档位表,必须与 common/commands.h GetQualityProfile 的内置默认一致(maxFPS/maxWidth/algorithm/bitRate) */
private val DEFAULT_PROFILES = intArrayOf(
25, 0, 1, 0, // Ultra
20, 0, 3, 0, // High
20, 1920, 2, 3000, // Good
15, 1600, 2, 2000, // Medium
12, 1280, 2, 1200, // Low
8, 1024, 2, 800, // Minimal
)
private fun prefs(): SharedPreferences? =
CaptureService.appContext?.getSharedPreferences(PREFS, Context.MODE_PRIVATE)
/** 建立屏幕子连接时上报给服务端(ScreenSettings.QualityLevel) */
@JvmStatic
fun getQualityLevel(): Int {
val v = prefs()?.getInt(K_LEVEL, QUALITY_GOOD) ?: QUALITY_GOOD
return if (v in 0 until QUALITY_COUNT || v == QUALITY_ADAPTIVE || v == QUALITY_DISABLED)
v else QUALITY_GOOD
}
/** 建立屏幕子连接时上报给服务端(ScreenSettings.MaxFPS) */
@JvmStatic
fun getMaxFps(): Int {
val stored = prefs()?.getInt(K_FPS, 0) ?: 0
if (stored > 0) return stored
val level = getQualityLevel()
if (level == QUALITY_DISABLED) return LEGACY_FPS // 与 startupRate 的编码参数保持一致
return profile(if (level in 0 until QUALITY_COUNT) level else QUALITY_GOOD)[0]
}
/**
* 开抓时的初始码率(kbps)与帧率。自适应从 Good 起步——服务端在
* m_Settings.QualityLevel == QUALITY_ADAPTIVE 时也是把 currentLevel 定成 Good。
*
* Android 一律编 H.264,而配置表里 Ultra/High 的默认算法是 DIFF/RGB565、bitRate=0
* (那是 Windows 的软编路径)。这两档没有码率可用,按改动前的 H.264 参数 4000kbps 起步;
* 主控端若在自定义表里给了码率则以它为准。
*/
fun startupRate(): Pair<Int, Int> {
val level = getQualityLevel()
val p = if (level in 0 until QUALITY_COUNT) profile(level) else profile(QUALITY_GOOD)
val kbps = if (level == QUALITY_DISABLED || p[3] <= 0) LEGACY_BITRATE_KBPS else p[3]
// CMD_FPS 的落点:主控端显式选过帧率就以它为准,否则按档位表的 maxFPS。
// 255=不限帧,编码器按 60 给码控提示(surface 编码器本来就按画面出帧)。
return Pair(kbps, getMaxFps().coerceIn(1, 60))
}
/** 服务端 CMD_QUALITY_LEVEL:level 有符号;persist=1 表示这是主控端选择的档位,要落盘 */
@JvmStatic
fun onQualityLevel(level: Int, persist: Boolean) {
if (persist) {
prefs()?.edit()?.putInt(K_LEVEL, level)?.apply()
Log.i(TAG, "quality level persisted: $level")
}
if (level in 0 until QUALITY_COUNT) {
val p = profile(level)
val kbps = if (p[3] > 0) p[3] else LEGACY_BITRATE_KBPS // Ultra/High 无 H.264 码率
Log.i(TAG, "quality level $level -> bitrate=${kbps}kbps fps=${p[0]}")
CaptureService.applyBitRate(kbps)
} else {
// -1 自适应:具体档位由服务端后续按网络状况下发;-2 关闭:不动原有设置
Log.i(TAG, "quality level mode: $level (rate unchanged)")
}
}
/** 服务端 CMD_FPS:255 = 不限帧;帧率只能在下次开抓时生效(见类注释)。原样保存,与 Windows 一致 */
@JvmStatic
fun onFps(fps: Int) {
val v = fps.coerceIn(1, 255)
prefs()?.edit()?.putInt(K_FPS, v)?.apply()
Log.i(TAG, "max fps -> $v (takes effect at next capture start)")
}
/** 服务端 CMD_QUALITY_PROFILES:6 档 × 4 字段,分号分隔档、逗号分隔字段 */
@JvmStatic
fun onQualityProfiles(csv: String) {
val rows = csv.split(';')
if (rows.size != QUALITY_COUNT) {
Log.e(TAG, "quality profiles: bad payload '${csv.take(64)}'")
return
}
val values = IntArray(QUALITY_COUNT * 4)
for ((i, row) in rows.withIndex()) {
val f = row.split(',')
if (f.size != 4) {
Log.e(TAG, "quality profiles: bad row $i")
return
}
for (j in 0 until 4) {
val n = f[j].toIntOrNull()
if (n == null || n < 0) {
Log.e(TAG, "quality profiles: bad number in row $i")
return
}
values[i * 4 + j] = n
}
}
prefs()?.edit()?.putString(K_PROFILES, values.joinToString(","))?.apply()
Log.i(TAG, "quality profiles stored: $csv")
}
/** 取某一档的 maxFPS/maxWidth/algorithm/bitRate(档位越界时按最近的合法档) */
private fun profile(level: Int): IntArray {
val raw = prefs()?.getString(K_PROFILES, null)
val table = if (raw != null) {
val v = raw.split(',').mapNotNull { it.toIntOrNull() }
if (v.size == QUALITY_COUNT * 4) v.toIntArray() else DEFAULT_PROFILES
} else DEFAULT_PROFILES
val base = level.coerceIn(0, QUALITY_COUNT - 1) * 4
return intArrayOf(table[base], table[base + 1], table[base + 2], table[base + 3])
}
}
+15
View File
@@ -270,6 +270,21 @@ MediaProjection 抓的是合成后的帧缓冲,无障碍 overlay 无论是否
**不建议合并**:两族走不同子连接、不同能力位、不同门控,合并会破坏既有安全模型(`McpMachine` 开关的意义就是「单独关掉一套」)。 **不建议合并**:两族走不同子连接、不同能力位、不同门控,合并会破坏既有安全模型(`McpMachine` 开关的意义就是「单独关掉一套」)。
### 3.3 Android 主机的远程桌面调参(`CMD_QUALITY_LEVEL` / `CMD_FPS` / `CMD_QUALITY_PROFILES`)
这一项不新增工具,修的是**Android 客户端把调参命令当日志丢弃**的问题,但直接决定
`remote_open` 会话的画质:Android 上报的 `ScreenSettings.QualityLevel` 原来写死为 Good,
服务端据此把会话判定为「固定档位」,`m_AdaptiveQuality.enabled` 永远为 false——
自适应质量在 Android 主机上从未启动。
客户端现在(`ScreenTuning.kt` + `ScreenHandler.h`):档位/帧率/6 档配置表存
SharedPreferences,下次建子连接时随 `TOKEN_BITMAPINFO` 上报;具体档位切换用
`MediaCodec.setParameters(PARAMETER_KEY_VIDEO_BITRATE)` 平滑改码率。帧率只进
`KEY_FRAME_RATE` 码控提示(下次开抓生效),`CMD_SCREEN_SIZE` 与
`CMD_SCREEN_TUNING` 的四个开关在 Android 上没有对应实现,客户端不声明
`SCREEN_CAP_TUNING_FLAGS`,服务端继续用主控端按设备保存的副本。设计与限制详见
`Mcp_RemoteControl_Design.md §5.4`。
--- ---
## 4. 大能力(需独立立项,不建议并入本轮) ## 4. 大能力(需独立立项,不建议并入本轮)
+34
View File
@@ -166,6 +166,40 @@ AI 无法「看懂」视频流。把 25fps 的远程画面直接丢给视觉模
| 编码 | JPEG,quality ~70–85 | 客户端 `jpegQuality` 已有,按 `ChooseScreenPreviewParams` 自适应 | | 编码 | JPEG,quality ~70–85 | 客户端 `jpegQuality` 已有,按 `ChooseScreenPreviewParams` 自适应 |
| 返回 | base64 `image/jpeg` | 已实现 | | 返回 | base64 `image/jpeg` | 已实现 |
### 5.4 远程桌面画质/帧率协商(Android 主机)
主控端在屏幕子连接建立前后下发 `CMD_QUALITY_PROFILES` / `CMD_QUALITY_LEVEL` / `CMD_FPS`
(`ScreenSpyDlg`),决定编码器的分辨率、码率与帧率。Android 客户端此前把这四个命令当
日志丢弃,并且把上报的 `ScreenSettings.QualityLevel` 写死成 Good——服务端据此判定为
「固定档位会话」并把 `m_AdaptiveQuality.enabled` 置 false,**自适应质量在 Android 主机上
永远不启动**。现在的处理(状态在 `ScreenTuning.kt`,C++ 只做协议解析):
| 命令 | 客户端行为 |
|---|---|
| `CMD_QUALITY_PROFILES` | 6 档 × (maxFPS, maxWidth, algorithm, bitRate) 存 SharedPreferences,主控端自定义表权威 |
| `CMD_QUALITY_LEVEL` | `persist=1` 落盘;具体档位(0-5)按配置表立即 `MediaCodec.setParameters(PARAMETER_KEY_VIDEO_BITRATE)`;`-1` 自适应由服务端后续下发具体档位;`-2` 关闭改用原有 4 Mbps/30 fps |
| `CMD_FPS` | 原样存盘(255=不限帧,与 Windows 一致),只在**下次开抓**时带进 `KEY_FRAME_RATE`(255 按 60 提示) |
| `CMD_SCREEN_TUNING` | 只记日志:四位开关都是 Windows 抓屏/软编管线的,Android 无对应实现,不声明 `SCREEN_CAP_TUNING_FLAGS` |
| `CMD_SCREEN_SIZE` | 忽略:Android 源为整屏抓取,编码尺寸在开抓时固定(变更需重建 MediaCodec) |
服务端在发出 `CMD_SCREEN_SIZE` 后会按「客户端可能重启」置分辨率切换标记,1 秒后超时
自愈(`CheckResolutionChangeTimeout`:刷新解码器并请求 IDR),表现为 Android 切档后画面
停约 1 秒。回发 `TOKEN_BITMAPINFO` 能立刻清标记,但会连带 `PrepareDrawing`、通知 web 端
重建解码器,比 1 秒停顿扰动更大,因此不做。
下次建立屏幕子连接时,`TOKEN_BITMAPINFO` 里的 `ScreenSettings.MaxFPS` / `QualityLevel`
直接从 SharedPreferences 读回,主控端据此恢复菜单勾选与自适应状态。
两条必须知道的后果:
- **码率是唯一能在运行中平滑改变的参数**(`PARAMETER_KEY_VIDEO_BITRATE`)。帧率只是码控
提示,surface 编码器按画面内容出帧,没有运行时限帧开关;分辨率变化要重建编码器,
会丢一个 IDR(画面闪一下),因此自适应质量只下发码率、不发 `CMD_SCREEN_SIZE`。
- **MediaCodec 侧不存在 `algorithm`**:配置表的 0=GRAY / 1=DIFF / 3=RGB565 都落到 H.264
(Android 一律用 `video/avc`),只有 `bitRate` 与 `maxFPS` 有实际意义。Ultra/High 的
默认 `bitRate` 是 0(Windows 走 DIFF/RGB565 软编),Android 按改动前的 4000 kbps 回退;
主控端在自定义表里给了码率则以它为准。
--- ---
## 6. 输入注入(Act) ## 6. 输入注入(Act)