Commit Graph
12 Commits
Author SHA1 Message Date
yuanyuanxiang 5eb44307ee Feature: Capture the virtual desktop for multi-monitor remote control
The MCP Observe/Act loop requires screenshots and mouse injection to share
one coordinate space. Previously the client sent only the primary monitor
while injection used virtual-desktop coordinates, so multi-monitor hosts
were rejected outright.

- Client: COMMAND_SCREEN_SPY param buffer[11] (wire index 12) now forces
  whole-virtual-desktop GDI capture; preview JPEGs also cover the virtual
  desktop including negative origins.
- Server: advertise CLIENT_CAP_VIRTUAL_DESKTOP, gate multi-monitor remote
  control on that capability, and pass the flag when starting the session.
  Linux stays exempt (X root capture/injection are already consistent).
2026-11-10 08:21:55 +01:00
yuanyuanxianganddeepseek-flash 2e34e49186 Fix: Keep the captured monitor stable across restarts
GDI and DXGI each kept their own static monitor index, and DXGI
committed the index before its initialization had succeeded. When a
quality change restarted capture and DXGI fell back to GDI, the next
DXGI attempt read the failed attempt's index and jumped to the other
monitor. Both backends now share one process-wide index with separate
peek and commit steps, and the commit only happens after the backend
is actually up.

Manual switching is made a one-shot request: InitScreenSpy consumes
the flag and RestartScreen only sets it, so an adaptive restart that
lands in the middle of a switch no longer drops the request or
re-applies it on every later rebuild. Window capture no longer
touches the index at all, so leaving window mode returns to the
monitor that was on screen.

Also: restart capture on a bitrate change only when the target level
is still H.264 (switching to RGB565/DIFF never uses the bitrate), and
bound the DXGI output fallback so a session without any enumerable
output cannot spin forever in InitDXGI.

Co-Authored-By: deepseek-flash
2026-11-08 20:44:49 +01:00
yuanyuanxianganddeepseek-flash a190c21217 Feature: make remote desktop tuning per client
The tuning switches lived in the master's Advanced menu and were saved
globally, so two hosts could not use different settings. The client already
sends ScreenSettings when a remote desktop starts, so the state belongs
there: the four items now sit in the remote desktop window's system menu,
directly below Screen quality, and are saved per client.

ScreenSettings had a 40-byte Reserved area at offset 56; four bytes became
TuningFlags, keeping the struct at 100 bytes with Capabilities at 96. A new
Capabilities bit marks a value written by a versioned master; old clients,
and new clients never configured, report 0 and are not trusted.

CMD_SCREEN_TUNING now ends with a version byte. A five-byte command comes
from an older master that does not know the server-only adaptive bit: the
client applies those three bits to the session but does not save them. A
fresh client starts at 0, so an old master still gets pre-feature
behaviour; the new master's 0xE default is persisted on its first session.
The per-client master copy stays as the old-client fallback, and the old
global [settings] Tuning* keys are honoured when neither exists, so an
explicit opt-out survives the migration.

Verified on the Debug server with the in-process test client: a fresh
client stored 0xE on first use; toggling VBV wrote 0xF to both sides; after
deleting the master copy and reopening, the menu restored VBV from the
client report.

Co-Authored-By: deepseek-flash
2026-11-07 14:11:47 +01:00
yuanyuanxiang ca4e85826f Feature: remote desktop tuning flags, on by default with per-item opt-out
Adds "Advanced -> Remote desktop tuning" with three independently
checkable items, all enabled by default:

  x264 bitrate cap (VBV)       TUNING_X264_VBV        0x1
  GDI capture: no slice delay  TUNING_GDI_NO_SLEEP    0x2
  Send pacing FPS floor        TUNING_PACE_FPS_FLOOR  0x4

Unticking an item clears its bit and restores the pre-commit behaviour
exactly, so the escape hatch is one click per measure -- no rebuild, no
config edit. Each bit is independent; a problem with one does not force
turning all three off.

Plumbing
--------
Menu state -> [settings] Tuning* -> CMy2015RemoteDlg::GetScreenTuningFlags()
  -> CScreenSpyDlg::SendScreenTuning()  (right after TOKEN_BITMAPINFO)
  -> CMD_SCREEN_TUNING = 178 [cmd:1][flags:4 LE]
  -> CScreenManager::OnReceive -> m_nTuningFlags -> ScreenCapture::SetTuningFlags

Why it is safe to send to old clients: CScreenManager::OnReceive has no
default branch in its switch, so an unknown command falls straight out
(verified). 178 was unassigned (177 = COMMAND_SET_PROTO, 199 =
TOKEN_DECRYPT), and the enum re-anchors explicitly so nothing auto-
increments onto it.

Flags are sent even when all-zero on purpose: the client's flag lives in
CScreenManager, which survives closing and reopening the remote desktop
window on the same connection. Without the zero send, unticking the menu
would leave the previous flags in effect.

Per-measure notes
-----------------
VBV: capped CRF (peak = target, 0.5s buffer, init 0.9). The hardware
paths already cap via rc_max_rate (1.5x, 1s buffer); this is tighter and
only affects x264. After an IDR the buffer drains and takes up to 0.5s
to refill, so a brief quality dip follows every keyframe -- if that shows
up, relaxing to 1.5x/1s would match NVENC/QSV.

GDI: keeps the 50-line BitBlt slicing (large single BitBlts hit driver
limits) and only drops the 5ms sleep between slices -- 1080p is 22 slices
= a forced 110ms per frame against a 50ms budget at 20fps.

Pacing: clamps s0 to 2x the base interval instead of 4x (2fps -> 4fps),
taken as max(2x base, 1000/MaxFPS) so it never exceeds the configured
MaxFPS. The growth/decay code itself is untouched. This one works against
the loop's own congestion signal, so it is the first to untick if a bad
link feels laggier rather than smoother.

SetTuningFlags only rebuilds the encoder when TUNING_ENCODER_BITS
(currently just VBV) changes -- the GDI and pacing bits are read live by
the capture and send threads, and rebuilding for them would drop an IDR
and visibly flash the picture.

Defaults: the on-by-default value lives in the third argument of
THIS_CFG.GetInt(), which only applies when the key is absent. A machine
that already has an explicit 0 keeps it, so the flip is not retroactive
and a test machine can end up with a mixed state.

Verified on the Debug build: with no Tuning* keys all three items start
checked; unticking VBV alone leaves the other two checked and survives a
restart; en_US/de_DE/zh_TW translations all resolve. Not yet measured end
to end on a real remote desktop session.
2026-11-04 14:47:40 +01:00
yuanyuanxiang bc75e1ae99 fix(client): fix DXGI black screen on rotated displays and slow first frame
1. Black screen on rotated displays

Root cause: with DXGI Desktop Duplication on a rotated output,
DXGI_OUTDUPL_DESC.ModeDesc reports the logical desktop size (e.g. 1440x2560)
while the surface returned by AcquireNextFrame is the unrotated landscape
bitmap (2560x1440). The old code created cpuTexture from the ModeDesc size.
On a mismatch ID3D11DeviceContext::CopyResource returns void and silently
copies nothing, leaving the destination texture all zeros, so the first
screen and every H.264 frame were fully black (IDR was only 5KB).
The GDI path does not go through DXGI and was unaffected; the web client uses
the DIFF algorithm and was hit too, but looked different.

Fixes:
- Create cpuTexture from the real D3D11_TEXTURE2D_DESC; when the probe fails,
  derive the real surface size from Rotation instead of falling back to
  ModeDesc, which would reintroduce the black screen
- Add BlitRotated() to upright the image for ROTATE90/180/270, with a whole-row
  memcpy fast path for IDENTITY; m_ulFullWidth/Height now hold the upright
  logical size
- Treat DXGI_ERROR_WAIT_TIMEOUT as "no new frame" instead of a failure, so a
  static desktop reuses the previous frame instead of always returning nullptr
- GetFirstScreenData: fix the write offset (start at +1, previously the color
  channels were all wrong) and report the scaled size instead of the full size,
  which used to send about 8.5MB of garbage
- Validate the frame format against a whitelist; fall back to BGRA and warn for
  formats that are not 4 bytes per pixel

2. Slow first frame (the probe frame consumed the initial frame)

Root cause: the probe frame AcquireNextFrame(500), added to read the real
texture description, was released right after use and therefore discarded.
But the frame DXGI pushes after DuplicateOutput is often the only frame
available on a static desktop. After discarding it, the first-screen
AcquireNextFrame always timed out (and that timeout was 1000ms at the time),
ScreenManager had already set m_SendFirst = TRUE so it never retried, and the
encoder had no input to produce an IDR, so the picture only appeared once the
desktop changed. Symptom: MDLL/EXE builds with this fix took a long time to
show the remote desktop, while the old build without the probe frame was fast.

Fixes:
- Keep the probe frame: stash it in m_probeRes without calling ReleaseFrame and
  let the new ConsumeProbeFrame() fill the first screen once the buffers are
  allocated, so the first screen no longer depends on the desktop changing
- Probe timeout 500 -> 200ms; verify both size and format of the probe frame
  against cpuTexture and skip consumption on mismatch, otherwise a silently
  failed CopyResource would be taken as a valid all-black first screen
- CaptureFrame: drop "reuseOk ? 100 : 1000" and use a fixed 100ms timeout. The
  AcquireNextFrame timeout is an upper bound rather than a fixed wait, and
  blocking on a static desktop only slowed the capture loop down to 1Hz
- Retry the first screen up to 3 times on "no new frame" (100ms timeout plus a
  20ms gap); skip the capture entirely when the probe frame already filled it
- Release m_probeRes in CleanupDXGI and ReinitDuplication so the duplication
  object can be released

3. Incomplete picture

- Add ScreenCapture::FitDefaultSendSize(), which scales proportionally so the
  long edge stays at or below 1920 and the short edge at or below 1080, for
  both landscape and portrait. The old min(1920,w) and min(1080,h) clamped the
  axes independently and broke the aspect ratio: a 1440x2560 portrait desktop
  was squeezed to 1440x1080, keeping only 42% of the height
- Use the same helper for both the GDI and DXGI paths; common landscape
  resolutions (1920x1080, 2560x1440, 3840x2160, 1280x720) are unchanged

4. Incidental fixes

- CaptureFrame leaked desktopResource when QueryInterface failed, one DXGI
  resource per failure
- Drop the frame and warn when the surface size changes, for example after the
  user changed the resolution, so BlitRotated does not read out of bounds using
  stale sizes
2026-10-30 21:18:57 +01:00
yuanyuanxiang 66c950cecb Refactor: split ScanScreen window-capture into 3 private methods 2026-06-16 21:03:45 +02:00
yuanyuanxiangandClaude Sonnet 4.6 abafd673a2 Opt: skip 8MB raw first-frame in H264 mode; server unlocks on first IDR instead
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-15 21:29:59 +02:00
yuanyuanxiang 2765d95950 Feat: Support viewing active window via online-host popup menu 2026-06-15 19:15:31 +02:00
yuanyuanxiang d3b9e7faae Feat: window capture via PrintWindow with server-side HWND routing by clientID 2026-06-15 14:11:42 +02:00
yuanyuanxiangandClaude Sonnet 4.6 63ef75b7ce Feat: ROI screen capture with remote control support via COMMAND_SCREEN_ROI
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-14 00:24:46 +02:00
yuanyuanxiang 8c7f612449 Feature: Implement H.264 and AV1 hardware encoding for remote control
Remark: Need to update FFmpeg static libraries to take effort
2026-05-30 00:12:38 +02:00
yuanyuanxiang 5a325a202b Init: Migrate SimpleRemoter (Since v1.3.1) to Gitea 2026-04-19 22:55:21 +02:00