Every quality change restarts the capturer on both sides, and two defects
showed up along that path.
Monitor jump: on a multi-monitor client the view moved to another monitor.
switchScreen=false means "reuse the index", but index is the *next*
monitor to switch to, not the one being shown. Track `current` and read it
on the non-switch path; the switch path still advances.
H264 stall: switching from RGB565/DIFF back to H264 froze the view for up
to ten seconds. The client encoder kept its reference frames across the
gap, so it emitted P frames the server could not decode until the next
natural IDR. It is now rebuilt on the first H264 frame after an algorithm
or tuning change. The rebuild is requested with an atomic dirty flag that
the capture thread consumes in ensureEncoder(), because resetting the
encoder from the network thread raced with encode().
On the server, ApplyQualityLevel() only sets m_bResolutionChanging when
the client will really restart the capture: it compares the target
maxWidth with the client's reported bitmap width (cached atomically on
TOKEN_BITMAPINFO) and stays conservative while a previous resize is still
in flight. The fallback timeout drops from 5s to 1s, flushes the decoder
and asks the client for a keyframe (COMMAND_SCREEN_KEYFRAME_REQ).
Verified: Debug client and server build clean; the 1920-wide High->Good
switch no longer waits out the gate. Not yet retested on a live session.
Co-Authored-By: deepseek-flash
Adds "Advanced -> Remote desktop tuning" with three independently
checkable items, all enabled by default:
x264 bitrate cap (VBV) TUNING_X264_VBV 0x1
GDI capture: no slice delay TUNING_GDI_NO_SLEEP 0x2
Send pacing FPS floor TUNING_PACE_FPS_FLOOR 0x4
Unticking an item clears its bit and restores the pre-commit behaviour
exactly, so the escape hatch is one click per measure -- no rebuild, no
config edit. Each bit is independent; a problem with one does not force
turning all three off.
Plumbing
--------
Menu state -> [settings] Tuning* -> CMy2015RemoteDlg::GetScreenTuningFlags()
-> CScreenSpyDlg::SendScreenTuning() (right after TOKEN_BITMAPINFO)
-> CMD_SCREEN_TUNING = 178 [cmd:1][flags:4 LE]
-> CScreenManager::OnReceive -> m_nTuningFlags -> ScreenCapture::SetTuningFlags
Why it is safe to send to old clients: CScreenManager::OnReceive has no
default branch in its switch, so an unknown command falls straight out
(verified). 178 was unassigned (177 = COMMAND_SET_PROTO, 199 =
TOKEN_DECRYPT), and the enum re-anchors explicitly so nothing auto-
increments onto it.
Flags are sent even when all-zero on purpose: the client's flag lives in
CScreenManager, which survives closing and reopening the remote desktop
window on the same connection. Without the zero send, unticking the menu
would leave the previous flags in effect.
Per-measure notes
-----------------
VBV: capped CRF (peak = target, 0.5s buffer, init 0.9). The hardware
paths already cap via rc_max_rate (1.5x, 1s buffer); this is tighter and
only affects x264. After an IDR the buffer drains and takes up to 0.5s
to refill, so a brief quality dip follows every keyframe -- if that shows
up, relaxing to 1.5x/1s would match NVENC/QSV.
GDI: keeps the 50-line BitBlt slicing (large single BitBlts hit driver
limits) and only drops the 5ms sleep between slices -- 1080p is 22 slices
= a forced 110ms per frame against a 50ms budget at 20fps.
Pacing: clamps s0 to 2x the base interval instead of 4x (2fps -> 4fps),
taken as max(2x base, 1000/MaxFPS) so it never exceeds the configured
MaxFPS. The growth/decay code itself is untouched. This one works against
the loop's own congestion signal, so it is the first to untick if a bad
link feels laggier rather than smoother.
SetTuningFlags only rebuilds the encoder when TUNING_ENCODER_BITS
(currently just VBV) changes -- the GDI and pacing bits are read live by
the capture and send threads, and rebuilding for them would drop an IDR
and visibly flash the picture.
Defaults: the on-by-default value lives in the third argument of
THIS_CFG.GetInt(), which only applies when the key is absent. A machine
that already has an explicit 0 keeps it, so the flip is not retroactive
and a test machine can end up with a mixed state.
Verified on the Debug build: with no Tuning* keys all three items start
checked; unticking VBV alone leaves the other two checked and survives a
restart; en_US/de_DE/zh_TW translations all resolve. Not yet measured end
to end on a real remote desktop session.
1. Black screen on rotated displays
Root cause: with DXGI Desktop Duplication on a rotated output,
DXGI_OUTDUPL_DESC.ModeDesc reports the logical desktop size (e.g. 1440x2560)
while the surface returned by AcquireNextFrame is the unrotated landscape
bitmap (2560x1440). The old code created cpuTexture from the ModeDesc size.
On a mismatch ID3D11DeviceContext::CopyResource returns void and silently
copies nothing, leaving the destination texture all zeros, so the first
screen and every H.264 frame were fully black (IDR was only 5KB).
The GDI path does not go through DXGI and was unaffected; the web client uses
the DIFF algorithm and was hit too, but looked different.
Fixes:
- Create cpuTexture from the real D3D11_TEXTURE2D_DESC; when the probe fails,
derive the real surface size from Rotation instead of falling back to
ModeDesc, which would reintroduce the black screen
- Add BlitRotated() to upright the image for ROTATE90/180/270, with a whole-row
memcpy fast path for IDENTITY; m_ulFullWidth/Height now hold the upright
logical size
- Treat DXGI_ERROR_WAIT_TIMEOUT as "no new frame" instead of a failure, so a
static desktop reuses the previous frame instead of always returning nullptr
- GetFirstScreenData: fix the write offset (start at +1, previously the color
channels were all wrong) and report the scaled size instead of the full size,
which used to send about 8.5MB of garbage
- Validate the frame format against a whitelist; fall back to BGRA and warn for
formats that are not 4 bytes per pixel
2. Slow first frame (the probe frame consumed the initial frame)
Root cause: the probe frame AcquireNextFrame(500), added to read the real
texture description, was released right after use and therefore discarded.
But the frame DXGI pushes after DuplicateOutput is often the only frame
available on a static desktop. After discarding it, the first-screen
AcquireNextFrame always timed out (and that timeout was 1000ms at the time),
ScreenManager had already set m_SendFirst = TRUE so it never retried, and the
encoder had no input to produce an IDR, so the picture only appeared once the
desktop changed. Symptom: MDLL/EXE builds with this fix took a long time to
show the remote desktop, while the old build without the probe frame was fast.
Fixes:
- Keep the probe frame: stash it in m_probeRes without calling ReleaseFrame and
let the new ConsumeProbeFrame() fill the first screen once the buffers are
allocated, so the first screen no longer depends on the desktop changing
- Probe timeout 500 -> 200ms; verify both size and format of the probe frame
against cpuTexture and skip consumption on mismatch, otherwise a silently
failed CopyResource would be taken as a valid all-black first screen
- CaptureFrame: drop "reuseOk ? 100 : 1000" and use a fixed 100ms timeout. The
AcquireNextFrame timeout is an upper bound rather than a fixed wait, and
blocking on a static desktop only slowed the capture loop down to 1Hz
- Retry the first screen up to 3 times on "no new frame" (100ms timeout plus a
20ms gap); skip the capture entirely when the probe frame already filled it
- Release m_probeRes in CleanupDXGI and ReinitDuplication so the duplication
object can be released
3. Incomplete picture
- Add ScreenCapture::FitDefaultSendSize(), which scales proportionally so the
long edge stays at or below 1920 and the short edge at or below 1080, for
both landscape and portrait. The old min(1920,w) and min(1080,h) clamped the
axes independently and broke the aspect ratio: a 1440x2560 portrait desktop
was squeezed to 1440x1080, keeping only 42% of the height
- Use the same helper for both the GDI and DXGI paths; common landscape
resolutions (1920x1080, 2560x1440, 3840x2160, 1280x720) are unchanged
4. Incidental fixes
- CaptureFrame leaked desktopResource when QueryInterface failed, one DXGI
resource per failure
- Drop the frame and warn when the surface size changes, for example after the
user changed the resolution, so BlitRotated does not read out of bounds using
stale sizes
Add a third view to the web UI, the screen wall, which shows several
devices at once in one tab with 1x1 / 2x2 / 3x3 / 4x4 / 5x5 layouts and a
device sidebar on the left. Tiles are monitoring-only; clicking one
enlarges that device into the existing single-screen page for control, and
Back returns to the wall. The same device can be added several times: all
tiles of one device share a single decoder and a single stream.
The wall decodes at most 9 streams on desktop and 4 on touch devices.
Tiles beyond the budget keep receiving frames but have their decoder
closed and show the last frame plus a Paused badge; they are promoted
automatically when a live tile goes away. Subscriptions are reference
counted on the client (streams map with owner sets), so a stream is only
connected on the 0 -> 1 transition and disconnected on 1 -> 0, and the
wall and the single-screen page can share one device without tearing the
stream down.
Server side, WebClient now holds a set of watched devices instead of a
single id. A new multiWatch command flag selects set semantics, while
clients without it keep the old replace-all behaviour, so an old frontend
against the new server is unchanged. connect and disconnect responses echo
id and req so the frontend can route them, input commands carry an id and
are resolved against the watch set (dropped when the set has several
devices and no id), and the per-client watch set is capped at 32.
For fast recovery after a pause or when joining late, the server caches
each device's raw IDR payload and replays it on connect and on a new
keyframe command, then asks the client to force a new IDR. The client gets
COMMAND_SCREEN_KEYFRAME_REQ = 169 and an atomic request flag that is only
cleared after an IDR actually came out of the encoder, so a failed capture
or EAGAIN does not lose the request. The frontend retries the keyframe
request up to three times; an old client that ignores the command simply
recovers on the next natural GOP.
Also fix the keyframe cache double wrapping: BroadcastH264Frame used to
cache the full packet header and HandleConnect wrapped it again. The cache
now stores data + 9, len - 9 and every sender wraps it through
BuildFramePacket, and the cache is cleared on cold start and on resolution
change. StopRemoteDesktop is always called outside m_ClientsMutex, with
the ids collected under the lock first.
Co-Authored-By: deepseek-flash