The adaptive quality controller counted only complete frames per second, so when
a single frame took several seconds to arrive the sample was 0 fps and the
throughput criterion abstained forever: Ultra and High kept flipping while a
foreground video played. The audio watchdog had the opposite problem: it rebuilt
the WASAPI capture every three seconds on an idle endpoint, and its sleep
detector measured the whole loop including a blocking send.
Sample the last ten seconds in a ring buffer and feed the criterion a windowed
rate, guarded so a scroll burst, a stale window or a few KB clock patch is not
mistaken for congestion. Apply the same per-frame floor to the one-second
fallback and to the post-downgrade local-bottleneck probe. Normalize delayed
timer ticks by real elapsed milliseconds, and make the 40 second
upgrade-failure window outlast the resolution cooldown it has to cross.
On the client, rebuild only when an audio session is actually rendering, and
rebuild immediately when the captured endpoint is no longer the default output;
refresh the sleep timestamps on failed as well as successful init.
Tests cover the window and fallback helpers directly; the full suite passes
845/845, and both the Debug server and Debug client build.
Co-Authored-By: deepseek-flash
The tuning switches lived in the master's Advanced menu and were saved
globally, so two hosts could not use different settings. The client already
sends ScreenSettings when a remote desktop starts, so the state belongs
there: the four items now sit in the remote desktop window's system menu,
directly below Screen quality, and are saved per client.
ScreenSettings had a 40-byte Reserved area at offset 56; four bytes became
TuningFlags, keeping the struct at 100 bytes with Capabilities at 96. A new
Capabilities bit marks a value written by a versioned master; old clients,
and new clients never configured, report 0 and are not trusted.
CMD_SCREEN_TUNING now ends with a version byte. A five-byte command comes
from an older master that does not know the server-only adaptive bit: the
client applies those three bits to the session but does not save them. A
fresh client starts at 0, so an old master still gets pre-feature
behaviour; the new master's 0xE default is persisted on its first session.
The per-client master copy stays as the old-client fallback, and the old
global [settings] Tuning* keys are honoured when neither exists, so an
explicit opt-out survives the migration.
Verified on the Debug server with the in-process test client: a fresh
client stored 0xE on first use; toggling VBV wrote 0xF to both sides; after
deleting the master copy and reopening, the menu restored VBV from the
client report.
Co-Authored-By: deepseek-flash
The client's audio thread starts before the capture thread sends
TOKEN_BITMAPINFO, so its first packet - the only one carrying the
AudioFormat - reaches the master before CScreenSpyDlg exists and is
dropped as an unknown command. Without a format the master cannot open
waveOut, and the client re-announced only on an audio off/on, so a
remote desktop opened with audio enabled stayed silent until the toolbar
button was toggled twice. The master now asks for a re-announce when
data arrives with no format on record (CMD_AUDIO_CTRL enable,
persist=0, one request per 2 s, 15 attempts max), and the client
attaches the format to its next packet.
Entering Modern Standby tears down the render endpoint. The client got
AUDCLNT_E_DEVICE_INVALIDATED and rebuilt its loopback capture in that
same instant; the new client initialized and started fine but was bound
to the dying audio engine, so after resume GetNextPacketSize returned
zero packets forever without error, and only a fresh screen session
recovered. The audio thread now measures its own loop gap (GetTickCount64
plus wall clock, 2 s; a normal pass is ~10 ms) and rebuilds the capture
client after the machine was frozen, waits a second after an
invalidation, and rebuilds on an audio off-to-on.
The master rebuilds waveOut on resume from WM_POWERBROADCAST. The format
recovery was verified live; the resume recovery is build-verified and
awaits a field check on a sleeping machine.
Co-Authored-By: deepseek-flash
Remote desktop audio could stop permanently after an idle period and only
came back by closing and reopening the remote desktop. The ring buffer and
WAVEHDRs were shared without locking between the IOCP worker thread
(OnAudioData) and the UI thread (MM_WOM_DONE): headers were queued twice
(waveOutWrite returning WAVERR_STILLPLAYING 33) and the non-atomic
m_nRingDataLen underflowed, leaving the buffer "full" of stale data.
Audio state is now serialized by a recursive m_AudioMutex, and opening,
resetting and closing the waveOut device happens only on the window's UI
thread: waveOutReset/Close can re-enter the MM_WOM_* handlers on that
thread, and a lock-holding worker would deadlock against the UI thread.
Workers post WM_AUDIO_INIT_DEVICE instead. Rebuilds are triggered by
format changes, waveOutWrite failures (throttled to 2 s), MM_WOM_CLOSE
with a 1 s guard against our own close notices, and a 2-second retry
while audio is enabled. Failed header preparation now tears the device
down instead of leaving a handle that rejects every write.
The client re-creates its Opus encoder when the endpoint channel count or
sample rate changes, downmixes more than two channels, and clamps
oversized WASAPI packets to the PCM staging buffer. Web audio forwarding
still runs outside the lock and is gated exactly as before. Verified by
building the Debug x64 server and client; the silence scenario itself was
not reproduced at runtime.
Co-Authored-By: deepseek-flash
The cap is capped CRF (VBV): when a frame needs more bits than the
profile's nominal bitrate allows, x264 has to raise QP to stay under the
ceiling. Scenes with lots of change (video, scrolling, window dragging)
therefore lose detail and look blurry, while static scenes stay sharp -
the cap only bites on motion. That trade is worth it on intercontinental
or otherwise constrained links, where an uncapped frame can take seconds
to transmit and stall everything queued behind it, but it is a pure
quality loss on a LAN.
The tuning item now defaults to unchecked, so a master sends 0x3 (GDI
no-sleep and send pacing) by default; machines that already stored
TuningX264Vbv keep their explicit choice. The menu entry is renamed to
"x264 码率封顶 (VBV,仅限洲际弱网)" and the en_US, zh_TW and de_DE
translations were updated to match the new source string.
Only newly opened remote desktops pick up the flag, since CMD_SCREEN_TUNING
is sent when the session starts. Verified by rebuilding the Debug x64
server and client, and by finding the new menu and translation strings in
the built binary.
Co-Authored-By: deepseek-flash
Adds "Advanced -> Remote desktop tuning" with three independently
checkable items, all enabled by default:
x264 bitrate cap (VBV) TUNING_X264_VBV 0x1
GDI capture: no slice delay TUNING_GDI_NO_SLEEP 0x2
Send pacing FPS floor TUNING_PACE_FPS_FLOOR 0x4
Unticking an item clears its bit and restores the pre-commit behaviour
exactly, so the escape hatch is one click per measure -- no rebuild, no
config edit. Each bit is independent; a problem with one does not force
turning all three off.
Plumbing
--------
Menu state -> [settings] Tuning* -> CMy2015RemoteDlg::GetScreenTuningFlags()
-> CScreenSpyDlg::SendScreenTuning() (right after TOKEN_BITMAPINFO)
-> CMD_SCREEN_TUNING = 178 [cmd:1][flags:4 LE]
-> CScreenManager::OnReceive -> m_nTuningFlags -> ScreenCapture::SetTuningFlags
Why it is safe to send to old clients: CScreenManager::OnReceive has no
default branch in its switch, so an unknown command falls straight out
(verified). 178 was unassigned (177 = COMMAND_SET_PROTO, 199 =
TOKEN_DECRYPT), and the enum re-anchors explicitly so nothing auto-
increments onto it.
Flags are sent even when all-zero on purpose: the client's flag lives in
CScreenManager, which survives closing and reopening the remote desktop
window on the same connection. Without the zero send, unticking the menu
would leave the previous flags in effect.
Per-measure notes
-----------------
VBV: capped CRF (peak = target, 0.5s buffer, init 0.9). The hardware
paths already cap via rc_max_rate (1.5x, 1s buffer); this is tighter and
only affects x264. After an IDR the buffer drains and takes up to 0.5s
to refill, so a brief quality dip follows every keyframe -- if that shows
up, relaxing to 1.5x/1s would match NVENC/QSV.
GDI: keeps the 50-line BitBlt slicing (large single BitBlts hit driver
limits) and only drops the 5ms sleep between slices -- 1080p is 22 slices
= a forced 110ms per frame against a 50ms budget at 20fps.
Pacing: clamps s0 to 2x the base interval instead of 4x (2fps -> 4fps),
taken as max(2x base, 1000/MaxFPS) so it never exceeds the configured
MaxFPS. The growth/decay code itself is untouched. This one works against
the loop's own congestion signal, so it is the first to untick if a bad
link feels laggier rather than smoother.
SetTuningFlags only rebuilds the encoder when TUNING_ENCODER_BITS
(currently just VBV) changes -- the GDI and pacing bits are read live by
the capture and send threads, and rebuilding for them would drop an IDR
and visibly flash the picture.
Defaults: the on-by-default value lives in the third argument of
THIS_CFG.GetInt(), which only applies when the key is absent. A machine
that already has an explicit 0 keeps it, so the flip is not retroactive
and a test machine can end up with a mixed state.
Verified on the Debug build: with no Tuning* keys all three items start
checked; unticking VBV alone leaves the other two checked and survives a
restart; en_US/de_DE/zh_TW translations all resolve. Not yet measured end
to end on a real remote desktop session.
iniFile/binFile::GetCachedKey used RegCreateKeyExA for every access, so a plain
GetStr/GetInt on a missing section materialised an empty subkey. CScreenManager's
constructor calls LoadQualityProfiles() unconditionally, which reads profile0..
profile5 x4 keys, so every client start left six zero-valued subkeys behind:
126 of them on the dev box, all with zero values. The server's HKCU\SoftwareYAMA\quality key is the same artefact.
Split the helper by intent: writes go through RegCreateKeyExA, reads through
RegOpenKeyExA. Both still ask for KEY_READ|KEY_WRITE, because the handle cache is
shared and a read-only cached handle would make a later SetStr on the same
section fail.
Also fixes a second bug in SaveQualityProfiles: when a level matched the default
it simply continued, so a level the operator reset to default kept the old
custom values in the registry and the reset never took effect. Add
SectionExists() (open-only, never creates) and write the defaults back when the
section already exists.
A host whose DXGI initialisation hangs never opens the remote desktop window at
all. WorkThreadProc runs InitScreenSpy() and SendBitMapInfo() back to back, so a
capturer that never finishes constructing means TOKEN_BITMAPINFO is never sent,
and the master - which only opens the window on that token - waits forever. On
the affected Server 2022 box the screen thread stopped right after logging its
type and algorithm, printing neither the DXGI init summary nor the existing
"DXGI SPY init failed" fallback, so it was stuck inside the capturer
constructor; a healthy machine on the same build reaches the summary in under a
second. Six clicks left six stuck threads, each holding a D3D device.
DXGI desktop duplication needs a display to duplicate. With no interactive
desktop it can return no frames at best, and on some drivers it never returns
from the initialisation calls at all. BitBlt has no such dependency and keeps
working, which is why switching the screen type to GDI made the same host
connect. Detect that up front and pick GDI instead of relying on a hang to be
survivable:
- IsWorkstationLocked() - the input desktop is Winlogon. Moved out of
CKernelManager into Manager.cpp so the capturer can use it too; the manager
keeps its private wrapper for the "Locked:"/"Inactive:" activity string.
- IsCurrentSessionDisconnected() - the process's own session is
WTSDisconnected. That is the state the reported host was actually in: its
client runs in session 2, which was disconnected, and the desktop only came
back after the master's "RDP session reset" ran tscon. Session 0 has no
disconnected state and is left alone.
Either condition means the same thing for capture, so both route to the plain
GDI capturer. An active session keeps DXGI untouched, so hosts that work today
are unaffected; the new log line names which condition fired.
Co-Authored-By: deepseek-flash
Add a third view to the web UI, the screen wall, which shows several
devices at once in one tab with 1x1 / 2x2 / 3x3 / 4x4 / 5x5 layouts and a
device sidebar on the left. Tiles are monitoring-only; clicking one
enlarges that device into the existing single-screen page for control, and
Back returns to the wall. The same device can be added several times: all
tiles of one device share a single decoder and a single stream.
The wall decodes at most 9 streams on desktop and 4 on touch devices.
Tiles beyond the budget keep receiving frames but have their decoder
closed and show the last frame plus a Paused badge; they are promoted
automatically when a live tile goes away. Subscriptions are reference
counted on the client (streams map with owner sets), so a stream is only
connected on the 0 -> 1 transition and disconnected on 1 -> 0, and the
wall and the single-screen page can share one device without tearing the
stream down.
Server side, WebClient now holds a set of watched devices instead of a
single id. A new multiWatch command flag selects set semantics, while
clients without it keep the old replace-all behaviour, so an old frontend
against the new server is unchanged. connect and disconnect responses echo
id and req so the frontend can route them, input commands carry an id and
are resolved against the watch set (dropped when the set has several
devices and no id), and the per-client watch set is capped at 32.
For fast recovery after a pause or when joining late, the server caches
each device's raw IDR payload and replays it on connect and on a new
keyframe command, then asks the client to force a new IDR. The client gets
COMMAND_SCREEN_KEYFRAME_REQ = 169 and an atomic request flag that is only
cleared after an IDR actually came out of the encoder, so a failed capture
or EAGAIN does not lose the request. The frontend retries the keyframe
request up to three times; an old client that ignores the command simply
recovers on the next natural GOP.
Also fix the keyframe cache double wrapping: BroadcastH264Frame used to
cache the full packet header and HandleConnect wrapped it again. The cache
now stores data + 9, len - 9 and every sender wraps it through
BuildFramePacket, and the cache is cleared on cold start and on resolution
change. StopRemoteDesktop is always called outside m_ClientsMutex, with
the ids collected under the lock first.
Co-Authored-By: deepseek-flash
Add a novice-friendly "File Transfer Speed" dialog under the Advanced menu
that controls file-transfer speed in both directions without touching
SimplePlugins. Speed is passed through InitFileUpload's existing
chunkSizeKb/sendDurationMs arguments, so the precompiled library stays
untouched. Seven presets run from ~0.3 MB/s up to a new 10 MB/s tier plus an
unlimited tier; the chosen value is stored in [transfer] ChunkKb/DurationMs.
Upload reads THIS_CFG at the master's InitFileUpload (2015RemoteDlg.cpp:2225)
and takes effect on restart. Download is pushed to the client over a new
CMD_FILE_SPEED (252) command, sent from OnOnlineFileManager and
OnOnlineDesktopManager before each session opens, and read by all four client
InitFileUpload call sites via GetTransferSpeed so the process-global refcount
cannot latch a stale speed. All Chinese strings go through LangManager, with
en_US/zh_TW/de_DE entries added for the tier names, format strings and hint
text.
Co-Authored-By: deepseek-v4-pro
ProcessCommand chose the MSG record size by testing "ulLength % 28 == 0"
before "% 48 == 0". The two sizes' least common multiple is 336 (7*48 =
12*28), so a 48-byte batch whose record count is a multiple of 7 was
misclassified as 28-byte MSG32 records, shifting every field and garbling
(or silently dropping) injected input.
Check "% 48 == 0" first. The modern controller always emits 48-byte MSG64;
the 28-byte MSG32 path is legacy 32-bit-controller compatibility and is only
reached when 48 does not divide evenly. This is the first feature (MCP
remote_keyboard "type", and the upcoming remote_mouse drag) to emit
multi-record batches, which is what made the latent bug reachable.
Co-Authored-By: deepseek-v4-pro
Replacing the old connection on a duplicate login made the client
reconnect in a tight loop. Instead, park the new connection in a standby
list keyed by clientID and promote it into the main host list only after
the old connection is removed by heartbeat timeout or disconnect.
Fix heartbeat attribution so a parked standby refreshes its own
last-heartbeat time rather than the main connection's, otherwise the main
never times out and the standby can never take over. Gate the offline
notification on an actual removal with no promoted standby to avoid a
spurious "host offline" when takeover happens. Skip promoting a standby
already marked removed, and reuse the normal login sign/settings path when
a standby is promoted.
Co-Authored-By: deepseek-v4-pro
Two changes to reduce unnecessary CScreenSpy restarts when connecting:
1. Client (ScreenManager.cpp): Initialize CScreenSpy with bitrate from
the locally-saved quality profile, so CMD_QUALITY_LEVEL arriving from
the server (same bitrate as default) hits SetBitRate(3000)==3000 and
skips the restart instead of comparing against the hard-coded 0.
Also fixes QualityLevel init to use the already-computed `quality`
variable (which honours the QUALITY_DISABLED override when algo!=NUL)
rather than re-reading the cfg key a second time.
2. Server (ScreenSpyDlg.cpp): Only send CMD_SCREEN_SIZE strategy=2 when
the session is in QUALITY_ADAPTIVE mode and a cached maxWidth exists.
Fixed quality levels already carry resolution via CMD_QUALITY_PROFILES,
so unconditionally sending CMD_SCREEN_SIZE caused a second restart when
the screen spy was still rebuilding from the first one.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Server: clamp web session adaptive quality to H264-only levels (>=Good) in EvaluateQuality and ApplyQualityLevel; Ultra/High (DIFF/RGB565) caused the browser to freeze ~1 min into a session
- Server: move session-type detection to the top of ScreenSpyDlg::OnInitDialog and skip SetWindowPlacement/EnterFullScreen for hidden web sessions, eliminating the MFC dialog flash on web-triggered opens
- Linux client: default QualityLevel from QUALITY_ADAPTIVE to QUALITY_GOOD to match Windows/macOS so the server's adaptive controller doesn't auto-upgrade to non-H264 algorithms
- Web: clear the floating quick-action toolbar on fullscreen exit so its row of buttons (RDP reset / Mouse / Close) doesn't stay pinned to the top of the page
- Web: route F11 to the remote in control mode instead of toggling local fullscreen
- Web: route Esc to the remote in control mode via the Keyboard Lock API instead of exiting native fullscreen