Feature: Sniff WebSocket upgrades on the MFC server's device port
This commit is contained in:
1 parent
d4f312c9d8
commit
cc891e3670
11 files changed
+915
-22
No files matched your search
@@ -11,7 +11,8 @@
|
||||
> 3. 「6543 我可以修改,但是只能改为 80,443,8080,但是这些端口都已经在使用」
|
||||
> 4. 「B 方案更适合目前场景。xxx.com 的 nginx 配置尽量少改,不要大改」
|
||||
>
|
||||
> 已核实前提:目标服务端是 **Go 版(`server/go`)**;客户端当前**都已启用 TLS**。
|
||||
> 已核实前提:客户端当前**都已启用 TLS**。服务端两套实现均可接入:Go 版(`server/go`)
|
||||
> 与 MFC 版(`server/2015Remote`,见 §4.2)。
|
||||
|
||||
## 1. 方案选型
|
||||
|
||||
@@ -128,7 +129,9 @@ Host 头同理:用**配置里的地址(域名)**,而不是解析后的 I
|
||||
`client/KernelManager.cpp::ApplyTlsAnchorChange` 调用,两者都只在 `TlsEnabled()`
|
||||
前后值发生变化时 `Disconnect()`,让重连循环用新传输方式重新登录。
|
||||
|
||||
## 4. 服务端实现(Go)
|
||||
## 4. 服务端实现
|
||||
|
||||
### 4.1 Go 服务端(`server/go`)
|
||||
|
||||
`server/go/server/ws.go`,在 `handleConnection` 里分层嗅探:
|
||||
|
||||
@@ -158,6 +161,40 @@ raw conn --wrapTLS(嗅 0x16 0x03)--> [TLS] --wrapWebSocket(嗅 "GET ")--> [WS
|
||||
| `YAMA_WS_PATH` | 只接受该路径的 upgrade(空 = 任意路径) |
|
||||
| `YAMA_TRUSTED_PROXIES` | 允许上报真实 IP 的代理地址,逗号分隔的 IP/CIDR,默认 `127.0.0.0/8,::1/128` |
|
||||
|
||||
### 4.2 MFC 服务端(`server/2015Remote`)
|
||||
|
||||
MFC 服务端此前就在设备端口上嗅探 TLS(首字节 `0x16 0x03` → `ServerTlsSession`),
|
||||
WS 叠在同一层状态机之上,**与 TLS 是分层而不是并列**(`wss` = TLS 里的 WebSocket):
|
||||
|
||||
```
|
||||
子套接字 -- TLS 嗅探 --+-- 明文 --+-- "GET " 嗅探 --+-- WS 帧 --+--> ParseReceivedData
|
||||
| | |
|
||||
+-- TLS ---+----------------+(WSS 时帧在解密的明文里判)
|
||||
```
|
||||
|
||||
新增两个文件,其余改动都在既有握手/收发路径上:
|
||||
|
||||
| 文件 | 职责 |
|
||||
|---|---|
|
||||
| `ServerWsSession.h/.cpp` | 每连接 WS 会话:升级握手(逐行解析、8 KiB 上限、`Sec-WebSocket-Accept`)、流式解帧(不整帧缓冲,大帧按片交付)、控制帧(ping→pong、close 回显后断开)、转发真实 IP |
|
||||
| `IOCPServer.cpp` | `WsHandshakeThreadProc` / `StartWsHandshake`(专用阻塞线程,与 TLS 握手线程同一模式)、`DispatchPlain`(明文/WSS 共用入口)、`OnClientPreSending` 里的「先封帧、再 TLS 加密」 |
|
||||
|
||||
- 状态机:`ServerWsState { WS_Undecided → WS_Handshaking → WS_Ready | WS_None }` 与既有的
|
||||
`ServerTlsState` 并存;`CONTEXT_OBJECT` 复用(`InitMember`)时一并复位并释放会话。
|
||||
- 判定点:**明文的第一段有 ≥6 字节**才开始分类(与 ClientHello 嗅探同一下限),
|
||||
避免把「请求头还没到齐」误判掉;阈值内的字节攒在 `tlsPreClassify` 里。
|
||||
- 升级线程与 TLS 握手线程同构:`FIONBIO=0`(该套接字常是非阻塞的,`ioctlsocket` 会以
|
||||
`WSAEINVAL` 失败),15 s 收发超时,遇 `WSAEWOULDBLOCK` 用 `select` 等可读/可写再试;
|
||||
线程持有一枚 `IoRefCount`,结束后把 socket 交还 IOCP(`PostRecv`)。升级响应之后同批
|
||||
字节里若已带客户端首帧(WSS 分批解密时常见),先在交还前喂给解帧路径。
|
||||
- 发送方向:`OnClientPreSending` 里把本次新追加的尾部封成一帧(`TruncateBuffer` +
|
||||
`WriteBuffer`,与既有的「只加密新尾部」同一手法规避在途数据),然后才走 TLS 加密;
|
||||
控制帧应答(pong / close 回显)走同一条发送路径(`SendWsReplies`,持 `SendLock`)。
|
||||
- 反代真实 IP:**仅当直接对端是回环地址**时才采信 `X-Forwarded-For` 的**最后一段**
|
||||
(`X-Real-IP` 兜底),且必须是 IP 字面量;结果写回 `PeerName`,在线列表、封禁与
|
||||
白名单看到的就是设备本身。非回环对端的转发头一律忽略(防伪造)。
|
||||
- 既有客户端零影响:非 `GET ` 首包在攒够 6 字节后原样进入 `ParseReceivedData`。
|
||||
|
||||
## 5. nginx 最小改动
|
||||
|
||||
前提:客户端地址填 `ws.xxx.com`、端口 **443**、协议选 **WebSocket+TLS**。
|
||||
@@ -281,13 +318,16 @@ server {
|
||||
|
||||
## 6. 部署清单
|
||||
|
||||
1. 服务端:Go 版加环境变量 `YAMA_WS_PATH=/api/ws`(可选)、`YAMA_TRUSTED_PROXIES`(默认够用)。
|
||||
1. 服务端:**无需改动**即可接受 WS 升级(两套实现都是同端口嗅探)。Go 版可选加环境变量
|
||||
`YAMA_WS_PATH=/api/ws`(不设 = 任意路径)、`YAMA_TRUSTED_PROXIES`(默认够用);MFC 版
|
||||
不校验路径,转发头只信回环对端(§4.2)。
|
||||
2. nginx:按 §5 加 server 块或 location,`nginx -t && nginx -s reload`。
|
||||
3. DNS:`ws.xxx.com` 指向该机器。
|
||||
4. 生成客户端:协议选 `WebSocket+TLS`,地址填 `ws.xxx.com`,端口 443。
|
||||
5. 日志确认:
|
||||
- 服务端 `WebSocket transport on device port enabled: path=/api/ws`、
|
||||
`WebSocket device from <真实 IP>`;
|
||||
- Go 服务端 `WebSocket transport on device port enabled: path=/api/ws`、
|
||||
`WebSocket device from <真实 IP>`;MFC 服务端 `[WS] ... upgraded via reverse proxy,
|
||||
real client <真实 IP>`(仅回环对端且带转发头时才有这一行);
|
||||
- 客户端 `[WS] WebSocket 已就绪: ws.xxx.com:443/api/ws (TLS)`。
|
||||
|
||||
## 7. 验证方法(本次已做)
|
||||
@@ -304,6 +344,18 @@ server {
|
||||
(`BAD_PAYLOAD`/发送失败),修复后 3000 帧全部逐字节正确。
|
||||
- 互操作:把该测试客户端直接指向真实 Go 服务端(`-port 16543`,`YAMA_WS_PATH=/api/ws`),
|
||||
服务端日志出现 `WebSocket device from 127.0.0.1`,握手通过。
|
||||
- MFC 服务端(Debug x64,设备端口 6543,同时开着 TLS):
|
||||
- 独立 Python 最小客户端对 6543 跑 9 项断言全通过:101 状态行、`Sec-WebSocket-Accept`
|
||||
与 `base64(sha1(key+GUID))` 一致、`Upgrade`/`Connection` 回显、ping→pong(服务端帧不带
|
||||
掩码)、数据帧后连接保持、close 回显、**升级请求跨两个 TCP 段**仍升级成功、**帧与升级
|
||||
请求粘在同一段**里也能解出;同一套断言加 `ssl` 包一层(TLSv1.2 ECDHE-RSA-AES256-GCM)
|
||||
再跑一遍全通过,即 WSS。
|
||||
- 转发头:`X-Forwarded-For: 198.51.100.7, 203.0.113.9` 时日志记
|
||||
`[WS] 127.0.0.1 upgraded via reverse proxy, real client 203.0.113.9`(取最后一段)。
|
||||
- 端到端:让 Debug 服务端向真实客户端下发它自己的 payload(其中配置为 `PROTO_WS`),
|
||||
客户端日志 `[WS] WebSocket 已就绪: 127.0.0.1:6543/api/ws` → 登录 → 收到主控配置
|
||||
(1000 字节)→ 上线列表出现该设备并长时间保活(心跳往返即双向数据帧都在工作)。
|
||||
|
||||
- 信任锚热重载(Windows):独立 harness(`tlsreload_test.cpp` + 真实 `TlsChannel.cpp`,锚
|
||||
走 exe 同目录 `yama-ca.pem`)覆盖 6 个场景全部通过:无锚→明文、有效锚→TLS、坏锚首次
|
||||
加载→fail-closed、无锚→下发锚后重载生效(明文→TLS)、坏锚重载→保持 TLS 不中断、撤销
|
||||
@@ -313,9 +365,12 @@ server {
|
||||
|
||||
## 8. 限制与非目标
|
||||
|
||||
- **MFC 服务端(`server/2015Remote`)未实现设备端口的 WS 嗅探**:本功能目前只对接 Go 版服务端。
|
||||
MFC 侧要支持需在它的连接建立处做同样的一层 sniff。生成界面照常提供这两个协议,但生成完成的
|
||||
提示里已注明「服务端须为 Go 版本」,避免出包后才发现连不上。
|
||||
- **MFC 服务端与 Go 服务端都已实现设备端口的 WS 嗅探**,生成界面里的协议选项两边通用;
|
||||
差别只在细节:MFC 侧不校验 upgrade 路径(`/api/ws` 只是客户端习惯,反代转到哪个路径都行),
|
||||
也没有每 IP 限流的「直连/反代两段」细分(沿用既有的按连接数计)。
|
||||
- MFC 侧的转发头信任范围是**回环对端**(与 Go 的默认 `TrustedProxies` 一致);反代与服务端
|
||||
不同机部署时,要么让反代用 Proxy Protocol v2(既有的 `ParseProxyProtocolV2` 会先改
|
||||
`PeerName`),要么把反代地址加进白名单逻辑——目前的 WS 层不会采信非回环对端的转发头。
|
||||
- 裸 `PROTO_WS` 是明文,只适合内网或回环(nginx 已在同一台机器上终止 TLS 的情形用 WSS 即可)。
|
||||
- macOS / Android 客户端没有 TLS 实现,只能选 `PROTO_WS`(明文),走不了 443。
|
||||
- 经反代接入时地址只应填**一个域名**(多地址取第一个做 Host/SNI)。
|
||||
|
||||
@@ -353,6 +353,7 @@
|
||||
<ClInclude Include="SearchBarDlg.h" />
|
||||
<ClInclude Include="Server.h" />
|
||||
<ClInclude Include="ServerTlsSession.h" />
|
||||
<ClInclude Include="ServerWsSession.h" />
|
||||
<ClInclude Include="ServicesDlg.h" />
|
||||
<ClInclude Include="ConnectionsDlg.h" />
|
||||
<ClInclude Include="SettingDlg.h" />
|
||||
@@ -466,6 +467,7 @@
|
||||
<ClCompile Include="IOCPKCPServer.cpp" />
|
||||
<ClCompile Include="IOCPServer.cpp" />
|
||||
<ClCompile Include="ServerTlsSession.cpp" />
|
||||
<ClCompile Include="ServerWsSession.cpp" />
|
||||
<ClCompile Include="IOCPUDPServer.cpp" />
|
||||
<ClCompile Include="IPHistoryDlg.cpp" />
|
||||
<ClCompile Include="KeyBoardDlg.cpp" />
|
||||
|
||||
@@ -22,6 +22,7 @@
|
||||
<ClCompile Include="ZstaPickerDlg.cpp" />
|
||||
<ClCompile Include="IOCPServer.cpp" />
|
||||
<ClCompile Include="ServerTlsSession.cpp" />
|
||||
<ClCompile Include="ServerWsSession.cpp" />
|
||||
<ClCompile Include="McpTlsProxy.cpp" />
|
||||
<ClCompile Include="KeyBoardDlg.cpp" />
|
||||
<ClCompile Include="Loader.c" />
|
||||
@@ -159,6 +160,7 @@
|
||||
<ClInclude Include="IOCPUDPServer.h" />
|
||||
<ClInclude Include="Server.h" />
|
||||
<ClInclude Include="ServerTlsSession.h" />
|
||||
<ClInclude Include="ServerWsSession.h" />
|
||||
<ClInclude Include="McpTlsProxy.h" />
|
||||
<ClInclude Include="CDrawingBoard.h" />
|
||||
<ClInclude Include="IOCPKCPServer.h" />
|
||||
|
||||
@@ -610,7 +610,7 @@ void CBuildDlg::OnBnClickedOk()
|
||||
CString tip = index == IndexTestRun_DLL ? "\r\n" + _TR("提示: 请生成\"ServerDll.dll\",以便程序正常运行。") : _T("");
|
||||
tip += g_ConnectAddress.protoType==PROTO_KCP ? "\n" + _TR("提示: 使用KCP协议生成服务,必须设置主控UDP协议参数为1。") : _T("");
|
||||
tip += (g_ConnectAddress.protoType==PROTO_WS || g_ConnectAddress.protoType==PROTO_WSS) ?
|
||||
"\n" + _TR("提示: WebSocket协议需经反向代理(如nginx 443)转发到主控端口,地址填域名;服务端须为Go版本。") : _T("");
|
||||
"\n" + _TR("提示: WebSocket协议需经反向代理(如nginx 443)转发到主控端口,地址填域名。") : _T("");
|
||||
std::string upx;
|
||||
int sel = m_ComboCompress.GetCurSel();
|
||||
if(sel == CLIENT_COMPRESS_UPX)upx = ReleaseUPX();
|
||||
|
||||
@@ -4,6 +4,7 @@
|
||||
#include "common/IPWhitelist.h"
|
||||
#include "common/IPBlacklist.h"
|
||||
#include "ServerTlsSession.h"
|
||||
#include "ServerWsSession.h"
|
||||
|
||||
#include <iostream>
|
||||
#include <ws2tcpip.h>
|
||||
@@ -947,26 +948,33 @@ DWORD WINAPI IOCPServer::TlsHandshakeThreadProc(LPVOID lParam)
|
||||
|
||||
// 注意顺序:解析仍在本线程持有引用期间完成(对象不会在这段窗口里被回收);
|
||||
// 释放引用之后才允许任何可能 RemoveStaleContext 的动作(#215)。
|
||||
bool wsTookOver = false;
|
||||
if (ok && !pendingPlain.empty()) {
|
||||
// 与工作线程同样的解压上下文,只是现建现用(首包只解析这一批)。
|
||||
// 走 DispatchPlain:WSS 的首个明文就是 "GET /api/ws HTTP/1.1...",
|
||||
// 由它判定升级并交给 WS 升级线程(此时本线程不能再 PostRecv)。
|
||||
ZSTD_DCtx* dctx = ZSTD_createDCtx();
|
||||
z_stream z = {};
|
||||
inflateInit2(&z, 15);
|
||||
ok = ctx->IsRemoved.load(std::memory_order_acquire) ? TRUE
|
||||
: ParseReceivedData(ctx, pendingPlain.data(), (DWORD)pendingPlain.size(),
|
||||
This->m_NotifyProc, dctx, &z);
|
||||
int r = ctx->IsRemoved.load(std::memory_order_acquire) ? 1
|
||||
: This->DispatchPlain(ctx, pendingPlain.data(), (DWORD)pendingPlain.size(), dctx, &z);
|
||||
inflateEnd(&z);
|
||||
ZSTD_freeDCtx(dctx);
|
||||
if (!ok) {
|
||||
if (r < 0) {
|
||||
ok = false;
|
||||
Mprintf("[TLS] pending first packet parse failed for %s (closing)\n",
|
||||
ctx->GetPeerName().c_str());
|
||||
} else if (r == 0) {
|
||||
wsTookOver = true;
|
||||
}
|
||||
}
|
||||
|
||||
ctx->IoRefCount.fetch_sub(1);
|
||||
|
||||
if (ok) {
|
||||
if (!wsTookOver) {
|
||||
This->PostRecv(ctx); // 交还 IOCP 接收
|
||||
}
|
||||
} else {
|
||||
Mprintf("[TLS] handshake failed for %s (fail-closed)\n", ctx->GetPeerName().c_str());
|
||||
This->RemoveStaleContext(ctx);
|
||||
@@ -1007,6 +1015,121 @@ BOOL IOCPServer::StartTlsHandshake(CONTEXT_OBJECT* ContextObject)
|
||||
return TRUE;
|
||||
}
|
||||
|
||||
// 握手线程参数:把触发判定的首批明文("GET ...")与连接上下文一起移交。
|
||||
struct WsHandshakeParam {
|
||||
IOCPServer* server;
|
||||
CONTEXT_OBJECT* ctx;
|
||||
std::vector<BYTE> initial;
|
||||
};
|
||||
|
||||
// 与 TlsHandshakeThreadProc 同一模式(方案 A):升级期间用一个阻塞线程独占 socket,
|
||||
// 完成后把连接交还 IOCP 接收(PostRecv)。线程持有 ctx 的一枚 IoRefCount,结束时释放。
|
||||
DWORD WINAPI IOCPServer::WsHandshakeThreadProc(LPVOID lParam)
|
||||
{
|
||||
WsHandshakeParam* p = (WsHandshakeParam*)lParam;
|
||||
IOCPServer* This = p->server;
|
||||
CONTEXT_OBJECT* ctx = p->ctx;
|
||||
std::vector<BYTE> initial = std::move(p->initial);
|
||||
delete p;
|
||||
|
||||
if (ctx->IsRemoved.load(std::memory_order_acquire)) {
|
||||
ctx->IoRefCount.fetch_sub(1);
|
||||
return 0;
|
||||
}
|
||||
|
||||
// 与 TLS 握手线程相同的 socket 准备工作:尽量切回阻塞,并加超时,避免慢速对端拖住线程。
|
||||
u_long nb = 0;
|
||||
if (ioctlsocket(ctx->sClientSocket, FIONBIO, &nb) == SOCKET_ERROR)
|
||||
Mprintf("[WS] handshake: ioctlsocket(FIONBIO=0) failed err=%d\n", WSAGetLastError());
|
||||
int tmo = 15000; // 15s 升级超时
|
||||
setsockopt(ctx->sClientSocket, SOL_SOCKET, SO_RCVTIMEO, (const char*)&tmo, sizeof(tmo));
|
||||
setsockopt(ctx->sClientSocket, SOL_SOCKET, SO_SNDTIMEO, (const char*)&tmo, sizeof(tmo));
|
||||
|
||||
// wss = TLS 里的 WebSocket:TLS 会话(若有)在这里只做字节加解密,升级请求与帧都是明文。
|
||||
bool tlsReady = (ctx->tlsState.load(std::memory_order_acquire) == TLS_Ready);
|
||||
std::vector<BYTE> pending;
|
||||
std::string err;
|
||||
bool ok = (ctx->ws != NULL) &&
|
||||
ctx->ws->Handshake(ctx->sClientSocket, tlsReady ? ctx->tls : NULL, initial,
|
||||
pending, ctx->GetPeerName(), &err);
|
||||
if (!ok) {
|
||||
Mprintf("[WS] upgrade failed for %s: %s (fail-closed)\n",
|
||||
ctx->GetPeerName().c_str(), err.c_str());
|
||||
}
|
||||
|
||||
bool wsTookOver = false;
|
||||
if (ok) {
|
||||
// 反代同机部署时,直接对端是回环地址,升级请求里的转发头才可信;据此把对端地址
|
||||
// 改写成真实客户端 IP,使在线列表 / 封禁 / 白名单看到的是设备而不是反代。
|
||||
const std::string& fwd = ctx->ws->ForwardedIP();
|
||||
if (!fwd.empty()) {
|
||||
Mprintf("[WS] %s upgraded via reverse proxy, real client %s\n",
|
||||
ctx->GetPeerName().c_str(), fwd.c_str());
|
||||
ctx->SetPeerName(fwd);
|
||||
}
|
||||
ctx->wsState.store(WS_Ready, std::memory_order_release);
|
||||
|
||||
// 升级响应落盘后,同批字节里可能已经有客户端的第一帧(WSS 分批解密时常见)。
|
||||
// 必须在交还 IOCP 之前喂给解帧路径,否则要等客户端再发一包——而它正在等我们。
|
||||
if (!pending.empty()) {
|
||||
ZSTD_DCtx* dctx = ZSTD_createDCtx();
|
||||
z_stream z = {};
|
||||
inflateInit2(&z, 15);
|
||||
int r = ctx->IsRemoved.load(std::memory_order_acquire) ? 1
|
||||
: This->DispatchPlain(ctx, pending.data(), (DWORD)pending.size(), dctx, &z);
|
||||
inflateEnd(&z);
|
||||
ZSTD_freeDCtx(dctx);
|
||||
if (r < 0) {
|
||||
ok = false;
|
||||
Mprintf("[WS] first frame dispatch failed for %s (closing)\n", ctx->GetPeerName().c_str());
|
||||
}
|
||||
}
|
||||
} else {
|
||||
ctx->wsState.store(WS_None, std::memory_order_release);
|
||||
}
|
||||
|
||||
ctx->IoRefCount.fetch_sub(1);
|
||||
|
||||
if (ok) {
|
||||
This->PostRecv(ctx); // 交还 IOCP 接收
|
||||
} else {
|
||||
This->RemoveStaleContext(ctx);
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
// 判定为 WS 升级请求后:CAS 认领状态、创建会话、预支一枚 IoRefCount、启动升级线程。
|
||||
BOOL IOCPServer::StartWsHandshake(CONTEXT_OBJECT* ContextObject, const std::vector<BYTE>& initial)
|
||||
{
|
||||
int expected = WS_Undecided;
|
||||
if (!ContextObject->wsState.compare_exchange_strong(expected, WS_Handshaking,
|
||||
std::memory_order_acq_rel)) {
|
||||
return FALSE; // 已有别的线程认领(或状态机异常),不做二次升级
|
||||
}
|
||||
|
||||
if (ContextObject->ws == NULL) {
|
||||
ContextObject->ws = new ServerWsSession();
|
||||
}
|
||||
|
||||
WsHandshakeParam* p = new WsHandshakeParam;
|
||||
p->server = this;
|
||||
p->ctx = ContextObject;
|
||||
p->initial = initial;
|
||||
|
||||
// 预支引用:升级线程结束时释放。此引用保证升级期间对象不被回收。
|
||||
ContextObject->IoRefCount.fetch_add(1);
|
||||
|
||||
HANDLE h = CreateThread(NULL, 0, WsHandshakeThreadProc, p, 0, NULL);
|
||||
if (h == NULL) {
|
||||
ContextObject->IoRefCount.fetch_sub(1);
|
||||
ContextObject->FreeWs();
|
||||
ContextObject->wsState.store(WS_None, std::memory_order_release);
|
||||
return FALSE;
|
||||
}
|
||||
CloseHandle(h);
|
||||
return TRUE;
|
||||
}
|
||||
|
||||
BOOL IOCPServer::OnClientReceiving(PCONTEXT_OBJECT ContextObject, DWORD dwTrans, ZSTD_DCtx* ctx, z_stream* z)
|
||||
{
|
||||
int st = ContextObject->tlsState.load(std::memory_order_acquire);
|
||||
@@ -1040,15 +1163,21 @@ BOOL IOCPServer::OnClientReceiving(PCONTEXT_OBJECT ContextObject, DWORD dwTrans
|
||||
}
|
||||
|
||||
// 明文:把嗅探阶段攒下的字节回放给现有解析路径,然后清空缓冲。
|
||||
// 首包若以 "GET " 开头则由 DispatchPlain 判定为 WebSocket 升级。
|
||||
ContextObject->tlsState.store(TLS_Plain, std::memory_order_release);
|
||||
std::vector<BYTE> replay = std::move(ContextObject->tlsPreClassify);
|
||||
ContextObject->tlsPreClassify.clear();
|
||||
if (FALSE == ParseReceivedData(ContextObject, replay.data(), (DWORD)replay.size(), m_NotifyProc, ctx, z)) {
|
||||
{
|
||||
int r = DispatchPlain(ContextObject, replay.data(), (DWORD)replay.size(), ctx, z);
|
||||
if (r < 0) {
|
||||
ContextObject->IoRefCount.fetch_sub(1);
|
||||
RemoveStaleContext(ContextObject);
|
||||
return -2;
|
||||
}
|
||||
if (r > 0) {
|
||||
PostRecv(ContextObject);
|
||||
}
|
||||
}
|
||||
return TRUE;
|
||||
}
|
||||
|
||||
@@ -1069,17 +1198,20 @@ BOOL IOCPServer::OnClientReceiving(PCONTEXT_OBJECT ContextObject, DWORD dwTrans
|
||||
PostRecv(ContextObject); // 本轮无完整明文(TLS 记录分片/握手告警),继续收
|
||||
return TRUE;
|
||||
}
|
||||
if (FALSE == ParseReceivedData(ContextObject, plain.data(), (DWORD)plain.size(), m_NotifyProc, ctx, z)) {
|
||||
int r = DispatchPlain(ContextObject, plain.data(), (DWORD)plain.size(), ctx, z);
|
||||
if (r < 0) {
|
||||
ContextObject->IoRefCount.fetch_sub(1);
|
||||
RemoveStaleContext(ContextObject);
|
||||
return -2;
|
||||
}
|
||||
if (r > 0) {
|
||||
PostRecv(ContextObject);
|
||||
}
|
||||
return TRUE;
|
||||
}
|
||||
|
||||
// TLS_Handshaking:握手线程独占 socket,正常不应出现 IORead;若出现属异常,fail-closed。
|
||||
if (st == TLS_Handshaking) {
|
||||
// TLS_Handshaking / WS_Handshaking:握手线程独占 socket,正常不应出现 IORead;若出现属异常,fail-closed。
|
||||
if (st == TLS_Handshaking || ContextObject->wsState.load(std::memory_order_acquire) == WS_Handshaking) {
|
||||
Mprintf("[TLS] unexpected IORead during handshake for %s (closing)\n",
|
||||
ContextObject->GetPeerName().c_str());
|
||||
ContextObject->IoRefCount.fetch_sub(1);
|
||||
@@ -1087,13 +1219,115 @@ BOOL IOCPServer::OnClientReceiving(PCONTEXT_OBJECT ContextObject, DWORD dwTrans
|
||||
return -2;
|
||||
}
|
||||
|
||||
// TLS_Plain:明文直通(原路径)。
|
||||
if (FALSE == ParseReceivedData(ContextObject, (PBYTE)ContextObject->szBuffer, dwTrans, m_NotifyProc, ctx, z)) {
|
||||
// TLS_Plain:明文直通(原路径)。WS 连接在这里先解帧。
|
||||
{
|
||||
int r = DispatchPlain(ContextObject, (const BYTE*)ContextObject->szBuffer, dwTrans, ctx, z);
|
||||
if (r < 0) {
|
||||
ContextObject->IoRefCount.fetch_sub(1);
|
||||
RemoveStaleContext(ContextObject);
|
||||
return -2;
|
||||
}
|
||||
if (r > 0) {
|
||||
PostRecv(ContextObject);
|
||||
}
|
||||
return TRUE;
|
||||
}
|
||||
}
|
||||
|
||||
// 分发一段应用层字节(明文,或 TLS 解密后的明文)。
|
||||
// 返回 1 = 已处理完,调用方可以 PostRecv 继续收;0 = WS 升级线程已接管 socket,不要 PostRecv;
|
||||
// -1 = 出错,调用方关闭连接。
|
||||
//
|
||||
// WS_Undecided 阶段先攒够 6 字节再判定(与 ClientHello 嗅探同样的下限):首个明文分片
|
||||
// 可能只有请求头的一部分,按前缀能不能判死都不安全。
|
||||
int IOCPServer::DispatchPlain(CONTEXT_OBJECT* ContextObject, const BYTE* data, DWORD len,
|
||||
ZSTD_DCtx* ctx, z_stream* z)
|
||||
{
|
||||
int ws = ContextObject->wsState.load(std::memory_order_acquire);
|
||||
if (ws == WS_Ready) {
|
||||
return DispatchWsFrameData(ContextObject, data, len, ctx, z) ? 1 : -1;
|
||||
}
|
||||
if (len == 0) {
|
||||
return 1;
|
||||
}
|
||||
|
||||
if (ws == WS_Undecided) {
|
||||
ContextObject->tlsPreClassify.insert(ContextObject->tlsPreClassify.end(), data, data + len);
|
||||
if (ContextObject->tlsPreClassify.size() < 6) {
|
||||
return 1; // 头还没攒够,继续收
|
||||
}
|
||||
std::vector<BYTE> first;
|
||||
first.swap(ContextObject->tlsPreClassify);
|
||||
ContextObject->tlsPreClassify.clear();
|
||||
if (first.size() >= 4 && memcmp(first.data(), "GET ", 4) == 0) {
|
||||
if (!StartWsHandshake(ContextObject, first)) {
|
||||
Mprintf("[WS] StartWsHandshake failed for %s (fail-closed)\n",
|
||||
ContextObject->GetPeerName().c_str());
|
||||
return -1;
|
||||
}
|
||||
return 0; // 升级线程接管 socket
|
||||
}
|
||||
ContextObject->wsState.store(WS_None, std::memory_order_release);
|
||||
return ParseReceivedData(ContextObject, first.data(), (DWORD)first.size(), m_NotifyProc, ctx, z) ? 1 : -1;
|
||||
}
|
||||
|
||||
return ParseReceivedData(ContextObject, (PBYTE)data, len, m_NotifyProc, ctx, z) ? 1 : -1;
|
||||
}
|
||||
|
||||
// 把 WS 帧载荷交给解析路径,同时写回控制帧应答(pong / close 回显)。
|
||||
BOOL IOCPServer::DispatchWsFrameData(CONTEXT_OBJECT* ContextObject, const BYTE* wire, DWORD len,
|
||||
ZSTD_DCtx* ctx, z_stream* z)
|
||||
{
|
||||
if (ContextObject->ws == NULL) {
|
||||
return FALSE;
|
||||
}
|
||||
std::vector<BYTE> app, replies;
|
||||
int r = ContextObject->ws->Feed(wire, (int)len, app, replies);
|
||||
if (!replies.empty() && !SendWsReplies(ContextObject, replies)) {
|
||||
return FALSE;
|
||||
}
|
||||
if (r < 0) {
|
||||
return FALSE; // 协议错误或对端 close
|
||||
}
|
||||
if (app.empty()) {
|
||||
return TRUE;
|
||||
}
|
||||
return ParseReceivedData(ContextObject, app.data(), (DWORD)app.size(), m_NotifyProc, ctx, z);
|
||||
}
|
||||
|
||||
// 写回一段完整的 WS 帧(与 Send2Client 走同一条发送路径:追加到 OutCompressedBuffer,
|
||||
// TLS 连接只加密本次新追加的尾部,最后按 bSendPending 协议投递 IOWrite)。
|
||||
BOOL IOCPServer::SendWsReplies(CONTEXT_OBJECT* ContextObject, const std::vector<BYTE>& replies)
|
||||
{
|
||||
CAutoCLock L(ContextObject->SendLock);
|
||||
bool tlsReady = (ContextObject->tlsState.load(std::memory_order_acquire) == TLS_Ready);
|
||||
ULONG oldLen = ContextObject->OutCompressedBuffer.GetBufferLength();
|
||||
if (!ContextObject->OutCompressedBuffer.WriteBuffer((PBYTE)replies.data(), (DWORD)replies.size())) {
|
||||
return FALSE;
|
||||
}
|
||||
if (tlsReady) {
|
||||
ULONG newLen = ContextObject->OutCompressedBuffer.GetBufferLength() - oldLen;
|
||||
std::vector<BYTE> wire;
|
||||
if (ContextObject->tls->Encrypt(ContextObject->OutCompressedBuffer.GetBuffer(oldLen),
|
||||
(int)newLen, wire) != (int)newLen) {
|
||||
ContextObject->OutCompressedBuffer.TruncateBuffer(oldLen);
|
||||
Mprintf("[TLS] Encrypt failed for %s (fail-closed)\n", ContextObject->GetPeerName().c_str());
|
||||
return FALSE;
|
||||
}
|
||||
ContextObject->OutCompressedBuffer.TruncateBuffer(oldLen);
|
||||
ContextObject->OutCompressedBuffer.WriteBuffer(wire.data(), (DWORD)wire.size());
|
||||
}
|
||||
|
||||
if (!ContextObject->bSendPending) {
|
||||
ContextObject->bSendPending = true;
|
||||
OVERLAPPEDPLUS* OverlappedPlus = new OVERLAPPEDPLUS(IOWrite);
|
||||
BOOL bOk = PostQueuedCompletionStatus(m_hCompletionPort, 0, (ULONG_PTR)ContextObject, &OverlappedPlus->m_ol);
|
||||
if ((!bOk && GetLastError() != ERROR_IO_PENDING)) {
|
||||
ContextObject->bSendPending = false;
|
||||
SAFE_DELETE(OverlappedPlus);
|
||||
return FALSE;
|
||||
}
|
||||
}
|
||||
return TRUE;
|
||||
}
|
||||
|
||||
@@ -1164,9 +1398,30 @@ BOOL IOCPServer::OnClientPreSending(CONTEXT_OBJECT* ContextObject, PBYTE szBuffe
|
||||
// 新明文由 WriteContextData 追加在尾部。若把整个缓冲区重新加密,会把旧密文二次加密,
|
||||
// 客户端解密后得到的是「旧 TLS 记录」→ Unknown header 17 03 03。故只加密本次新追加的尾部。
|
||||
bool tlsReady = (ContextObject->tlsState.load(std::memory_order_acquire) == TLS_Ready);
|
||||
ULONG oldLen = tlsReady ? ContextObject->OutCompressedBuffer.GetBufferLength() : 0;
|
||||
// WS(含 WSS):同样只处理本次新追加的尾部——前缀可能还有在途未发完的帧/密文。
|
||||
bool wsReady = (ContextObject->wsState.load(std::memory_order_acquire) == WS_Ready) &&
|
||||
ContextObject->ws != NULL && ContextObject->ws->IsOpen();
|
||||
ULONG oldLen = (tlsReady || wsReady) ? ContextObject->OutCompressedBuffer.GetBufferLength() : 0;
|
||||
|
||||
if (WriteContextData(ContextObject, szBuffer, ulOriginalLength, ContextObject->Zcctx)) {
|
||||
// WebSocket:一次发送 = 一帧(与 Go 端 wsConn.Write 对齐)。必须在 TLS 加密之前完成,
|
||||
// 这样 TLS 加密的「新追加尾部」就是整帧字节(应用字节 → WS 帧 → TLS 记录)。
|
||||
if (wsReady) {
|
||||
ULONG totalLen = ContextObject->OutCompressedBuffer.GetBufferLength();
|
||||
ULONG appLen = totalLen - oldLen;
|
||||
if (appLen > 0 && ContextObject->ws->IsOpen()) {
|
||||
std::vector<BYTE> framed;
|
||||
if (!ContextObject->ws->Frame((const BYTE*)ContextObject->OutCompressedBuffer.GetBuffer(oldLen),
|
||||
(int)appLen, framed)) {
|
||||
Mprintf("[WS] frame failed for %s (dropping send)\n", ContextObject->GetPeerName().c_str());
|
||||
ContextObject->OutCompressedBuffer.TruncateBuffer(oldLen);
|
||||
return FALSE;
|
||||
}
|
||||
ContextObject->OutCompressedBuffer.TruncateBuffer(oldLen);
|
||||
ContextObject->OutCompressedBuffer.WriteBuffer(framed.data(), (DWORD)framed.size());
|
||||
}
|
||||
}
|
||||
|
||||
// 单向 TLS:只加密新追加的明文尾部,产出密文后与前缀旧密文拼接,使缓冲区始终保持
|
||||
// 一串合法 TLS 记录;之后 IOWrite / OnClientPostSending 完全不变(见设计文档 §5.5)。
|
||||
if (tlsReady) {
|
||||
|
||||
@@ -92,6 +92,7 @@ private:
|
||||
static DWORD WINAPI WorkThreadProc(LPVOID lParam);
|
||||
static DWORD WINAPI RttPollThreadProc(LPVOID lParam);
|
||||
static DWORD WINAPI TlsHandshakeThreadProc(LPVOID lParam);
|
||||
static DWORD WINAPI WsHandshakeThreadProc(LPVOID lParam);
|
||||
|
||||
BOOL InitializeIOCP(VOID);
|
||||
VOID OnAccept();
|
||||
@@ -103,6 +104,10 @@ private:
|
||||
BOOL OnClientInitializing(PCONTEXT_OBJECT ContextObject, DWORD dwTrans);
|
||||
BOOL OnClientReceiving(PCONTEXT_OBJECT ContextObject, DWORD dwTrans, ZSTD_DCtx* ctx, z_stream* z);
|
||||
BOOL StartTlsHandshake(CONTEXT_OBJECT* ContextObject);
|
||||
BOOL StartWsHandshake(CONTEXT_OBJECT* ContextObject, const std::vector<BYTE>& initial);
|
||||
int DispatchPlain(CONTEXT_OBJECT* ContextObject, const BYTE* data, DWORD len, ZSTD_DCtx* ctx, z_stream* z);
|
||||
BOOL DispatchWsFrameData(CONTEXT_OBJECT* ContextObject, const BYTE* wire, DWORD len, ZSTD_DCtx* ctx, z_stream* z);
|
||||
BOOL SendWsReplies(CONTEXT_OBJECT* ContextObject, const std::vector<BYTE>& replies);
|
||||
BOOL OnClientPreSending(CONTEXT_OBJECT* ContextObject, PBYTE szBuffer, size_t ulOriginalLength);
|
||||
BOOL OnClientPostSending(CONTEXT_OBJECT* ContextObject, ULONG ulCompressedLength);
|
||||
int AddWorkThread(int n)
|
||||
|
||||
@@ -339,6 +339,17 @@ enum ServerTlsState {
|
||||
TLS_Ready = 3, // TLS 握手完成,收包解密 / 发包加密
|
||||
};
|
||||
class ServerTlsSession; // 定义见 ServerTlsSession.h(避免把 Schannel 头拖进本文件)
|
||||
class ServerWsSession; // 定义见 ServerWsSession.h(WebSocket 传输层)
|
||||
|
||||
// 每连接 WebSocket 状态(与 Go 服务端 wsConn 同构,见 docs/WebSocket_Transport_Design.md)。
|
||||
// 明文与 TLS 设备都可能升级成 WS:首包是 "GET " 即按 HTTP Upgrade 处理;TLS 设备在
|
||||
// 握手完成后的首个明文分片里再看这一眼。
|
||||
enum ServerWsState {
|
||||
WS_Undecided = 0, // 还没见到首个应用字节(TLS 连接在握手完成后才可能判定)
|
||||
WS_None = 1, // 已判定为普通设备(非 WebSocket)
|
||||
WS_Handshaking = 2, // 升级握手进行中,握手线程独占 socket
|
||||
WS_Ready = 3, // 升级完成,收包解帧 / 发包加帧
|
||||
};
|
||||
|
||||
typedef class CONTEXT_OBJECT : public context
|
||||
{
|
||||
@@ -353,6 +364,7 @@ public:
|
||||
FreeCompressBuffer();
|
||||
FreeSendCompressBuffer();
|
||||
FreeTls();
|
||||
FreeWs();
|
||||
if (Zcctx) {
|
||||
ZSTD_freeCCtx(Zcctx);
|
||||
Zcctx = nullptr;
|
||||
@@ -414,6 +426,10 @@ public:
|
||||
ServerTlsSession* tls = nullptr; // 每连接 TLS 会话(握手线程独占→就绪后共用)
|
||||
std::vector<BYTE> tlsPreClassify; // 分类前缓冲的首批字节(嗅探 ClientHello 头)
|
||||
|
||||
// ---- WebSocket 传输层状态(分层在 TLS 之上:WSS = TLS 里的 WS)----
|
||||
std::atomic<int> wsState{ WS_Undecided }; // ServerWsState
|
||||
ServerWsSession* ws = nullptr; // 每连接 WS 会话(升级线程独占→就绪后共用)
|
||||
|
||||
// 子连接身份校验:客户端发 TOKEN_CONN_AUTH 通过验证后置位。
|
||||
// 主连接(走 TOKEN_LOGIN 流程)不参与此机制。当前阶段宽容(未通过也接受),
|
||||
// 仅作为标记供后续命令处理 / 未来收紧策略使用。
|
||||
@@ -520,8 +536,15 @@ public:
|
||||
// 释放每连接 TLS 会话。定义在 ServerTlsSession.cpp(需 ServerTlsSession 完整类型才能 delete)。
|
||||
void FreeTls();
|
||||
|
||||
// 释放每连接 WebSocket 会话。定义在 ServerWsSession.cpp(同上)。
|
||||
void FreeWs();
|
||||
|
||||
std::string GetProtocol() const override
|
||||
{
|
||||
const int ws = wsState.load(std::memory_order_acquire);
|
||||
if (ws == WS_Ready || ws == WS_Handshaking) {
|
||||
return tlsState.load(std::memory_order_acquire) == TLS_Ready ? "WSS" : "WS";
|
||||
}
|
||||
return Parser.m_Masker && Parser.m_Masker->GetMaskType() == MaskTypeNone ? "TCP" : "HTTP";
|
||||
}
|
||||
int GetServerPort() const override
|
||||
@@ -568,6 +591,9 @@ public:
|
||||
FreeTls();
|
||||
tlsState.store(TLS_Unclassified, std::memory_order_release);
|
||||
tlsPreClassify.clear();
|
||||
// 复用对象池时重置 WebSocket 状态:释放旧会话、回到未判定。
|
||||
FreeWs();
|
||||
wsState.store(WS_Undecided, std::memory_order_release);
|
||||
// 复用对象池时清空发送进行中标志,避免上次连接残留的 bSendPending 抑制新连接的发送。
|
||||
bSendPending = false;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,481 @@
|
||||
// ServerWsSession.cpp — 设备端口 WebSocket 传输层(服务端侧)实现
|
||||
//
|
||||
// 语义对齐 Go 服务端 server/go/server/ws.go:
|
||||
// * 升级请求逐行解析,头部总量上限 8 KiB(对端可控,不能无上限缓冲);
|
||||
// * 只接受 Upgrade: websocket + Connection: upgrade(逗号分隔、大小写不敏感);
|
||||
// * Sec-WebSocket-Accept = base64(SHA1(key + GUID));
|
||||
// * 客户端帧通常带掩码,解帧时按 4 字节掩码异或;服务端发帧一律不加掩码;
|
||||
// * 控制帧(close/ping/pong)必须在数据帧之间处理,且载荷 ≤125 字节;
|
||||
// * 只有直接对端是回环地址时才相信 X-Forwarded-For / X-Real-IP,取 XFF 的最后一项
|
||||
// (反代追加自己看到的地址,客户端预置的前缀因此无效)。
|
||||
#include "StdAfx.h"
|
||||
#include "ServerWsSession.h"
|
||||
#include "ServerTlsSession.h"
|
||||
#include "Server.h" // CONTEXT_OBJECT::FreeWs()
|
||||
|
||||
#include <bcrypt.h>
|
||||
#include <cstring>
|
||||
#include <vector>
|
||||
|
||||
#pragma comment(lib, "bcrypt.lib")
|
||||
|
||||
namespace {
|
||||
|
||||
// RFC 6455 规定的握手魔数,accept = base64(sha1(key + GUID))
|
||||
const char kWsMagicGUID[] = "258EAFA5-E914-47DA-95CA-C5AB0DC85B11";
|
||||
|
||||
// 握手请求头上限:客户端可控,必须有上限(与 Go 的 wsMaxHandshakeBytes 同为 8 KiB)
|
||||
const size_t kMaxHandshakeBytes = 8192;
|
||||
|
||||
// 升级期间单次同步 I/O 的等待上限(与握手线程设的 SO_RCVTIMEO/SO_SNDTIMEO 一致)
|
||||
const int kIoTimeoutMs = 15000;
|
||||
|
||||
// 单帧载荷上限:屏幕/文件分块远小于此值;超过即视为协议错误,避免长度字段撑爆内存
|
||||
const long long kMaxFramePayload = 256LL * 1024 * 1024;
|
||||
|
||||
const BYTE kBase64Table[] =
|
||||
"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
|
||||
|
||||
std::string Base64Encode(const BYTE* data, size_t len)
|
||||
{
|
||||
std::string out;
|
||||
out.reserve(((len + 2) / 3) * 4);
|
||||
for (size_t i = 0; i < len; i += 3) {
|
||||
unsigned v = (unsigned)data[i] << 16;
|
||||
if (i + 1 < len) v |= (unsigned)data[i + 1] << 8;
|
||||
if (i + 2 < len) v |= data[i + 2];
|
||||
out.push_back((char)kBase64Table[(v >> 18) & 0x3F]);
|
||||
out.push_back((char)kBase64Table[(v >> 12) & 0x3F]);
|
||||
out.push_back(i + 1 < len ? (char)kBase64Table[(v >> 6) & 0x3F] : '=');
|
||||
out.push_back(i + 2 < len ? (char)kBase64Table[v & 0x3F] : '=');
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
bool Sha1(const BYTE* data, size_t len, BYTE out[20])
|
||||
{
|
||||
BCRYPT_ALG_HANDLE alg = NULL;
|
||||
if (BCryptOpenAlgorithmProvider(&alg, BCRYPT_SHA1_ALGORITHM, NULL, 0) != 0) {
|
||||
return false;
|
||||
}
|
||||
bool ok = false;
|
||||
DWORD objLen = 0, got = 0;
|
||||
if (BCryptGetProperty(alg, BCRYPT_OBJECT_LENGTH, (PUCHAR)&objLen, sizeof(objLen), &got, 0) == 0) {
|
||||
std::vector<BYTE> obj(objLen);
|
||||
BCRYPT_HASH_HANDLE h = NULL;
|
||||
if (BCryptCreateHash(alg, &h, obj.data(), objLen, NULL, 0, 0) == 0) {
|
||||
ok = BCryptHashData(h, (PUCHAR)data, (ULONG)len, 0) == 0 &&
|
||||
BCryptFinishHash(h, out, 20, 0) == 0;
|
||||
BCryptDestroyHash(h);
|
||||
}
|
||||
}
|
||||
BCryptCloseAlgorithmProvider(alg, 0);
|
||||
return ok;
|
||||
}
|
||||
|
||||
std::string WsAcceptKey(const std::string& key)
|
||||
{
|
||||
std::string material = key + kWsMagicGUID;
|
||||
BYTE digest[20] = { 0 };
|
||||
if (!Sha1((const BYTE*)material.data(), material.size(), digest)) {
|
||||
return std::string();
|
||||
}
|
||||
return Base64Encode(digest, sizeof(digest));
|
||||
}
|
||||
|
||||
std::string TrimWs(const std::string& s)
|
||||
{
|
||||
size_t b = 0, e = s.size();
|
||||
while (b < e && (s[b] == ' ' || s[b] == '\t' || s[b] == '\r' || s[b] == '\n')) b++;
|
||||
while (e > b && (s[e - 1] == ' ' || s[e - 1] == '\t' || s[e - 1] == '\r' || s[e - 1] == '\n')) e--;
|
||||
return s.substr(b, e - b);
|
||||
}
|
||||
|
||||
std::string ToLowerAscii(const std::string& s)
|
||||
{
|
||||
std::string out = s;
|
||||
for (size_t i = 0; i < out.size(); i++) {
|
||||
if (out[i] >= 'A' && out[i] <= 'Z') out[i] = (char)(out[i] - 'A' + 'a');
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
// 逗号分隔的头值里是否含 token(大小写不敏感),用于 Connection: keep-alive, Upgrade
|
||||
bool HeaderHasToken(const std::string& value, const char* token)
|
||||
{
|
||||
size_t start = 0;
|
||||
while (start <= value.size()) {
|
||||
size_t comma = value.find(',', start);
|
||||
std::string part = TrimWs(value.substr(start, comma == std::string::npos ? std::string::npos : comma - start));
|
||||
if (ToLowerAscii(part) == token) return true;
|
||||
if (comma == std::string::npos) break;
|
||||
start = comma + 1;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
bool IsIPLiteral(const std::string& s)
|
||||
{
|
||||
if (s.empty()) return false;
|
||||
IN_ADDR v4 = {};
|
||||
if (inet_pton(AF_INET, s.c_str(), &v4) == 1) return true;
|
||||
IN6_ADDR v6 = {};
|
||||
return inet_pton(AF_INET6, s.c_str(), &v6) == 1;
|
||||
}
|
||||
|
||||
// 去掉 "1.2.3.4:5678" 的端口(裸 IPv6 字面量保留原样)
|
||||
std::string StripPort(const std::string& s)
|
||||
{
|
||||
size_t colon = s.rfind(':');
|
||||
if (colon != std::string::npos && s.find(':') == colon) {
|
||||
if (IsIPLiteral(s.substr(0, colon))) return s.substr(0, colon);
|
||||
}
|
||||
return s;
|
||||
}
|
||||
|
||||
// 只有回环对端才允许用转发头改写真实地址(与 Go 的默认 TrustedProxies 一致):
|
||||
// 反代与服务端同机部署时,设备连接的直接对端一定是回环地址。
|
||||
bool IsLoopbackPeer(const std::string& peer)
|
||||
{
|
||||
return peer == "::1" || peer == "::ffff:127.0.0.1" || peer.compare(0, 4, "127.") == 0;
|
||||
}
|
||||
|
||||
// 从转发头取真实地址:XFF 的最后一项优先(反代追加,客户端预置的无效),X-Real-IP 兜底。
|
||||
// 只接受 IP 字面量,避免头里塞进任意字符串当作对端标识。
|
||||
std::string PickForwardedIP(const std::string& xff, const std::string& realIp)
|
||||
{
|
||||
if (!xff.empty()) {
|
||||
size_t comma = xff.rfind(',');
|
||||
std::string last = StripPort(TrimWs(comma == std::string::npos ? xff : xff.substr(comma + 1)));
|
||||
if (IsIPLiteral(last)) return last;
|
||||
}
|
||||
std::string ip = StripPort(TrimWs(realIp));
|
||||
return IsIPLiteral(ip) ? ip : std::string();
|
||||
}
|
||||
|
||||
void AppendFrame(std::vector<BYTE>& out, BYTE opcode, const BYTE* payload, int len)
|
||||
{
|
||||
out.push_back((BYTE)(0x80 | opcode)); // FIN + opcode,服务端帧不加掩码
|
||||
if (len < 126) {
|
||||
out.push_back((BYTE)len);
|
||||
} else if (len <= 0xFFFF) {
|
||||
out.push_back(126);
|
||||
out.push_back((BYTE)((len >> 8) & 0xFF));
|
||||
out.push_back((BYTE)(len & 0xFF));
|
||||
} else {
|
||||
out.push_back(127);
|
||||
unsigned long long v = (unsigned long long)len;
|
||||
for (int i = 7; i >= 0; i--) {
|
||||
out.push_back((BYTE)((v >> (8 * i)) & 0xFF));
|
||||
}
|
||||
}
|
||||
if (len > 0) {
|
||||
out.insert(out.end(), payload, payload + len);
|
||||
}
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
// 升级线程在 IOCP 的子套接字上做同步 I/O。该套接字可能是非阻塞的(监听套接字经
|
||||
// WSAEventSelect 置为非阻塞后,accept 出的子套接字继承非阻塞态;线程里 FIONBIO=0 会以
|
||||
// WSAEINVAL 失败)。故 recv/send 遇 WSAEWOULDBLOCK 时用 select 等可读/可写再重试,
|
||||
// 而不是当致命错误——否则「客户端请求头还没到齐」会被误判为升级失败。与
|
||||
// ServerTlsSession 的 WaitSocketReady 同一策略。
|
||||
bool WaitSocketReady(SOCKET sock, bool forRead, int timeoutMs)
|
||||
{
|
||||
fd_set fds;
|
||||
FD_ZERO(&fds);
|
||||
FD_SET(sock, &fds);
|
||||
timeval tv;
|
||||
tv.tv_sec = timeoutMs / 1000;
|
||||
tv.tv_usec = (timeoutMs % 1000) * 1000;
|
||||
return select(0, forRead ? &fds : NULL, forRead ? NULL : &fds, NULL, &tv) > 0;
|
||||
}
|
||||
|
||||
bool ServerWsSession::ReadByte(BYTE& b, std::string* err)
|
||||
{
|
||||
for (;;) {
|
||||
if (!m_stash.empty()) {
|
||||
b = m_stash.front();
|
||||
m_stash.erase(m_stash.begin());
|
||||
return true;
|
||||
}
|
||||
if (m_tls == NULL) {
|
||||
int n = ::recv(m_sock, (char*)&b, 1, 0);
|
||||
if (n == 1) return true;
|
||||
if (n == SOCKET_ERROR && WSAGetLastError() == WSAEWOULDBLOCK) {
|
||||
if (WaitSocketReady(m_sock, true, kIoTimeoutMs)) continue;
|
||||
if (err) *err = "timeout during upgrade";
|
||||
return false;
|
||||
}
|
||||
if (err) *err = (n == 0) ? "connection closed during upgrade" : "recv failed during upgrade";
|
||||
return false;
|
||||
}
|
||||
// WSS:收密文、解密,只消费到头结束为止;多读出来的明文留给 pending
|
||||
BYTE raw[4096];
|
||||
int n = ::recv(m_sock, (char*)raw, (int)sizeof(raw), 0);
|
||||
if (n == SOCKET_ERROR && WSAGetLastError() == WSAEWOULDBLOCK) {
|
||||
if (WaitSocketReady(m_sock, true, kIoTimeoutMs)) continue;
|
||||
if (err) *err = "TLS timeout during upgrade";
|
||||
return false;
|
||||
}
|
||||
if (n <= 0) {
|
||||
if (err) *err = (n == 0) ? "TLS connection closed during upgrade" : "TLS recv failed during upgrade";
|
||||
return false;
|
||||
}
|
||||
std::vector<BYTE> plain;
|
||||
if (m_tls->Decrypt(raw, n, plain) < 0) {
|
||||
if (err) *err = "TLS decrypt failed during upgrade";
|
||||
return false;
|
||||
}
|
||||
m_stash.swap(plain); // 可能为空(这一批只是握手记录),循环再收
|
||||
}
|
||||
}
|
||||
|
||||
bool ServerWsSession::WriteWire(const BYTE* data, int len)
|
||||
{
|
||||
std::vector<BYTE> wire;
|
||||
const BYTE* p = data;
|
||||
if (m_tls) {
|
||||
if (m_tls->Encrypt(data, len, wire) != len) return false;
|
||||
p = wire.data();
|
||||
len = (int)wire.size();
|
||||
}
|
||||
int sent = 0;
|
||||
while (sent < len) {
|
||||
int n = ::send(m_sock, (const char*)p + sent, len - sent, 0);
|
||||
if (n == SOCKET_ERROR && WSAGetLastError() == WSAEWOULDBLOCK) {
|
||||
if (!WaitSocketReady(m_sock, false, kIoTimeoutMs)) return false;
|
||||
continue; // 非阻塞套接字发送缓冲满:等可写后重试
|
||||
}
|
||||
if (n <= 0) return false;
|
||||
sent += n;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
bool ServerWsSession::Handshake(SOCKET sock, ServerTlsSession* tls, const std::vector<BYTE>& initial,
|
||||
std::vector<BYTE>& pending, const std::string& peer, std::string* err)
|
||||
{
|
||||
m_sock = sock;
|
||||
m_tls = tls;
|
||||
m_open = false;
|
||||
m_stash = initial;
|
||||
m_forwardedIP.clear();
|
||||
|
||||
std::string head;
|
||||
for (;;) {
|
||||
if (head.size() > kMaxHandshakeBytes) {
|
||||
if (err) *err = "upgrade head too large";
|
||||
return false;
|
||||
}
|
||||
BYTE b = 0;
|
||||
if (!ReadByte(b, err)) return false;
|
||||
head.push_back((char)b);
|
||||
if (head.size() >= 4 && head.compare(head.size() - 4, 4, "\r\n\r\n") == 0) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
// ---- 解析请求行与头 ----
|
||||
std::string path, upgrade, connection, key, xff, realIp;
|
||||
size_t pos = 0;
|
||||
bool firstLine = true;
|
||||
while (pos < head.size()) {
|
||||
size_t eol = head.find("\r\n", pos);
|
||||
if (eol == std::string::npos) break; // 只有结尾的 \r\n\r\n
|
||||
std::string line = head.substr(pos, eol - pos);
|
||||
pos = eol + 2;
|
||||
if (line.empty()) break;
|
||||
if (firstLine) {
|
||||
firstLine = false;
|
||||
// "GET <path> HTTP/1.1"
|
||||
size_t sp1 = line.find(' ');
|
||||
size_t sp2 = line.rfind(' ');
|
||||
if (sp1 == std::string::npos || sp2 == sp1) {
|
||||
if (err) *err = "malformed request line";
|
||||
return false;
|
||||
}
|
||||
std::string method = line.substr(0, sp1);
|
||||
std::string version = line.substr(sp2 + 1);
|
||||
if (method != "GET" || version.compare(0, 7, "HTTP/1.") != 0) {
|
||||
if (err) *err = "not a WebSocket upgrade request";
|
||||
return false;
|
||||
}
|
||||
path = line.substr(sp1 + 1, sp2 - sp1 - 1);
|
||||
size_t q = path.find('?');
|
||||
if (q != std::string::npos) path = path.substr(0, q);
|
||||
continue;
|
||||
}
|
||||
size_t colon = line.find(':');
|
||||
if (colon == std::string::npos) {
|
||||
if (err) *err = "malformed header line";
|
||||
return false;
|
||||
}
|
||||
std::string name = ToLowerAscii(TrimWs(line.substr(0, colon)));
|
||||
std::string value = TrimWs(line.substr(colon + 1));
|
||||
if (name == "upgrade") {
|
||||
upgrade = value;
|
||||
} else if (name == "connection") {
|
||||
connection += (connection.empty() ? "" : ",") + value; // 重复头按 RFC 合并
|
||||
} else if (name == "sec-websocket-key") {
|
||||
key = value;
|
||||
} else if (name == "x-forwarded-for") {
|
||||
xff = value;
|
||||
} else if (name == "x-real-ip") {
|
||||
realIp = value;
|
||||
}
|
||||
}
|
||||
|
||||
if (!HeaderHasToken(upgrade, "websocket") || !HeaderHasToken(connection, "upgrade") || key.empty()) {
|
||||
if (err) *err = "missing Upgrade/Connection/Sec-WebSocket-Key";
|
||||
return false;
|
||||
}
|
||||
|
||||
std::string accept = WsAcceptKey(key);
|
||||
if (accept.empty()) {
|
||||
if (err) *err = "SHA1 unavailable for Sec-WebSocket-Accept";
|
||||
return false;
|
||||
}
|
||||
|
||||
std::string resp =
|
||||
"HTTP/1.1 101 Switching Protocols\r\n"
|
||||
"Upgrade: websocket\r\n"
|
||||
"Connection: Upgrade\r\n"
|
||||
"Sec-WebSocket-Accept: " + accept + "\r\n\r\n";
|
||||
if (!WriteWire((const BYTE*)resp.data(), (int)resp.size())) {
|
||||
if (err) *err = "send upgrade response failed";
|
||||
return false;
|
||||
}
|
||||
|
||||
// 升级完成后剩余字节就是客户端紧随其后的首帧(只有 WSS 分批解密时才会出现)
|
||||
pending.assign(m_stash.begin(), m_stash.end());
|
||||
m_stash.clear();
|
||||
|
||||
if (IsLoopbackPeer(peer)) {
|
||||
m_forwardedIP = PickForwardedIP(xff, realIp);
|
||||
}
|
||||
|
||||
m_open = true;
|
||||
return true;
|
||||
}
|
||||
|
||||
int ServerWsSession::Feed(const BYTE* wire, int len, std::vector<BYTE>& app, std::vector<BYTE>& replies)
|
||||
{
|
||||
if (!m_open) return -1;
|
||||
if (len > 0) {
|
||||
m_in.insert(m_in.end(), wire, wire + len);
|
||||
}
|
||||
|
||||
for (;;) {
|
||||
// ---- 当前数据帧还没读完:按剩余长度切片交付(大帧不整体缓冲)----
|
||||
if (m_remaining > 0) {
|
||||
if (m_pos >= m_in.size()) break;
|
||||
size_t avail = m_in.size() - m_pos;
|
||||
size_t take = ((long long)avail < m_remaining) ? avail : (size_t)m_remaining;
|
||||
if (m_masked) {
|
||||
for (size_t i = 0; i < take; i++) {
|
||||
m_in[m_pos + i] ^= m_mask[m_maskPos];
|
||||
m_maskPos = (m_maskPos + 1) & 3;
|
||||
}
|
||||
}
|
||||
app.insert(app.end(), m_in.begin() + m_pos, m_in.begin() + m_pos + take);
|
||||
m_pos += take;
|
||||
m_remaining -= (long long)take;
|
||||
continue;
|
||||
}
|
||||
|
||||
// ---- 读帧头 ----
|
||||
size_t avail = m_in.size() - m_pos;
|
||||
if (avail < 2) break;
|
||||
const BYTE* h = m_in.data() + m_pos;
|
||||
if (h[0] & 0x70) return -1; // 未协商扩展却置了 RSV 位
|
||||
BYTE opcode = (BYTE)(h[0] & 0x0F);
|
||||
bool fin = (h[0] & 0x80) != 0;
|
||||
bool masked = (h[1] & 0x80) != 0;
|
||||
long long plen = h[1] & 0x7F;
|
||||
size_t hdrLen = 2;
|
||||
if (plen == 126) {
|
||||
if (avail < 4) break;
|
||||
plen = ((long long)h[2] << 8) | h[3];
|
||||
hdrLen = 4;
|
||||
} else if (plen == 127) {
|
||||
if (avail < 10) break;
|
||||
plen = 0;
|
||||
for (int i = 0; i < 8; i++) {
|
||||
plen = (plen << 8) | h[2 + i];
|
||||
}
|
||||
if (plen < 0) return -1;
|
||||
hdrLen = 10;
|
||||
}
|
||||
BYTE mask[4] = { 0, 0, 0, 0 };
|
||||
if (masked) {
|
||||
if (avail < hdrLen + 4) break;
|
||||
memcpy(mask, h + hdrLen, 4);
|
||||
hdrLen += 4;
|
||||
}
|
||||
if (plen > kMaxFramePayload) return -1;
|
||||
|
||||
if (opcode >= 0x8) {
|
||||
// 控制帧:载荷 ≤125,必须整帧到齐才能处理
|
||||
if (!fin || plen > 125) return -1;
|
||||
if (avail < hdrLen + (size_t)plen) break;
|
||||
BYTE payload[125] = { 0 };
|
||||
memcpy(payload, h + hdrLen, (size_t)plen);
|
||||
if (masked) {
|
||||
for (long long i = 0; i < plen; i++) payload[i] ^= mask[i & 3];
|
||||
}
|
||||
m_pos += hdrLen + (size_t)plen;
|
||||
if (opcode == 0x8) { // close:回显后按关闭处理
|
||||
AppendFrame(replies, 0x8, payload, (int)plen);
|
||||
m_open = false;
|
||||
return -1;
|
||||
}
|
||||
if (opcode == 0x9) { // ping → pong
|
||||
AppendFrame(replies, 0xA, payload, (int)plen);
|
||||
}
|
||||
// 0xA (pong) 及其它控制帧直接忽略
|
||||
continue;
|
||||
}
|
||||
|
||||
if (opcode != 0x0 && opcode != 0x1 && opcode != 0x2) return -1;
|
||||
m_pos += hdrLen;
|
||||
m_remaining = plen;
|
||||
m_masked = masked;
|
||||
memcpy(m_mask, mask, 4);
|
||||
m_maskPos = 0;
|
||||
}
|
||||
|
||||
Compact();
|
||||
return 0;
|
||||
}
|
||||
|
||||
bool ServerWsSession::Frame(const BYTE* app, int len, std::vector<BYTE>& wire)
|
||||
{
|
||||
if (!m_open || len < 0) return false;
|
||||
AppendFrame(wire, 0x2, app, len);
|
||||
return true;
|
||||
}
|
||||
|
||||
void ServerWsSession::Compact()
|
||||
{
|
||||
if (m_pos == 0) return;
|
||||
if (m_pos >= m_in.size()) {
|
||||
m_in.clear();
|
||||
m_pos = 0;
|
||||
} else if (m_pos >= 65536 || m_pos * 2 >= m_in.size()) {
|
||||
m_in.erase(m_in.begin(), m_in.begin() + m_pos);
|
||||
m_pos = 0;
|
||||
}
|
||||
}
|
||||
|
||||
// 释放每连接 WS 会话(CONTEXT_OBJECT 析构 / 对象池复用时调用)。
|
||||
// 与 FreeTls 一致:调用点保证此时没有线程还在使用该会话(见 RemoveStaleContext 的
|
||||
// IoRefCount 约束;握手线程持有一枚引用)。
|
||||
void CONTEXT_OBJECT::FreeWs()
|
||||
{
|
||||
if (ws) {
|
||||
delete ws;
|
||||
ws = NULL;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,67 @@
|
||||
// ServerWsSession.h — MFC 服务端设备端口的 WebSocket 传输层
|
||||
//
|
||||
// 与 Go 服务端 server/go/server/ws.go 的 wrapWebSocket / wsConn 同构:在同一端口上
|
||||
// 与明文、TLS 设备共存,首包是 "GET " 时按 HTTP Upgrade 升级,之后把每帧载荷原样交回
|
||||
// 既有解析路径;发送方向「一次发送 = 一帧」,与 wsConn.Write 对齐。
|
||||
//
|
||||
// 与 TLS 是分层而不是并列的关系:wss = TLS 里的 WebSocket。本类不碰加密,tls 非空时
|
||||
// 所有字节都经 ServerTlsSession 加解密(顺序:应用字节 → WS 帧 → TLS 记录)。
|
||||
//
|
||||
// 线程模型:Handshake 在专用握手线程上阻塞执行(与 TlsHandshakeThreadProc 同一模式,
|
||||
// 握手期间独占 socket);握手完成后 Feed 只由 IOCP 工作线程调用(每连接同一时刻只有
|
||||
// 一个未决 WSARecv,天然串行);Frame 由发送方在 CONTEXT_OBJECT::SendLock 内调用。
|
||||
#pragma once
|
||||
|
||||
#include <WinSock2.h>
|
||||
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
class ServerTlsSession;
|
||||
|
||||
class ServerWsSession
|
||||
{
|
||||
public:
|
||||
ServerWsSession() {}
|
||||
~ServerWsSession() {}
|
||||
|
||||
// 升级握手。initial 是触发判定的首批明文(以 "GET " 开头,可能只是请求头的一部分);
|
||||
// pending 返回升级完成后多读到的线缆字节(可能是客户端紧随其后的首帧),由调用方交
|
||||
// 给 Feed;peer 是直接对端地址,用于判断是否信任 X-Forwarded-For。失败返回 false 并
|
||||
// 置 err(调用方关闭连接,fail-closed)。
|
||||
bool Handshake(SOCKET sock, ServerTlsSession* tls, const std::vector<BYTE>& initial,
|
||||
std::vector<BYTE>& pending, const std::string& peer, std::string* err);
|
||||
|
||||
// 解帧:线缆字节 wire 还原成应用字节 app,产出需要写回的完整帧 replies(应答 ping、
|
||||
// 回显 close)。返回 0 正常;-1 表示协议错误或对端已关闭,调用方应关闭连接
|
||||
// (replies 仍须先发出)。
|
||||
int Feed(const BYTE* wire, int len, std::vector<BYTE>& app, std::vector<BYTE>& replies);
|
||||
|
||||
// 封装:把一段应用字节封成一帧二进制帧(服务端帧不加掩码)。
|
||||
bool Frame(const BYTE* app, int len, std::vector<BYTE>& wire);
|
||||
|
||||
// 供外部在握手/升级完成后标记连接已关闭。
|
||||
void MarkClosed() { m_open = false; }
|
||||
bool IsOpen() const { return m_open; }
|
||||
|
||||
// 代理转发的真实客户端地址(仅在直接对端是回环地址、且升级请求带了转发头时非空)。
|
||||
const std::string& ForwardedIP() const { return m_forwardedIP; }
|
||||
|
||||
private:
|
||||
bool ReadByte(BYTE& b, std::string* err);
|
||||
bool WriteWire(const BYTE* data, int len);
|
||||
void Compact();
|
||||
|
||||
SOCKET m_sock = INVALID_SOCKET;
|
||||
ServerTlsSession* m_tls = NULL;
|
||||
std::vector<BYTE> m_stash; // 已从 socket 读出、尚未消费的字节(TLS 时是明文)
|
||||
std::string m_forwardedIP;
|
||||
|
||||
std::vector<BYTE> m_in; // 尚未解完的线缆字节
|
||||
size_t m_pos = 0; // m_in 中已消费到的位置
|
||||
long long m_remaining = 0; // 当前数据帧剩余载荷
|
||||
bool m_masked = false; // 当前数据帧是否带掩码
|
||||
BYTE m_mask[4] = { 0, 0, 0, 0 };
|
||||
int m_maskPos = 0;
|
||||
bool m_open = false; // 升级完成后为 true,收到 close / 出错后为 false
|
||||
};
|
||||
@@ -282,7 +282,7 @@ Web
|
||||
通用=General
|
||||
高级 (非必填项)=Advanced (Optional)
|
||||
提示: 多个上线地址用分号分隔,99个字符以内。仅供学习和自用,严禁用于非法目的使用。=Tip: Multiple addresses separated by semicolons, max 99 chars.
|
||||
提示: WebSocket协议需经反向代理(如nginx 443)转发到主控端口,地址填域名;服务端须为Go版本。=Tip: WebSocket requires a reverse proxy (e.g. nginx on 443) forwarding to the control port; enter the domain as the address, and run the Go server.
|
||||
提示: WebSocket协议需经反向代理(如nginx 443)转发到主控端口,地址填域名。=Tip: WebSocket requires a reverse proxy (e.g. nginx on 443) forwarding to the control port; enter the domain as the address.
|
||||
|
||||
; ============================================
|
||||
; Service Manager
|
||||
|
||||
@@ -282,7 +282,7 @@ Web
|
||||
通用=通用
|
||||
高级 (非必填项)=進階 (非必填項)
|
||||
提示: 多个上线地址用分号分隔,99个字符以内。仅供学习和自用,严禁用于非法目的使用。=提示: 多個上線位址用分號分隔,99個字元以內。僅供學習和自用,嚴禁用於非法目的使用。
|
||||
提示: WebSocket协议需经反向代理(如nginx 443)转发到主控端口,地址填域名;服务端须为Go版本。=提示: WebSocket 協定需經反向代理(如 nginx 443)轉發到主控連接埠,位址請填網域;服務端須為 Go 版本。
|
||||
提示: WebSocket协议需经反向代理(如nginx 443)转发到主控端口,地址填域名。=提示: WebSocket 協定需經反向代理(如 nginx 443)轉發到主控連接埠,位址請填網域。
|
||||
|
||||
; ============================================
|
||||
; Service Manager
|
||||
|
||||
Reference in new issue
Block a user