Perf: Optimize macOS screen capture with CGDisplayStream

Core optimization:
- Use CGDisplayStream instead of per-frame CGDisplayCreateImage
- Push model: CPU sleeps when screen is static (condition_variable wait)
- IOSurface capture avoids expensive image creation per frame
- ~47% CPU reduction during active remote desktop (45% → 24%)

Additional optimizations:
- vImageVerticalReflect (SIMD) replaces manual row-by-row flip
- Cache CGColorSpaceRef to avoid per-frame creation/release
- Cache tempBuffer to avoid per-frame memory allocation
- Throttle getCursorTypeIndex to 250ms (Accessibility API is expensive)

Bug fixes:
- Fix unreliable screen capture permission check (use actual capture test)
- Improve permission logging

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
yuanyuanxiang
2026-05-03 23:18:30 +02:00
parent b732f841d0
commit 92f3df8464
7 changed files with 483 additions and 43 deletions

View File

@@ -3,6 +3,7 @@
#import <CoreGraphics/CoreGraphics.h>
#import <dispatch/dispatch.h>
#import <IOKit/pwr_mgt/IOPMLib.h>
#import <IOSurface/IOSurface.h>
#import "../client/IOCPClient.h"
#import "../common/commands.h" // QualityLevel, QualityProfile, ALGORITHM_*
#include <vector>
@@ -11,6 +12,7 @@
#include <thread>
#include <cstdint>
#include <memory>
#include <condition_variable>
// Forward declarations
class IOCPClient;
@@ -118,6 +120,7 @@ private:
std::vector<uint8_t> m_prevFrame;
std::vector<uint8_t> m_currFrame;
std::vector<uint8_t> m_diffBuffer;
std::vector<uint8_t> m_tempBuffer; // 临时缓冲区,避免每帧分配
// Quality settings
std::atomic<uint8_t> m_algorithm;
@@ -133,4 +136,25 @@ private:
// Power management: prevent display sleep during remote desktop
IOPMAssertionID m_displayAssertionID;
// Cached color space (avoid per-frame creation)
CGColorSpaceRef m_colorSpace;
// CGDisplayStream (efficient continuous capture)
CGDisplayStreamRef m_displayStream;
dispatch_queue_t m_streamQueue;
IOSurfaceRef m_latestSurface;
std::mutex m_surfaceMutex;
std::condition_variable m_surfaceCond;
std::atomic<bool> m_hasNewFrame;
// Initialize/cleanup display stream
bool initDisplayStream();
void cleanupDisplayStream();
// Process frame from IOSurface (called by stream callback)
void processIOSurface(IOSurfaceRef surface);
// Capture from IOSurface to buffer (with vertical flip)
bool captureFromIOSurface(IOSurfaceRef surface, std::vector<uint8_t>& buffer);
};