9486292

Merge pull request #21 from ehfd/vaapi-sei

fix(core): Cursor latency improvements

2.0.0

PixelFlux 2.0.0

A major release featuring a complete rewrite of the X11 capture core from C++ to Rust, unified encoding pipelines, and significant performance and reliability improvements.

Breaking Changes

• X11 capture core rewritten in Rust : both X11 and Wayland backends now share the same encoding profiles and pipeline logic in a single unified extension
• Python bindings migrated from ctypes to PyO3 : import pixelflux now loads a native CPython extension directly. CaptureSettings and ScreenCapture are both native (PyO3) classes — CaptureSettings carries a __dict__, so you set fields as attributes and may stash extras — and callbacks receive StripeFrame buffer-protocol objects instead of ctypes pointers. No more ctypes.byref() or manual memory management
• pixelflux_wayland module removed : Wayland code is merged into the single pixelflux crate
• Wayland display no longer auto-starts : callers must explicitly call ensure_wayland_display() before capture if they need WAYLAND_DISPLAY available early for companion applications
• WaylandBackend constructor changed : now requires (width, height, dri_node, auto_gpu_selected=False, cursor_size=-1) instead of no arguments
• start_capture(callback, settings) on both backends : ScreenCapture and WaylandBackend both take the callback first, then the settings (X11's ScreenCapture.start_capture was realigned to match Wayland)
• All h264_* settings renamed to video_* : e.g. h264_crf → video_crf, h264_bitrate_kbps → video_bitrate_kbps
• vaapi_render_node_index replaced by encode_node_index : -2 = auto-detect (new default), -1 = forced software, >=0 = specific GPU by index
• VBV buffer changed from fixed kilobits to a multiplier : h264_vbv_buffer_size_kb (fixed value) replaced by video_vbv_multiplier (multiplier of one frame's bit budget, auto-rescales with live bitrate/fps changes; <=0 = policy default: 1.5 infinite-GOP, 3 with scheduled keyframes)
• Python requirement raised to >=3.9

New Features

• Computer Use API : Wayland-only HTTP server (tiny_http, bound to 0.0.0.0:<port>) that dispatches on a JSON action field — screenshot, mouse move/click/drag/down-up, keyboard type/key-combos/hold, scroll, cursor-position query, wait, and region zoom — injecting input into the in-process compositor. Enabled via PIXELFLUX_CU=<port>
• StripeFrame buffer-protocol class : zero-copy frame access across Python 3.9–3.14 via memoryview(frame), with metadata fields for data_type, frame_id, stripe_y_start, and stripe_height
• OpenH264 software encoder : Cisco's BSD-licensed OpenH264 as an alternative to x264, selectable via use_openh264=True (built from vendored source and statically linked — no runtime download)
• Auto GPU selection : new auto_gpu setting that walks /sys/class/drm and picks the first render node whose token matches by kernel driver name, PCI vendor ID, or devicetree/OF vendor prefix ("true"/any truthy value = first GPU). With the default encode_node_index=-2, an enabled auto_gpu triggers auto-detection (otherwise -2 falls back to device 0)
• stripe_frame_from_buffer() : wrap pre-encoded bytes into a StripeFrame for testing or alternative pipelines
• atexit handler : all live captures are cleanly stopped before interpreter shutdown
• Out-of-band X11 cursor delivery : X11 cursor images are delivered through the same set_cursor_callback API as Wayland (advertised by the X11_CURSOR_CALLBACK module flag) instead of being composited into the framebuffer
• Expanded WaylandBackend control surface : clipboard get/set with copy notifications, xkb keymap get/set plus key and pointer injection, live update_rate(bitrate_kbps, vbv_multiplier, fps), and request_idr_frame() for client (re)joins

New CaptureSettings Fields

Field	Type	Default	Purpose
use_openh264	bool	false	Select OpenH264 over x264
omit_stripe_headers	bool	false	Emit raw payload without the 10-byte per-stripe wire header
keyframe_interval_s	f64	0.0	Periodic IDR cadence in seconds (0 = infinite GOP)
video_min_qp / video_max_qp	i32	0 / 0	CBR QP clamp range (0 = encoder default; CRF/CQP ignore)
encode_node_path	str/bytes	None	Explicit encoder device path
render_node_path	str/bytes	None	Explicit render device (Wayland)
recording_socket	str	None	Unix socket path for H.264 recording tap
cursor_size	int	-1	Compositor cursor theme size (≤0 = theme default, 24)
cursor_size_cap	int	32	Longest edge (px) of X11 out-of-band cursor images; larger are downscaled (≤0 = uncapped)
auto_gpu	str	None	Auto-select a GPU render node ("" = off, "true" = first, else a vendor/driver/PCI-vendor/devicetree token)
use_wayland	bool	None	Backend select: True = Wayland, False = X11, None = follow WAYLAND_DISPLAY

Encoding & Pipeline

• H.264 striping with threaded encode (CPU/x264 path) : per-stripe threading (via rayon) trades a slight CPU increase for approximately 30% reduction in frame delivery latency to the client
• NVENC completely rewritten : runtime loading via libloading (no compile-time NVIDIA packages required), API version negotiation probing from 13.0 down to 10.0 (13.0, 12.1, 12.0, 11.1, 11.0, 10.0), ARGB direct encoding with driver-side BT.709 colorspace conversion (no CUDA Toolkit/nvcc — the CUDA driver libcuda.so.1 is still loaded at runtime), 4:4:4 High profile support, multi-GPU container support via GPU enumeration filtering, and in-place reconfiguration on resolution/rate changes without session rebuild. H.264 only (HEVC/AV1 GUIDs exist in the bindings but are unused)
• VA-API encoder rewritten : uses FFmpeg (ffmpeg-sys-next, avcodec/avfilter only) with encode_raw() support for NV12 planar input and live rate reconfiguration. 4:4:4 full-color is not supported on VA-API and falls back to the CPU x264 encoder
• Shared pipeline decision logic : unified send/QP/keyframe decisions for all full-frame encoders (NVENC, VA-API, OpenH264) with periodic IDR scheduling via keyframe_interval_s (the striped x264 path mirrors this policy in its own code)
• libx264 global state protection : encoder open/close serialized under a process-global mutex (X264_OPEN_CLOSE_LOCK) to prevent races between concurrent capture instances (never held around encode)
• Thread priority boost : encode and delivery threads run at elevated priority (nice -10; capture/compositor threads at -15) for more consistent frame pacing

Resilience & Fixes

• GBM buffer resize failures degrade gracefully instead of panicking the compositor thread
• Wayland encode delivery uses non-blocking try_send with a parking slot : a full channel no longer blocks the calloop event loop
• Pending damage from paused ticks is folded into the next encode so no screen changes are lost
• Stale LiveTunables snapshot is cleared on StartCapture to prevent old settings from overriding fresh configuration
• Frame buffer allocation clamps dimensions to max(0) to prevent overflow on negative sizes
• Encode thread ID can be late-resolved from a channel receiver, preventing re-entrant-stop deadlock cycles
• Recording sink simplified : the recording sink now intercepts already-encoded frames at the output rather than plumbing write_frame calls into every encoder individually. This introduces a small amount of additional CPU load (a write_all per connected client per frame) but works uniformly across all full-frame encoding paths, including X11, which was previously unsupported
• NVENC compatible sessions reconfigure in place (milliseconds) instead of destroy-and-rebuild
• PY_SHUTDOWN flag prevents detached threads from calling into a finalizing Python interpreter
• auto_adjust_screen_capture_size now defaults to false : callers must opt in explicitly

Build & CI

• Wheels built and published as GitHub release artifacts via cibuildwheel (v4.1.0) : pre-release builds on every push to master, full releases on semver tags
• manylinux_2_28 and musllinux wheels, for both x86_64 and aarch64 : glibc and musl (Alpine), Intel and ARM64 — four platform combinations
• No CPU baseline forced : nothing sets target-cpu (e.g. x86-64-v3), so wheels run on baseline x86_64 / aarch64 for wider hardware support
• Builds for Python 3.9 through 3.14 (CPython only; free-threaded and PyPy builds are skipped)
• Rust installed via rustup across all build targets for consistent toolchain versions
• Removed system build dependencies: libyuv, libjpeg-turbo, libev, libXcursor, libXcomposite (now pure-Rust yuv, calloop, and xcursor crates; JPEG via the vendored, statically-linked turbojpeg-sys)
• FFmpeg bumped to n8.1, built pure LGPL (no --enable-gpl); only avcodec/avfilter are consumed (for h264_vaapi)
• New local nvcodec-sys subcrate for NVIDIA Video Codec SDK 13.0 headers with committed bindgen output (its build.rs is a no-op unless the opt-in regen feature is set)

a9f1993

simplify recording sink to intercept frames at module delivery and strip headers if needed, introduces small amount of mem copies while working universally for full frame encoding methods

0eccb31

Merge pull request #22 from ehfd/vaapi-sei

fix(core): Cleanup API and Docs

0de2d13

restore env var ingestion for recording socket

8c872cf

Merge pull request #20 from ehfd/vaapi-sei

Remove SEI option in VAAPI + Other Fixes

3cb5b48

fix readback cpu churn on no damage, send cached frame to encoder

5956367

add initial 2.0 changelog

c83d9bc

restore gl read pixels async to not deadlock nvidia encoders and intel on pixel readback and no damage

23aa571

use current rust on musl