Skip to content

Audio, Screen, and Webcam Recording Backends#1081

Merged
jonoomph merged 30 commits into
developfrom
audio-recording
Jul 22, 2026
Merged

Audio, Screen, and Webcam Recording Backends#1081
jonoomph merged 30 commits into
developfrom
audio-recording

Conversation

@jonoomph

@jonoomph jonoomph commented Jul 22, 2026

Copy link
Copy Markdown
Member

This PR adds cross-platform recording support to libopenshot, including microphone, screen, webcam, and
supported system-audio capture. It also introduces higher-density peak/RMS waveform data, accurate live-
capture timestamps, a new Timer effect, and webcam layout properties.

Recording Support

  • Linux/X11 ✅ V4L2 ✅ PulseAudio
  • Linux/Wayland ✅ PipeWire portal ✅ V4L2 ✅ PulseAudio
  • Windows ✅ GDI ✅ DirectShow ✅ WASAPI loopback
  • macOS ✅ AVFoundation ✅ AVFoundation

Audio Recording

  • Add a JUCE-based AudioRecorder
  • Support WAV, FLAC, and MP3 output
  • Provide live peak and RMS level monitoring
  • Generate configurable peak/RMS waveform samples while recording
  • Fix FFmpegWriter FLAC duration metadata
  • Keep microphone audio in an independently editable file

Screen Capture

  • Add Linux X11 screen capture
  • Add Wayland capture through xdg-desktop-portal and PipeWire
  • Support Wayland crop metadata and cropped window streams
  • Drop invalid PipeWire frames safely
  • Add Windows GDI screen capture
  • Add macOS AVFoundation screen capture
  • Support full-screen, window, and region bounds where available
  • Interrupt blocking capture reads during shutdown

System Audio

  • Add PulseAudio monitor capture on Linux
  • Add WASAPI loopback capture on Windows
  • Synchronize initial audio with captured video frames
  • Preserve system-audio timing during frame writing
  • Fix MinGW Core Audio GUID linking
  • Report backend support so clients can disable unavailable controls
  • macOS system-audio capture is not currently supported

Webcam Capture

  • Add Linux V4L2 camera capture
  • Add Windows DirectShow camera capture and COM enumeration
  • Add macOS AVFoundation camera capture
  • Support device-provided default capture modes
  • Improve capture buffer cleanup and shutdown behavior

Timing and Synchronization

  • Preserve source capture timestamps on live frames
  • Preserve timing gaps instead of compressing dropped or delayed frames
  • Support sparse video timestamps in FFmpegWriter
  • Align microphone, screen, webcam, and system-audio recordings
  • Add timestamp compatibility tests

Waveforms

  • Add configurable waveform sample density
  • Store both peak and RMS envelopes
  • Preserve absolute signal levels instead of always normalizing quiet audio
  • Fix recording waveform extraction and FLAC metadata handling

Clip Layout Properties

  • Add animatable clip margin
  • Add animatable corner radius
  • Support inset and rounded webcam overlays
  • Include JSON serialization and rendering tests

Timer Effect

  • Add a configurable Timer effect
  • Support count-up and count-down modes
  • Provide text, font, color, background, outline, position, and timing controls
  • Register the effect across C++, Python, Java, and Ruby bindings

Compatibility and Build Improvements

  • Support older FFmpeg avdevice headers and input-format APIs
  • Preserve compatibility with the existing macOS Python build environment
  • Add required public headers and SWIG bindings
  • Add optional Wayland/PipeWire build detection
  • Link required Windows COM and multimedia libraries
  • Install libavdevice-dev in GitHub Actions
  • Expand Linux, Windows, and macOS build configuration

Validation

  • Full libopenshot build succeeds
  • All 515 unit tests pass
  • Python bindings expose the new recorder, capture readers, settings, and Timer effect
  • Manually verified Linux/X11 and Windows recording
  • Manually verified Linux and Windows system-audio capture
  • GitLab CI passes across the supported builders

jonoomph added 30 commits May 28, 2026 06:47
- Add AudioRecorder with input device capture, monitoring, recording, stats, level snapshots, and waveform snapshots
- Add AudioRecorder settings, level data, waveform chunk data, and frame factory helpers
- Expose AudioRecorder and related APIs to Python/SWIG
- Add AudioDevices input device listing support
- Register AudioRecorder in CMake and OpenShot public headers
- Write recorded audio through FFmpegWriter using configured codec, sample rate, channels, layout, and bitrate
- Finalize recorder writers cleanly on stop/close, including prepared recordings that never start
- Update FFmpegWriter audio timestamps, packet durations, encoder flush behavior, and stream duration metadata
- Fix FLAC outputs so FFmpegReader reports valid duration and video length after recording
- Improve AudioWaveformer extraction for audio files with missing/unknown duration metadata
- Add AudioRecorder unit tests for validation, monitoring, waveform, levels, and synthetic recording behavior
- Add AudioWaveformer/FFmpegWriter regression coverage for FLAC duration and waveform extraction
…ak our Mac builder by including the wrong version of python at this moment. We'll revisit in the future.
Introduce FFmpeg-backed live capture readers for X11 screen capture and v4l2 camera devices, including JSON/settings support, Python bindings, public OpenShot includes, and focused metadata validation tests.
@jonoomph
jonoomph merged commit eac81cf into develop Jul 22, 2026
11 checks passed
@jonoomph
jonoomph deleted the audio-recording branch July 22, 2026 19:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant