Skip to content

Add Frame coalesce limit - #32

Merged
sedawwk merged 1 commit into
masterfrom
feature/frame-coalesce-limit
Sep 4, 2026
Merged

Add Frame coalesce limit#32
sedawwk merged 1 commit into
masterfrom
feature/frame-coalesce-limit

Conversation

@heshaoqiong-tuya

@heshaoqiong-tuya heshaoqiong-tuya commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

What

Small transport frames now go out as a single write. A frame whose whole wire
size — frame header + app header + payload + signature — is strictly under
TAI_FRAME_COALESCE_LIMIT (new compile-time knob, default 512 B) is gathered
into tx_ctrl_buf and emitted with one ctx_io_send:

scatter path:  [hdr‖app-hdr] [payload] [sig]  →  2–3 tls_write
coalesced:     [hdr‖app-hdr‖payload‖sig]      →  1 tls_write

Ping, every control packet (SessionNew/Close, the Event series, ChatBreak) and
small audio chunks all fall inside the window.

Why

Each ctx_io_send is a locked tls_write plus an mbedtls record build —
tens of microseconds per call. The typical TAI frame is small (Ping ≈ 47 B on
the wire; a 20 ms Opus chunk + header + signature ≈ 200–400 B), so per-write
overhead dominated the send path for exactly the packets sent most often. The
gather is a bounded memmove/memcpy of at most 511 B — noise next to one
saved TLS record.

Design notes

  • The scratch buffer IS the control packet. A control packet's payload is
    already tx_ctrl_buf (send_app hands it straight through), so the gather
    relocates the payload to its final offset first (memmove, overlap-safe)
    before the frame header overwrites its front. The frame HMAC is computed
    over the original bytes before any relocation; the in-place shift preserves
    them, so the signature stays valid.
  • Zero-length payloads arrive as NULL. Header-only packets (Audio END)
    pass pay = NULL, pay_len = 0; the relocation is length-guarded — glibc
    declares memmove non-NULL and UBSan flags the call even for length 0.
    (Caught by Linux CI: Darwin's string.h carries no non-NULL attributes, so
    this class is invisible to a Mac-hosted UBSan.)
  • Two independent knobs. Coalescing is capped at the smaller of
    TAI_FRAME_COALESCE_LIMIT and TAI_TX_CTRL_BUF_SIZE, so a
    memory-constrained build that shrinks the control buffer narrows the
    coalesce window instead of writing past tai_ctx. Defaults stay
    independent (512 / 1024): the coalesce limit is a performance window, the
    control buffer's job is holding the largest control packet (escaped
    session/event JSON + framing) — deriving one from the other would turn a
    performance knob into a capacity knob.
  • §6.3 semantics unchanged. Pre-wire failures still return the build
    error with the sequence rolled back; any failure after the first byte on
    the wire returns TAI_ERR_NET with the sequence consumed. For a coalesced
    frame that first byte is the single write.

@heshaoqiong-tuya
heshaoqiong-tuya marked this pull request as draft September 3, 2026 08:18
@heshaoqiong-tuya
heshaoqiong-tuya force-pushed the feature/frame-coalesce-limit branch from 8e8a550 to 72d0a4d Compare September 4, 2026 02:54
heshaoqiong-tuya added a commit that referenced this pull request Sep 4, 2026
Frames whose whole wire size (frame hdr + app hdr + payload + signature)
is strictly under TAI_FRAME_COALESCE_LIMIT (default 512 B) are gathered
into tx_ctrl_buf and sent as a single ctx_io_send — one TLS record
instead of 2-3. Ping, the control packets and small audio chunks all
fall inside the window.

A control packet's payload IS tx_ctrl_buf (send_app hands it straight
through), so the gather relocates the payload to its final offset FIRST
— memmove is overlap-safe — before the frame header overwrites its
front; the HMAC is computed over the original bytes before any of this,
and the in-place shift preserves them. Coalescing is capped at the
smaller of TAI_FRAME_COALESCE_LIMIT and TAI_TX_CTRL_BUF_SIZE: both are
user-overridable knobs, and a memory-constrained build that shrinks the
control buffer must narrow the window, not write past tai_ctx. The two
knobs stay independent — tx_ctrl_buf's job is holding the largest
control packet (escaped session/event JSON + framing), and deriving it
from the coalesce limit would turn a performance knob into a capacity
knob.

test_sg_coalesce_boundary pins the edge — one byte under the limit
coalesces, at the limit scatters — HMAC-verifying every frame of both
events (the aliased control frames are the hazard) and byte-comparing
the Text payload. test_sg_send_failure case A stays above the limit so
its fail-after-1-write arithmetic still targets the scatter path.

CHANGELOG: adds the Unreleased entry (PR #32); while touching the
section it also corrects the UEAZ entry's PR number (#32 -> #31, it
merged as #31) and the "Chaneged" header typo.

Verified: full ctest suite 15/15 (tai_integration_tests 361 passed,
0 failed); ASan+UBSan build clean.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@heshaoqiong-tuya
heshaoqiong-tuya force-pushed the feature/frame-coalesce-limit branch from 72d0a4d to 25ecd78 Compare September 4, 2026 02:56
Frames whose whole wire size (frame hdr + app hdr + payload + signature)
is strictly under TAI_FRAME_COALESCE_LIMIT (default 512 B) are gathered
into tx_ctrl_buf and sent as a single ctx_io_send — one TLS record
instead of 2-3. Ping, the control packets and small audio chunks all
fall inside the window.

A control packet's payload IS tx_ctrl_buf (send_app hands it straight
through), so the gather relocates the payload to its final offset FIRST
— memmove is overlap-safe — before the frame header overwrites its
front; the HMAC is computed over the original bytes before any of this,
and the in-place shift preserves them. Header-only packets (Audio END)
arrive with pay=NULL/len=0, so the relocation is length-guarded —
glibc's memmove is declared non-NULL and UBSan flags the call even for
a zero length. Coalescing is capped at the smaller of
TAI_FRAME_COALESCE_LIMIT and TAI_TX_CTRL_BUF_SIZE: both are
user-overridable knobs, and a memory-constrained build that shrinks the
control buffer must narrow the window, not write past tai_ctx. The two
knobs stay independent — tx_ctrl_buf's job is holding the largest
control packet (escaped session/event JSON + framing), and deriving it
from the coalesce limit would turn a performance knob into a capacity
knob.

test_sg_coalesce_boundary pins the edge — one byte under the limit
coalesces, at the limit scatters — HMAC-verifying every frame of both
events (the aliased control frames are the hazard) and byte-comparing
the Text payload. test_sg_send_failure case A stays above the limit so
its fail-after-1-write arithmetic still targets the scatter path.

CHANGELOG: adds the Unreleased entry (PR #32); while touching the
section it also corrects the UEAZ entry's PR number (#32 -> #31, it
merged as #31) and the "Chaneged" header typo.

Verified: full ctest suite 15/15 (tai_integration_tests 361 passed,
0 failed); ASan+UBSan build clean locally (macOS). The zero-length
memmove was caught by Linux UBSan in CI — Darwin's string.h carries no
non-NULL attributes, so that class of check is invisible on a Mac.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@heshaoqiong-tuya
heshaoqiong-tuya force-pushed the feature/frame-coalesce-limit branch from 25ecd78 to febf980 Compare September 4, 2026 03:01
@heshaoqiong-tuya
heshaoqiong-tuya marked this pull request as ready for review September 4, 2026 03:27
@sedawwk
sedawwk merged commit b8c943d into master Sep 4, 2026
8 checks passed
@heshaoqiong-tuya
heshaoqiong-tuya deleted the feature/frame-coalesce-limit branch September 7, 2026 03:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants