tokio-quiche: pool the IO egress buffer per worker

[LynnLi100]

Before, each H3Driver held a persistent 64 KiB buffer io_worker_buf for the connection's entire lifetime, so every idle connection pinned 64 KiB and total egress scratch memory scaled with the connection count.

What this PR does is removing the per-connection buffer and instead borrowing a transient scratch buffer from a per-worker-thread pool for the duration of each send burst, returning it before the worker sleeps. Idle connections now hold no egress buffer, while the pooled buffers' pages stay resident across bursts so steady-state sending avoids per-burst page faults and kernel zero-fill.

• SEND_BUF_POOL is a thread_local free-list; PooledSendBuf::acquire() pops a buffer (allocating only on a miss) and its Drop returns it.
• The pool is capped at SEND_BUF_POOL_CAP = 16 buffers per worker thread (<= 1 MiB per runtime thread), independent of the connection count.
• Buffers are not re-zeroed on reuse: only send_buf[..bytes_written] is ever transmitted, so stale bytes are never sent.
• Work-stealing migration across the flush .await is benign -- a buffer acquired on one thread may be returned to another; the per-thread cap keeps the total bounded by cap * workers.
• ApplicationOverQuic::buffer() now returns an empty slice (kept only to satisfy the trait); the unused ConnectionStageContext::buffer() helper is removed.

Adds unit tests for PooledSendBuf and driver tests asserting buffer() stays empty before/after the handshake and with an active stream.

From doing A/B testing for stalled active requests, this PR deduct heap usage by ~35%.

[gregor-cf]

I think it should be possible to get same / similar saving by tuning jemalloc (lg_tcache_max:16). That sets the max allocation size for which jemalloc will use the thread-local cache to 64KB (default is 32KB)

[jannes]

if we can lean on jemalloc to achieve the same, then i strongly prefer that. have only skimmed this MR so far, but it's non-trivial and introduces further complexity. maybe you can try the specific jemalloc tuning Gregor suggested in your testing?

[gregor-cf]

Very nice.
It would be cool if we could have metrics to track how many total allocations we've made and how many buffers where discarded because the TL cache is full. So one counter increment in alloc_send_buf and one on Drop (only if TL cache is full). We want to avoid counting on the hot-path (re-using a buffer) I think

[LynnLi100]

Thank you for the suggestion, added them. There are now two counters, both on the cold paths. The first one is send_buffer_pool_allocated, bumps in acquire() only when the pool is empty and we actually have to allocate a new buffer. The second, send_buffer_pool_discarded, bumps in Drop, but only when the pool is already full and we throw the buffer away instead of parking it. The hot path stays untouched, we reuse a parked buffer or hand one back to a pool that still has room, so we're not counting on every burst. I wired them up as global foundations quic:: counters rather than going through the Metrics trait, that way nothing has to be threaded through PooledSendBuf itself.

[jannes]

the approach lgtm, need to give it a second pass to grok all the details

[jannes]

had another pass and looks all solid to me