Backport call-site aware memory tracking to release-7.4 (PR #13344)

[gxglass]

Backport of PR #13344 ("call-site aware memory tracking", main commit aa705c8) to the release-7.4 line.

Adds a sampled, always-compiled, knob-controlled per-call-site memory tracker (flow/MemoryTracker.{cpp,h}) hooked into the three FDB-owned allocation paths: the global operator new/delete set, FastAllocator, and ArenaBlock::create/destroyLeaf. A periodic dump from SystemMonitor emits MemoryTrackerSite / MemoryTrackerSummary TraceEvents. Prod default is off; simulation samples 1-in-10 so the path is exercised. The whole feature is gated at compile time by FDB_MEMORY_TRACKER (on by default; build with -DFDB_MEMORY_TRACKER=OFF to compile it out).

The replaced global operator new / operator delete (and the legacy ALLOC_INSTRUMENTATION accounting variants) are moved into fdbserver/GlobalNewDelete.cpp so the interposition is compiled into the fdbserver executable only, never into flow / libfdb_c / the client bindings. Verified with nm: the strong operator-new/delete definitions are present in the fdbserver binary and absent from libfdb_c.so and libflow.a.

Differences from the main-branch PR, due to release-7.4 divergence:

• operator-new removal and memTrackerInit() wiring applied to fdbserver/fdbserver.actor.cpp (main: fdbserver.cpp); knob re-init uses g_knobs.initialize() rather than initializeServerKnobs().
• forceLinkMemoryTrackerTests() wired into fdbserver/workloads/UnitTests.actor.cpp (main: UnitTests.cpp).
• AGENTS.md added, adapted to 7.4 (flat fdbserver/ layout, ServerKnobs under fdbclient/, fdbserver -r unittests, absent design docs dropped).
• Omitted: design/memory-tracker.md, the fdbserver/bench microbenchmark (needs main's benchmark harness, absent from 7.4), and the two contrib/mako_ab_*.py scripts (need contrib/mako_storage_bench.sh, absent from 7.4).

Testing, correctness:

• All 14 /flow/MemoryTracker/ unit tests pass; ran 1000x with randomized seeds, 1000/1000 passed, 0 failures.
• 100K Joshua correctness run, clean:
  20260805-213736-gglass-36215d1242f71327 compressed=True data_size=41598074 duration=4226513 ended=100000 fail_fast=10 max_runs=100000 pass=100000 priority=100 remaining=0 runtime=0:41:34 sanity=False started=100000 stopped=20260805-221910 submitted=20260805-213736 timeout=5400 username=gglass

Testing, performance:

Attachments show saturation throughput of {{compiled out vs compiled-in but disabled}, {compiled in, disabled vs compiled in, enabled at 1% sampling}}; and p50 latency of {{compiled out vs compiled-in but disabled}, {{compiled-in, disabled vs compiled-in, enabled at 1% sampling}}.

Summary:

• Throughput reduction: O(1%) (compiled out --> compiled in) and O(10%) (disabled --> enabled at 1% sampling).
• Latency increase at p50: O(1%) (compiled out --> compiled in) and O(7%) (disabled --> enabled at 1% sampling).

The intent is to be able to compile in by default and ship that, and only enable when needed (e.g. interesting end to end test scenarios, or for leak debugging as a last resort in actual production).

FDB Memory Tracker A_B_ release-7.4 tracker compiled out vs in (tracking off).pdf

FDB Memory Tracker A_B_ off vs 1_100 sampling.pdf

FDB Memory Tracker A_B_ compiled-out vs compiled-in-disabled latency @ 2500 tps.pdf

FDB Memory Tracker A_B_ latency @ 2500 tps (sub-saturation, 3 runs_arm).pdf

[foundationdb-ci]

Result of foundationdb-pr-macos-m1 on macOS 14.x

• Commit ID: bddf107
• Duration 0:35:55
• Result: ✅ SUCCEEDED
• Error: N/A
• Build Log terminal output (available for 30 days)
• Build Workspace zip file of the working directory (available for 30 days)

[foundationdb-ci]

Result of foundationdb-pr-clang on Linux RHEL 9

• Commit ID: bddf107
• Duration 0:43:58
• Result: ✅ SUCCEEDED
• Error: N/A
• Build Log terminal output (available for 30 days)
• Build Workspace zip file of the working directory (available for 30 days)

[foundationdb-ci]

Result of foundationdb-pr-clang-arm on Linux RHEL 9

• Commit ID: bddf107
• Duration 0:45:54
• Result: ✅ SUCCEEDED
• Error: N/A
• Build Log terminal output (available for 30 days)
• Build Workspace zip file of the working directory (available for 30 days)

[foundationdb-ci]

Result of foundationdb-pr on Linux RHEL 9

• Commit ID: bddf107
• Duration 1:00:46
• Result: ✅ SUCCEEDED
• Error: N/A
• Build Log terminal output (available for 30 days)
• Build Workspace zip file of the working directory (available for 30 days)

[foundationdb-ci]

Result of foundationdb-pr-macos on macOS 14.x

• Commit ID: bddf107
• Duration 1:12:47
• Result: ✅ SUCCEEDED
• Error: N/A
• Build Log terminal output (available for 30 days)
• Build Workspace zip file of the working directory (available for 30 days)

[foundationdb-ci]

Result of foundationdb-pr-cluster-tests on Linux RHEL 9

• Commit ID: bddf107
• Duration 1:28:51
• Result: ✅ SUCCEEDED
• Error: N/A
• Build Log terminal output (available for 30 days)
• Build Workspace zip file of the working directory (available for 30 days)
• Cluster Test Logs zip file of the test logs (available for 30 days)