fix(labels,qa): #109 — drop hardcoded tts_engine, derive single_backend from generation_metadata.tts_backend

[shaypal5]

Problem

ClipMetadata.tts_engine was always written as the literal string
"azure_he_IL" regardless of which TTS provider actually rendered the
clip. This mis-labeled every google-rendered clip and caused
qa-report's run-level single_backend warning to fire as a false
positive on mixed-provider runs.

Fingerprint from avdp-synth-corpus#4 (delivery-003):

$ jq '.tts_engine, .generation_metadata.tts_backend' \
    data/he/agg_m_30-45_002/sp_sv_a_0003_00.json
# → "azure_he_IL"                                          ← WRONG (this clip is Google)
# → {"AGG_M_30-45_002": "google", "VIC_F_25-40_003": "google"}  ← correct, per-speaker

And the matching qa-report.json:

"clips_by_tts_engine": {"azure_he_IL": 20},
"run_warnings": ["single_backend", ...]

single_backend is a false positive — the run genuinely has two
backends.

⚠️ Breaking changes

This PR is intentionally a schema and qa-output API break, scoped to
fix the bug correctly rather than perpetuate the dead-weight field. If
you have downstream consumers in avdp-synth-corpus or elsewhere, grep
them for the patterns below before pulling this in.

Surface	Before	After
ClipMetadata.tts_engine (Python attribute)	str, always "azure_he_IL"	Removed. Access raises AttributeError.
Clip JSON "tts_engine" field	always present	Not written. Legacy JSON parses fine (Pydantic extra="ignore"); the value is silently dropped on read.
RunSummary.clips_by_tts_engine (Python attribute)	dict[str, int] keyed on "azure_he_IL"	Renamed to clips_by_tts_backend, keyed on "azure" / "google" / "unknown".
qa-report JSON key "clips_by_tts_engine"	always present	Renamed to "clips_by_tts_backend".
Backend histogram value-space	"azure_he_IL"	"azure", "google", "unknown"
sum(clips_by_tts_backend.values()) invariant	== total_clips	>= total_clips (a mixed-backend clip contributes +1 to each backend)

The rename is deliberate (not scope creep): keeping the old name
clips_by_tts_engine while changing the value-space from "azure_he_IL"
→ "azure" would have been a silent value-shape change. Renaming makes
both the value and key changes visible to downstream consumers in one
place. Anyone consuming the qa-report JSON will get a KeyError rather
than wrong counts under the old key — which is what we want.

Solution

Per the issue's recommended option (a): drop tts_engine as the
primary signal. The structured per-speaker generation_metadata.tts_backend
map already lives in clip JSON and carries richer information (per-speaker,
not per-clip); the flat field was dead weight.

• Remove tts_engine from ClipMetadata entirely.
• Drop the hardcoded tts_engine="azure_he_IL" line in
  LabelGenerator.generate_clip_metadata.
• In qa.py, derive backend diversity from
  generation_metadata.tts_backend.values() instead of clip.tts_engine.
• Rename RunSummary.clips_by_tts_engine → clips_by_tts_backend. A
  clip with mixed providers contributes one count to each backend.
• Bucket pre-bug(labels,qa): tts_engine hardcoded "azure_he_IL" regardless of actual provider — misleads single_backend QA #109 clips (no generation_metadata) under "unknown"
  so they remain visible to backend-diversity metrics. Without this, an
  old corpus mixed with new clips would silently lose the old clips
  from backend_count.
• Update docs/spec.md §5.1 example + add a field note pointing readers
  to generation_metadata.tts_backend.
• Update AGENTS.md (= CLAUDE.md via symlink) to drop tts_engine
  from the ascii-safe validator list.

Test plan

• .venv/bin/pytest tests/unit/ -q — 1702/1702 passing
• Explicit AttributeError test on both fresh and legacy-parsed models
• "unknown" bucket coverage for pre-bug(labels,qa): tts_engine hardcoded "azure_he_IL" regardless of actual provider — misleads single_backend QA #109 clips (two new qa tests)
• End-to-end test: full mocked _run_generate_pipeline → run_qa, asserts "azure" lands in clips_by_tts_backend
• ruff check synthbanshee/ tests/unit/ — clean
• mypy synthbanshee/cli.py synthbanshee/package/qa.py synthbanshee/labels/schema.py synthbanshee/labels/generator.py — clean
• Pre-commit hooks (ruff, ruff-format, mypy) — clean at commit time

Tier-3 ASR sanity (local)

Not applicable — this PR touches synthbanshee/labels/,
synthbanshee/package/qa.py, the qa-report rendering in
synthbanshee/cli.py, and docs. None of synthbanshee/tts/,
synthbanshee/script/, synthbanshee/augment/, nor any speaker / scene
/ acoustic / project YAML config is modified. Per CLAUDE.md "ASR sanity
check policy", this PR is exempt from the local qa-report --asr run.

Corpus impact

Existing on-disk corpus clips (delivery-003 included) still have the
wrong tts_engine value baked in. After this PR lands, the next regen
will simply drop the field. For a no-regen retrofit, a small follow-up
PR on avdp-synth-corpus can post-process the 21 affected JSONs (drop
tts_engine; derive any consumer logic from
generation_metadata.tts_backend). Otherwise, the new "unknown"
bucket keeps them visible to qa-report diversity metrics, and
Pydantic's extra="ignore" makes the stale field harmless on read.

Fixes #109.

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR fixes incorrect backend labeling in clip metadata and QA reporting by removing the hardcoded per-clip tts_engine field and deriving backend diversity from the structured per-speaker generation_metadata.tts_backend map instead, eliminating false-positive single_backend warnings on mixed-provider runs.

Changes:

• Remove ClipMetadata.tts_engine and stop writing the hardcoded "azure_he_IL" value during label generation.
• Update QA aggregation + CLI output to use generation_metadata.tts_backend and rename clips_by_tts_engine → clips_by_tts_backend.
• Update docs and unit tests/fixtures to reflect the schema and QA-report changes.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
synthbanshee/labels/schema.py	Drops tts_engine from the schema and updates ASCII-safe validation fields.
synthbanshee/labels/generator.py	Stops emitting the hardcoded tts_engine value when generating metadata.
synthbanshee/package/qa.py	Computes backend diversity/histogram from generation_metadata.tts_backend and renames the run-summary field accordingly.
synthbanshee/cli.py	Updates qa-report Rich table and JSON output to use clips_by_tts_backend.
docs/spec.md	Removes tts_engine from the example and documents the new source of truth.
AGENTS.md	Updates the list of string fields enforced by the ASCII-safe validator.
tests/unit/test_qa.py	Updates clip fixtures to always emit generation_metadata.tts_backend and adds run-summary backend-diversity tests.
tests/unit/test_labels.py	Adds coverage asserting tts_engine is absent and legacy JSON still parses.
tests/unit/test_manifest.py	Removes tts_engine from JSON fixtures.
tests/unit/test_generation_metadata.py	Removes tts_engine from minimal-clip fixture.
tests/unit/test_cli.py	Removes tts_engine from minimal-clip fixture.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[github-actions]

pr-agent-context report:

No unresolved review comments, failing checks, or actionable patch coverage gaps were found on PR #112 in repository https://github.com/DataHackIL/SynthBanshee. Treat this PR as all clear unless new signals appear.

Run metadata:

Tool ref: v4
Tool version: 4.0.21
Trigger: commit pushed
Workflow run: 25745193064 attempt 1
Comment timestamp: 2026-05-12T15:41:36.349620+00:00
PR head commit: a14755dbe920e9a5f81686b2516ca6959133e053

[Copilot]

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated no new comments.