Activate glom's MONDO close-match guard, with concord-reader symmetry tests

[gaurav]

What

Isolates the MONDO close-match guard fix from #883 onto a clean, main-based branch so it can be reviewed on its own. #883 also bundled the babel-clique-diff tool (already merged via #885) and the add-mpo (#790) work; neither is included here.

The bug

compute_cliques_for_impact_report() (which build_compendium() calls to do the real production build) read the MONDO_close concord with close_mondos[x[0]].add(x[1]), keying each MONDO subject to column 2 — the predicate, oio:closeMatch — instead of column 3, the close-match object CURIE.

Full write-up of the bug and its history: #912.

glom()'s close= guard (src/babel_utils.py) rejects a merge only when a MONDO CURIE's recorded close-match object also turns up in the same newly-merged set. Because the recorded value was the literal predicate string (which never appears in any clique), the guard never fired — a silent no-op on every build. Close-but-not-exact matches were free to collapse into exact MONDO cliques.

How long has this been broken?

git blame traces close_mondos[x[0]].add(x[1]) to commit 93a6c898 ("disease/phenotype", 2021-05-25) — the very same commit that introduced the close-match guard. Its parent had no close-match logic at all, and MONDO_close was already a 3-column concord (subject predicate object) written by ubergraph.build_sets() from that commit. So this was never a regression from a later format change: the reader keyed on the predicate (x[1]) from day one, while the regular-concord loop in the same function already keyed on subject + object (stuff[0]/stuff[2]).

The guard has therefore been a silent no-op for its entire ~4-year existence — no disease build has ever actually enforced it, and close-but-not-exact matches have been collapsing into exact MONDO cliques the whole time.

Why it went unnoticed / how to prevent the next one

• It failed open, not loud: a mis-keyed guard silently permits merges rather than crashing, so nothing in the build ever complained.
• The two column conventions lived side by side in one function (x[1] for the close reader vs stuff[0]/stuff[2] for every other concord) — a symmetry a reviewer or a linter can't easily see, and no test exercised the guard end-to-end.
• Prevention baked into this PR: both disease concord readers are now single named helpers — load_concord_pairs() (regular concords) and load_close_mondos() (MONDO_close) — each with a docstring pinning the column semantics, and a set of concord-reader column-symmetry tests asserts the shared convention across all of them (see the Tests section). Any future concord reader should likewise route through a shared, tested parse rather than re-deriving column indices inline.

The fix

Extract the parse into load_close_mondos() and key on x[2], matching the (stuff[0], stuff[2]) parsing every other concord in compute_cliques_for_impact_report already uses. Also skip blank lines and raise RuntimeError on any row that is not exactly 3 columns, so x[2] is always safe and a malformed file fails loudly.

To remove the asymmetry at its root, the regular-concord reader that lived inline in compute_cliques_for_impact_report is also pulled out into a named load_concord_pairs() helper, so both disease concord readers now share one documented "subject + object, skip the predicate" parse (this also fixes a pre-existing malformed RuntimeError in that inline loop that passed a tuple of args instead of a message).

No change to src/babel_utils.py — the guard already worked; the bug was in what was fed to it.

Impact (SME review)

Activating the guard is a deliberate change to disease clique merging: close-but-not-exact matches no longer merge into exact MONDO cliques.

A build-vs-build clique diff isolating exactly this change (both sides built from identical cached disease intermediates, only src/createcompendia/diseasephenotype.py differs) is committed under docs/sources/MONDO/mondo-close-match-guard/:

• 1,218 members dropped from Disease.txt, every one a MEDDRA identifier — the close-match MEDDRA terms that were reaching MONDO disease cliques transitively through the UMLS concord, now pulled back out.
• 1,228 before-cliques changed (of 365,466); 114 members regrouped within Disease.txt, 72 moved to PhenotypicFeature.txt; net clique count +20.
• PhenotypicFeature.txt: +13 cliques (14 new HP-led cliques formed from groups shed by disease cliques, 1 pre-existing absorbed); 0 dropped members. The diff tool emits only 1 row here because these are brand-new cliques with no before-counterpart.

The 1,218 figure matches the ~1,219 MEDDRA drop from the earlier combined analysis in #883.

Cliques that actually split — needs SME review

MEDDRA is barely used downstream, so the 1,218 dropped MEDDRA identifiers are low-stakes. The changes worth an SME's attention are the 37 cliques that structurally split, moving a real (non-MEDDRA) identifier out of a MONDO clique. The full list with members is in docs/sources/MONDO/mondo-close-match-guard/README.md and split-cliques.csv. Three kinds: 19 eject a piece into a brand-new Disease.txt clique, 4 re-attach it to a different existing MONDO clique, and 14 shed a group that re-types into a brand-new PhenotypicFeature.txt clique (these are the +13 phenotype additions the diff tool doesn't emit as rows, since it iterates before-cliques).

Mechanism: when the guard fires, glom() drops the entire pairwise concord link, not just the offending MEDDRA term. So a legitimate UMLS/DOID exact-equivalent that happens to ride in the same group as a close-matched MEDDRA is ejected as collateral damage. That is why many splits pair a MONDO term with an identically-labelled UMLS/DOID concept — a strong regression signal:

• ⚠ likely regressions (near-identical labels), e.g. MONDO:0000889 "haemophilus meningitis" splitting off UMLS:C0276028 "Haemophilus influenzae meningitis"; the two multiple endocrine neoplasia DOID splits (MONDO:0007540→DOID:10017, MONDO:0017169→DOID:3125); toxic oil syndrome, argyria, pyomyositis, secondary polycythemia, macular corneal dystrophy, Spasmus nutans each splitting off their identically-named UMLS concept.
• Likely improvements, e.g. MONDO:0009297 "familial renal glucosuria" correctly no longer subsuming the broader UMLS:C0017980 "Glycosuria, Renal".

The 4 re-attach cases land the ejected group in a different, already-existing MONDO clique. That happens because MONDO is a unique_prefix (one MONDO per clique): when a group is linked to two MONDO terms at once, only one wins, and blocking the winning path hands the group to the runner-up. Example: MONDO:0016702 "oligoastrocytoma" → MONDO:0003268 "mixed glioma" — a UMLS:C0280793 group (incl. NCIT:C129323 "Oligoastrocytoma, NOS") linked to both; the guard blocks the oligoastrocytoma path, so it lands under mixed glioma instead. These re-attachments can also be regressions (here an oligoastrocytoma NCIT term now sits under mixed glioma). See the README for the full trace.

The 14 phenotype cases eject a group carrying an HP term and no MONDO, so it re-types as biolink:PhenotypicFeature. Here a near-identical MONDO↔HP label does not imply a regression — Babel deliberately keeps disease and phenotype in separate compendia, so pulling HP:0001543 "Gastroschisis" out of MONDO:0009264 "gastroschisis" is arguably correct typing; the SME question is whether the UMLS/SNOMEDCT that moved with the HP belong on the phenotype side.

split-cliques.csv carries a first-pass sme_assessment for the 23 disease splits: the 2 confirmed cases, the 8 identical-label splits marked suspected regression, and the remaining 13 left blank as genuine SME calls. The 14 phenotype splits are left blank.

Tests

tests/createcompendia/test_diseasephenotype.py adds two groups of tests:

• MONDO close-match guard — load_close_mondos keys on the object column, skips blanks, and rejects malformed rows; glom's close= guard blocks a close-match MEDDRA term from collapsing into the exact MONDO clique end-to-end; and a companion test documents the original bug (a predicate-keyed close dict leaves the guard a no-op).

• Concord-reader column symmetry — parametrized over every disease concord reader (load_concord_pairs and load_close_mondos), asserting each surfaces the subject (column 1) and object (column 3), never the predicate (column 2), and raises RuntimeError on a non-3-column row. This pins the shared column convention whose violation silently disabled the guard, so a new reader (or a regression of an old one) that reads the predicate column fails loudly. Adding a future reader to the CONCORD_READERS list automatically holds it to the same invariant.

• uv run pytest tests/createcompendia/test_diseasephenotype.py -q — passes

• uv run pytest -m unit -q — 251 passed

• uv run ruff check / ruff format --check — clean

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR fixes how the disease/phenotype pipeline reads the MONDO_close concord so that glom()’s MONDO close-match guard is actually enforced (it was previously a silent no-op due to keying on the predicate column instead of the object column). It also extracts concord parsing into named helpers and adds regression + symmetry tests to prevent future column-index drift.

Changes:

• Fix MONDO_close parsing by keying MONDO subjects to close-match objects (column 3), with blank-line skipping and strict 3-column validation.
• Extract regular concord parsing into load_concord_pairs() with strict 3-column validation and bad-pair filtering.
• Add unit tests covering the close-match guard end-to-end and parametrized “reader column symmetry” invariants across concord readers.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
src/createcompendia/diseasephenotype.py	Adds load_close_mondos() / load_concord_pairs() helpers and wires them into build_compendium() so the MONDO close-match guard can trigger correctly.
tests/createcompendia/test_diseasephenotype.py	Adds regression tests proving the guard blocks close-match merges and enforces subject/object column symmetry across concord readers.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.