Rebuild SSSOM to current filenames + revalidate CHEBI (OAK+OLS)#117
Merged
Conversation
The published mapping set had drifted from the source YAMLs. Two causes: 1. Stale subject_ids. Commit 3f7093e (2026-05-30) re-slugified ~101 ingredient filenames (e.g. `2-(Methylthio)ethanol.yaml` → `2-_Methylthioethanol.yaml`); only incremental row edits followed, so the SSSOM kept the old-slug subject_ids. A full rebuild from the current filenames re-derives them, so every mapped YAML now emits exactly one subject and reconciles 1:1 (1875 YAMLs ↔ 1875 subjects, was 1873). 2. Reverted curation. 18 CONSIDER_SPECIFIC decisions in the residual-P2.5 cache (Xylose, Mannitol, Trehalose, …) had been reconciled to the generic MIM CHEBI by cfd643c but never pruned from the cache; the rebuild would re-apply the specific-form swap (e.g. Xylose CHEBI:18222 "xylose" → CHEBI:15936 "aldehydo-D-xylose"). Pruned those entries so the build honors the curated term. The 18 now emit exactMatch to the generic CHEBI, and their orphaned kgmicrobe.compound B1 rows (only required for narrowMatch) are dropped. CHEBI revalidated against OAK + EBI OLS: 861 CONFIRMED, 597 SYNONYM_ENRICH (chebi), stamps refreshed to 2026-07-07. Passes sssom validate and validate_sssom_invariants (Rules A/B1/B2/B3). Row delta vs prior publish: -17 (131/132 rows re-keyed to corrected subject_ids; 18 orphan kgmicrobe.compound rows removed; +1 cas identity row for Glycogen_From_Bovine_Liver). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ject-ids-revalidate-chebi # Conflicts: # mappings/ingredient_mappings.sssom.tsv
Bacto Soytone and Sodium L-lactate are REJECTED in both their per-record YAMLs and data/curated/mapped_ingredients.yaml, but the claw builder emitted rows for them because it didn't honor mapping_status. MIM's reconcile_sssom.py qc gate flagged them as ORPHANs. Builder now skips REJECTED records (CultureBotAI/culturebotai-claw#16); regenerated SSSOM is 2200 rows and reconcile_sssom.py --check passes (0 GAP/ORPHAN/STALE). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
realmarcin
added a commit
that referenced
this pull request
Jul 7, 2026
Regenerate the unified mapping so it reflects the current MIM/CultureMech state after the SSSOM rebuild (#117) and #115/#116. 20 rows updated: - cholesterol CHEBI:140435 → CHEBI:16113 - Carnitine Hydrochloride CHEBI:48601 → kgmicrobe.compound:carnitine_hydrochloride - Phytone / Soya pepton(e) / Tryptone peptone remapped to FOODON/MICRO - Glycerol synonyms + kg_microbe_node_id refreshed - stale cas_rn cleared on beef-extract variants and Sulfur Row count unchanged (3938). Built via `build_unified_ingredient_mapping.py` (run under uv for the GitPython dep). Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rebuilds the canonical
mappings/ingredient_mappings.sssom.tsvfrom the current ingredient YAMLs and re-grounds every CHEBI row against OAK + EBI OLS. The published mapping set had drifted from its source.Why
1. Stale subject_ids. Commit
3f7093e(2026-05-30) re-slugified ~101 ingredient filenames (e.g.2-(Methylthio)ethanol.yaml→2-_Methylthioethanol.yaml). Only incremental row edits followed, so the SSSOM kept the old-slugsubject_ids and no longer matched the files. A full rebuild re-derives them, so every mapped YAML emits exactly one subject and reconciles 1:1 (1875 YAMLs ↔ 1875 subjects, was 1873).2. Reverted curation. 18
CONSIDER_SPECIFICdecisions in the residual-P2.5 cache (Xylose, Mannitol, Trehalose, …) were reconciled to the generic MIM CHEBI bycfd643cbut never pruned from the cache; a rebuild re-applied the specific-form swap (e.g. XyloseCHEBI:18222"xylose" →CHEBI:15936"aldehydo-D-xylose"), reverting the curation. Fixed on the tooling side (see companion claw PR); the 18 now emitexactMatchto the generic CHEBI, and their orphanedkgmicrobe.compoundB1 rows (only required fornarrowMatch) are dropped.CHEBI revalidation
Re-grounded against OAK + EBI OLS4: 861 CONFIRMED, 597 SYNONYM_ENRICH (chebi),
validation_methodrefreshed to 2026-07-07. Passessssom validateandvalidate_sssom_invariants(Rules A/B1/B2/B3; B4 skipped without local kg-microbe transforms, as in CI).Row delta
−17 vs prior publish: 131/132 rows re-keyed to corrected
subject_ids; 18 orphankgmicrobe.compoundrows removed; +1cas:identity row forGlycogen_From_Bovine_Liver.Companion
Tooling that produced this and prevents recurrence: CultureBotAI/culturebotai-claw PR (residual-prune wiring + self-maintaining guard).
🤖 Generated with Claude Code