Skip to content

Fail closed on computed dataset exports#974

Merged
MaxGhenis merged 1 commit into
mainfrom
codex/fail-closed-computed-export
May 16, 2026
Merged

Fail closed on computed dataset exports#974
MaxGhenis merged 1 commit into
mainfrom
codex/fail-closed-computed-export

Conversation

@MaxGhenis
Copy link
Copy Markdown
Contributor

@MaxGhenis MaxGhenis commented May 14, 2026

Summary

  • Replace final Extended CPS formula-variable pruning with a fail-closed export contract.
  • Drop or rename computed construction variables at their final stage boundary, including Social Security totals after subcomponents reconcile.
  • Extend built-dataset validation so upload checks reject policyengine-us computed variables in exported H5s.

Root Cause

The old _drop_formula_variables step silently removed any exported variable that policyengine-us classified as computed. That hid cross-repo contract breaks when a data-backed variable later gained a formula, adds, or subtracts definition upstream.

This PR changes that behavior: us-data now drops only explicitly classified construction variables, then fails the build if computed model variables still reach the final export surface.

Validation

  • uv run ruff format policyengine_us_data/datasets/cps/extended_cps.py policyengine_us_data/utils/dataset_validation.py tests/unit/test_extended_cps.py tests/unit/test_dataset_validation.py
  • uv run ruff check policyengine_us_data/datasets/cps/extended_cps.py policyengine_us_data/utils/dataset_validation.py tests/unit/test_extended_cps.py tests/unit/test_dataset_validation.py
  • uv run pytest tests/unit/test_extended_cps.py tests/unit/test_dataset_validation.py tests/unit/calibration/test_calibration_puf_impute.py -q
  • uv run pytest tests/integration/test_cps_generation.py tests/unit/test_upload_completed_datasets.py -q

Related Discussions

@MaxGhenis MaxGhenis force-pushed the codex/fail-closed-computed-export branch from f552b45 to 3e202dc Compare May 14, 2026 12:16
@MaxGhenis MaxGhenis marked this pull request as ready for review May 16, 2026 15:43
@MaxGhenis MaxGhenis merged commit 2b348a0 into main May 16, 2026
12 checks passed
@MaxGhenis MaxGhenis deleted the codex/fail-closed-computed-export branch May 16, 2026 15:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant