Clean up the repo for new readers: remove refactor scaffolding, bundle the paper, rewrite the reproduction guide - #1
Merged
Conversation
Remove "Phase N" labels, dead cross-references to files being deleted (HANDOFF.md, REFACTOR_PLAN.md, paper/PHASE9B_INVESTIGATION.md), and "moved/ported from the paper code's <internal filename>" archaeology that only makes sense with the private research repo's history. Keep every technical rationale that survives that history (equation citations, the kernel-mutation-during-sampling contract in mixture.py/acquisition.py/kernels.py, why the McCabe-Thiele reboiler equation looks the way it does, etc.) -- comments should explain the code and cite the paper, not the project's development narrative. Also fixes two real bugs in the prose: full_reproduction.py's docstring claimed the full adaptive-loop rerun is "hours-long" when every other mention in the repo (and direct measurement) puts it at ~25 min, and paper/reference/README.md claimed the bulk archive "goes to Zenodo" when paper/DATA.md says the opposite (no Zenodo deposit). No numerical values, algorithms, defaults, or paper/data|reference contents changed. pytest -q: 204 passed, 2 deselected (unchanged); ruff check: clean. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Phase 9b investigation log These are AI-driven-refactor session logs and a bug-investigation write-up referencing a private, unpublished research repo -- the kind of development history that belongs in git history, not the working tree. The durable technical content they carried is already present elsewhere and was cleaned (not rewritten) in the prior commit: - The GPmodel-kernel mutation-during-sampling hazard and its save/restore fix: mixture.py/acquisition.py/kernels.py docstrings, and examples/vle_distillation/phase_diagram.py's surrogate_gamma/surrogate_gamma_averaged docstrings. - The per-dimension-Parameter kernel design rationale: kernels.py's module docstring. - The disk-as-state retirement rationale: sampler.py/state.py. - The archive-free paper/data/ policy: paper/DATA.md, paper/data/README.md. - The from-scratch stochastic-reproduction numbers (R-hat/ESS, hyperparameter posterior, entropy decay, phase diagram/stage table agreement): folded into paper/REPRODUCTION.md in the prior commit. paper/phase9_validation/'s two JSON files and three PNGs backed only the investigation log's narrative and are not read by any test or script -- removed with it. Nothing in pytest -q (204 passed, 2 deselected) reads any of these paths. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v0.1.1 is released on PyPI and the docs are live on ReadTheDocs, so: - README: drop the "pre-1.0, under active refactor" banner and the HTML comment claiming the PyPI/Docs badges are inert placeholders. Lead the Install section with `pip install bits_for_gaps` (previously buried behind a from-source dev workflow); point Docs at the live RTD URL instead of only "build it locally". - docs/index.md: replace the "Pre-1.0 ... has not yet had a tagged release" warning with a one-line honest 0.x API-stability caveat. - docs/installation.md: `pip install bits_for_gaps` is now the primary instruction, not something gated behind "once published, see HANDOFF.md for status"; fixed the stale test count (193 -> 204, verified) and the deselected-tests explanation (they need Julia, not "the private archived run" -- that framing predates archive-free figure reproduction). Also drops now-dead REFACTOR_PLAN.md citations from docs/conf.py and pyproject.toml comments (substance kept, citation dropped). ruff check: clean; sphinx-build -W: clean, zero warnings. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
phase narrative
- CHANGELOG.md: strip "(Phase N)" tags from every 0.1.0 bullet; drop
the one bullet that was itself about doc staleness during
development (not user-facing software behavior); describe what
changed for a user, not the port's process.
- RELEASE.md: was written as a one-time record of preparing v0.1.0
("recorded 2026-08-06", "NOT performed by this phase") -- but
trusted publishing, PyPI, and RTD are all live now (v0.1.1 shipped
after v0.1.0). Restructured into a reusable maintainer checklist:
"Already set up" / "Before every release" / "Cutting a release" /
"Optional hardening", so it stays correct for the next release
instead of describing one that already happened.
- docs/improvements_over_paper.md: same content, phase framing
removed. Compares the package to the paper's original code on
technical merit (bugs fixed, hardening, faithfulness, architecture)
-- kept the "Phase 9b" bug's technical substance, dropped the
reviewer-facing "46 new tests were added" / "behavior-preserving"
assurances and the now-deleted paper/PHASE9B_INVESTIGATION.md
citation.
ruff check: clean; sphinx-build -W: clean, zero warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Previously wrote for someone already inside the project's development
history ("As of Phase 9...", pointers to HANDOFF.md and the
now-deleted paper/PHASE9B_INVESTIGATION.md). Rewritten for a reader
who has read the paper once and nothing else:
- Leads with a comparison table distinguishing the two things
"reproduce the paper" can mean: regenerating the 11 figures from
paper/data/ (fast, deterministic) vs. re-running the full
15-iteration adaptive loop from scratch (slow, stochastic).
- Figure regeneration cost is measured, not guessed: ~80s wall-clock
on this machine for all 11 figures via `python paper/reproduce.py`
(timed this session). The full-loop cost is stated as the
documented ~25-30 min (matches every existing mention in the repo;
full_reproduction.py's own docstring wrongly said "hours-long" --
fixed in the scrub-source commit).
- Prominently warns the full-loop run is stochastic and will not
reproduce the paper's exact numbers, then gives concrete
what-should-match (R-hat < 1.1, healthy ESS, temperature lengthscale
> mole-fraction lengthscale, entropy decaying, test error dropping,
phase diagram/stage table within a few 0.01 mole fraction of Wilson)
vs. what-will-differ (exact sampled points, exact posterior values,
the non-seedable TF ambient-RNG-driven test-RMSE trajectory) lists.
- States exactly which parts need Julia (only Fig 8/9 for figure
regeneration; the full loop throughout) and which don't.
Verified by rendering the built HTML in a browser: table, admonition,
and cross-references all render correctly.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The docs described the authors' private research repository as the "archive of record" and said "author access" was needed for some figures, which wrongly implies a reader is missing something. They are not: paper/data/ contains everything the figure scripts read, so `python paper/reproduce.py` works from a fresh clone. - Reframe every such reference: --archive/$BFG_ARCHIVE_DIR is now described for what it is generally useful for (pointing at any directory of run artifacts with the same layout, e.g. your own paper/full_reproduction.py output), not as a private archive. Rewrote paper/DATA.md around "everything needed is committed here". - README gains a short Provenance section: the code was originally developed in a private repository, then migrated here and reorganized into an installable, tested package; the private repository holds only development history, nothing a user needs. - Remove hardcoded personal filesystem paths (~/DowlingLab/...) from paper/extract_reference.py and paper/data/README.md. extract_reference.py now defaults to the committed paper/data/, so it actually runs from a fresh clone -- verified to regenerate paper/reference/*.json byte-identically. - Add paper/bits_for_gaps_paper.pdf (the published article, redistributed under CC BY 4.0) with paper/PAPER_LICENSE.md giving the citation and license statement, and link it from the README, docs/index, docs/theory, and docs/reproduce_paper. The DOI remains the canonical citation. The PDF ships in neither the wheel nor the sdist. Gates: 204 passed / 2 deselected, pytest -m vle unaffected, ruff clean, sphinx -W clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Completes the scrub: no "Phase N"/phaseN reference remains anywhere in the tree. - paper/full_reproduction.py, paper/REPRODUCTION.md: the run's default output directory and exp_name were literally "phase9_fullrun" -> "full_reproduction" (matches the script name). Safe: exp_name only sets an opt-in default checkpoint dir that this script overrides via --out-dir. - tests/integration/test_end_to_end.py: test_matches_pre_phase4_baseline -> test_matches_synthetic_baseline (matches its data file, synthetic_baseline.json); referenced nowhere else, and verified still collected and passing. - examples/vle_distillation/phase_diagram.py: dropped a dangling ``PhaseDiagram.n_draws`` cross-reference -- that class belonged to the original research code and exists nowhere in this repository, so a reader would have searched for it in vain. Every remaining "phase" in the tree is thermodynamics (phase diagram, phase equilibrium, liquid/vapor phases) and is left alone. Gates: 204 passed / 2 deselected; pytest -m vle 2 passed; ruff clean; sphinx -W clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reads the repository as a new user would — someone who has read the paper once and wants
to use the package — and removes everything that only made sense to its authors.
Why
The package shipped (v0.1.1 on PyPI, docs live on ReadTheDocs), but the working tree still
read like a project journal: development-phase labels on ~100+ comments and docstrings, two
whole-file process logs, and a README that still announced "pre-1.0, under active refactor."
None of that helps a reader, and some of it actively misled.
What changed
Removed the process scaffolding. Deleted
HANDOFF.md(1064 lines),REFACTOR_PLAN.md,and the bug-investigation log after confirming nothing unique lived only in them. Scrubbed
development-phase labels from
src/,tests/,examples/, andpaper/— keeping everytechnical rationale (equation citations, the in-place kernel-mutation contract, why the
McCabe–Thiele reboiler equation is written the way it is) and dropping the archaeology.
No remaining reference to a development phase anywhere in the tree; every surviving "phase"
is thermodynamics (phase diagram, phase equilibrium, liquid/vapor phases).
Fixed misleading status and framing. The README's "pre-1.0, under active refactor" banner
and "not yet published" notes were false. The docs also described the authors' private
research repository as the "archive of record" and said "author access" was needed for some
figures — which tells readers they are missing something. They are not:
paper/data/contains everything the figure scripts read.
--archive/$BFG_ARCHIVE_DIRis now documentedfor what it is generally good for — pointing at any directory of run artifacts, such as
your own
full_reproduction.pyoutput. The README gains a short Provenance sectionexplaining the private-repo → public-package migration honestly.
Rewrote the paper-reproduction guide for a newcomer (
docs/reproduce_paper.md). It nowseparates the two things "reproduce the paper" can mean, with measured costs — regenerating
the figures (~1–2 min, deterministic) versus re-running the adaptive loop (~25–30 min,
stochastic) — and states explicitly what should match qualitatively versus what will
differ run to run.
Bundled the paper.
paper/bits_for_gaps_paper.pdf, redistributed under CC BY 4.0 withcitation and license in
paper/PAPER_LICENSE.md, linked from the README and docs. Ships inneither the wheel nor the sdist. The DOI remains the canonical citation.
Fixed two things that would have bitten a user:
~/DowlingLab/...) inpaper/extract_reference.pyand
paper/data/README.md. The former meantextract_reference.pywas unrunnable foranyone but the authors; it now defaults to the committed
paper/data/and is verified toregenerate
paper/reference/*.jsonbyte-identically.PhaseDiagram.n_drawscross-reference to a class that belongs to the originalresearch code and exists nowhere in this repository.
Scope
Documentation, comments, and tests only. No algorithm, default, numerical value, or
dependency changed;
paper/data/andpaper/reference/values are untouched.Verification
pytest -qpytest -m vleruff check .sphinx-build -W docsimport bits_for_gapsloads neitherjuliacallnortensorflowextract_reference.pyNo version bump in this PR — see the discussion on the release below.
🤖 Generated with Claude Code