Align brainscore pipeline with official Brain-Score (closes #9)

[c-mulliken]

Addresses all substantive deviations catalogued in #9, using the hybrid default policy: align the low-risk mechanical defaults to Brain-Score, but keep ceiling normalization opt-in (it changes the score's meaning most).

Stacked on #8 (test-suite-and-precommit-gate) — base is that branch so the diff shows only the alignment work. Merge after #8.

Changes per issue item

#	Item	Fix	Default
1	No ceiling normalization	--ceiling_normalize divides the score by the median per-neuroid noise ceiling; --ceiling_sb_correct applies Spearman-Brown to the reliability first	opt-in (off)
4	No Spearman-Brown	utils.brainscore.spearman_brown() helper + spearman_brown= option on TVSD_TestDataset._compute_reliability (test-set path can now produce an SB-corrected ceiling)	n/a
3	Extra StandardScaler	standardization now gated behind --standardize; default off to match Brain-Score's scale=False + internal PLS centering	changed → aligned
5	CV strategy/count	default ShuffleSplit(n_splits=10, train_size=0.9); --cv_strategy kfold retained	changed → aligned
2	PCA leakage	in-benchmark PCA is fit on the train fold only (leak-free); the generate-time IncrementalPCA leakage is documented; --skip_pca for pre-reduced features	aligned + documented

The conflated --preprocessed flag is replaced by --standardize / --skip_pca. scripts/benchmark.sh updated accordingly (now --skip_pca --n_splits 10, with a comment on the IPCA leakage caveat).

Not changed (intentional)

Low-reliability neuroid dropping (reliability > threshold) is kept — it follows the original TVSD paper, per maintainer direction. Ceiling normalization is independent of it.

Test plan

• New tests/test_brainscore.py: spearman_brown values/monotonicity, _make_splitter, compute_brain_score predicts strongly-correlated data, ceiling normalization scales the score, ceiling_normalize without a ceiling raises, reproducibility.
• New SB reliability test in tests/test_reliability.py.
• Full gate green: ruff, ruff-format, mypy, pytest 91 passed.

Closes #9.

🤖 Generated with Claude Code

[c-mulliken]

Update: reworked to fit-train / score-test with a real noise ceiling (commit 6a3a409)

Inspecting THINGS_normMUA.mat directly settled the ceiling question and changed the design:

• reliab is mean pairwise single-trial reliability (435 = C(30,2) rep-pairs, averaged; median ≈ 0.37) — not split-half, not SB-corrected. So the earlier --ceiling_sb_correct (n=2 2r/(1+r)) was simply wrong for this field.
• test_MUA (100×1024) is exactly the 30-rep average — the held-out responses to predict already exist.
• The true noise ceiling of that average ≈ 0.95 (15v15 split-half + SB).

What changed

• score_train_test (new): fit the mapping on train_MUA, predict the held-out test_MUA, per-neuroid Pearson → median; std via bootstrap over the 100 test stimuli. The train/test split is the evaluation — no within-train CV. Shared _fit_predict/_correlations/_reduce helpers; compute_brain_score (CV) kept as a legacy path.
• benchmark.py: fits on train, scores on test; neuron selection still uses single-trial reliab > 0.3 (per TVSD paper); ceiling = split-half+SB internal consistency via TVSD_TestDataset(recompute_reliability=True, spearman_brown=True). Removed the incorrect --ceiling_sb_correct.
• generate_activations.py: --split {train,test}, saved under TVSD_train/TVSD_test. IPCA is fit on train only and reused for test (Activations.load_ipca_models) → shared basis, no train→test leakage.
• Scripts generate both splits; benchmark runs --skip_pca --ceiling_normalize.

Gate green (ruff/ruff-format/mypy) and 96 tests pass. Not yet run on real activations — that's the next step (blocked only on a working torch env on Oscar).

[c-mulliken]

Smoke-tested end-to-end on Oscar ✅

Ran the reworked pipeline on real TVSD data (alexnet, monkeyF, region IT) — deliberately tiny (512 train images) just to validate the plumbing, not to produce real numbers.

Generation (--split train then --split test):

• Both splits saved under TVSD_train/ and TVSD_test/.
• [IPCA] loaded 11 models from …/TVSD_train/alexnet — the train-fit IPCA is reused for test (leak-free shared basis) as intended.

Benchmark (--skip_pca --ceiling_normalize):

• Shapes align: X(512, 100) IPCA-reduced features, Y(…, 157) IT neuroids surviving the reliab > 0.3 mask.
• 157 neuroids retained … median ceiling 0.970 — matches the empirical .mat finding (~0.95), so the split-half + Spearman-Brown ceiling from test_MUA_reps is computed correctly on real data.
• Ceiling-normalized scores show the expected cortical hierarchy:
  features.0 = 0.11 → deep conv ~0.53 → classifier.1 = 0.57 (all sensibly below 1.0).

Null control (--noise_test): every layer collapses to ≈ 0 (−0.04 to +0.05). ✓

Env note for anyone reproducing on Oscar: the default PyPI torch wheel is cu130, which is too new for Oscar's CUDA 12.9 GPU driver — install from the cu124 index instead (pip install --index-url https://download.pytorch.org/whl/cu124 torch torchvision).

Not run at scale yet (full 22,248-image train generation across models/regions is a separate sbatch job).