Skip to content

Add repeated stratified CV and learning-curve evaluation for supervised CEBRA embeddings -Siddharth_Chauhan#3

Open
SIDDHARTH1-1CHAUHAN wants to merge 3 commits into
ML4SCI:mainfrom
SIDDHARTH1-1CHAUHAN:feature/stratified-cv-learning-curves
Open

Add repeated stratified CV and learning-curve evaluation for supervised CEBRA embeddings -Siddharth_Chauhan#3
SIDDHARTH1-1CHAUHAN wants to merge 3 commits into
ML4SCI:mainfrom
SIDDHARTH1-1CHAUHAN:feature/stratified-cv-learning-curves

Conversation

@SIDDHARTH1-1CHAUHAN

Copy link
Copy Markdown

Problem

The current supervised workflow saves embeddings, consistency scores, and a single train/test decoding score per pairing, but it does not provide repeated stratified cross-validation or learning-curve outputs for decoding stability. That makes the evaluation sensitive to one split and limits how easily supervised runs can be compared.

What I changed

  • added a small reusable utils/cv_eval.py helper for repeated stratified cross-validation and learning-curve evaluation on learned embeddings
  • added optional flags to train_cebra_cut_supervised.py to run these evaluations without changing the default training flow
  • exported per-fold and aggregate CSV/JSON metrics under results/evaluation/supervised/
  • updated the README with the new supervised evaluation command and output location

Why this matters

Repeated stratified CV provides a more stable estimate of decoding performance than a single split, and the learning-curve outputs make it possible to inspect how decoding performance changes as the amount of training data increases. This makes the supervised embedding analysis more reproducible and better aligned with the NeuroDyads goal of evaluating stable neural dyad decoding.

How to run

python prepare_cebra_input.py
python train_cebra_cut_supervised.py --run-cv --cv-folds 5 --cv-repeats 1 --run-learning-curve --lc-fracs 0.2,0.4,0.6,0.8,1.0


## By: Siddharth Chauhan

@SIDDHARTH1-1CHAUHAN SIDDHARTH1-1CHAUHAN changed the title Adding repeated stratified CV and learning-curve evaluation for supervised CEBRA embeddings -Siddharth_Chauhan Add repeated stratified CV and learning-curve evaluation for supervised CEBRA embeddings -Siddharth_Chauhan Mar 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant