Feature/real data metrics conference original#190

Open

tobias-liaudat wants to merge 146 commits intodevelopfrom

feature/real_data_metrics_conference_original

Commits on Aug 5, 2025

Correct syntax in docstring and generalise exception message
Jennifer Pollack
committed
Add inference and test_inference packages
Jennifer Pollack
committed
Refactor: Encapsulate logic in psf_models package with subpackages: models and tf_modules, add/rm modules, update import statements and tests
Jennifer Pollack
committed
Remove unused module with duplicate zernike_generator function
Jennifer Pollack
committed
Correct syntax in docstrings and logger messages
Jennifer Pollack
committed
Refactor file structure; update import statements in tests; remove unit test due to refactoring
Jennifer Pollack
committed
Update package name in import statement
Jennifer Pollack
committed
Reorder imports; Refactor MetricsConfigHandler class attributes, methods and variable names|
Jennifer Pollack
committed
Move psf_model weights loader to psf_model_loader.py module
Jennifer Pollack
committed
Add psf_model_loader module
Jennifer Pollack
committed
Remove weights_path arg from evaluate_model method; Update logger.info statement and comment

jeipollack
authored and
Jennifer Pollack
committed
Update variable name and logger statement

jeipollack
authored and
Jennifer Pollack
committed
Add import logging and create logger object

jeipollack
authored and
Jennifer Pollack
committed
Create psf_inference.py module
Jennifer Pollack
committed
Remove arg from evaluate_model unit test
Jennifer Pollack
committed
Refactor: reorganise modules, relocate utility functions, rename modules, update import statements and unit tests
Jennifer Pollack
committed
Update import statements to new module names
Jennifer Pollack
committed
Update DataHandler class docstring to include option for inference dataset handling
Jennifer Pollack
committed
Refactor data_handler with new utility functions to validate and process datasets and update docstrings
Jennifer Pollack
committed
Update unit tests associated to changes in data_handler.py
Jennifer Pollack
committed
Change exception handling in DataConfigHandler; modify args to DataHandler
Jennifer Pollack
committed
Add data and psf_model_imports into inference and sketch out methods
Jennifer Pollack
committed
add base psf inference

tobias-liaudat
authored and
Jennifer Pollack
committed
add common call interface through PSF models

tobias-liaudat
authored and
Jennifer Pollack
committed
add handling of inference params

tobias-liaudat
authored and
Jennifer Pollack
committed
automatic formatting

tobias-liaudat
authored and
Jennifer Pollack
committed
add first completed class draft

tobias-liaudat
authored and
Jennifer Pollack
committed
add inference config file

tobias-liaudat
authored and
Jennifer Pollack
committed
remove unused code

tobias-liaudat
authored and
Jennifer Pollack
committed
update params

tobias-liaudat
authored and
Jennifer Pollack
committed
update params

tobias-liaudat
authored and
Jennifer Pollack
committed
update inference

tobias-liaudat
authored and
Jennifer Pollack
committed
reduce arguments and add compute psfs when appropiate

tobias-liaudat
authored and
Jennifer Pollack
committed
add config handler class

tobias-liaudat
authored and
Jennifer Pollack
committed
set up inference config handler and simplify PSFInferenc init

tobias-liaudat
authored and
Jennifer Pollack
committed
remove unused imports

tobias-liaudat
authored and
Jennifer Pollack
committed
update inference

tobias-liaudat
authored and
Jennifer Pollack
committed
Add single-space lines to improve readability; Remove duplicated static method
Jennifer Pollack
committed
Add additional PSFInference class attributes; update set_source_parameters to use class attributes; assign variable names in get_trained_psf_model; Update class and __init__ docstrings
Jennifer Pollack
committed
Add checks to convert to np.ndarray and expand dimensions if needed

jeipollack
authored and
Jennifer Pollack
committed
Update pyproject.toml with numpy dependency limits - sdc-uk

jeipollack
authored and
Jennifer Pollack
committed
Revert "Update pyproject.toml with numpy dependency limits - sdc-uk"

jeipollack
authored and
Jennifer Pollack
committed
Correct name of psf_inference_test module to follow repo naming convention
Jennifer Pollack
committed
Correct config subkey names for defining trained_model_path and trained_model_config_path
Jennifer Pollack
committed
Refactor psf_inference adding PSFInferenceEngine to separate concerns, enabling isolated testing; implement lazy loaders for config handling, model loading, and inference
Jennifer Pollack
committed
Add unit tests for psf_inference
Jennifer Pollack
committed
Bugfix: Ensure updated training_config.training.model_params are passed to simPSF
Jennifer Pollack
committed
test(simPSF): add unit test to verify updated model_params are passed
Jennifer Pollack
committed
Bug: replace self.data_conf with self.data_config

jeipollack
authored and
Jennifer Pollack
committed
Change logger.warnings to ValueErrors for missing fields in datasets & remove redundant check
Jennifer Pollack
committed
Update unit tests with changes to data_handler.py
Jennifer Pollack
committed

Commits on Aug 18, 2025

Refactor: reorganise modules, relocate utility functions, rename modules, update import statements and unit tests

Jennifer Pollack
authored and
jeipollack
committed
Refactor data_handler with new utility functions to validate and process datasets and update docstrings

Jennifer Pollack
authored and
jeipollack
committed
Update unit tests associated to changes in data_handler.py

Jennifer Pollack
authored and
jeipollack
committed
automatic formatting

tobias-liaudat
authored and
jeipollack
committed
Refactor: add ZernikeInputs dataclass, ZernikeInputsFactory, helper methods for assembling zernike contributions according to run_type mode: training, simulation, or inference

Jennifer Pollack
authored and
jeipollack
committed
Update docstring describing data_conf types permitted

Jennifer Pollack
authored and
jeipollack
committed
Move imports to method to avoid circular imports

Jennifer Pollack
authored and
jeipollack
committed
Remove batch_size arg from ZernikeInputsFactory ; raise ValueError to check Zernike contributions have the same number of samples

Jennifer Pollack
authored and
jeipollack
committed
Add and set run_type attribute to DataConfigHandler object in TrainingConfigHandler constructor

Jennifer Pollack
authored and
jeipollack
committed
Add and set run_type attribute ; Replace var name end with end_sample in PSFInferenceEngine

Jennifer Pollack
authored and
jeipollack
committed
Refactor TFPhysicalPolychromaticField to lazy load property objects and attributes dynamically at run-time according to the run_type: training or inference

Jennifer Pollack
authored and
jeipollack
committed
Update/Add unit tests to test refactoring changes to psf_model_physical_polychromatic.py and data_zernike_utils.py

Jennifer Pollack
authored and
jeipollack
committed
Replace arg: data in compute_ccd_misalignment with positions

Jennifer Pollack
authored and
jeipollack
committed
Correct object attributes for DataConfigHandler in ZernikeInputsFactory
jeipollack
committed
Add missing return for tf_physical_layer property
jeipollack
committed
Add tf_utils.py module to tf_modules subpackage
jeipollack
committed
Use ensure_tensor method from tf_utils.py to check/convert to tensorflow type; Remove get_obs_positions and replace with ensure_tensor method; add property tf_positions
jeipollack
committed
Refactor: Add eager-mode helpers and avoid lazy-loading obscurations in graph mode
jeipollack
committed
Replace deprecated get_obs_positions with get_np_obs_positions and apply ensure_tensor to convert obs_pos to tensorflow float32
jeipollack
committed
Remove tf.convert_to_tensor from all Zernike list contributors
jeipollack
committed
Add and set self.data_conf.run_type value to 'metrics' in MetricsConfigHandler
jeipollack
committed
Eagerly precompute Zernike components; add support for 'metrics' run_type
jeipollack
committed
Correct value error: train in dataset_type with training
jeipollack
committed
fix: pass random seed to TFNonParametricPolynomialVariationsOPD constructor in psf_model_physical_polychromatic.py
jeipollack
committed
Refactor to suppress TensorFlow debug msgs: replace lambda in call method with a proper function: find_position_indices enable batch processing for better graph optimization
jeipollack
committed
Match old behaviour with conditional and float64 accumulation
jeipollack
committed
Add helper to stack x/y field coordinates into (N, 2) positions array
jeipollack
committed
Add helper method to prepare dataset for inference & handle empty/None fields
jeipollack
committed
Update data_handler_test replacing "get_obs_positions" (deprecation) with "get_np_obs_positions"

Jennifer Pollack
authored and
jeipollack
committed
Remove deprecated code from rebase

Jennifer Pollack
authored and
jeipollack
committed
Remove duplicated checks on arg existance

Jennifer Pollack
authored and
jeipollack
committed
Improve Zernike prior handling in assemble_zernike_contributions

Jennifer Pollack
authored and
jeipollack
committed
Fix bug where Tensor zernike_prior was not appended after eager conversion

Jennifer Pollack
authored and
jeipollack
committed
Update unit tests with latest changes to fixtures and data_zernike_utils.py

Jennifer Pollack
authored and
jeipollack
committed
Set mock Zernike priors to None in test_data_utils.py helper module

Jennifer Pollack
authored and
jeipollack
committed
Remove -1.0 multiplicative factor applied to Zernike tip and tilt values

Jennifer Pollack
authored and
jeipollack
committed
Update unit tests with changes to compute_centroid_correction

Jennifer Pollack
authored and
jeipollack
committed
Move TFPhysicalPolychromaticField.pad_zernikes to helper method pad_tf_zernikes in data_zernike_utils.py

Jennifer Pollack
authored and
jeipollack
committed
Correct bug in test_load_inference_model

Jennifer Pollack
authored and
jeipollack
committed
Revert sign change applied to compute_centroid_correction

Jennifer Pollack
authored and
jeipollack
committed

Commits on Aug 19, 2025

Refactor _prepare_positions_and_seds to enforce shape consistency and add validation
Jennifer Pollack
committed

Commits on Aug 21, 2025

Fix tensor handling in ZernikeInputsFactory
jeipollack
committed
Reformat and remove unused import
Jennifer Pollack
committed

Commits on Aug 22, 2025

Correct zernike_prior extraction when dataset is a dict, reformat file
Jennifer Pollack
committed
Replace np.array with Tensorflow tensors in unit test and fixtures, reformat files
Jennifer Pollack
committed

Commits on Aug 27, 2025

Commits on Sep 3, 2025

refactor: centralise PSF data extraction in data_handler
jeipollack
committed

Commits on Sep 4, 2025

Commits on Sep 5, 2025

Reformat with black
Jennifer Pollack
committed
Correct type hint errors
jeipollack
committed
Remove unused import
jeipollack
committed
Replace call to deprecated get_np_obs_positions with get_data_array
jeipollack
committed

Commits on Oct 31, 2025

Remove unused imports and reformat
Jennifer Pollack
committed
Update fixtures and unit tests
Jennifer Pollack
committed
Reformat with black
Jennifer Pollack
committed

Commits on Nov 26, 2025

refactor the noise std dev calculation

tobias-liaudat
authored and
Jennifer Pollack
committed
improve refactoring

tobias-liaudat
authored and
Jennifer Pollack
committed
add chi2 metric function

tobias-liaudat
authored and
Jennifer Pollack
committed
added chi2 metric to list and automatic black formating

tobias-liaudat
authored and
Jennifer Pollack
committed
improve naming

tobias-liaudat
authored and
Jennifer Pollack
committed
improve docsting

tobias-liaudat
authored and
Jennifer Pollack
committed
add chi2 evaluation function

tobias-liaudat
authored and
Jennifer Pollack
committed
add evalutation flag

tobias-liaudat
authored and
Jennifer Pollack
committed
fix parameter name bug

tobias-liaudat
authored and
Jennifer Pollack
committed
fix data type problem

tobias-liaudat
authored and
Jennifer Pollack
committed
Refactor: reorganise modules, relocate utility functions, rename modules, update import statements and unit tests
Jennifer Pollack
committed
Refactor data_handler with new utility functions to validate and process datasets and update docstrings
Jennifer Pollack
committed
Update unit tests associated to changes in data_handler.py
Jennifer Pollack
committed
Refactor TFPhysicalPolychromaticField to lazy load property objects and attributes dynamically at run-time according to the run_type: training or inference
Jennifer Pollack
committed
Use ensure_tensor method from tf_utils.py to check/convert to tensorflow type; Remove get_obs_positions and replace with ensure_tensor method; add property tf_positions

jeipollack
authored and
Jennifer Pollack
committed
add median noise calculation and todo comment

tobias-liaudat
authored and
Jennifer Pollack
committed
update docstring

tobias-liaudat
authored and
Jennifer Pollack
committed
update tabs

tobias-liaudat
authored and
Jennifer Pollack
committed
add info about the reference simulated datasets

tobias-liaudat
authored and
Jennifer Pollack
committed
add noisy stars and masked noisy stars for the test dataset

tobias-liaudat
authored and
Jennifer Pollack
committed
add noshift generation for the test dataset

tobias-liaudat
authored and
Jennifer Pollack
committed
update to new paths

tobias-liaudat
authored and
Jennifer Pollack
committed
update the intrapixel shift range

tobias-liaudat
authored and
Jennifer Pollack
committed
fix small bug for unitary masks

tobias-liaudat
authored and
Jennifer Pollack
committed
fix chi2 bug

tobias-liaudat
authored and
Jennifer Pollack
committed
add chi2 per image calculation

tobias-liaudat
authored and
Jennifer Pollack
committed
add per image results

tobias-liaudat
authored and
Jennifer Pollack
committed
add more output statistics for the chi2 metric

tobias-liaudat
authored and
Jennifer Pollack
committed
update default metric config to include chi2 option

tobias-liaudat
authored and
Jennifer Pollack
committed
merge feature/159-psf-output-from-trained-model with real data metrics

tobias-liaudat
authored and
Jennifer Pollack
committed
compute the metrics on `noisy_stars` if `stars` is not in the data. Regenerate if it is not there

tobias-liaudat
authored and
Jennifer Pollack
committed
fix small bug in nested conditionals

tobias-liaudat
authored and
Jennifer Pollack
committed
Update evaluate_model unit test with patch for chi2 metric
Jennifer Pollack
committed
Update train_util tests with mock data including binary mask
Jennifer Pollack
committed
Remove duplicated function and unnecessary variable re-assignment
Jennifer Pollack
committed
Remove functions left over from previous rebase
Jennifer Pollack
committed