Interest in Foundation Model Fine-tuning for Dark Matter Classification

[Valyrian-Code]

Introduction

Hi everyone, I’m Rajveer Bishnoi and I’m interested in contributing to DeepLense.

I’ve been exploring the projects in this repository and reading through the existing work on Lensiformer, LensCoAt, ViT-based models, equivariant networks, and SSL approaches.

While going through the repository, I noticed that large pre-trained vision foundation models like DINOv2 or MAE do not seem to have been explored yet on the DeepLense datasets.

Idea

I’m interested in experimenting with:

• Fine-tuning DINOv2-base on Models 1–3
• Comparing results with existing models already benchmarked in the repository
• Studying whether large-scale visual pretraining helps with dark matter substructure classification

A possible extension could be evaluating the same model on Model 4 (real galaxy images) to test transfer from natural-image pretraining to astrophysical data.

Why this seems interesting

• Foundation models are now widely used in computer vision
• DINOv2 learns strong spatial features that may work well for localized lensing signals
• A direct benchmark against existing DeepLense models could be useful for future contributors

Questions

1. Would this be a useful direction to explore within DeepLense?
2. Is there already any ongoing work related to this?
3. Is there a preferred communication channel for discussing ideas further?

I’m happy to start with smaller contributions first and improve the scope based on feedback.

Thank you.