Skip to content

Benchmarking#410

Open
jeilealr wants to merge 4 commits into
GeoStat-Framework:mainfrom
jeilealr:benchmarking
Open

Benchmarking#410
jeilealr wants to merge 4 commits into
GeoStat-Framework:mainfrom
jeilealr:benchmarking

Conversation

@jeilealr
Copy link
Copy Markdown

Write Benchmarks for Current DS-MPS Implementation

Summary

This PR adds a reproducible benchmarking setup for GSTools using
Airspeed Velocity plus a small complementary
cProfile helper.

The goal is to establish a measurement baseline before optimizing or extending
the DS-MPS implementation. The benchmarks focus on representative GSTools
workflows rather than isolated micro-functions:

  • variogram estimation
  • global kriging
  • spatial random field generation
  • conditioned spatial random field generation

The suite can compare the current Cython fallback backend with the Rust backend
from gstools_core, and it can report both runtime and peak-memory results.

What Changed

  • Added the default ASV configuration in asv.conf.json.
  • Added benchmark workflows in benchmarks/benchmark_backends.py.
  • Added helper scripts for:
    • Rust-vs-Cython speedup summaries
    • cProfile workflow profiling
    • Cython OpenMP verification
    • macOS OpenMP-enabled gstools-cython installation inside ASV
  • Added an optional macOS OpenMP ASV configuration in
    asv.macos-openmp.conf.json.
  • Added a detailed benchmark guide in benchmarks/README.md.
  • Added the benchmark optional dependency extra.
  • Added a GitHub Actions workflow for benchmark checks.

Most of the detailed usage instructions, benchmark case explanations, and
interpretation notes are intentionally kept in benchmarks/README.md to avoid
duplicating the guide in this PR description.

Why ASV and cProfile?

ASV is used for repeatable benchmark results across commits. This makes it
useful for detecting performance regressions and tracking how workflows change
over time. It is also the standard benchmarking tool used by several scientific
Python projects, including NumPy, SciPy, pandas, scikit-image, and Astropy.

cProfile is used for a different purpose: it profiles one selected workflow in
the current checkout and shows where runtime is spent internally. In this PR,
the cProfile helper imports the same benchmark classes used by ASV, so the
profiling cases stay aligned with the benchmark suite.

Initial Observations

The initial benchmark runs show that Rust is generally faster for larger
variogram and kriging workloads. For very small cases, Rust can be similar or
slightly slower because fixed overhead dominates. With one thread, Rust is not
always faster for every SRF case, so the backend comparison is workflow- and
problem-size-dependent.

The cProfile results suggest that the slowest parts of the measured workflows
are mostly inside compiled backend/numerical routines. That means Python-level
cleanup is unlikely to be the main optimization path for these cases. The next
useful optimization work should focus on algorithmic cost reduction and backend
implementation details.

Follow-Up Tasks

  • Test and document the OpenMP benchmark setup on Linux.
  • Test and document the OpenMP benchmark setup on Windows.
  • Test and document the OpenMP benchmark setup on an HPC environment.
  • Decide whether Linux, Windows, and HPC need separate ASV config files or can
    share one generalized OpenMP setup.
  • Connect these benchmarks more directly to the DS-MPS implementation once the
    current migration work is ready to benchmark.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant