Skip to content

Add DiskANN benchmark pipeline to GitHub Actions #857

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
YuanyuanTian-hh wants to merge 31 commits into
base: main
Choose a base branch
from
Open

Add DiskANN benchmark pipeline to GitHub Actions #857

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
c6402fb
Migrate DiskANN benchmark pipeline from ADO to GitHub Actions
Mar 19, 2026
2217353
Fix openai-100K distance metric and add gitignore patterns
Mar 20, 2026
6b7b250
Add push trigger to benchmarks workflow for pre-merge testing
Mar 20, 2026
d4a6abd
Fix baseline run: use input config from current branch checkout
Mar 20, 2026
c6d54a9
Fix markdown conversion: replace broken csvtomd with inline Python
Mar 20, 2026
5775198
Update benchmark configs: align build/search params to fix low recall
Mar 20, 2026
7ac841e
Fix recall_at: set to 100 to match groundtruth file K=100
Mar 23, 2026
7d5a72e
Remove stale absolute contracts (qps/recall/total_time): calibrated f…
Mar 23, 2026
c294c16
remove comments
Mar 23, 2026
b8e1d6d
Fix build_ram_limit_gb: reduce 10->4 to fit GitHub runner RAM (7GB li…
Mar 23, 2026
137cae0
Fix wikipedia distance: cosine_normalized->cosine (vectors are not L2…
Mar 23, 2026
96b63a8
Fix wikipedia distance: cosine->inner_product (groundtruth uses raw i…
Mar 23, 2026
1b4cfc8
Align build threads
Mar 24, 2026
5e6a6e0
Speed up benchmarks: build threads 1->4, openai pq_chunks 384->192
Mar 24, 2026
59d25b9
Temp: disable concurrency cancellation for A/A batch testing
Mar 24, 2026
751e775
revert A/A test settings, update OpenAI config to SQ_1_2.0
Mar 26, 2026
2c4d235
Remove micro-benchmark-iai comments
Mar 27, 2026
f58e084
use GitHub Release assets for benchmark datasets
Mar 27, 2026
0f5c277
extract csv-to-markdown into reusable script
Mar 27, 2026
b8cddb5
calibrate contract thresholds from GitHub runner data
Mar 27, 2026
accdf2c
add daily A/A benchmark stability test with failure notification
Mar 27, 2026
5b0e7f7
widen mean_cpus threshold to 15% for shared-runner CPU noise
Mar 27, 2026
639d4bb
move benchmark datasets to separate repo (YuanyuanTian-hh/diskann-ben…
Mar 30, 2026
1b52de2
consolidate 3 benchmark scripts into single benchmark_validate.py
Mar 30, 2026
9779e3c
switch benchmark jobs to self-hosted 1ES runner pool (diskann-github)
Mar 31, 2026
db7e97a
replace gh CLI with curl for dataset downloads (gh not available on 1…
Mar 31, 2026
ea91c62
fix latency_95: read p95_latency instead of p999_latency from benchma…
Mar 31, 2026
360cdc7
revert to ubuntu-latest runners, switch dataset source to BAB v0.4.0
Apr 1, 2026
9d635be
address PR review: reduce search_list to 200, remove hardcoded Rust v…
Apr 2, 2026
4ca80d1
widen latency_95 threshold to 15% for shared-runner noise
Apr 2, 2026
d168932
replace push trigger with pull_request trigger targeting main
Apr 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Loading