Skip to content

chore(bench): add vm-linux-x86_64 benchmark run 2026-09-02 - #2370

Merged
chaliy merged 1 commit into
mainfrom
claude/benchmarks-stend-vl81zw
Sep 2, 2026
Merged

chore(bench): add vm-linux-x86_64 benchmark run 2026-09-02#2370
chaliy merged 1 commit into
mainfrom
claude/benchmarks-stend-vl81zw

Conversation

@chaliy

@chaliy chaliy commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

What changed

Adds a fresh bashkit-bench run (bashkit in-process vs bash 5.2.21 subprocess) to crates/bashkit-bench/results/ and refreshes the generated /benches site timeline from it. No code changes.

Published reports:

Why

The last saved vm-linux-x86_64 run is from 2026-05-26. This refreshes the baseline on current main (v0.17.1) per the result-location contract in knowledge/operations/performance-results.md.

Before / After

Same machine class (vm, linux x86_64, 4 CPUs), 96 cases × 10 iterations, 0 errors, 100% output match in both runs.

2026-05-26 (v0.9.0 era) 2026-09-02 (this PR)
bashkit avg / case 0.447 ms 0.539 ms
bash avg / case 9.363 ms 11.174 ms
bashkit vs bash (avg) 20.9× 20.7×
bashkit vs bash (geo-mean) 24.4× 29.8×

The companion 3-month comparison (3 alternating rounds × 30 iterations per version, same machine) shows a split rather than a uniform change: tool builtins are 20–55% faster (grep/sed, tool_jq_map 1.46×) and control is up 29%, while every case in the large category regressed 12–33% (category geo-mean 0.83×; e.g. large_function_calls_500 5.48 → 6.59 ms, large_string_append_100 0.286 → 0.372 ms). Overall geo-mean 1.02×, total time 40.9 → 47.3 ms. Worth a bisect on large_function_calls_500; not addressed in this PR.

Risk

  • Low
  • Results-only change. Site timeline regenerated with site/scripts/build-performance-data.mjs and verified deterministic on re-run.

Checklist

  • Tests added or updated — N/A, no code changed
  • Backward compatibility considered — N/A

Generated by Claude Code

96 cases, 10 iterations, bashkit vs bash 5.2.21. 0 errors, 100% output
match, 20.7x faster on average per case (0.539 ms vs 11.174 ms).
Refreshes site performance timeline.
@chaliy
chaliy merged commit ab04bca into main Sep 2, 2026
25 checks passed
@chaliy
chaliy deleted the claude/benchmarks-stend-vl81zw branch September 2, 2026 19:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant