chore(bench): add vm-linux-x86_64 benchmark run 2026-09-02 - #2370
Merged
Conversation
96 cases, 10 iterations, bashkit vs bash 5.2.21. 0 errors, 100% output match, 20.7x faster on average per case (0.539 ms vs 11.174 ms). Refreshes site performance timeline.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Adds a fresh
bashkit-benchrun (bashkit in-process vs bash 5.2.21 subprocess) tocrates/bashkit-bench/results/and refreshes the generated/benchessite timeline from it. No code changes.Published reports:
Why
The last saved
vm-linux-x86_64run is from 2026-05-26. This refreshes the baseline on currentmain(v0.17.1) per the result-location contract inknowledge/operations/performance-results.md.Before / After
Same machine class (vm, linux x86_64, 4 CPUs), 96 cases × 10 iterations, 0 errors, 100% output match in both runs.
The companion 3-month comparison (3 alternating rounds × 30 iterations per version, same machine) shows a split rather than a uniform change: tool builtins are 20–55% faster (grep/sed,
tool_jq_map1.46×) andcontrolis up 29%, while every case in thelargecategory regressed 12–33% (category geo-mean 0.83×; e.g.large_function_calls_5005.48 → 6.59 ms,large_string_append_1000.286 → 0.372 ms). Overall geo-mean 1.02×, total time 40.9 → 47.3 ms. Worth a bisect onlarge_function_calls_500; not addressed in this PR.Risk
site/scripts/build-performance-data.mjsand verified deterministic on re-run.Checklist
Generated by Claude Code