Prefilter VCS permalink lines#2253
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #2253 +/- ##
==========================================
+ Coverage 92.73% 92.75% +0.01%
==========================================
Files 127 127
Lines 27486 27518 +32
==========================================
+ Hits 25489 25523 +34
+ Misses 1997 1995 -2 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4e64c23d35
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
📦 Cargo Bloat ComparisonBinary size change: +1.12% (26.7 MiB → 27.0 MiB) Expand for cargo-bloat outputHead Branch ResultsBase Branch Results |
⚡️ Hyperfine BenchmarksSummary: 0 regressions, 0 improvements above the 10% threshold. Environment
CLI CommandsBenchmarking basic commands in the main repo:
|
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
prek-base --version |
2.2 ± 0.2 | 2.0 | 3.7 | 1.04 ± 0.12 |
prek-head --version |
2.1 ± 0.1 | 2.0 | 3.0 | 1.00 |
prek list
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
prek-base list |
9.2 ± 3.1 | 8.6 | 40.2 | 1.01 ± 0.38 |
prek-head list |
9.1 ± 1.4 | 8.6 | 22.8 | 1.00 |
prek validate-config .pre-commit-config.yaml
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
prek-base validate-config .pre-commit-config.yaml |
3.0 ± 0.1 | 2.8 | 3.4 | 1.01 ± 0.05 |
prek-head validate-config .pre-commit-config.yaml |
2.9 ± 0.1 | 2.8 | 3.1 | 1.00 |
prek sample-config
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
prek-base sample-config |
2.4 ± 0.0 | 2.3 | 2.5 | 1.00 |
prek-head sample-config |
2.4 ± 0.0 | 2.3 | 2.5 | 1.01 ± 0.03 |
Cold vs Warm Runs
Comparing first run (cold) vs subsequent runs (warm cache):
prek run --all-files (cold - no cache)
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
prek-base run --all-files |
72.0 ± 2.6 | 69.6 | 76.7 | 1.00 |
prek-head run --all-files |
72.6 ± 2.2 | 70.0 | 75.8 | 1.01 ± 0.05 |
prek run --all-files (warm - with cache)
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
prek-base run --all-files |
72.8 ± 2.4 | 68.1 | 76.5 | 1.00 |
prek-head run --all-files |
73.3 ± 2.7 | 68.0 | 77.1 | 1.01 ± 0.05 |
Full Hook Suite
Running the builtin hook suite on the benchmark workspace:
prek run --all-files (full builtin hook suite)
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
prek-base run --all-files |
73.2 ± 2.5 | 68.2 | 79.5 | 1.00 ± 0.05 |
prek-head run --all-files |
73.1 ± 2.6 | 68.5 | 79.0 | 1.00 |
Individual Hook Performance
Benchmarking each hook individually on the test repo:
prek run trailing-whitespace --all-files
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
prek-base run trailing-whitespace --all-files |
19.3 ± 0.4 | 18.9 | 20.4 | 1.00 ± 0.03 |
prek-head run trailing-whitespace --all-files |
19.3 ± 0.4 | 18.6 | 19.9 | 1.00 |
prek run end-of-file-fixer --all-files
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
prek-base run end-of-file-fixer --all-files |
24.5 ± 1.3 | 23.2 | 28.8 | 1.00 |
prek-head run end-of-file-fixer --all-files |
25.3 ± 1.5 | 23.4 | 29.4 | 1.03 ± 0.08 |
prek run check-json --all-files
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
prek-base run check-json --all-files |
7.4 ± 0.3 | 7.0 | 8.1 | 1.01 ± 0.05 |
prek-head run check-json --all-files |
7.4 ± 0.3 | 7.0 | 8.0 | 1.00 |
prek run check-yaml --all-files
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
prek-base run check-yaml --all-files |
7.2 ± 0.1 | 6.9 | 7.4 | 1.00 ± 0.03 |
prek-head run check-yaml --all-files |
7.1 ± 0.1 | 6.9 | 7.5 | 1.00 |
prek run check-toml --all-files
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
prek-base run check-toml --all-files |
7.5 ± 0.3 | 6.8 | 8.4 | 1.02 ± 0.06 |
prek-head run check-toml --all-files |
7.3 ± 0.3 | 7.0 | 8.1 | 1.00 |
prek run check-xml --all-files
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
prek-base run check-xml --all-files |
7.3 ± 0.2 | 6.9 | 7.7 | 1.00 |
prek-head run check-xml --all-files |
7.4 ± 0.3 | 6.8 | 7.8 | 1.01 ± 0.05 |
prek run detect-private-key --all-files
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
prek-base run detect-private-key --all-files |
12.9 ± 1.1 | 11.4 | 15.7 | 1.00 |
prek-head run detect-private-key --all-files |
13.0 ± 1.0 | 11.3 | 14.9 | 1.01 ± 0.12 |
prek run fix-byte-order-marker --all-files
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
prek-base run fix-byte-order-marker --all-files |
17.3 ± 0.8 | 16.1 | 18.9 | 1.00 |
prek-head run fix-byte-order-marker --all-files |
17.4 ± 0.8 | 16.4 | 19.3 | 1.01 ± 0.06 |
Installation Performance
Benchmarking hook installation (fast path hooks skip Python setup):
prek install-hooks (cold - no cache)
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
prek-base install-hooks |
4.3 ± 0.1 | 4.2 | 4.4 | 1.02 ± 0.03 |
prek-head install-hooks |
4.2 ± 0.1 | 4.1 | 4.3 | 1.00 |
prek install-hooks (warm - with cache)
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
prek-base install-hooks |
4.3 ± 0.1 | 4.2 | 4.3 | 1.00 |
prek-head install-hooks |
4.4 ± 0.1 | 4.3 | 4.4 | 1.02 ± 0.02 |
File Filtering/Scoping Performance
Testing different file selection modes:
prek run (staged files only)
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
prek-base run |
37.5 ± 1.4 | 35.2 | 41.0 | 1.01 ± 0.05 |
prek-head run |
37.0 ± 1.1 | 35.3 | 39.6 | 1.00 |
prek run --files '*.json' (specific file type)
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
prek-base run --files '*.json' |
7.7 ± 0.1 | 7.5 | 7.9 | 1.00 |
prek-head run --files '*.json' |
8.0 ± 0.2 | 7.7 | 8.3 | 1.04 ± 0.02 |
Workspace Discovery & Initialization
Benchmarking hook discovery and initialization overhead:
prek run --dry-run --all-files (measures init overhead)
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
prek-base run --dry-run --all-files |
6.5 ± 0.1 | 6.4 | 6.6 | 1.00 ± 0.01 |
prek-head run --dry-run --all-files |
6.5 ± 0.1 | 6.4 | 6.7 | 1.00 |
Meta Hooks Performance
Benchmarking meta hooks separately:
prek run check-hooks-apply --all-files
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
prek-base run check-hooks-apply --all-files |
11.2 ± 0.6 | 10.0 | 11.8 | 1.10 ± 0.06 |
prek-head run check-hooks-apply --all-files |
10.2 ± 0.1 | 10.0 | 10.4 | 1.00 |
prek run check-useless-excludes --all-files
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
prek-base run check-useless-excludes --all-files |
10.1 ± 0.1 | 10.0 | 10.3 | 1.00 |
prek-head run check-useless-excludes --all-files |
10.2 ± 0.1 | 10.0 | 10.6 | 1.01 ± 0.02 |
prek run identity --all-files
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
prek-base run identity --all-files |
9.4 ± 0.1 | 9.3 | 9.5 | 1.00 |
prek-head run identity --all-files |
9.6 ± 0.1 | 9.4 | 9.8 | 1.01 ± 0.01 |
```
❯ hyperfine --warmup 5 --runs 30 "prek run check-vcs-permalinks --all-files" "./target/profiling/prek run check-vcs-permalinks --all-files" -i
Benchmark 1: prek run check-vcs-permalinks --all-files
Time (mean ± σ): 107.5 ms ± 8.3 ms [User: 81.3 ms, System: 29.2 ms]
Range (min … max): 97.6 ms … 134.6 ms 30 runs
Warning: Ignoring non-zero exit code.
Benchmark 2: ./target/profiling/prek run check-vcs-permalinks --all-files
Time (mean ± σ): 39.3 ms ± 1.1 ms [User: 20.4 ms, System: 27.5 ms]
Range (min … max): 36.9 ms … 41.8 ms 30 runs
Warning: Ignoring non-zero exit code.
Summary
./target/profiling/prek run check-vcs-permalinks --all-files ran
2.74 ± 0.23 times faster than prek run check-vcs-permalinks --all-files
```
Uh oh!
There was an error while loading. Please reload this page.