Skip to content

Improve benchmark test image - #9831

Open
akx wants to merge 1 commit into
python-pillow:mainfrom
akx:better-benchmark-image
Open

Improve benchmark test image#9831
akx wants to merge 1 commit into
python-pillow:mainfrom
akx:better-benchmark-image

Conversation

@akx

@akx akx commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

The old image was highly channel-correlated, low-color, low-structure and non-representative of anything real. My bad in #9654 :)

The new image contains both low-energy, high-energy (full noise) and zero-energy segments.
If someone has an idea for an even better synthetic image, I'm all ears!

Let's see how the Codspeed numbers change - at least locally, test_quantize and test_histogram got a lot slower, because they actually need to do work 😄

Old New
old new

The old image was (e.g. in the case of palette optimization) pessimal in some cases.
@codspeed-hq

codspeed-hq Bot commented Jul 29, 2026

Copy link
Copy Markdown

Merging this PR will degrade performance by 29.32%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
❌ 1 regressed benchmark
✅ 343 untouched benchmarks
⏩ 2 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
test_quantize[1024x1024-RGB] 233.7 ms 527.8 ms -55.72%
test_scale[1024x1024-RGB-0.01-NEAREST] 68.3 µs 60.5 µs +12.82%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing akx:better-benchmark-image (69f73df) with main (72f6e83)

Open in CodSpeed

Footnotes

  1. 2 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@akx

akx commented Jul 29, 2026

Copy link
Copy Markdown
Contributor Author

Merging this PR will degrade performance by 29.32%

As predicted! But no, code performance didn't change, just measurements for the more accurate...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants