Pipeline StageIterator: Improve performance and simplify

[danotorrey]

Description

Refactor the pipeline StageIterator to skip gaps between stages, improving performance by avoiding unnecessary iteration and eliminating map.containsKey() checks for stage numbers that have no associated pipeline.

StageIterator in particular can be a hotspot, since it previously iterates all stages for each message that is processed (on each processor thread), this delay could add up with messages processing in serial on each processor thread.

Testing

The existing unit tests still pass with this refactor (good initial sign). Initial performance testing shows roughly the execution time between two contiguous stages and far apart stages (e.g 0,int.max). Some additional tests have also been added. We should do some solid testing with this within a running application too.

[Copilot]

Pull request overview

This PR refactors the pipeline StageIterator to improve performance by eliminating unnecessary iteration through gaps between stages. Instead of iterating through all stage numbers from minimum to maximum and checking for existence, the new implementation directly iterates only over stage numbers that actually have associated pipelines.

Key Changes:

• Replaced iteration-based approach with direct stage enumeration using ImmutableSortedSet and ImmutableListMultimap
• Eliminated containsKey() checks by pre-computing only existing stage numbers
• Simplified iterator logic by delegating to Iterators.transform()

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File	Description
StageIterator.java	Refactored to use immutable collections and direct iteration over existing stages only, removing gap-checking logic
StageIteratorTest.java	Added three new test cases verifying iterator behavior with disparate stage numbers, multiple pipelines, and empty pipelines
pr-24528.toml	Added changelog entry documenting the performance improvement

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.