Skip to content

Conversation

@teskje
Copy link
Contributor

@teskje teskje commented Jan 29, 2026

This PR removes the invalidate_for_index logic that was responsible for invalidating all expr cache entries for objects that could potentially make use of a newly added index.

There are two reasons to think we don't want this logic:

  • Currently, bootstrapping plans dataflows in ID order. A new index will always have a greater ID than existing objects, so will never be available during the planning of those objects, regardless of whether or not the cache entries for them are invalidated.
  • Even if the replanning worked as intended, we don't actually want it. Changing the plan means breaking compute reconciliation, which means causing unavailability until the affected dataflows have rehydrated. We do want replanning across version upgrades, but in that case the entire expr cache is cleared anyway, so no special handling is needed.

Because of the first point, this is not expected to change anything about the current behavior. Because of the second point, this is a necessary (though not sufficient) precondition for merging #34859.

Motivation

  • This PR refactors existing code.

Tips for reviewer

Checklist

  • This PR has adequate test coverage / QA involvement has been duly considered. (trigger-ci for additional test/nightly runs)
  • This PR has an associated up-to-date design doc, is a design doc (template), or is sufficiently small to not require a design.
  • If this PR evolves an existing $T ⇔ Proto$T mapping (possibly in a backwards-incompatible way), then it is tagged with a T-proto label.
  • If this PR will require changes to cloud orchestration or tests, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label (example).
  • If this PR includes major user-facing behavior changes, I have pinged the relevant PM to schedule a changelog post.

This commit removes the `invalidate_for_index` logic that was
responsible for invalidating all expr cache entries for objects that
could potentially make use of a newly added index.

There are two reasons to think we don't want this logic:

 * Currently, bootstrapping plans dataflows in ID order. A new index
   will always have a greater ID than existing objects, so will never be
   available during the planning of those objects, regardless of whether
   or not the cache entries for them are invalidated.
 * Even if the replanning worked as intended, we don't actually want it.
   Changing the plan means breaking compute reconciliation, which means
   causing unavailability until the affected dataflows have rehydrated.
   We do want replanning across version upgrades, but in that case the
   entire expr cache is cleared anyway, so no special handling is
   needed.
@teskje teskje force-pushed the remove-invalidate_for_index branch from e9ae259 to f80a5ee Compare January 29, 2026 18:13
@teskje teskje marked this pull request as ready for review January 29, 2026 20:31
@teskje teskje requested a review from a team as a code owner January 29, 2026 20:31
@teskje teskje requested a review from ggevay January 29, 2026 20:31
Copy link
Contributor

@ggevay ggevay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

new_local_expressions,
new_global_expressions,
invalidate_ids,
Default::default(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All calls to update are now passing an empty set to the invalidate_ids parameter, so you could consider removing that parameter, and downstream things. (But it's also ok to leave that to follow-up PRs.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left it in because I want to follow up with a PR that also updates the cache in response to DDL. And I suspect there is a good chance we'll need to pass this argument when removing plans on DROP.

@teskje teskje merged commit 36aef2c into MaterializeInc:main Jan 30, 2026
133 checks passed
@teskje
Copy link
Contributor Author

teskje commented Jan 30, 2026

TFTR!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants