Skip to content

Support updating PCGroup filters dynamically#45

Open
colprog wants to merge 1 commit into
synadia-io:mainfrom
colprog:DyanmicFilter
Open

Support updating PCGroup filters dynamically#45
colprog wants to merge 1 commit into
synadia-io:mainfrom
colprog:DyanmicFilter

Conversation

@colprog
Copy link
Copy Markdown
Contributor

@colprog colprog commented Mar 27, 2026

This PR adds runtime partitioning-filter management for Elastic PCGroups and fixes NatsPcgPartitioningFilter value semantics.

Changes
Added:
AddPcgElasticFiltersAsync(...)
DeletePcgElasticFiltersAsync(...)
NatsPcgPartitioningFilter now compares by value (Filter + PartitioningWildcards) via custom Equals/GetHashCode.

@mtmk
Copy link
Copy Markdown
Collaborator

mtmk commented Mar 27, 2026

thanks @colprog for sorting out git

cc @jnmoyne, this adds runtime filter add/delete for elastic PCGroups which isn't in orbit.go yet. Worth a look before we go ahead with it.

@mtmk
Copy link
Copy Markdown
Collaborator

mtmk commented Mar 27, 2026

@jnmoyne
Copy link
Copy Markdown
Collaborator

jnmoyne commented Apr 20, 2026

Hi and thanks for the contribution, however at this point I'm not convinced this would be a good idea to merge. It certainly wasn't the initial intention to be able to modify the filters of an existing consumer group.

While on the face of it I don't see an issue with adding a filter, there's a fundamental issue with orphaned messages on filter delete. Messages already sourced (under a now-removed filter) stay in the work-queue indefinitely.
It is not cross-compatible with the Go and Java versions which see any change in the config of an existing consumer group as an error and all quit the group when it happens.
Also note it should update the KV first and then update the stream config (it seems it does it the other way around).

If I may ask what is the use case? Can it not be done with for example deleting and re-creating the consumer group?

@colprog
Copy link
Copy Markdown
Contributor Author

colprog commented Apr 20, 2026

Thanks for your reply.
I am working on message broker based on nats. the server would forward messages from nats to other systems based on dynamic generated rules. This commit is to support our last iteration where when we update filter in response to incoming requests. We have now moved to using > as filter and adding/removing subjects from stream instead. this seems cleaner and avoids the orphaned message issue.
if filters are considered static by design. please feel free to close this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants