query: introduce GC delay for gRPC conns

[GiedriusS]

See query/endpointset.go for rationale. We currently immediately kill
gRPC connections on Close() but they might still be used. :/ Wait until
the query timeout duration before closing them.

Signed-off-by: Giedrius Statkevičius [email protected]

[Copilot]

Pull request overview

This pull request introduces a garbage collection delay mechanism for gRPC connections to prevent premature closure while connections might still be in use. The implementation adds a queryTimeout parameter to the EndpointSet constructor and delays connection closure by using the maximum of queryTimeout, endpointInfoTimeout, and unhealthyEndpointTimeout.

Key changes:

• Modified EndpointSet to track a gcTimeout calculated as the maximum of query, endpoint info, and unhealthy endpoint timeouts
• Updated endpointRef.Close() to accept a gcDelay parameter and defer actual connection closure using time.AfterFunc
• Added goleak ignore rules for gRPC connection goroutines that are intentionally delayed

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
pkg/query/endpointset.go	Added gcTimeout field to EndpointSet, modified Close() signature to accept delay parameter, implemented delayed closure using time.AfterFunc
pkg/query/endpointset_test.go	Updated test helper calls with new queryTimeout parameter, added TestEndpointCloseGCTime to verify GC delay behavior
cmd/thanos/query.go	Passed queryTimeout parameter to setupEndpointSet
cmd/thanos/rule.go	Passed evalInterval as query timeout to setupEndpointSet
cmd/thanos/endpointset.go	Added queryTimeout parameter to setupEndpointSet and passed it to NewEndpointSet
pkg/testutil/custom/custom.go	Added goleak ignore rules for gRPC connection goroutines that are not immediately reaped

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[fpetkovski]

Looks like unit test failure is related to this change

[GiedriusS]

Added back synctest.Wait() so that the test would properly wait until the other goroutine closes the gRPC connection. I cannot reproduce the failure anymore even with -count=50.

[fpetkovski]

🚀