Bump version to 6.2.0, resolve integration test flakiness#219
Merged
austin-denoble merged 12 commits intomainfrom Apr 1, 2026
Merged
Bump version to 6.2.0, resolve integration test flakiness#219austin-denoble merged 12 commits intomainfrom
6.2.0, resolve integration test flakiness#219austin-denoble merged 12 commits intomainfrom
Conversation
…me of the integration test flakiness: remove unnecessary sleeps and poll instead, fix error handling in retry mechanism, isolate namespaces across tests, use unique prefixes in namespace tests
src/integration/java/io/pinecone/integration/dataPlane/NamespacesTest.java
Show resolved
Hide resolved
…ll ListEndpointTests, use Isolated for ConfigureIndexTest, fix waitUntilIndexStateIsReady always waiting for the full amount of time in some cases, clean up ReadCapacityAndSchemaTest
...egration/java/io/pinecone/integration/controlPlane/serverless/ReadCapacityAndSchemaTest.java
Show resolved
Hide resolved
…lessAsync, pod, and podAsync index connections, synchronize all the getOrCreate*Connection() methods, cleanupResources now closes all connections before deleting indexes, removed AfterAll -> cleanUp from individual test files
...egration/java/io/pinecone/integration/dataPlane/ResponseMetadataListenerIntegrationTest.java
Show resolved
Hide resolved
src/integration/java/io/pinecone/integration/dataPlane/UpsertAndSearchRecordsTest.java
Outdated
Show resolved
Hide resolved
…ut going through the shared static Pinecone.connectionsMap
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Autofix Details
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: Java
assertsilently replaces JUnit assertions for cleanup verification- Replaced Java assert statements with JUnit assertFalse() to ensure cleanup verification always executes regardless of JVM flags.
Or push these changes by commenting:
@cursor push f85faa77e1
Preview (f85faa77e1)
diff --git a/src/integration/java/io/pinecone/clients/ConnectionsMapTest.java b/src/integration/java/io/pinecone/clients/ConnectionsMapTest.java
--- a/src/integration/java/io/pinecone/clients/ConnectionsMapTest.java
+++ b/src/integration/java/io/pinecone/clients/ConnectionsMapTest.java
@@ -14,6 +14,7 @@
import static io.pinecone.helpers.TestUtilities.waitUntilIndexIsReady;
import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertThrows;
public class ConnectionsMapTest {
@@ -114,8 +115,8 @@
index1_2.close();
// Verify the specific entries for this test's indexes were removed
- assert !connectionsMap.containsKey(indexName1);
- assert !connectionsMap.containsKey(indexName2);
+ assertFalse(connectionsMap.containsKey(indexName1));
+ assertFalse(connectionsMap.containsKey(indexName2));
// Delete the indexes
pinecone1.deleteIndex(indexName1);This Bugbot Autofix run was free. To enable autofix for future PRs, go to the Cursor dashboard.
src/integration/java/io/pinecone/clients/ConnectionsMapTest.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
src/integration/java/io/pinecone/clients/ConnectionsMapTest.java
Outdated
Show resolved
Hide resolved
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Problem
We need to bump version packaging and constants for releasing v6.2.0. There's also a significant amount of integration test flakiness to address, and the suite was taking ~20 minutes to run.
Solution
Version bump
Bump version to 6.2.0 in
gradle.propertiesandCHANGELOG.md.NOTE: All of the following changes are within the integration testing harness, so none of the client code is actually touched. I spent a bit of time working with claude to tighten up the testing harness itself. Test execution went from 20-24mins per run to 7-8mins, it looks like. Primarily, this was achieved through parallelizing the test classes since a lot of them are reusing the TestResourcesManagerSingle already. The singleton class also needed a lot of work to make it a bit more robust when being accessed by parallel testing threads. I summarized most of this with claude since it was done across numerous commits.
Parallel test execution
Added
junit-platform.propertiesenabling concurrent class-level execution at parallelism 4, with methods within each class remaining sequential. This alone brought the suite runtime from ~20 minutes to ~8 minutes.AssertRetry — exception handling gap
The retry helper only caught
AssertionError,ExecutionException, andIOException. Any other exception (most commonlyNullPointerExceptionwhen a namespace key is absent from the stats map during eventual consistency) escaped the retry loop immediately. Widened the catch and theAssertionRunnableinterface toException. Also addedexplicit
InterruptedExceptionre-throw before the broad catch so that cancellation signals propagate correctly rather than being silently retried.TestResourcesManager — thread safety
Enabling parallel class execution exposed multiple races in the shared singleton:
getInstance()was unsynchronized; two classes could both seenulland construct duplicate instances. Fixed withvolatile+synchronized.getOrCreatePodIndex(),getOrCreateServerlessIndex(), andgetOrCreateCollection()had check-then-act races where two threads could both pass the null guard and create duplicate indexes. All markedsynchronized.getOrCreate*Connection()methods now cache their results and aresynchronized, ensuring parallel classes that both callgetOrCreateServerlessIndexConnection()share the sameIndexobject rather than creating duplicate channels.cleanupResources()now closes all cached connections before deleting indexes.!= "ready") in the collection-readiness loop that caused it to always run to the full 120-second timeout. Fixed to!"ready".equals(...).Parallel tests closing shared connections early
Index.close()shuts down the underlyingPineconeConnection, which is shared via astatic final ConcurrentHashMapkeyed by index name across allPineconeinstances. With parallel class execution, one class callingclose()in@AfterAllcould terminate the channel while another class was mid-request. Removedclose()calls from@AfterAllin all test classes that obtained connections fromTestResourcesManager. Connections are now closed exclusively inTestResourcesManager.cleanupResources()at the end of the run.ResponseMetadata tests — static connection map race
The
ResponseMetadata*tests create their ownPineconeclient with a listener interceptor, butgetIndexConnection()usescomputeIfAbsenton the shared static map — whichever class first creates a connection for the shared index name wins. If another class won the race, theResponseMetadata*tests got a listener-less connection andcaptured nothing. Fixed by constructing
PineconeConnectiondirectly (bypassingcomputeIfAbsent) using a host exposed viaTestResourcesManager.getOrCreateServerlessIndexHost(). EachResponseMetadata*test now owns its connection and closes it in its own@AfterAll.UpsertDescribeIndexStatsAndDeletePodTest / ...ServerlessTest — shared mutable counter
A
static namespaceVectorCountfield was shared and mutated across all test methods. Each test now creates its own isolated namespace and tracks its own local count, with null-safe assertions for the case where a deleted namespace is removed from the stats map.NamespacesTest — cross-test namespace count interference
Both sync and async tests asserted on exact total namespace counts against the same shared index. Each test now uses a unique random prefix and counts only its own namespaces via prefix-filtered
listNamespaces, making assertions fully isolated.Thread.sleepreplaced withassertWithRetry. Async upserts now properly await with.get().ListEndpointTest — concurrent write interference
Count assertions against the shared serverless index were vulnerable to concurrent writes from parallel classes inflating the results. All assertions now use prefix-filtered
list()calls that only count vectors seeded byTestResourcesManagerunder known prefixes.ReadCapacityAndSchemaTest — unnecessary waits + missing cleanup
Removed
waitUntilIndexIsReadyfrom tests 1–5 (assertions are on the create response, not index state). Added@AfterAllthat deletes all indexes created by the class, with logged warnings on deletion failure rather than silent swallowing.ConfigureIndexTest — parallel isolation
Added
@Isolatedsince this test mutates a shared pod index's replica count and pod type, which cannot safely run concurrently with any other class. Also fixed a string identity comparison (!= "ready") in the local readiness poll.ConnectionsMapTest — size assertions replaced
assertEquals(size, 1)/assertEquals(size, 2)assumed the static map contained only this test's entries, which breaks under parallel execution when other classes have their own connections in the map. Replaced with key-presence checks (assertFalse(containsKey(...))) and object-identity checks (assertSame) to verify connection reuse withoutcounting map entries. Also replaced bare
assertkeywords (silently skipped without-ea) withassertFalse/assertSame. Fixed a pre-existing typo whereconfig1.setHost(host2)was overwriting the wrong config.UpsertAndSearchRecordsTest — cleanup masking failures
createIndexForModelwas called before thetryblock, so if it failed thefinallyclause would calldeleteIndexon a non-existent index, replacing the original exception with a not-found error. Moved creation insidetrywith anindexCreatedflag sodeleteIndexonly runs when there is actually an index to clean up. Also replacedhardcoded sleeps with
waitUntilIndexIsReadyandassertWithRetry.