Skip to content

Bump version to 6.2.0, resolve integration test flakiness#219

Merged
austin-denoble merged 12 commits intomainfrom
adenoble/bump-version-6.2.0
Apr 1, 2026
Merged

Bump version to 6.2.0, resolve integration test flakiness#219
austin-denoble merged 12 commits intomainfrom
adenoble/bump-version-6.2.0

Conversation

@austin-denoble
Copy link
Copy Markdown
Contributor

@austin-denoble austin-denoble commented Mar 31, 2026

Problem

We need to bump version packaging and constants for releasing v6.2.0. There's also a significant amount of integration test flakiness to address, and the suite was taking ~20 minutes to run.

Solution

Version bump

Bump version to 6.2.0 in gradle.properties and CHANGELOG.md.

NOTE: All of the following changes are within the integration testing harness, so none of the client code is actually touched. I spent a bit of time working with claude to tighten up the testing harness itself. Test execution went from 20-24mins per run to 7-8mins, it looks like. Primarily, this was achieved through parallelizing the test classes since a lot of them are reusing the TestResourcesManagerSingle already. The singleton class also needed a lot of work to make it a bit more robust when being accessed by parallel testing threads. I summarized most of this with claude since it was done across numerous commits.

Parallel test execution

Added junit-platform.properties enabling concurrent class-level execution at parallelism 4, with methods within each class remaining sequential. This alone brought the suite runtime from ~20 minutes to ~8 minutes.

AssertRetry — exception handling gap

The retry helper only caught AssertionError, ExecutionException, and IOException. Any other exception (most commonly NullPointerException when a namespace key is absent from the stats map during eventual consistency) escaped the retry loop immediately. Widened the catch and the AssertionRunnable interface to Exception. Also added
explicit InterruptedException re-throw before the broad catch so that cancellation signals propagate correctly rather than being silently retried.

TestResourcesManager — thread safety

Enabling parallel class execution exposed multiple races in the shared singleton:

  • getInstance() was unsynchronized; two classes could both see null and construct duplicate instances. Fixed with volatile + synchronized.
  • getOrCreatePodIndex(), getOrCreateServerlessIndex(), and getOrCreateCollection() had check-then-act races where two threads could both pass the null guard and create duplicate indexes. All marked synchronized.
  • getOrCreate*Connection() methods now cache their results and are synchronized, ensuring parallel classes that both call getOrCreateServerlessIndexConnection() share the same Index object rather than creating duplicate channels.
  • cleanupResources() now closes all cached connections before deleting indexes.
  • Fixed a string identity comparison bug (!= "ready") in the collection-readiness loop that caused it to always run to the full 120-second timeout. Fixed to !"ready".equals(...).

Parallel tests closing shared connections early

Index.close() shuts down the underlying PineconeConnection, which is shared via a static final ConcurrentHashMap keyed by index name across all Pinecone instances. With parallel class execution, one class calling close() in @AfterAll could terminate the channel while another class was mid-request. Removed close() calls from
@AfterAll in all test classes that obtained connections from TestResourcesManager. Connections are now closed exclusively in TestResourcesManager.cleanupResources() at the end of the run.

ResponseMetadata tests — static connection map race

The ResponseMetadata* tests create their own Pinecone client with a listener interceptor, but getIndexConnection() uses computeIfAbsent on the shared static map — whichever class first creates a connection for the shared index name wins. If another class won the race, the ResponseMetadata* tests got a listener-less connection and
captured nothing. Fixed by constructing PineconeConnection directly (bypassing computeIfAbsent) using a host exposed via TestResourcesManager.getOrCreateServerlessIndexHost(). Each ResponseMetadata* test now owns its connection and closes it in its own @AfterAll.

UpsertDescribeIndexStatsAndDeletePodTest / ...ServerlessTest — shared mutable counter

A static namespaceVectorCount field was shared and mutated across all test methods. Each test now creates its own isolated namespace and tracks its own local count, with null-safe assertions for the case where a deleted namespace is removed from the stats map.

NamespacesTest — cross-test namespace count interference

Both sync and async tests asserted on exact total namespace counts against the same shared index. Each test now uses a unique random prefix and counts only its own namespaces via prefix-filtered listNamespaces, making assertions fully isolated. Thread.sleep replaced with assertWithRetry. Async upserts now properly await with .get().

ListEndpointTest — concurrent write interference

Count assertions against the shared serverless index were vulnerable to concurrent writes from parallel classes inflating the results. All assertions now use prefix-filtered list() calls that only count vectors seeded by TestResourcesManager under known prefixes.

ReadCapacityAndSchemaTest — unnecessary waits + missing cleanup

Removed waitUntilIndexIsReady from tests 1–5 (assertions are on the create response, not index state). Added @AfterAll that deletes all indexes created by the class, with logged warnings on deletion failure rather than silent swallowing.

ConfigureIndexTest — parallel isolation

Added @Isolated since this test mutates a shared pod index's replica count and pod type, which cannot safely run concurrently with any other class. Also fixed a string identity comparison (!= "ready") in the local readiness poll.

ConnectionsMapTest — size assertions replaced

assertEquals(size, 1) / assertEquals(size, 2) assumed the static map contained only this test's entries, which breaks under parallel execution when other classes have their own connections in the map. Replaced with key-presence checks (assertFalse(containsKey(...))) and object-identity checks (assertSame) to verify connection reuse without
counting map entries. Also replaced bare assert keywords (silently skipped without -ea) with assertFalse/assertSame. Fixed a pre-existing typo where config1.setHost(host2) was overwriting the wrong config.

UpsertAndSearchRecordsTest — cleanup masking failures

createIndexForModel was called before the try block, so if it failed the finally clause would call deleteIndex on a non-existent index, replacing the original exception with a not-found error. Moved creation inside try with an indexCreated flag so deleteIndex only runs when there is actually an index to clean up. Also replaced
hardcoded sleeps with waitUntilIndexIsReady and assertWithRetry.

…me of the integration test flakiness: remove unnecessary sleeps and poll instead, fix error handling in retry mechanism, isolate namespaces across tests, use unique prefixes in namespace tests
…ll ListEndpointTests, use Isolated for ConfigureIndexTest, fix waitUntilIndexStateIsReady always waiting for the full amount of time in some cases, clean up ReadCapacityAndSchemaTest
…lessAsync, pod, and podAsync index connections, synchronize all the getOrCreate*Connection() methods, cleanupResources now closes all connections before deleting indexes, removed AfterAll -> cleanUp from individual test files
Copy link
Copy Markdown

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Autofix Details

Bugbot Autofix prepared a fix for the issue found in the latest run.

  • ✅ Fixed: Java assert silently replaces JUnit assertions for cleanup verification
    • Replaced Java assert statements with JUnit assertFalse() to ensure cleanup verification always executes regardless of JVM flags.

Create PR

Or push these changes by commenting:

@cursor push f85faa77e1
Preview (f85faa77e1)
diff --git a/src/integration/java/io/pinecone/clients/ConnectionsMapTest.java b/src/integration/java/io/pinecone/clients/ConnectionsMapTest.java
--- a/src/integration/java/io/pinecone/clients/ConnectionsMapTest.java
+++ b/src/integration/java/io/pinecone/clients/ConnectionsMapTest.java
@@ -14,6 +14,7 @@
 
 import static io.pinecone.helpers.TestUtilities.waitUntilIndexIsReady;
 import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
 import static org.junit.jupiter.api.Assertions.assertThrows;
 
 public class ConnectionsMapTest {
@@ -114,8 +115,8 @@
         index1_2.close();
 
         // Verify the specific entries for this test's indexes were removed
-        assert !connectionsMap.containsKey(indexName1);
-        assert !connectionsMap.containsKey(indexName2);
+        assertFalse(connectionsMap.containsKey(indexName1));
+        assertFalse(connectionsMap.containsKey(indexName2));
 
         // Delete the indexes
         pinecone1.deleteIndex(indexName1);

This Bugbot Autofix run was free. To enable autofix for future PRs, go to the Cursor dashboard.

Copy link
Copy Markdown

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

@austin-denoble austin-denoble merged commit 6072a1d into main Apr 1, 2026
13 checks passed
@austin-denoble austin-denoble deleted the adenoble/bump-version-6.2.0 branch April 1, 2026 00:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant