[flink] Fix ClassNotFoundException for RoaringBitmap in shaded Flink connector JAR

[pbanakar]

Purpose

Linked issue: close #3980

Fix ClassNotFoundException: org.roaringbitmap.RoaringBitmap at runtime when using FlussCatalog bitmap functions introduced in FIP-37 (rb_build_agg, rb_cardinality, rb_or_agg).

Brief change log

Root cause: RoaringBitmap is declared as compile scope in fluss-flink-common/pom.xml, but Maven's dependency mediation resolves it as test scope in the version-specific Flink connector modules. This happens because fluss-flink-common:test-jar is a test-scoped dependency in those modules, and its transitive RoaringBitmap dependency overrides the compile scope from the main artifact. As a result, the maven-shade-plugin excludes org/roaringbitmap/* classes from the final shaded connector JAR.

Confirmed by: ./mvnw dependency:tree -pl fluss-flink/fluss-flink-1.20 | grep roaring
→ org.roaringbitmap:RoaringBitmap:jar:1.3.0:test ← wrong scope

jar tf fluss-flink-1.20-*.jar | grep "org/roaringbitmap" → (empty) ← classes missing from JAR

Fix: Add an explicit compile-scope RoaringBitmap dependency in fluss-flink-1.18, fluss-flink-1.19, fluss-flink-1.20, and fluss-flink-2.2 pom.xml files to override the test scope resolution.

After fix: jar tf fluss-flink-1.20-*.jar | grep "org/roaringbitmap" → org/roaringbitmap/RoaringBitmap.class (and all other classes present)

Tests

Verified end-to-end locally on Flink 1.20:

• Built apache/fluss-quickstart-flink from main branch with fix applied
• Ran full user profile quickstart using rb_build_agg and rb_cardinality
• Confirmed correct results — unique_visitor_count and total_clicks accumulating in real time with no errors

API and Format

No API or storage format changes. This is a packaging fix only — 4 pom.xml files modified.

Documentation

No documentation changes. This fix unblocks the Real-Time User Profile quickstart tutorial (PR #2669 ) which demonstrates these functions.

[platinumhamburg]

@pbanakar Thanks for investigating and fixing this runtime packaging issue.

At first glance, adding a third-party dependency to the shade configuration of all four version-specific connector modules looks unusual, since these modules previously only assembled Fluss artifacts. After reviewing the dependency and packaging structure again, however, I think handling RoaringBitmap at the final connector level is reasonable in this case.

The version-specific modules exclude all transitive dependencies of fluss-flink-common, so RoaringBitmap must be declared explicitly before it can be included in the final connector JAR. Moving the shade step into fluss-flink-common would instead change that library's published artifact and create inconsistencies between its main JAR, test JAR, sources, and published POM. Introducing a separate shaded module also seems unnecessarily heavy for this fix.

There are two remaining issues:

1. Please relocate org.roaringbitmap to org.apache.fluss.shaded.org.roaringbitmap, consistent with the server-side packaging, to avoid potential dependency conflicts.

2. Please add the corresponding NOTICE file to each version-specific connector module. This should also resolve the current License Check failures.

Since this is the first time these version-specific connector modules explicitly include a third-party artifact in their own shade configuration, it would be helpful to get another opinion on this dependency boundary. @polyzos @wuchong , could you please take a look as well?