HDDS-14226. [DO NOT MERGE] Get rid of Native Libs and use JNI for SSTFileTableIterator in rocksdb

[swamirishi]

What changes were proposed in this pull request?

Patch : facebook/rocksdb#12511 in rocksdb creates a new JNI for the tableIterator API in SSTFileReader thus eliminating the need for the patched rocksdb native lib written in ozone for efficient snapshot diff. This patch would be responsible for bumping up rocksdb to a version containing the afforementioned rocksdb patch and removing all usages of the native lib from the code base which needs to be done in a single patch since the native lib won't build with the new rocksdb version and making the native lib compatible with newer rocksdb version is a duplication of effort.
This patch will not build currently since this needs a future version of rocksdb with the patch. This code has been tested with a local build of rocksdb containing the patch.

To test this on local follow this step:

• Clone rocksdb code repo and add my remote fork [email protected]:swamirishi/rocksdb.git and checkout to the patch.
  git checkout swamirishi/JniReaderForTableIterator
• Build rocksdb jar.
  make -j8 rocksdbjavastatic
• Install the rocksdb jar in local maven repo.

 mvn install:install-file \
  -Dfile=java/target/rocksdbjni-10.10.0-osx.jar \ 
  -DgroupId=org.rocksdb \
  -DartifactId=rocksdbjni \
  -Dversion=10.10.0 \
  -Dpackaging=jar

• Now the local ozone code should be compilable and all tests can be run.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-14226

How was this patch tested?

Added unit tests for the changed code.

[swamirishi]

@adoroszlai I was wondering if I can do something similar for running CI with newer rocksdb version from a different maven repo. Any idea how I can do this?

[peterxcli]

@swamirishi I tried to the same thing before to build my own rocksdb include the patch to fix the bug in the getPropertiesOfTablesInRange JNI: peterxcli/rocksdb@d9abdf0

• https://github.com/peterxcli/rocksdb/packages/2527529 (maven repo on github, include the build step)
• https://github.com/peterxcli/rocksdb/releases/tag/v7.7.3.2

It was used in my PR: peterxcli#2 (able to pass the ci: https://github.com/peterxcli/ozone/actions/runs/15602888062)

[swamirishi]

Thanks @peterxcli I will take a look at this. I am somehow not able to build rocksdb all compatible fat jar from arm mac-osx. Are there any special flags I need to pass? That is the reason I just built a mac only jar containing only the mac native lib. The build for OSX-x86 is failing when trying to build that. I can maybe just disable OSX-x86 build and build the rest from docker.

[swamirishi]

@rnblough This is the patch for upgrading our rocksdb version

[smengcl]

TODO: Remove this

[smengcl]

TODO: Remove this