libmigraphx_c unconditionally loads the TensorFlow parser and causes duplicate protobuf descriptor registration

[ibuki2003]

Description

Loading the ONNX Runtime MIGraphX Execution Provider in a process that
already contains OpenCV DNN aborts during dlopen():

File already exists in database: tensor_shape.proto
Check failed: GeneratedDatabase()->Add(encoded_file_descriptor, size)

The crash occurs before model execution, while loading the MIGraphX
provider.

Environment

• MIGraphX: 7.2.3
• ROCm: 7.2.3
• ONNX Runtime: reproduced with 1.26.0 and 1.27.1
• protobuf: 35.1
• OpenCV: 4.13.0
• Application: darktable 5.6.0 on NixOS
• Provider: MIGraphX
• CPU provider works correctly

Stack trace

google::protobuf::DescriptorPool::InternalAddGeneratedFile()
google::protobuf::internal::AddDescriptors()
call_init
_dl_init
dlopen
onnxruntime::PosixEnv::LoadDynamicLibrary()
onnxruntime::ProviderLibrary::Load()
OrtSessionOptionsAppendExecutionProvider_MIGraphX

Analysis

darktable loads G'MIC, which brings OpenCV DNN into the process.
OpenCV DNN registers TensorFlow-related protobuf descriptors.

MIGraphX's stable C API library links both parsers unconditionally:

https://github.com/ROCm/AMDMIGraphX/blob/rocm-7.2.3/src/api/CMakeLists.txt#L40

target_link_libraries(migraphx_c PRIVATE migraphx migraphx_tf migraphx_onnx)

Consequently, loading libmigraphx_c.so.3 for ONNX inference also loads
libmigraphx_tf, although the TensorFlow parser is not used. This causes
tensor_shape.proto to be registered for a second time and protobuf
aborts.

Removing the TensorFlow API implementation from src/api/api.cpp and
removing migraphx_tf from the migraphx_c link dependencies fixes the
crash. The ONNX Runtime MIGraphX provider then loads successfully after
OpenCV DNN.

Disabling G'MIC/OpenCV in darktable also avoids the crash, but this is an
application-level workaround.

Expected behavior

Using the MIGraphX C API for ONNX inference should not load the TensorFlow
parser or register its protobuf descriptors.

Would it be possible to split the TensorFlow parser API from
libmigraphx_c, or otherwise make parser dependencies optional?

Minimal reproduction

The crash can be reproduced without darktable or ONNX Runtime by loading
OpenCV DNN followed by the unmodified MIGraphX C API:

import ctypes

ctypes.CDLL(
    "/path/to/libopencv_dnn.so",
    mode=ctypes.RTLD_GLOBAL,
)
ctypes.CDLL("/path/to/libmigraphx_c.so.3")

Result:

File already exists in database: tensor_shape.proto
Check failed: GeneratedDatabase()->Add(encoded_file_descriptor, size)
Aborted (exit status 134)

Tested with:

• OpenCV 4.13.0
• MIGraphX 7.2.3
• protobuf 35.1

After removing migraphx_tf from the libmigraphx_c link dependencies
and removing the TensorFlow C API implementation from src/api/api.cpp,
the same reproducer succeeds with exit status 0.

[ambiso]

specifically for nixos I was able to work around this with

    (darktable.override {
      withAi = true;
      onnxruntime = onnxruntime.override { rocmSupport = true; };
      gmic = gmic.overrideAttrs (old: {
        buildInputs = builtins.filter (dependency: dependency != opencv) (old.buildInputs or [ ]);

        cmakeFlags =
          builtins.filter (flag: !(lib.hasPrefix "-DENABLE_OPENCV=" flag)) (old.cmakeFlags or [ ])
          ++ [
            (lib.cmakeBool "ENABLE_OPENCV" false)
          ];
      });
    })

since darktable doesn't appear to use any of the opencv functionality of gmic

note that compiling the kernels on first use takes a long time and uses all cores for that time.