Resolve model package paths and path-valued session options through ONNX Runtime#2255
Draft
jambayk wants to merge 4 commits into
Draft
Resolve model package paths and path-valued session options through ONNX Runtime#2255jambayk wants to merge 4 commits into
jambayk wants to merge 4 commits into
Conversation
Model package path resolution is delegated to ORT's ModelPackage_ResolveStringRef. Config gains a package_resolver that captures the OrtModelPackageContext and routes path-shaped genai_config.json values through it, so a sha256: tokenizer_dir resolves to a content-addressed shared asset (honoring manifest overrides) and relative paths resolve against the variant directory. OpenAndSelectVariant returns the context to keep it alive for the config's lifetime. Test fixtures declare a single inline component with variant directories at the package root, and the end-to-end tokenizer test references its tokenizer through a sha256: shared asset. The docs describe the inline-component layout and the sha256: scheme. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ion 1.0 The Ort::Experimental typed getters for the OrtModelPackageApi functions are generated in this header from the experimental .inc using OrtApi::GetExperimentalFunction, instead of relying on ONNX Runtime's onnxruntime_experimental_cxx_api.h, which pulls in onnxruntime_cxx_api.h that genai does not vendor. Model package test fixtures author schema_version as the "1.0" major.minor string. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Route the external initializers folder and ep.context_file_path session options through Config::ResolvePath when applying config entries, so a variant's genai_config can reference weights or the EP context file by sha256: shared-asset URI or relative path. For memory-loaded models, only default the external initializers folder to the config directory when the config did not already set it. ResolvePath now rejects sha256: references outside a package instead of producing a bogus path. Add HasConfigEntry to the session-options wrapper and tests covering external-data resolution and the sha256 rejection. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR extends the model-package loader so that all path references originating from a package variant’s genai_config.json (including sha256: shared-asset URIs and relative paths) are resolved via ONNX Runtime’s model package resolver, and additionally resolves path-valued session options before applying them to ORT.
Changes:
- Delegate
genai_config.jsonpath resolution to ORT’sModelPackage_ResolveStringRefwhen loading from a package, and rejectsha256:references when loading from a flat directory. - Resolve path-valued session option entries (notably
session.model_external_initializers_file_folder_pathandep.context_file_path) viaConfig::ResolvePathbefore applying them. - Update tests + docs for the inline-component package layout and
sha256:shared assets.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| test/model_package_test.cpp | Updates package fixture authoring and adds E2E tests for sha256: tokenizer + external initializer folder resolution. |
| src/models/onnxruntime_inline.h | Adds HasConfigEntry wrapper and resolves model package experimental function pointers by name (incl. ResolveStringRef). |
| src/models/onnxruntime_api.h | Exposes HasConfigEntry and extends the model package API wrapper with ModelPackage_ResolveStringRef. |
| src/models/model.cpp | Resolves path-valued session options and adjusts external-initializers defaulting for in-memory loads; wires package resolver into Config. |
| src/models/model_package.h | Keeps the package context alive in the load result for deferred path resolution. |
| src/models/model_package.cpp | Stores the opened package context in the load result. |
| src/config.h | Adds package_resolver callback used when loading from a package. |
| src/config.cpp | Reworks ResolvePath to delegate to ORT when in a package; rejects sha256: in flat directories. |
| docs/model_package.md | Updates documentation to describe inline components and sha256: shared assets. |
Comments suppressed due to low confidence (1)
src/models/onnxruntime_inline.h:1575
Ort::GetModelPackageApi()only checksCreateModelPackageContextfor availability. If an ONNX Runtime build has the model package API but lacks the newerModelPackage_ResolveStringRefentry, later calls throughResolveStringRef()will dereference a null function pointer and crash. Guard forModelPackage_ResolveStringRef(and ideally any other required entries) and fail with a clear error early.
if (fns.CreateModelPackageContext == nullptr) {
throw std::runtime_error(
"Model package API is not available in this ONNX Runtime build. "
"Model packages require ONNX Runtime with the OrtModelPackageApi experimental functions.");
}
Comment on lines
+107
to
+123
| fs_std::path ExportModelWithExternalData(const fs_std::path& src_onnx, const fs_std::path& out_dir) { | ||
| // The low-level ORT wrappers below run in this test binary, which has its own copy of the | ||
| // Ort::api pointer (the genai .so initializes its own). Initialize ours before using them. | ||
| Ort::InitApi(); | ||
| fs_std::create_directories(out_dir); | ||
| const auto model_path = out_dir / "model.onnx"; | ||
| auto session_options = OrtSessionOptions::Create(); | ||
| session_options->SetGraphOptimizationLevel(ORT_DISABLE_ALL); | ||
| session_options->SetOptimizedModelFilePath(model_path.c_str()); | ||
| session_options->AddConfigEntry("session.optimized_model_external_initializers_file_name", | ||
| "model.onnx.data"); | ||
| session_options->AddConfigEntry("session.optimized_model_external_initializers_min_size_in_bytes", | ||
| "0"); | ||
| // Creating the session writes the optimized model + external data as a side effect. | ||
| auto session = OrtSession::Create(Generators::GetOrtEnv(), src_onnx.c_str(), session_options.get()); | ||
| return model_path; | ||
| } |
| const char* values[] = {"false", "true"}; | ||
|
|
||
| // Resolve tokenizer_dir (may be empty, relative, absolute, or "package:"-scheme). | ||
| // Resolve tokenizer_dir (may be empty, relative, or a "sha256:" shared-asset reference). |
| ```jsonc | ||
| // manifest.json | ||
| { | ||
| "schema_version": 1, |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Builds on model package loading (#2227) so onnxruntime-genai resolves every path reference in a variant's
genai_config.jsonthrough ONNX Runtime's model package resolver. A variant can reference its weights, an EP context file, and shared assets (such as tokenizer or processor config) bysha256:shared-asset URI or relative path, and ONNX Runtime owns shared-asset resolution and path confinement.What changed
genai_config.jsonpath resolution to ORT'sModelPackage_ResolveStringRefwhen the model is loaded from a package, sosha256:shared-asset references and relative paths resolve consistently with the package rules.Config::ResolvePathnow rejects asha256:reference when the model is a plain directory, with a clear error, instead of producing a bogus path.session.model_external_initializers_file_folder_path) and the EP context file path (ep.context_file_path) declared in a variant'ssession_optionsare resolved before being applied, so a variant can point at weights or a context file stored in a shared asset. For models loaded from memory, the external initializers folder defaults to the config directory only when the config did not already set it.OrtModelPackageApiexperimental functions self-contained inGetModelPackageApi(), so genai does not vendor ONNX Runtime's experimental C++ header.schema_version"1.0".Dependencies
This requires building against an ONNX Runtime that includes:
Testing
unit_tests --gtest_filter='ModelPackage.*'(CPU, built against an ONNX Runtime with the dependencies above): 10 tests pass, covering variant selection, tokenizer resolution through asha256:shared asset, external-initializers folder resolution through a shared asset, and rejection ofsha256:references in a plain directory.