Adding compilation mode by pnikolic-amd · Pull Request #4770 · ROCm/AMDMIGraphX

pnikolic-amd · 2026-04-09T19:50:16Z

Motivation

Adding compilation mode which defines factor between compilation time and quality of model optimization. This is defined by compile option "compile_mode" which is defined by int value 0-100 (0 - fast, minimal/no optimization, 100 slow, best optimization quality).

Technical Details

compile_mode is represented by enum class compile_modes, and int value is mapped in the closest value. There are three modes defined for now (more fine-grained system can be implemented later):

BALANCED (50) - Default execution of model compilation
MAX (100) - Default execution with exhaustive_tune flag enabled
EAGER (0) - Removed most of gpu independent passes except fusions and normalize_ops and skiped benchmarking for gpu kernels

Tested on both platforms, Eager mode improves compilation time up to 10x while decreasing inference performance for 20-50%.

Changelog Category

Add a CHANGELOG.md entry for any option other than Not Applicable

- Added: New enum class compile_modes that define compile options
- Added: New param compile_mode in compile_options struct
- Added: Added support to set compile mode using all supported APIs and added flag for migraphx-driver app.
- Changed: Changed get passes function for gpu target, to support execution in defined modes.
- TODO: Add support to skip fusions in eager mode
- TODO: Re-run mgx tests in eager mode

This reverts commit 0160c2b.

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

…ming and cppcheck warnings

gh-app-migraphx-bot-pr-write · 2026-06-11T13:48:50Z

Test	Status	Result
bert-mrpc-onnx	✅	PASSED: MIGraphX meets tolerance
bert-mrpc-tf	✅	PASSED: MIGraphX meets tolerance
pytorch-examples-wlang-gru	✅	PASSED: MIGraphX meets tolerance
pytorch-examples-wlang-lstm	✅	PASSED: MIGraphX meets tolerance
dlrm-criteoterabyte	✅	PASSED: MIGraphX meets tolerance
agentmodel	✅	PASSED: MIGraphX meets tolerance
unet	✅	PASSED: MIGraphX meets tolerance
resnet50v1	✅	PASSED: MIGraphX meets tolerance
bert_base_cased_fp16	✅	PASSED: MIGraphX meets tolerance
bert_large_uncased_fp16	🔴	FAILED: MIGraphX is not within tolerance - check verbose output
bert_large	✅	PASSED: MIGraphX meets tolerance
yolov5s	✅	PASSED: MIGraphX meets tolerance
tinyllama	✅	PASSED: MIGraphX meets tolerance
vicuna-fastchat	✅	PASSED: MIGraphX meets tolerance
whisper-tiny-encoder	✅	PASSED: MIGraphX meets tolerance
whisper-tiny-decoder	✅	PASSED: MIGraphX meets tolerance
distilgpt2_fp16	✅	PASSED: MIGraphX meets tolerance
llama2_7b	✅	PASSED: MIGraphX meets tolerance
qwen1.5-7b	✅	PASSED: MIGraphX meets tolerance
phi3-3.8b	✅	PASSED: MIGraphX meets tolerance
llama3-8b	✅	PASSED: MIGraphX meets tolerance
whisper-large-decoder	✅	PASSED: MIGraphX meets tolerance
mistral-7b	✅	PASSED: MIGraphX meets tolerance
FLUX.1-schnell	✅	PASSED: MIGraphX meets tolerance

pfultz2 · 2026-06-11T19:02:55Z

+        pipelines = {
+            p.dynamic_shapes_pipeline(),
+            p.required_pipeline(),
+            {optimize_module{},


I dont think we want to run optimize_module in eager mode.

If optimize_module is not included logsoftmax test fails. If we return it than we lose time improvements from eager mode, so I want to exclude it.
I didn't see this problem when testing with models, so can we skip this test right now, since you mentioned it will be removed.

I see the verification fails because it runs log(softmax(x)) without simplify_algebra. We could update rewrite_reduce to generate the logsoftmax correctly so simplify_algebra is not needed.

Because this requires additional work I want to leave it for next iterations, I have reverted all changes regarding removing passes, that will be handled in future tasks. Right now, in eager mode we have kernel benchmarking removed.

Adding only simplify_algebra does not solve failing for those tests, in order to add rewrite_reduce and rewrite_topk, optimize_module also needs to be added in order for tests to pass.

This has optimize_module before rewrite. But arent you going to add simplify_algebra after rewrite_reduce? You added the header for it but I dont see it being used.

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

…nto compile-modes

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

pfultz2 · 2026-08-04T19:50:05Z

+
+    // Eager mode drops precision-preserving passes (rewrite_low_precision, propagate_precision,
+    // rewrite_gelu)
+    rv.disable_eager_tests({"test_topk", "test_softmax", "test_logsoftmax"});


These tests should not be disabled.

pfultz2 · 2026-08-04T19:51:21Z

-        test::add_test_case(p.name, [=] { verify(p); });
+        test::add_test_case(p.name, [=] { verify(p, migraphx::compile_modes::balanced); });
+
+        if(check_skipped_eager_tests(p.name))


Remove this as well, we shouldnt be disabling a test for a compile mode.

…ny passes for now

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

…nto compile-modes

pfultz2 · 2026-08-07T17:25:25Z

To fix the CI failures, you can probably disable fuse_attention for eager mode.

pnikolic-amd added 9 commits April 9, 2026 16:46

Compile modes init version

6743461

Removed passes for eager mod

58b364c

Implement support for compile modes in mgx apis

755844d

Fix driver command and add compile mode to api

63a9c15

Remove layout_convolution pass for eager mode

e3aec3a

Add option for skiping benchmarks

a152941

Remove unecesary code

f85f74e

Refactor get passes to remove duplicates

6461326

Fix missing argument

91fd7bd

pnikolic-amd requested review from apwojcik, pfultz2 and tperry-amd April 9, 2026 19:50

pnikolic-amd self-assigned this Apr 9, 2026

pnikolic-amd requested a review from causten as a code owner April 9, 2026 19:50

pfultz2 reviewed Apr 10, 2026

View reviewed changes

pnikolic-amd and others added 15 commits April 28, 2026 15:07

Merge develop

308e237

Fix merge passes

0f1169a

Updated types and setting

6a39c41

Create pipeline factory to remove duplications and hadle passes list

8f7bb08

Context ptr fix

1f76942

Clang foramt chages

6719b2c

Update copyright

0014161

Merge branch 'develop' into compile-modes

99c72f2

Adding compile modes to migraphx.h

0160c2b

Revert "Adding compile modes to migraphx.h"

b3f334f

This reverts commit 0160c2b.

Merge branch 'develop' into compile-modes

f62d878

Merge branch 'develop' into compile-modes

df3b65d

Update src/api/include/migraphx/migraphx.hpp

4130813

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Update tools/api/api.cpp

f01d575

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Fix compile-modes CI failures: add missing C API declarations, fix na…

46e0756

…ming and cppcheck warnings

pfultz2 reviewed Jun 11, 2026

View reviewed changes

Comment thread src/targets/gpu/target.cpp

pnikolic-amd and others added 14 commits July 29, 2026 15:32

Merge develop

1178a67

Update pipleine for eager mode

e6f8fac

Skip some tests in eager mode

2c7d374

Update src/api/include/migraphx/migraphx.hpp

4a6665d

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Update src/include/migraphx/compile_options.hpp

807ca67

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Update src/py/migraphx_py.cpp

f63d33b

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Merge branch 'compile-modes' of https://github.com/ROCm/AMDMIGraphX i…

d22f478

…nto compile-modes

Merge branch 'develop' into compile-modes

9874a76

Fix mispelling

ab47383

Formating and remove test

d079644

Generate api

5b25ce2

Merge branch 'develop' into compile-modes

16a9ff7

Update test/verify/main.cpp

5fc2665

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Update test/verify/run_verify.cpp

6771c9c

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

pnikolic-amd requested a review from pfultz2 August 4, 2026 13:37

pfultz2 requested changes Aug 4, 2026

View reviewed changes

pnikolic-amd and others added 5 commits August 6, 2026 15:24

Remove skiping tests in eager mode, modify eager mode to not remove a…

9198f2a

…ny passes for now

Merge branch 'develop' into compile-modes

b74f5c6

Update src/targets/gpu/target.cpp

5ef99b3

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Fix formatting

0d567d7

Merge branch 'compile-modes' of https://github.com/ROCm/AMDMIGraphX i…

5c11284

…nto compile-modes

pnikolic-amd added the Windows Related changes for Windows Environments label Aug 6, 2026

pnikolic-amd added 3 commits August 6, 2026 17:44

Putting just necessary passes in eager mode for tests to pass

2f46744

Foramting fixes

6c516a0

Merge branch 'develop' into compile-modes

5c44724

pnikolic-amd requested a review from pfultz2 August 7, 2026 14:45

Uh oh!

Conversation

pnikolic-amd commented Apr 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Technical Details

Changelog Category

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gh-app-migraphx-bot-pr-write Bot commented Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pfultz2 commented Aug 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

pnikolic-amd commented Apr 9, 2026 •

edited

Loading

gh-app-migraphx-bot-pr-write Bot commented Jun 11, 2026 •

edited

Loading