Skip to content

Merge upstream→gfx11#1041

Merged
mgehre-amd merged 83 commits into
ROCm:gfx11from
eble-amd:merge-from-upstream
Jul 1, 2026
Merged

Merge upstream→gfx11#1041
mgehre-amd merged 83 commits into
ROCm:gfx11from
eble-amd:merge-from-upstream

Conversation

@eble-amd

@eble-amd eble-amd commented Jun 30, 2026

Copy link
Copy Markdown

Among the changes to torch bindings, there were some diff hunks where both upstream and local added lines. I kept both.

Finding what to merge was tricky this time. My initial attempt to stop at the first conflicting commit hit errors related to torch version and bindings. For more information, see the series of changes vllm-project#44334, vllm-project#44648, vllm-project#47128.

cleonard530 and others added 30 commits June 4, 2026 20:14
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
…llm-project#44571)

Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com>
… output plumbing (vllm-project#43720)

Signed-off-by: zixi-qi <zixi@inferact.ai>
Signed-off-by: jiang1.li <jiang1.li@intel.com>
…ect#40426)

Signed-off-by: hanlin12 <hanlin12@amd.com>
Signed-off-by: Han Lin <hanlin12@amd.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
…lm-project#41002)

Signed-off-by: Stig-Arne Grönroos <stig-arne.gronroos@amd.com>
Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com>
Co-authored-by: Stig-Arne Grönroos <stig-arne.gronroos@amd.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
Signed-off-by: viiccwen <viiccwen@gmail.com>
…detokenizer (vllm-project#44620)

Signed-off-by: Ting Sun <suntcrick@gmail.com>
…lm-project#44618)

Signed-off-by: Xu Zhou <xuzhou9417@163.com>
Co-authored-by: Xu Zhou <xuzhou9417@163.com>
Signed-off-by: UranusSeven <109661872+UranusSeven@users.noreply.github.com>
…vllm-project#44622)

Signed-off-by: Xu Zhou <xuzhou9417@163.com>
Co-authored-by: Xu Zhou <xuzhou9417@163.com>
Signed-off-by: RickyChen / 陳昭儒 <ricky.chen@infinirc.com>
Signed-off-by: chunyang.wen <chunyang.wen@gmail.com>
Signed-off-by: Hugh Ryan <197298026+HueCodes@users.noreply.github.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: tianyu-z <zhangtianyupro@gmail.com>
Signed-off-by: Tianyu Zhang <53099276+tianyu-z@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
…llm-project#43167)

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
…ct#44615)

Signed-off-by: ADHITHYA BALAKRISHNAN <adhithya.balakrishnan@multicorewareinc.com>
…l tags (vllm-project#44588)

Signed-off-by: rishitdholakia13 <rishit+github@cohere.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Yan Ma <yan.ma@intel.com>
…8804)

Signed-off-by: vikrantpalle <vikrantpalle@gmail.com>
Signed-off-by: StingLin <sting.lin@cienet.com>
Signed-off-by: Yifan Zong <yzong@redhat.com>
charlifu and others added 22 commits June 7, 2026 09:50
Signed-off-by: charlifu <charlifu@amd.com>
…ction (vllm-project#44454)

Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
…-project#42736)

Signed-off-by: Dobrzyniewicz, Agata <agata.dobrzyniewicz@intel.com>
)

Signed-off-by: SunskyXH <sunskyxh@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
…roject#44805)

Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
…t#39562)

Signed-off-by: Holworth <kangqihan17@mails.ucas.ac.cn>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: shen-shanshan <467638484@qq.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
… rare OOMs (vllm-project#44761)

Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: zengxian <xiangdong.zeng@intel.com>
…llm-project#44828)

Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>
Signed-off-by: Sungjae Lee <sung-jae.lee@navercorp.com>
…ct#44499)

Signed-off-by: Sahil Singh <sahiilsiingh37@gmail.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
…rence failures (vllm-project#44470)

Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
…eloaded (vllm-project#44419)

Signed-off-by: jmamou <jonathan.mamou@intel.com>
Signed-off-by: Jonathan Mamou <jonathan.mamou@intel.com>
Co-authored-by: Li, Jiang <bigpyj64@gmail.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: walterbm <walter.beller.morales@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
…om-upstream

Among the changes to torch bindings, there were some diff hunks where
both upstream and local added lines.  I kept both.

Signed-off-by: Dan Eble <Dan.Eble@amd.com>
@eble-amd

eble-amd commented Jun 30, 2026

Copy link
Copy Markdown
Author

Manual sanity test on gfx1151:

Arguments: --model Qwen/Qwen2.5-0.5B-Instruct-AWQ --num-prompts 10 --max-model-len 4096 --kill-e
xisting-vllm-processes --input-len 3968 --output-len 128 --target-gpu-memory-gb 10 --max-num-seq
s 1 -e TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1 -e FLASH_ATTENTION_TRITON_AMD_ENABLE=TRUE -e TO
RCH_BLAS_PREFER_HIPBLASLT=1
Prefill 23338.17 tokens/s; TTFT 170 ms
Decode 312.9 tokens/s (TPOT 3.20 ms)
End-to-end latency 575 ms (median)
Average CPU utilization 7%

@eble-amd

Copy link
Copy Markdown
Author

Performance regression tests on gfx1151:

python -m pytest -v tests/kernels/quantization/test_hybrid_w4a16_perf.py
...
========================= 128 passed, 17 warnings in 436.47s (0:07:16) =========================
...

@eble-amd eble-amd marked this pull request as ready for review June 30, 2026 20:34
@eble-amd eble-amd requested a review from mgehre-amd June 30, 2026 20:35

@mgehre-amd mgehre-amd left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@mgehre-amd mgehre-amd merged commit 23ac9e6 into ROCm:gfx11 Jul 1, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.