Clarify orchestration in heterogeneous compute report

Author: Andrey Cheptsov

mkdocs/blog/posts/state-of-heterogeneous-compute-2026.md

@@ -134,13 +134,15 @@ Use the chart as a starting point; the real position changes with the model, pro
 
 Heterogeneous compute is only useful if teams can turn scattered capacity into runnable workloads. That requires more than a scheduler queue. Teams need a way to describe training jobs, fine-tuning, batch jobs, services, images, secrets, resources, and placement constraints without rewriting the workflow for every accelerator pool.
 
-Orchestration does not make ROCm behave like CUDA or make a TPU workload portable by itself. Its job is to expose usable capacity, match workloads to compatible pools, keep task and service definitions portable where possible, and give platform teams one control layer across clouds, Kubernetes clusters, Slurm clusters, and on-prem infrastructure.
+The role of orchestration is to absorb operational complexity: provisioning, placement, images, resources, queues, scaling, service topology, logs, retries, and capacity constraints. It still has to respect runtime constraints. CUDA, ROCm, XLA, Neuron, collective libraries, serving engines, kernels, and model support are not interchangeable, so the control plane matches workloads to compatible capacity rather than treating every accelerator as equivalent.
 
 | Stack | Role and trade-off |
 | :---- | :---- |
-| Kubernetes | Strong substrate for containers, services, operators, and internal platforms. For AI, the stack is fragmented across device plugins, Dynamic Resource Allocation, GPU operators, Kueue, Volcano, Kubeflow, KubeRay, autoscaling, and serving layers. Teams still need a higher-level interface for accelerator discovery, quotas, procurement, placement, and workload submission. |
+| Kubernetes | Strong substrate for containers, services, operators, and internal platforms. For AI, it often requires stitching together device plugins, vendor GPU operators, Dynamic Resource Allocation, Kueue or Volcano, Kubeflow, KubeRay, autoscaling, and serving layers. Kubernetes-native multi-cluster tooling helps when capacity is already Kubernetes, but it still leaves accelerator discovery, quotas, procurement, runtime images, and cross-cloud placement as platform work. |
 | Slurm | Strong for large batch clusters, queueing, accounting, policy, and topology-aware placement. The limits are cloud-native and container-native workflows: real deployments often depend on local modules, queues, prolog/epilog scripts, plugins, and launch conventions. NVIDIA's SchedMD acquisition also makes Slurm vendor independence something teams should watch. |
-| Emerging | AI-native control planes such as dstack and SkyPilot focus on provisioning, placement, portable task and service definitions, and running workloads across providers or clusters. They can integrate with Kubernetes, Slurm, clouds, or on-prem infrastructure, but still depend on each accelerator stack for kernels, images, quotas, and performance. |
+| Emerging | AI-native control planes such as dstack and SkyPilot focus on making scattered accelerator capacity usable through higher-level workload definitions for training jobs, batch jobs, and services. They reduce the glue work created by fragmented clouds, clusters, accelerators, runtimes, and sites, which otherwise falls to platform teams. They still depend on each accelerator stack and serving runtime for kernels, images, collectives, routing, and performance. |
+
+Distributed inference makes this boundary especially important. Runtimes such as SGLang, Dynamo, vLLM, Mooncake, and NIXL own routing, batching, prefill/decode behavior, and KV cache movement; orchestration makes those topologies deployable and operable across compatible capacity.
 
 !!! info "Vendor independence"
     The orchestration and control-plane layer should not quietly become another accelerator dependency. NVIDIA acquired [Run:ai](https://blogs.nvidia.com/blog/runai/), a Kubernetes-based workload management platform, and [SchedMD](https://blogs.nvidia.com/blog/nvidia-acquires-schedmd/), the leading developer of Slurm. NVIDIA says Slurm will remain open-source and vendor-neutral, and its Run:ai messaging emphasizes choice and flexibility. Teams may still trust those projects, but heterogeneous compute makes vendor independence a practical requirement for the scheduler and control plane.
@@ -160,7 +162,7 @@ The open question is portability. Ethernet has momentum through [Spectrum-X](htt
 - **Compute shift.** Teams may not want more infrastructure complexity, but the market is moving there anyway. Capacity commitments, cloud-owned accelerators, merchant alternatives, and specialized inference systems are making more than one accelerator path relevant.
 - **Supply.** Agents, inference, and larger model deployments increase demand for usable capacity. The practical question is not only which accelerator is faster, but which one is available in the right volume, region, cloud or data center, timeline, and commercial terms.
 - **Software.** An accelerator path matters only if the model, framework, runtime, kernels, distributed training path, serving stack, containers, and debugging tools are ready enough for the workload.
-- **Orchestration.** Kubernetes and Slurm remain important, but heterogeneous compute needs a higher-level way to provision, place, and operate workloads across clouds, clusters, and accelerator stacks.
+- **Orchestration.** Kubernetes and Slurm remain important, but heterogeneous compute needs a higher-level way to provision, place, and operate workloads across fragmented clouds, clusters, datacenters, and accelerator stacks without leaving platform teams to stitch every path together themselves.
 
 ## Outside of the scope