MultiKueue: Support Elastic RayService

[JunchengMa]

Similarly as #8712,

Description

What would you like to be added:

Support elastic RayService (using UpgradeStrategy: NewClusterWithIncrementalUpgrade) via MultiKueue.

Background:

Basic MultiKueue support for RayService was added in #10025. However, elastic RayService — where UpgradeStrategy: NewClusterWithIncrementalUpgrade is used — is not yet covered.

The problem:

When NewClusterWithIncrementalUpgrade is used, KubeRay creates a second RayCluster alongside the existing one during an upgrade, gradually shifting traffic from the old cluster to the new one. At peak, both clusters run simultaneously.

The current MultiKueue adapter for RayService (rayservice_multikueue_adapter.go) copies the full spec to the worker cluster via copyJobSpec, which includes the UpgradeStrategy field. This means the worker cluster will also attempt an incremental upgrade and spin up a second RayCluster — but:

1. The Kueue workload on the manager cluster only accounts for one cluster's resources. The additional resource requirements of the second (upgrade) cluster are not reflected in the workload, leading to under-accounting of quota consumed on the worker.

2. MultiKueue has no visibility into the dual-cluster state during upgrade. The workload controller syncs status back from the worker's RayService, but does not model the transient two-cluster resource usage for admission or quota purposes.

Why is this needed:

To run elastic Ray serving workloads with zero-downtime incremental upgrades via MultiKueue, enabling multi-cluster scheduling for long-running RayService resources.

[olekzabl]

/area multikueue

[kevin85421]

/assign kevin85421

[kevin85421]

Hi @hiboyang @mimowo (based on #9059),

This issue affects not only RayService's incremental upgrade but also its zero-downtime upgrade. Both strategies create a new RayCluster and switch traffic to it. That is, during an upgrade, a RayService will own two RayCluster CRs.

I ran an experiment locally:

• Created a ClusterQueue with 1 CPU and 2 GB memory.
• Created a RayService that required only 1 head Pod, requesting 1 CPU and 2 GB memory.
• Updated the RayService's rayVersion to trigger a zero-downtime upgrade.
• KubeRay created a new RayCluster also requesting 1 CPU and 2 GB memory in total.
• The new RayCluster was created even though the ClusterQueue was already fully occupied by the old RayCluster.
• Once the new RayCluster was ready, traffic was switched over and KubeRay deleted the old one.

I'll take a look and propose a solution to discuss with you both.

[kevin85421]

I opened two PRs related to this issue:

• [RayService][Kueue] Support top-level Spec.Suspend for zero-downtime upgrade ray-project/kuberay#4841
• POC: gate RayService zero-downtime upgrade with workload slicing #11264

The first step is to make sure folks from both Kueue and KubeRay are aligned on the API introduced in ray-project/kuberay#4841.

[hiboyang]

@kevin85421 , yes, you are right, there is issue during upgrading time. Thanks for working on this!

[kevin85421]

Both KubeRay PRs ray-project/kuberay#4841 and ray-project/kuberay#4881 are merged. I will start to work on #11264.