Skip to main content
Version: Next

Run RayJob

Note

This example is how to setup KubeRay and run Ray Job with YuniKorn scheduler. It relies on an admission controller to configure the default applicationId and queue name. If you want more details, please refer to Yunikorn supported labels and Yunikorn queue setting.

Modify YuniKorn settings​

Follow YuniKorn install guide and modify YuniKorn configmap "yunikorn-defaults" to allow ray operator based on k8s service account.

kubectl patch configmap yunikorn-defaults -n yunikorn --patch '{"data":{"admissionController.accessControl.systemUsers": "^system:serviceaccount:kube-system:,^system:serviceaccount:default:"}}'

Setup a KubeRay operator​

helm repo add kuberay https://ray-project.github.io/kuberay-helm/
helm repo update
helm install kuberay-operator kuberay/kuberay-operator --version 1.2.2 --set batchScheduler.name=yunikorn
  • The result should be as shown below ray_cluster_operator

Configure your Ray CRD(optional)​

If you disable admission controller, you need to add the schedulerName: yunikorn in raycluster labels.

#example
metadata:
  labels:
    applicaionId: ray-<service_type>-0001
    queue: root.ray.<specific_queue_name>
spec:
  schedulerName: yunikorn # k8s will inform yunikorn based on this

Run a RayJob​

kubectl apply -f https://raw.githubusercontent.com/ray-project/kuberay/v1.1.1/ray-operator/config/samples/ray-job.sample.yaml
  • View the job status ray_job_job

Services in Kubernetes aren't directly accessible by default. However, you can use port-forwarding to connect to them locally.

kubectl port-forward service/raycluster-kuberay-head-svc 8265:8265

After port-forward set up, you can access the Ray dashboard by going to http://localhost:8265 in your web browser.

  • Ray Dashboard ray_job_ray_dashboard
  • YuniKorn UI ray_job_on_ui