[Core][UX] One-click user-space Ray cluster #7935

kevinmingtarja · 2025-11-12T09:09:40Z

This PR implements the first of many SkyPilot templates, a way to simplify the user experience of writing SkyPilot YAMLs. This first PR implements ~/sky_templates/ray/start_cluster.sh, a script to easily spin up your own Ray cluster, for running your own Ray programs, or when running multi-node vLLM inference, or RL with SkyRL, verl, etc.

To use it, simply call it from your run step:

resources:
  cpus: 2+

num_nodes: 2

workdir: .

run: |
  ~/sky_templates/ray/start_cluster.sh

  if [ "$SKYPILOT_NODE_RANK" == "0" ]; then
    python main.py
  fi

# ⚙︎ Job submitted, ID: 1
# ├── Waiting for task resources on 2 nodes.
# └── Job started. Streaming logs... (Ctrl-C to exit log streaming; job will not be killed)
# (head, rank=0, pid=3726) Starting Ray cluster...
# (head, rank=0, pid=3726) Ray is not installed. Installing...
# (head, rank=0, pid=3726) Using Python 3.10.10 environment at: /opt/conda
# ...
# (head, rank=0, pid=3726) Ray 2.51.1 installed successfully.
# (worker1, rank=1, pid=2680, ip=10.0.7.223) Ray 2.51.1 installed successfully.
# (head, rank=0, pid=3726) Starting Ray head node...
# (head, rank=0, pid=3726) 2025-11-12 09:28:59,475        INFO usage_lib.py:447 -- Usage stats collection is disabled.
# (head, rank=0, pid=3726) 2025-11-12 09:28:59,475        INFO scripts.py:914 -- Local node IP: 10.0.7.170
# (worker1, rank=1, pid=2680, ip=10.0.7.223) Starting Ray worker node...
# (worker1, rank=1, pid=2680, ip=10.0.7.223) Waiting for head node at 10.0.7.170:6379...                 
# (worker1, rank=1, pid=2680, ip=10.0.7.223) Head node not healthy yet. Retrying in 1s...
# (worker1, rank=1, pid=2680, ip=10.0.7.223) Head node is healthy. Starting worker node...               
# (head, rank=0, pid=3726) 2025-11-12 09:29:04,648        SUCC scripts.py:950 -- --------------------
# (head, rank=0, pid=3726) 2025-11-12 09:29:04,648        SUCC scripts.py:951 -- Ray runtime started.
# (head, rank=0, pid=3726) 2025-11-12 09:29:04,648        SUCC scripts.py:952 -- --------------------
# ...                             
# (worker1, rank=1, pid=2680, ip=10.0.7.223) 2025-11-12 09:29:05,139      INFO scripts.py:1095 -- Local node IP: 10.0.7.223
# (head, rank=0, pid=3726) Head node started successfully.
# (head, rank=0, pid=3726) Waiting for all 2 nodes to join...
# ...
# (worker1, rank=1, pid=2680, ip=10.0.7.223) Worker node started successfully.                           
# (head, rank=0, pid=3726) 2025-11-12 09:29:05,800        WARNING services.py:397 -- Found multiple active Ray instances: {'10.0.7.170:6379', '10.0.7.170:6380'}. Connecting to latest cluster at 10.0.7.170:6379. You can override this by setting the `--address` flag or `RAY_ADDRESS` environment variable.        
# (head, rank=0, pid=3726) All 2 nodes have joined.
# (head, rank=0, pid=3726) 2025-11-12 09:29:11,172        WARNING services.py:397 -- Found multiple active Ray instances: {'10.0.7.170:6380', '10.0.7.170:6379'}. Connecting to latest cluster at 10.0.7.170:6379. You can override this by setting the `--address` flag or `RAY_ADDRESS` environment variable.       
# (head, rank=0, pid=3726) 2025-11-12 09:29:11,172        INFO worker.py:1832 -- Connecting to existing Ray cluster at address: 10.0.7.170:6379...                                                              
# (head, rank=0, pid=3726) 2025-11-12 09:29:11,186        INFO worker.py:2003 -- Connected to Ray cluster. View the dashboard at 127.0.0.1:8265                                                                 
# (head, rank=0, pid=3726) /opt/conda/lib/python3.10/site-packages/ray/_private/worker.py:2051: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0                                                                    
# (head, rank=0, pid=3726)   warnings.warn(
# (head, rank=0, pid=3726) Sum of squares: 328350                                                        
# ✓ Job finished (status: SUCCEEDED).

Docs preview:

https://docs.skypilot.co/en/simple-ray/examples/training/ray.html
https://docs.skypilot.co/en/simple-ray/running-jobs/distributed-jobs.html#executing-a-distributed-ray-program

Tested (run the relevant ones):

Code formatting: install pre-commit (auto-check on commit) or bash format.sh
Any manual or new tests for this PR (please specify below)
- test_ray_basic
- Manually tested skyrl, verl, kimi-k2 example
All smoke tests: /smoke-test (CI) or pytest tests/test_smoke.py (local)
Relevant individual tests: /smoke-test -k test_name (CI) or pytest tests/test_smoke.py::test_name (local)
Backward compatibility: /quicktest-core (CI) or pytest tests/smoke_tests/test_backward_compat.py (local)

kevinmingtarja · 2025-11-12T09:15:22Z

/smoke-test

kevinmingtarja · 2025-11-12T23:42:22Z

/smoke-test -k nemo
/smoke-test -k ray
/smoke-test -k nemo --nebius
/smoke-test -k ray --nebius

kevinmingtarja · 2025-11-13T01:18:07Z

/smoke-test -k nemo
/smoke-test -k ray

kevinmingtarja · 2025-11-13T01:25:51Z

/smoke-test -k nemo

kevinmingtarja · 2025-11-13T01:38:18Z

/smoke-test -k nemo
/smoke-test -k ray

kevinmingtarja · 2025-11-13T02:25:43Z

sky_templates/ray/start_cluster.sh

+# Environment Variables:
+#   RAY_HEAD_PORT=6379                     - Ray head node port
+#   RAY_DASHBOARD_PORT=8265                - Ray dashboard port
+#   RAY_DASHBOARD_HOST=127.0.0.1           - Dashboard host (set to 0.0.0.0 to expose externally)
+#   RAY_DASHBOARD_AGENT_LISTEN_PORT=       - (Optional) Dashboard agent listen port
+#   RAY_NODE_IP_ADDRESS=                   - (Optional) Node IP address
+#   RAY_CMD_PREFIX=                        - (Optional) Command prefix (e.g., "uv run")


These are the minimal set of params needed to cover all our examples that uses Ray.

Michaelvll

Thanks @kevinmingtarja! It looks mostly good to me. Please see the comments below.

Also, we should modify this ray example page to include the new template: https://docs.skypilot.co/en/latest/examples/training/ray.html

examples/distributed_ray_train/ray_train.yaml

llm/verl/verl-grpo.yaml

sky/backends/wheel_utils.py

llm/verl/multinode.yaml

kevinmingtarja · 2025-11-13T06:40:05Z

/build-docs

github-actions · 2025-11-13T06:40:55Z

✅ ReadTheDocs build triggered for branch simple-ray

The documentation will be available at: https://docs.skypilot.co/en/simple-ray/

kevinmingtarja · 2025-11-13T06:43:54Z

/build-docs

github-actions · 2025-11-13T06:45:28Z

✅ ReadTheDocs build triggered for branch simple-ray

The documentation will be available at: https://docs.skypilot.co/en/simple-ray/

kevinmingtarja · 2025-11-13T06:45:34Z

Docs preview:

Added references on the start and stop templates script. Listed down the environment variables for passing arguments to the script.

kevinmingtarja · 2025-11-13T06:50:21Z

/build-docs

github-actions · 2025-11-13T06:52:52Z

✅ ReadTheDocs build triggered for branch simple-ray

The documentation will be available at: https://docs.skypilot.co/en/simple-ray/

kevinmingtarja · 2025-11-13T09:11:02Z

Note: I renamed ~/skypilot_templates -> ~/sky_templates to be more consistent with ~/sky_workdir and ~/sky_logs.

kevinmingtarja · 2025-11-13T09:12:01Z

/smoke-test -k nemo
/smoke-test -k ray

Michaelvll

Thanks @kevinmingtarja ! This is awesome! It looks quite good to me

docs/source/running-jobs/distributed-jobs.rst

examples/distributed_ray_train/README.md

docs/source/running-jobs/distributed-jobs.rst

llm/skyrl/train.yaml

llm/verl/multinode.yaml

kevinmingtarja · 2025-11-14T04:14:04Z

/smoke-test -k nemo
/smoke-test -k ray

kevinmingtarja · 2025-11-14T04:58:09Z

/smoke-test -k ray_basic
/smoke-test -k ray_basic --gcp
/smoke-test -k ray_basic --nebius
/smoke-test -k ray_basic --kubernetes

[Core] One-click user-space Ray cluster

fce5c19

kevinmingtarja changed the title ~~[Core] One-click user-space Ray cluster~~ [Core][UX] One-click user-space Ray cluster Nov 12, 2025

kevinmingtarja added 6 commits November 12, 2025 10:03

fix circular import

06afb0c

fix ray start script

8343863

improve start script

e8d5aa0

bundle templates in the python package instead of using cloud buckets

56e90ff

use new template in ray examples

3339bb2

rm

2a01d3e

kevinmingtarja added 7 commits November 12, 2025 15:48

fix verl example

1c690b6

revert k8s change

bfa4be5

Merge branch 'master' into simple-ray

e177016

revert schema change

a3aa4ba

revert some more stuff

6e3b752

add simple ray smoke test, robustify script

db00a50

fix wheel caching, ray installation

0743e93

fix

c61c1a8

fix ray install

469c57b

kevinmingtarja requested review from Michaelvll, SeungjinYang and romilbhardwaj November 13, 2025 01:43

Merge branch 'master' into simple-ray

0d63455

kevinmingtarja commented Nov 13, 2025

View reviewed changes

Michaelvll reviewed Nov 13, 2025

View reviewed changes

examples/distributed_ray_train/ray_train.yaml Outdated Show resolved Hide resolved

llm/verl/verl-grpo.yaml Show resolved Hide resolved

sky/backends/wheel_utils.py Outdated Show resolved Hide resolved

llm/verl/multinode.yaml Outdated Show resolved Hide resolved

kevinmingtarja added 2 commits November 12, 2025 18:40

small fix

dad0154

address pr review

d467530

update pkill cmd

f62e7c0

revert a bit

4cd171d

test sky exec too in test_ray_basic

ec82eeb

kevinmingtarja requested a review from Michaelvll November 13, 2025 07:40

kevinmingtarja added 4 commits November 12, 2025 23:43

cleanup

3be8049

simplify

3529579

fix RAY_HEAD_IP_ADDRESS description

c1225ad

rename ~/skypilot_templates -> ~/sky_templates

4175029

Michaelvll approved these changes Nov 13, 2025

View reviewed changes

kevinmingtarja added 5 commits November 13, 2025 16:54

RAY_CMD_PREFIX -> RAY_CMD

493f214

revert skyrl unintended diff

eef4205

cleanup verl multinode yaml

f3b404c

cleanup docs

0294d67

remove .sh extension

b430858

improve smoke test

68c16c1

kevinmingtarja enabled auto-merge (squash) November 14, 2025 05:06

kevinmingtarja merged commit 969e9c4 into master Nov 14, 2025
24 checks passed

kevinmingtarja deleted the simple-ray branch November 14, 2025 05:06

kevinmingtarja mentioned this pull request Nov 14, 2025

[Docs] Fix Ray start_cluster script name #7965

Merged

5 tasks

[Core][UX] One-click user-space Ray cluster #7935

[Core][UX] One-click user-space Ray cluster #7935

Uh oh!

Conversation

kevinmingtarja commented Nov 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kevinmingtarja commented Nov 12, 2025

Uh oh!

kevinmingtarja commented Nov 12, 2025

Uh oh!

kevinmingtarja commented Nov 13, 2025

Uh oh!

kevinmingtarja commented Nov 13, 2025

Uh oh!

kevinmingtarja commented Nov 13, 2025

Uh oh!

kevinmingtarja Nov 13, 2025

Choose a reason for hiding this comment

Uh oh!

Michaelvll left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kevinmingtarja commented Nov 13, 2025

Uh oh!

github-actions bot commented Nov 13, 2025

Uh oh!

kevinmingtarja commented Nov 13, 2025

Uh oh!

github-actions bot commented Nov 13, 2025

Uh oh!

kevinmingtarja commented Nov 13, 2025

Uh oh!

kevinmingtarja commented Nov 13, 2025

Uh oh!

github-actions bot commented Nov 13, 2025

Uh oh!

kevinmingtarja commented Nov 13, 2025

Uh oh!

kevinmingtarja commented Nov 13, 2025

Uh oh!

Michaelvll left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kevinmingtarja commented Nov 14, 2025

Uh oh!

kevinmingtarja commented Nov 14, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

kevinmingtarja commented Nov 12, 2025 •

edited

Loading