Skip to content

feat: Add force_restart option to SkypilotBackend #135

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 21, 2025
Merged

feat: Add force_restart option to SkypilotBackend #135

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 18 additions & 3 deletions src/art/skypilot/backend.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ async def initialize_cluster(
resources: sky.Resources | None = None,
art_version: str | None = None,
env_path: str | None = None,
force_restart: bool = False,
) -> "SkyPilotBackend":
self = cls.__new__(cls)
self._cluster_name = cluster_name
Expand Down Expand Up @@ -74,10 +75,19 @@ async def initialize_cluster(
else:
print(f"Cluster {self._cluster_name} exists, using it...")

if await is_task_created(
art_server_running = await is_task_created(
cluster_name=self._cluster_name, task_name="art_server"
):
print("Art server task already running, using it...")
)

if art_server_running and force_restart:
print("force_restart=True; cancelling existing art_server task…")
await to_thread_typed(
lambda: sky.cancel(cluster_name=self._cluster_name, all=True)
)
art_server_running = False

if art_server_running:
print("Art server task already running, using it…")
else:
art_server_task = sky.Task(name="art_server", run="uv run art")
resources = await to_thread_typed(
Expand All @@ -88,6 +98,11 @@ async def initialize_cluster(
art_server_task.set_resources(cast(sky.Resources, resources))
art_server_task.update_envs(self._envs)

# If a local path was provided for art_version, ensure it is mounted so the latest
# code is synced to the remote cluster every time we (re)launch the art_server task.
if art_version is not None and os.path.exists(art_version):
art_server_task.set_file_mounts({"~/sky_workdir": art_version})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, not sure we want to do this by default. It's conceivable that I might be editing code on the cluster itself, and I don't want that getting overwritten.

Maybe make this a parameter, and default to False?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed we'll experiment with keeping this on by default for now and can turn it off if it ends up being annoying for @arcticfly's workflow.


# run art server task
await to_thread_typed(
lambda: sky.exec(
Expand Down