Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added docs/assets/images/hopsworksai/add-worker.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/images/hopsworksai/remove-worker.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
84 changes: 84 additions & 0 deletions docs/hopsworksai/adding_removing_workers.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
#Adding and removing workers
Once you have started a hopsworks cluster you can add and remove workers from the cluster to accommodate your workload.

##Adding workers
If the computation you are running is using all the resources of your Hopsworks cluster you can add workers to your cluster.
To add workers to a cluster, go to the *Details* tab of this cluster and click on *Add workers*.

<p align="center">
<figure>
<a href="../../../assets/images/hopsworksai/add-worker.png">
<img src="../../../assets/images/hopsworksai/add-worker.png" alt="Add worker">
</a>
<figcaption>Add worker</figcaption>
</figure>
</p>

Select the number of workers you want to add (1). Select the type of instance you want the workers to run on (2). Select the local storage size for the workers (3). Click on *Next*.

<p align="center">
<figure>
<a href="../../../assets/images/hopsworksai/add-workers-config.png">
<img src="../../../assets/images/hopsworksai/add-workers-config.png" alt="Add workers config">
</a>
<figcaption>Add workers</figcaption>
</figure>
</p>

Review your request and click *Add*.

<p align="center">
<figure>
<a href="../../../assets/images/hopsworksai/add-workers-review.png">
<img src="../../../assets/images/hopsworksai/add-workers-review.png" alt="Add workers review">
</a>
<figcaption>Add workers</figcaption>
</figure>
</p>

Hopsworks.ai will start the new workers and you will be able to use them in your cluster as soon as they have finished starting.

##Removing workers

If the load on your Hopsworks cluster is low, you can decide to remove worker nodes from your cluster.

!!! warning
When removing workers Hopsworks.ai will try to select workers that can be removed while interfering as little as possible with any ongoing computation. It will also wait for the workers to be done with their computation before stopping them. But, if this computation lasts too long, the worker may get stopped before the computation properly finish. This could interfere with your ongoing computation.

!!! note
You can remove all the workers of your cluster. If you do so the cluster will be able to store data but not run any computations. This may affect feature store functionality.

To remove workers from a cluster, go to the *Details* tab of this cluster and click on *Remove workers*

<p align="center">
<figure>
<a href="../../../assets/images/hopsworksai/remove-worker.png">
<img src="../../../assets/images/hopsworksai/remove-worker.png" alt="Remove worker">
</a>
<figcaption>Remove workers</figcaption>
</figure>
</p>

For each of the types of instances existing in your cluster select the number of workers you want to remove and click on *Next*.

<p align="center">
<figure>
<a href="../../../assets/images/hopsworksai/remove-workers-config.png">
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo

Suggested change
<a href="../../../assets/images/hopsworksai/remove-workers-config.png">
<a href="../../../assets/images/hopsworksai/remove-worker-config.png">

<img src="../../../assets/images/hopsworksai/remove-workers-config.png" alt="Remove workers config">
</a>
<figcaption>Remove workers</figcaption>
</figure>
</p>

Review your request and click *Remove*.

<p align="center">
<figure>
<a href="../../../assets/images/hopsworksai/remove-workers-review.png">
<img src="../../../assets/images/hopsworksai/remove-workers-review.png" alt="Remove workers review">
</a>
<figcaption>Remove workers</figcaption>
</figure>
</p>

Hopsworks.ai will select the workers corresponding to your criteria which can be stopped with as little interferences as possible with any ongoing computation. It will set them to decommission and stop them when they have finished decommissioning.
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ nav:
- Cluster creation:
- AWS: hopsworksai/aws/cluster_creation.md
- Azure: hopsworksai/azure/cluster_creation.md
- Adding and Removing workers: hopsworksai/adding_removing_workers.md
# - Databricks Quickstart:
# - AWS: hopsworksai/aws/databricks_quickstart.md
# - Azure: hopsworksai/azure/databricks_quickstart.md
Expand Down