Skip to content

Commit d6dcc6d

Browse files
authored
docs: integration refactor (#83)
* init * docs: add swanlab into your lib
1 parent 494a342 commit d6dcc6d

File tree

4 files changed

+423
-1
lines changed

4 files changed

+423
-1
lines changed

.vitepress/en.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -164,6 +164,7 @@ function sidebarGuideCloud(): DefaultTheme.SidebarItem[] {
164164
{ text: 'XGBoost', link: 'integration/integration-xgboost'},
165165
{ text: 'Xtuner', link: 'integration/integration-xtuner'},
166166
{ text: 'ZhipuAI', link: 'integration/integration-zhipuai'},
167+
{ text: 'Add SwanLab into Your Lib', link: 'integration/integration-any-library'},
167168
]
168169
},
169170
{

.vitepress/zh.ts

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -136,8 +136,8 @@ function sidebarGuideCloud(): DefaultTheme.SidebarItem[] {
136136
text: '⚡️ 集成',
137137
// collapsed: false,
138138
items: [
139-
{ text: 'Ascend NPU & MindSpore', link: 'integration/integration-ascend.md' },
140139
{ text: 'Argparse', link:'integration/integration-argparse' },
140+
{ text: 'Ascend NPU & MindSpore', link: 'integration/integration-ascend.md' },
141141
{ text: 'Fastai', link: 'integration/integration-fastai' },
142142
{ text: 'HuggingFace Accelerate', link: 'integration/integration-huggingface-accelerate' },
143143
{ text: 'HuggingFace Transformers', link: 'integration/integration-huggingface-transformers' },
@@ -165,6 +165,7 @@ function sidebarGuideCloud(): DefaultTheme.SidebarItem[] {
165165
{ text: 'XGBoost', link: 'integration/integration-xgboost'},
166166
{ text: 'Xtuner', link: 'integration/integration-xtuner'},
167167
{ text: 'ZhipuAI', link: 'integration/integration-zhipuai'},
168+
{ text: '将SwanLab集成到你的库', link: 'integration/integration-any-library' },
168169
]
169170
},
170171
{
Lines changed: 210 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,210 @@
1+
# Add SwanLab into Your Library
2+
3+
This guide provides best practices on how to integrate SwanLab into your Python library to leverage powerful experiment tracking, GPU and system monitoring, hyperparameter logging, and more.
4+
5+
Below, we outline the best practices we've compiled for when you're working with a codebase more complex than a single Python training script or Jupyter Notebook.
6+
7+
**🪵 Table of Contents:**
8+
9+
[[toc]]
10+
11+
## 1. Supplementing Requirements
12+
13+
Before you begin, decide whether to require SwanLab in your library's dependencies:
14+
15+
### 1.1 Adding SwanLab as a Dependency
16+
17+
```plaintext
18+
torch==2.5.0
19+
...
20+
swanlab==0.4.*
21+
```
22+
23+
### 1.2 Making SwanLab an Optional Installation
24+
25+
There are two ways to set up SwanLab as an optional installation.
26+
27+
1. Use a try-except block in your code to raise an error when SwanLab is not installed by the user.
28+
29+
```python
30+
try:
31+
import swanlab
32+
except ImportError:
33+
raise ImportError(
34+
"You are trying to use SwanLab, which is not currently installed."
35+
"Please install it using pip install swanlab"
36+
)
37+
```
38+
39+
2. If you are building a Python package, add `swanlab` as an optional dependency in the `pyproject.toml` file:
40+
41+
```toml
42+
[project]
43+
name = "my_awesome_lib"
44+
version = "0.1.0"
45+
dependencies = [
46+
"torch",
47+
"transformers"
48+
]
49+
50+
[project.optional-dependencies]
51+
dev = [
52+
"swanlab"
53+
]
54+
```
55+
56+
## 2. User Login
57+
58+
Your users have several methods to log in to SwanLab:
59+
60+
::: code-group
61+
62+
```bash [Command Line]
63+
swanlab login
64+
```
65+
66+
```python [Python]
67+
import swanlab
68+
swanlab.login()
69+
```
70+
71+
```bash [Environment Variable (Bash)]
72+
export SWANLAB_API_KEY=$YOUR_API_KEY
73+
```
74+
75+
```python [Environment Variable (Python)]
76+
import os
77+
os.environ["SWANLAB_API_KEY"] = "zxcv1234..."
78+
```
79+
80+
:::
81+
82+
If the user is using `swanlab` for the first time without following any of the above steps, they will be automatically prompted to log in when your script calls `swanlab.init`.
83+
84+
## 3. Starting a SwanLab Experiment
85+
86+
An experiment is the computational unit of SwanLab. Typically, you can create an `Experiment` object for each experiment and start it using the `swanlab.init` method.
87+
88+
### 3.1 Initializing the Experiment
89+
90+
Initialize SwanLab and start an experiment in your code:
91+
92+
```python
93+
swanlab.init()
94+
```
95+
96+
You can provide parameters such as project name, experiment name, and workspace for this experiment:
97+
98+
```python
99+
swanlab.init(
100+
project="my_project",
101+
experiment_name="my_experiment",
102+
workspace="my_workspace",
103+
)
104+
```
105+
106+
::: warning Where to Best Place swanlab.init?
107+
108+
Your library should create the SwanLab experiment as early as possible, as SwanLab automatically collects any output from the console, making debugging easier.
109+
110+
:::
111+
112+
### 3.2 Configuring Three Startup Modes
113+
114+
You can configure SwanLab's startup mode using the `mode` parameter:
115+
116+
::: code-group
117+
118+
```python [Cloud Mode]
119+
swanlab.init(
120+
mode="cloud", # Default mode
121+
)
122+
```
123+
124+
```python [Local Mode]
125+
swanlab.init(
126+
mode="local",
127+
)
128+
```
129+
130+
```python [Disabled Mode]
131+
swanlab.init(
132+
mode="disabled",
133+
)
134+
```
135+
136+
:::
137+
138+
- **Cloud Mode**: The default mode. SwanLab uploads experiment data to a web server (SwanLab's official cloud or your privately deployed cloud).
139+
- **Local Mode**: SwanLab does not upload experiment data to the cloud but records a special `swanlog` directory that can be opened by the `dashboard` plugin for visualization.
140+
- **Disabled Mode**: SwanLab does not collect any data, and code execution will bypass any `swanlab` related code.
141+
142+
### 3.3 Defining Experiment Hyperparameters/Configuration
143+
144+
Using SwanLab's experiment configuration (config), you can provide metadata about your model, dataset, etc., when creating a SwanLab experiment. You can use this information to compare different experiments and quickly understand the main differences.
145+
146+
Typical configuration parameters you might log include:
147+
148+
- Model name, version, architecture parameters, etc.
149+
- Dataset name, version, number of training/test data points, etc.
150+
- Training parameters such as learning rate, batch size, optimizer, etc.
151+
152+
The following code snippet shows how to log configuration:
153+
154+
```python
155+
config = {"learning_rate": 0.001, ...}
156+
swanlab.init(..., config=config)
157+
```
158+
159+
**Updating Configuration**:
160+
161+
Use the `swanlab.config.update` method to update the configuration. This method is convenient for updating the config dictionary after defining it.
162+
163+
For example, you might want to add model parameters after instantiating the model:
164+
165+
```python
166+
swanlab.config.update({"model_params": "1.5B"})
167+
```
168+
169+
## 4. Logging Data to SwanLab
170+
171+
Create a dictionary where the key is the name of the metric and the value is the metric's value. Pass this dictionary object to `swanlab.log`:
172+
173+
::: code-group
174+
175+
```python [Logging a Set of Metrics]
176+
metrics = {"loss": 0.5, "accuracy": 0.8}
177+
swanlab.log(metrics)
178+
```
179+
180+
```python [Logging Metrics in a Loop]
181+
for epoch in range(NUM_EPOCHS):
182+
for input, ground_truth in data:
183+
prediction = model(input)
184+
loss = loss_fn(prediction, ground_truth)
185+
metrics = { "loss": loss }
186+
swanlab.log(metrics)
187+
```
188+
189+
:::
190+
191+
If you have many metrics, you can use prefixes in the metric names (e.g., `train/...` and `val/...`). In the UI, SwanLab will automatically group them to isolate different categories of chart data:
192+
193+
```python
194+
metrics = {
195+
"train/loss": 0.5,
196+
"train/accuracy": 0.8,
197+
"val/loss": 0.6,
198+
"val/accuracy": 0.7,
199+
}
200+
swanlab.log(metrics)
201+
```
202+
203+
For more information on `swanlab.log`, refer to the [Logging Metrics](../experiment_track/log-experiment-metric) section.
204+
205+
## 5. Advanced Integration
206+
207+
You can also explore advanced SwanLab integrations in the following:
208+
209+
- [HuggingFace Transformers](../integration/integration-huggingface-transformers.md)
210+
- [PyTorch Lightning](../integration/integration-pytorch-lightning.md)

0 commit comments

Comments
 (0)