Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
ef52f9a
Update main.py
SAKURA-CAT Jan 23, 2024
3ac1c50
feat:run path
Feudalman Jan 23, 2024
a972bc4
Merge branch 'feat/v0.1.5' of https://github.com/SwanHubX/SwanLab int…
Feudalman Jan 23, 2024
e1114c6
version limit
SAKURA-CAT Jan 23, 2024
37b45f7
change version
SAKURA-CAT Jan 23, 2024
3bcab31
fix: empty table border
Feudalman Jan 23, 2024
35f40ae
fix: check cursor
Feudalman Jan 23, 2024
c807e2f
Merge branch 'feat/v0.1.5' of https://github.com/SwanHubX/SwanLab int…
Feudalman Jan 23, 2024
a782de7
update message
SAKURA-CAT Jan 23, 2024
99d4a7e
Merge branch 'feat/v0.1.5' of https://github.com/SwanHubX/swanlab int…
SAKURA-CAT Jan 23, 2024
860a062
Feat/readme v0.1.5 (#251)
Zeyi-Lin Jan 23, 2024
a305cdd
fix: delete project
Feudalman Jan 23, 2024
cb7e01f
fix: export as csv
Feudalman Jan 23, 2024
7d291c6
update step
SAKURA-CAT Jan 23, 2024
e70f4b0
Merge branch 'feat/v0.1.5' of https://github.com/SwanHubX/swanlab int…
SAKURA-CAT Jan 23, 2024
d99302b
Touch up error message
Zeyi-Lin Jan 23, 2024
539e7c6
Update package.py
SAKURA-CAT Jan 23, 2024
a417ea7
Merge branch 'feat/v0.1.5' of https://github.com/SwanHubX/swanlab int…
SAKURA-CAT Jan 23, 2024
deb70f8
update version_limit
Zeyi-Lin Jan 23, 2024
1a6ec89
Merge branch 'feat/v0.1.5' of https://github.com/SwanHubX/swanlab int…
Zeyi-Lin Jan 23, 2024
60dd37a
fix: table width
Feudalman Jan 23, 2024
e4d864e
Merge branch 'feat/v0.1.5' of https://github.com/SwanHubX/SwanLab int…
Feudalman Jan 23, 2024
d0e434d
fix: table width
Feudalman Jan 23, 2024
d51a2ac
refactor: search delay
Feudalman Jan 23, 2024
be8020e
fix: text wrap
Feudalman Jan 23, 2024
664cebf
Update EnvItem.vue
Feudalman Jan 23, 2024
2e82253
empty chart
SAKURA-CAT Jan 23, 2024
45b5604
fix: empty summary table
Feudalman Jan 23, 2024
984a7d2
Merge branch 'feat/v0.1.5' of https://github.com/SwanHubX/swanlab int…
SAKURA-CAT Jan 23, 2024
114bd71
fix: table title
Feudalman Jan 23, 2024
7e98cd0
Merge branch 'feat/v0.1.5' of https://github.com/SwanHubX/SwanLab int…
Feudalman Jan 23, 2024
c7f4699
bugfix: version_limit
Zeyi-Lin Jan 23, 2024
908bd73
Update package.py
SAKURA-CAT Jan 23, 2024
f8c3f59
Update sdk.py
Zeyi-Lin Jan 23, 2024
2d8c218
modify version_limit
SAKURA-CAT Jan 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 67 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,9 @@ English | <a href="README_zh-hans.md">中文</a>

- [Key Function](#key-function)
- [Changelog](#changelog)
- [Use Case](#use-case)
- [Installation](#installation)
- [Getting Started](#getting-started)
- [Use Case](#use-case)
- [More Tips](#more-tips)
- [LICENSE](#license)

Expand Down Expand Up @@ -63,26 +64,33 @@ English | <a href="README_zh-hans.md">中文</a>

## Changelog

[24/01/23] 🚨 We use SQLite database and Peewee to replace the previous basic configuration information reading and writing solution by [#114](https://github.com/SwanHubX/SwanLab/issues/114). It's a major change that is highly beneficial for the future of the project, but the drawback is that it's not compatible with old versions (swanlab<=v0.1.4) of log data files. Therefore, if you need to visualize log files generated by older versions, please install version `<=0.1.4`.

Additionally, we supported export the experiment list as `CSV`, new environment record items `Run path` and `logdir`, added interactive quick copy function, and new API `swanlab.config`.

[24/01/14] 🔥 We supported a new UI, tracking additional environment information, including command, git commit/branch and memory. Additionally, we've added a `logdir` API, allowing developers to set the directory for log files.

[24/01/07] ✨ We supported delete experiment and edit experiment inforamation on Dashboard.

[24/01/01] We fixed some bugs to make SwanLab more stable.

[Full Changelog](https://github.com/SwanHubX/SwanLab/releases)

<br>

## Use Case
## Installation

Learn how to use SwanLab more effectively by following these use cases:
### with pip

| Code Cases | Description |
| ------- | ------- |
| [Hello World](https://github.com/SwanHubX/SwanLab-examples/tree/main/Hello_World) | Getting Started |
| [MNIST](https://github.com/SwanHubX/SwanLab-examples/tree/main/MNIST) | Handwriting recognition based on a plain net and MNIST dataset with pytroch, swanlab. |
| [Image Classification](https://github.com/SwanHubX/SwanLab-examples/blob/main/Resnet50) | Cat and dog classification based on ResNet50 with pytorch, swanlab and gradio. [Tutorial](https://zhuanlan.zhihu.com/p/676430630). |
| [Text Generation](https://github.com/SwanHubX/SwanLab-examples/blob/main/Word_language_model) | Text generation based on Word_language_model (RNN/LSTM/GRU/Transformer) |
Requirement:Python 3.8+.

Use [pip](https://pip.pypa.io/en/stable/) to install our stable release version. The installation command is as follows:

```bash
pip install -U swanlab
```

### with source code

If you need the bleeding edge of the code and can't wait for a new release, you must [install the library from source](https://geektechstudio.feishu.cn/wiki/DvxSweHUKiAe8yksci3cMflbnwh#SMXHdJ1c1o4jzTxcDticHcwvnHd)。

<br>

Expand All @@ -95,7 +103,7 @@ pip install -U swanlab
```

2. Second, Use the example code snippet below as a template to integrate SwanLab to your Python script:
```Python
```python
import swanlab

# Start a SwanLab Run with swanlab.init
Expand All @@ -120,9 +128,28 @@ That's it! Open http://127.0.0.1:5092 to view a dashboard of your first SwanLab

<br>

## Use Case

Learn how to use SwanLab more effectively by following these use cases:

| Code Cases | Description |
| ------- | ------- |
| [Hello World](https://github.com/SwanHubX/SwanLab-examples/tree/main/Hello_World) | Getting Started |
| [MNIST](https://github.com/SwanHubX/SwanLab-examples/tree/main/MNIST) | Handwriting recognition based on a plain net and MNIST dataset with pytroch, swanlab. |
| [Image Classification](https://github.com/SwanHubX/SwanLab-examples/blob/main/Resnet50) | Cat and dog classification based on ResNet50 with pytorch, swanlab and gradio. [Tutorial](https://zhuanlan.zhihu.com/p/676430630). |
| [Text Generation](https://github.com/SwanHubX/SwanLab-examples/blob/main/Word_language_model) | Text generation based on Word_language_model (RNN/LSTM/GRU/Transformer) |

<br>

## More Tips

- Set a log directory save path and run the Dashboard using it:
🏄‍♀️ Experiment Dashboard:

<details>
<summary>Set a log directory save path and run the Dashboard using it</summary>

Set a log directory save path, such as `./logs`:

```python
import swanlab

Expand All @@ -131,15 +158,37 @@ swanlab.init(
)
```

Run the Dashboard using it:

```bash
$ swanlab watch --logdir ./logs_path
```

- Set the Host and Port for the Dashboard:
</details>

<details>
<summary>Set the Host and Port for the Dashboard</summary>

```bash
$ swanlab watch --host 0.0.0.0 --port 8080
```
- Use Argparse init swanlab.config:

</details>

<details>
<summary>Remotely access Dashboard</summary>

- Link:[Access the SwanLab Dashboard While Training on a Remote Server](https://zhuanlan.zhihu.com/p/677224865)

</details>

⚙️ Other:

<details>
<summary>argparse init swanlab.config</summary>

`swanlab.config` supports directly passing variables of type `argparse.Namespace`, such as:

```python
import argparse
import swanlab
Expand All @@ -150,11 +199,12 @@ args = parser.parse_args()


swanlab.init(
config=vars(args)
config=args
)
```

- [Remotely access Dashboard](https://zhuanlan.zhihu.com/p/677224865): Access the SwanLab Dashboard While Training on a Remote Server.
</details>


<br>

Expand Down
83 changes: 63 additions & 20 deletions README_zh-hans.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,9 @@

- [核心功能](#核心功能)
- [更新日志](#更新日志)
- [案例](#案例)
- [安装](#安装)
- [快速开始](#快速开始)
- [案例](#案例)
- [更多技巧](#更多技巧)
- [协议](#协议)

Expand Down Expand Up @@ -57,32 +58,37 @@
- **🥔 离线支持**: SwanLab可以完全离线运行,无需任何对互联网的访问。例如,它可以在您的本地计算机上、企业防火墙后面或数据中心中运行。

> [!NOTE]
> See the SwanLab <a href="https://geektechstudio.feishu.cn/wiki/MwXmw9yDeiZWyQkPnNgcixwWnwu">Documentaion</a> and <a href="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/SwanHubX/SwanLab-examples">Examples</a> for a full description of the SwanLab.
> 请查看SwanLab的<a href="https://geektechstudio.feishu.cn/wiki/MwXmw9yDeiZWyQkPnNgcixwWnwu">文档</a><a href="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/SwanHubX/SwanLab-examples">示例</a>,以获取有关SwanLab的完整介绍。

<br>

## 更新日志

[24/01/23] 🚨 我们使用SQLite数据库和Peewee库替代了之前的基础配置信息读写方案([#114](https://github.com/SwanHubX/SwanLab/issues/114)),这是个极大有利于项目未来的改动,但缺陷是不兼容旧版本(swanlab<=v0.1.4)的日志数据文件,所以如需可视化旧版本产生的日志文件请安装<=0.1.4的版本。与此同时,我们增加了支持导出实验列表为CSV,新的环境记录项`Run path`和`logdir`,增加了快捷复制的交互,以及新的API `swanlab.config`。

[24/01/14] 🔥 我们发布了一个新的UI界面,以及跟踪更多环境信息,包括Command、git提交/分支、和机器内存。此外,我们还添加了一个`logdir` API,允许开发人员设置日志文件的目录。

[24/01/07] ✨ 我们支持在仪表板上删除实验和编辑实验信息。

[24/01/01] 我们修复了一些错误,使SwanLab更加稳定。

[完整更新日志](https://github.com/SwanHubX/SwanLab/releases)

<br>

## 案例
## 安装

通过以下用例学习如何更有效地使用SwanLab:
### pip安装

| 案例 | 介绍 |
| ------- | ------- |
| [Hello World](https://github.com/SwanHubX/SwanLab-examples/tree/main/Hello_World) | 简单入门 |
| [MNIST](https://github.com/SwanHubX/SwanLab-examples/tree/main/MNIST) | 基于神经网络的MNIST手写体识别(使用pytorch、swanlab库) |
| [图像分类](https://github.com/SwanHubX/SwanLab-examples/blob/main/Resnet50) | ResNet50猫狗分类(使用pytorch、swanlab、Gradio库) [图文教程](https://zhuanlan.zhihu.com/p/676430630) |
| [文本生成](https://github.com/SwanHubX/SwanLab-examples/blob/main/Word_language_model) | 基于自然语言模型的文本生成 (RNN/LSTM/GRU/Transformer) |
环境要求:Python 3.8+

使用[pip](https://pip.pypa.io/en/stable/)将安装我们稳定发布的版本,安装命令如下所示:

```bash
pip install -U swanlab
```

### 源码安装

如果您等不及发布,想体验最新的代码与特性,那么必须[从源代码安装此库](https://geektechstudio.feishu.cn/wiki/DvxSweHUKiAe8yksci3cMflbnwh#SMXHdJ1c1o4jzTxcDticHcwvnHd)。

<br>

Expand All @@ -95,7 +101,7 @@ pip install -U swanlab
```

2. 然后,使用下面的示例代码片段作为模板,将SwanLab集成到您的Python脚本中:
```Python
```python
import swanlab

# Start a SwanLab Run with swanlab.init
Expand All @@ -120,9 +126,28 @@ $ swanlab watch

<br>

## 案例

通过以下用例学习如何更有效地使用SwanLab:

| 案例 | 介绍 |
| ------- | ------- |
| [Hello World](https://github.com/SwanHubX/SwanLab-examples/tree/main/Hello_World) | 简单入门 |
| [MNIST](https://github.com/SwanHubX/SwanLab-examples/tree/main/MNIST) | 基于神经网络的MNIST手写体识别(使用pytorch、swanlab库) |
| [图像分类](https://github.com/SwanHubX/SwanLab-examples/blob/main/Resnet50) | ResNet50猫狗分类(使用pytorch、swanlab、Gradio库) [图文教程](https://zhuanlan.zhihu.com/p/676430630) |
| [文本生成](https://github.com/SwanHubX/SwanLab-examples/blob/main/Word_language_model) | 基于自然语言模型的文本生成 (RNN/LSTM/GRU/Transformer) |

<br>

## 更多技巧

- 设置1个日志文件保存路径,以及运行基于它的实验看板:
🏄‍♀️ 实验看板相关:

<details>
<summary>设置日志文件保存路径,并运行基于它的实验看板</summary>

设置日志文件保存路径,比如`./logs`:

```python
import swanlab

Expand All @@ -131,16 +156,35 @@ swanlab.init(
)
```

运行基于`./logs`的看实验看板:

```bash
$ swanlab watch --logdir ./logs_path
$ swanlab watch --logdir ./logs
```
</details>


<details>
<summary>设置实验看板的主机和端口</summary>

- 设置实验看板的主机和端口:
```bash
$ swanlab watch --host 0.0.0.0 --port 8080
```
</details>

<details>
<summary>远程访问实验看板</summary>

- 使用Argparse初始化swanlab.config:
- 链接:[在远程服务器上进行训练时,访问SwanLab实验看板](https://zhuanlan.zhihu.com/p/677224865)

</details>

⚙️ 其他:

<details>
<summary>argparse与swanlab.config</summary>

swanlab.config支持直接传入argparse.Namespace类型的变量,如:
```python
import argparse
import swanlab
Expand All @@ -151,11 +195,10 @@ args = parser.parse_args()


swanlab.init(
config=vars(args)
config=args
)
```

- [远程访问实验看板](https://zhuanlan.zhihu.com/p/677224865): 在远程服务器上进行训练时,访问SwanLab实验看板。
</details>

<br>

Expand Down
14 changes: 9 additions & 5 deletions swanlab/cli/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@

import click
from .utils import is_valid_ip, is_valid_port, is_valid_root_dir, URL
from ..utils import FONT
from ..utils import FONT, version_limit
from ..env import get_server_host, get_server_port, get_swanlog_dir
import time
from ..db import connect
Expand Down Expand Up @@ -66,12 +66,16 @@ def watch(log_level: str, **kwargs):
from ..server import app
import uvicorn

# 在此处已经完成了路径的检查,所以可以直接导入
connect()
log_dir = get_swanlog_dir()
print(log_dir)
version_limit(log_dir, mode="watch")

# debug一下当前日志文件夹的位置
swl.debug("Try to explore the swanlab experiment logs in: " + FONT.bold(get_swanlog_dir()))

swl.debug("Try to explore the swanlab experiment logs in: " + FONT.bold(log_dir))
try:
connect()
except FileNotFoundError:
swl.error("Can not find the swanlab db in: " + FONT.bold(log_dir))
# ---------------------------------- 日志等级处理 ----------------------------------
swl.setLevel(log_level)
# ---------------------------------- 服务地址处理 ----------------------------------
Expand Down
1 change: 1 addition & 0 deletions swanlab/cli/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
import click
from ..utils import FONT, file
from ..env import PORT, HOST, ROOT
import json


def is_valid_ip(ctx, param, ip: str) -> tuple:
Expand Down
8 changes: 4 additions & 4 deletions swanlab/data/run/exp.py
Original file line number Diff line number Diff line change
Expand Up @@ -83,10 +83,10 @@ def add(self, key: str, data: DataType, step: int = None):
"""
由于添加图表的同时会尝试转换data的类型,但这在注入step之前
所以此处需要手动注入依赖
关于step,如果step格式不正确,会在tag_obj.add的时候被拦截,此处如果格式不正确直接设置为1即可
关于step,如果step格式不正确,会在tag_obj.add的时候被拦截,此处如果格式不正确直接设置为0即可
"""
if isinstance(data, BaseType):
data.step = 1 if step is None or not isinstance(step, int) else step
data.step = 0 if step is None or not isinstance(step, int) else step
data.tag = tag_obj.tag
tag_obj.create_chart(tag, data)
# 检查tag创建时图表是否创建成功,如果失败则也没有写入数据的必要了,直接退出
Expand Down Expand Up @@ -154,9 +154,9 @@ def add(self, data: DataType, step: int = None):
if step is not None and not isinstance(step, int):
swanlog.warning(f"Step {step} is not int, SwanLab will set it automatically.")
step = None
# 转换step,如果step为None,则改为合适的长度,目前step从1开始
# 转换step,如果step为None,则改为合适的长度,目前step从0开始
if step is None:
step = len(self.__steps) + 1
step = len(self.__steps)
# 如果step已经存在,打印警告并退出
if step in self.__steps:
return swanlog.warning(f"Step {step} on tag {self.tag} already exists, ignored.")
Expand Down
6 changes: 4 additions & 2 deletions swanlab/data/sdk.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
from ..env import init_env, ROOT
from .utils.file import check_dir_and_create, formate_abs_path
from ..db import Project, connect
from ..utils import version_limit


run: Optional["SwanLabRun"] = None
Expand Down Expand Up @@ -82,7 +83,7 @@ def init(
except IOError:
raise IOError("logdir must be a path and have Write permission.")
os.environ[ROOT] = logdir
# 如果没有传入logdir,则使用默认的logdir,即当前工作目录,但是需要保证目录存在
# 如果没有传入logdir,则使用默认的logdir, 即当前工作目录,但是需要保证目录存在
else:
logdir = os.path.abspath("swanlog")
try:
Expand All @@ -91,7 +92,8 @@ def init(
raise IOError
except:
raise IOError("logdir must have Write permission.")

# 检查logdir内文件的版本,如果<=0.1.4则报错
version_limit(logdir, mode="init")
# 初始化环境变量
init_env()
# 连接数据库,要求路径必须存在,但是如果数据库文件不存在,会自动创建
Expand Down
2 changes: 1 addition & 1 deletion swanlab/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "swanlab",
"version": "0.1.4",
"version": "0.1.5",
"description": "",
"python": "true"
}
Loading