Skip to content

Conversation

SAKURA-CAT
Copy link
Member

@SAKURA-CAT SAKURA-CAT commented Feb 13, 2025

这个拉取请求包含对 swanlab/api/cos.py 文件的重大更新,主要集中在将云对象存储操作从腾讯云 COS SDK 迁移到 AWS Boto3 SDK。最重要的更改包括更新导入、修改初始化方法以及重构上传方法。

迁移到 AWS Boto3 SDK:

  • swanlab/api/cos.py:将腾讯云 COS SDK 的导入(CosConfigCosS3ClientSimpleThreadPool)替换为 AWS Boto3 SDK 的导入(boto3BotocoreConfigThreadPoolExecutor)。

初始化方法更新:

  • swanlab/api/cos.py:更新了 __init__ 方法,使用 Boto3 初始化 S3 客户端,并配置了适当的凭证和配置。

重构上传方法:

  • swanlab/api/cos.py:修改了 upload 方法,移除了 EnableMD5 参数,因为 Boto3 不支持该参数。
  • swanlab/api/cos.py:重构了 upload_files 方法,使用 ThreadPoolExecutor 进行并发上传,而不是 SimpleThreadPool

其他改进:

  • swanlab/api/cos.py:在 should_refresh 属性中添加了一个 FIXME 注释,建议使用时区感知对象来表示 UTC 时间。

close #767

@SAKURA-CAT SAKURA-CAT self-assigned this Feb 13, 2025
@SAKURA-CAT SAKURA-CAT added 🐛 bug Something isn't working 💪 enhancement New feature or request labels Feb 13, 2025
@SAKURA-CAT SAKURA-CAT marked this pull request as ready for review March 3, 2025 04:26
@SAKURA-CAT SAKURA-CAT changed the base branch from main to test/self-hosted March 3, 2025 05:52
@SAKURA-CAT SAKURA-CAT requested a review from Feudalman March 3, 2025 05:53
@SAKURA-CAT SAKURA-CAT merged commit 067740b into test/self-hosted Mar 3, 2025
5 checks passed
@SAKURA-CAT SAKURA-CAT deleted the feat/s3 branch March 3, 2025 05:56
SAKURA-CAT added a commit that referenced this pull request Mar 11, 2025
* feat: cloud only and subcontracting (#843)

* feat: cloud only and subcontracting

* fix: some comments and test

* delete some tips

for #777

* fix: watch tip

* fix: comment

* fix: mkdir

* callback -> callbacks

* fix: swanlog

---------

Co-authored-by: ZeYi Lin <[email protected]>

* feat: s3 sdk (#821)

* feat: s3 sdk

delete cos sdk for #767

* fix: thread pool

* chore: fit minio

* fix: console recorder not enabled in cloud mode (#849)

* fix: host formatter error

* fix: repeated login

* fix: get http err

* feat: swanlab.login default not save key

* chore: fit cos data structure

* fix: delete task run time

* fix: delete local print in cloud mode

---------

Co-authored-by: ZeYi Lin <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

🐛 bug Something isn't working 💪 enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[ADVICE] 统一对象存储依赖

1 participant