Skip to content

Commit 0bb9bc4

Browse files
committed
don't allow accessing /models during sleep, it is not thread-safe
1 parent fd09f88 commit 0bb9bc4

File tree

2 files changed

+3
-2
lines changed

2 files changed

+3
-2
lines changed

tools/server/README.md

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1630,7 +1630,6 @@ When the server enters sleep mode, the model and its associated memory (includin
16301630
Note that the following endpoints are exempt from being considered as incoming tasks. They do not trigger model reloading and do not reset the idle timer:
16311631
- `GET /health`
16321632
- `GET /props`
1633-
- `GET /models` and `GET /v1/models`
16341633

16351634
## More examples
16361635

tools/server/server-context.cpp

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3436,8 +3436,10 @@ void server_routes::init_routes() {
34363436
return res;
34373437
};
34383438

3439+
// TODO: this endpoint is unsafe to access during model reloading (i.e. wake up from sleeping)
3440+
// how to make it work even during load_model()?
34393441
this->get_models = [this](const server_http_req &) {
3440-
auto res = std::make_unique<server_res_generator>(ctx_server, true);
3442+
auto res = std::make_unique<server_res_generator>(ctx_server);
34413443
json model_meta = nullptr;
34423444
if (is_ready()) {
34433445
model_meta = ctx_server.json_server_model_meta;

0 commit comments

Comments
 (0)