Skip to content

Commit 39dea84

Browse files
SyakyrSyakyr
authored
Merge pull request #76 from aisingapore/dev
0.5.3 release
2 parents a18a0af + 1b7da90 commit 39dea84

File tree

18 files changed

+129
-158
lines changed

18 files changed

+129
-158
lines changed

‎{{cookiecutter.repo_name}}/.gitlab-ci.yml

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -217,9 +217,9 @@ build:cpu-image:
217217
script:
218218
- mkdir -p /kaniko/.docker
219219
{%- if cookiecutter.platform == 'onprem' %}
220-
- cat $HARBOR_ROBOT_CREDS_JSON > /kaniko/.docker/config.json
220+
- echo $HARBOR_ROBOT_CREDS_JSON > /kaniko/.docker/config.json
221221
{%- elif cookiecutter.platform == 'gcp' %}
222-
- cat $GCP_SERVICE_ACCOUNT_KEY > /kaniko/.docker/config.json
222+
- echo $GCP_SERVICE_ACCOUNT_KEY > /kaniko/.docker/config.json
223223
{%- endif %}
224224
- >-
225225
/kaniko/executor
@@ -259,9 +259,9 @@ build:gpu-image:
259259
script:
260260
- mkdir -p /kaniko/.docker
261261
{%- if cookiecutter.platform == 'onprem' %}
262-
- cat $HARBOR_ROBOT_CREDS_JSON > /kaniko/.docker/config.json
262+
- echo $HARBOR_ROBOT_CREDS_JSON > /kaniko/.docker/config.json
263263
{%- elif cookiecutter.platform == 'gcp' %}
264-
- cat $GCP_SERVICE_ACCOUNT_KEY > /kaniko/.docker/config.json
264+
- echo $GCP_SERVICE_ACCOUNT_KEY > /kaniko/.docker/config.json
265265
{%- endif %}
266266
- >-
267267
/kaniko/executor
@@ -302,11 +302,11 @@ build:retag-images:
302302
{%- endif %}
303303
script:
304304
{%- if cookiecutter.platform == 'onprem' %}
305-
- cat $HARBOR_ROBOT_CREDS_JSON > /root/.docker/config.json
305+
- echo $HARBOR_ROBOT_CREDS_JSON > /root/.docker/config.json
306306
- crane tag {{cookiecutter.registry_project_path}}/cpu:${CI_COMMIT_SHORT_SHA} ${CI_COMMIT_TAG}
307307
- crane tag {{cookiecutter.registry_project_path}}/gpu:${CI_COMMIT_SHORT_SHA} ${CI_COMMIT_TAG}
308308
{%- elif cookiecutter.platform == 'gcp' %}
309-
- cat $GCP_SERVICE_ACCOUNT_KEY > /gcp-sa.json
309+
- echo $GCP_SERVICE_ACCOUNT_KEY > /gcp-sa.json
310310
- gcloud container images add-tag "{{cookiecutter.registry_project_path}}/cpu:${CI_COMMIT_SHORT_SHA}" "{{cookiecutter.registry_project_path}}/data-prep:${CI_COMMIT_TAG}"
311311
- gcloud container images add-tag "{{cookiecutter.registry_project_path}}/gpu:${CI_COMMIT_SHORT_SHA}" "{{cookiecutter.registry_project_path}}/model-training:${CI_COMMIT_TAG}"
312312
{%- endif %}

‎{{cookiecutter.repo_name}}/aisg-context/guide-site/docs/appendix/09-cicd.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -117,7 +117,7 @@ so:
117117
```
118118

119119
Using the output from above, copy and paste the following content
120-
into a CI/CD environment variable of type `File`
120+
into a CI/CD environment variable of type `variable`
121121
(under `Settings` -> `CI/CD` -> `Variables` -> `Add variable`):
122122

123123
```json
@@ -410,9 +410,9 @@ that builds a Docker image:
410410
script:
411411
- mkdir -p /kaniko/.docker
412412
{%- if cookiecutter.platform == 'onprem' %}
413-
- cat $HARBOR_ROBOT_CREDS_JSON > /kaniko/.docker/config.json
413+
- echo $HARBOR_ROBOT_CREDS_JSON > /kaniko/.docker/config.json
414414
{%- elif cookiecutter.platform == 'gcp' %}
415-
- cat $GCP_SERVICE_ACCOUNT_KEY > /kaniko/.docker/config.json
415+
- echo $GCP_SERVICE_ACCOUNT_KEY > /kaniko/.docker/config.json
416416
{%- endif %}
417417
- >-
418418
/kaniko/executor
@@ -531,11 +531,11 @@ the default branch before this.
531531
{%- endif %}
532532
script:
533533
{%- if cookiecutter.platform == 'onprem' %}
534-
- cat $HARBOR_ROBOT_CREDS_JSON > /root/.docker/config.json
534+
- echo $HARBOR_ROBOT_CREDS_JSON > /root/.docker/config.json
535535
- crane tag {{cookiecutter.registry_project_path}}/cpu:${CI_COMMIT_SHORT_SHA} ${CI_COMMIT_TAG}
536536
- crane tag {{cookiecutter.registry_project_path}}/gpu:${CI_COMMIT_SHORT_SHA} ${CI_COMMIT_TAG}
537537
{%- elif cookiecutter.platform == 'gcp' %}
538-
- cat $GCP_SERVICE_ACCOUNT_KEY > /gcp-sa.json
538+
- echo $GCP_SERVICE_ACCOUNT_KEY > /gcp-sa.json
539539
- gcloud container images add-tag "{{cookiecutter.registry_project_path}}/cpu:${CI_COMMIT_SHORT_SHA}" "{{cookiecutter.registry_project_path}}/cpu:${CI_COMMIT_TAG}"
540540
- gcloud container images add-tag "{{cookiecutter.registry_project_path}}/gpu:${CI_COMMIT_SHORT_SHA}" "{{cookiecutter.registry_project_path}}/gpu:${CI_COMMIT_TAG}"
541541
{%- endif %}

‎{{cookiecutter.repo_name}}/aisg-context/guide-site/docs/appendix/assets/screenshots/gitlab-settings-cicd-set-file-var.png

20.6 KB
Loading

‎{{cookiecutter.repo_name}}/aisg-context/guide-site/docs/docker/07b-deployment.md

Lines changed: 7 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -125,12 +125,7 @@ that exists within the repository.
125125

126126
### Running the API Server
127127

128-
Run the FastAPI server using [Gunicorn](https://gunicorn.org)
129-
(for Linux/macOS) or [`uvicorn`][uvicorn] (for Windows):
130-
131-
!!! attention
132-
Gunicorn is only executable on UNIX-based or UNIX-like systems;
133-
this method would not be possible/applicable for Windows machines.
128+
Run the FastAPI server using [`uvicorn`][uvicorn]:
134129

135130
=== "Linux"
136131

@@ -147,9 +142,8 @@ Run the FastAPI server using [Gunicorn](https://gunicorn.org)
147142
-v ./models:/home/aisg/{{cookiecutter.repo_name}}/models \
148143
-w /home/aisg/{{cookiecutter.repo_name}}/src \
149144
{{cookiecutter.registry_project_path}}/gpu:0.1.0 \
150-
gunicorn {{cookiecutter.src_package_name}}_fastapi.main:APP \
151-
-k uvicorn.workers.UvicornWorker \
152-
-b 0.0.0.0:8080 -w 2 -t 90
145+
uvicorn {{cookiecutter.src_package_name}}_fastapi.main:APP \
146+
--host 0.0.0.0 --port 8080 --workers 2 --app-dir src
153147
```
154148

155149
=== "macOS"
@@ -160,9 +154,8 @@ Run the FastAPI server using [Gunicorn](https://gunicorn.org)
160154
-v ./models:/home/aisg/{{cookiecutter.repo_name}}/models \
161155
-w /home/aisg/{{cookiecutter.repo_name}}/src \
162156
{{cookiecutter.registry_project_path}}/gpu:0.1.0 \
163-
gunicorn {{cookiecutter.src_package_name}}_fastapi.main:APP \
164-
-k uvicorn.workers.UvicornWorker \
165-
-b 0.0.0.0:8080 -w 2 -t 90
157+
uvicorn {{cookiecutter.src_package_name}}_fastapi.main:APP \
158+
--host 0.0.0.0 --port 8080 --workers 2 --app-dir src
166159
```
167160

168161
=== "Windows PowerShell"
@@ -181,9 +174,8 @@ Run the FastAPI server using [Gunicorn](https://gunicorn.org)
181174
-v .\models:/home/aisg/{{cookiecutter.repo_name}}/models `
182175
-w /home/aisg/{{cookiecutter.repo_name}}/src `
183176
{{cookiecutter.registry_project_path}}/gpu:0.1.0 `
184-
gunicorn {{cookiecutter.src_package_name}}_fastapi.main:APP `
185-
-k uvicorn.workers.UvicornWorker `
186-
-b 0.0.0.0:8080 -w 2 -t 90
177+
uvicorn {{cookiecutter.src_package_name}}_fastapi.main:APP \
178+
--host 0.0.0.0 --port 8080 --workers 2 --app-dir src
187179
```
188180

189181
And with that, our document site for our server is viewable through
@@ -234,7 +226,6 @@ With the returned JSON object, we have successfully submitted a request
234226
to the FastAPI server and it returned predictions as part of the
235227
response.
236228

237-
[reason]: https://fastapi.tiangolo.com/deployment/server-workers/
238229
[uvicorn]: https://www.uvicorn.org/
239230
[rocm-wsl]: https://rocm.docs.amd.com/projects/radeon/en/latest/docs/install/wsl/howto_wsl.html
240231

‎{{cookiecutter.repo_name}}/aisg-context/guide-site/docs/local/07a-deployment.md

Lines changed: 5 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -104,33 +104,22 @@ that exists within the repository.
104104

105105
### Running the API Server
106106

107-
Run the FastAPI server using [Gunicorn](https://gunicorn.org)
108-
(for Linux/macOS) or [`uvicorn`][uvicorn] (for Windows):
109-
110-
!!! attention
111-
Gunicorn is only executable on UNIX-based or UNIX-like systems;
112-
this method would not be possible/applicable for Windows machines.
107+
Run the FastAPI server using [`uvicorn`][uvicorn]:
113108

114109
=== "Linux/macOS"
115110

116111
```bash
117112
conda activate {{cookiecutter.repo_name}}
118-
gunicorn {{cookiecutter.src_package_name}}_fastapi.main:APP \
119-
-k uvicorn.workers.UvicornWorker \
120-
-b 0.0.0.0:8080 -w 2 -t 90 --chdir src
113+
uvicorn {{cookiecutter.src_package_name}}_fastapi.main:APP \
114+
--host 0.0.0.0 --port 8080 --workers 2 --app-dir src
121115
```
122116

123-
!!! info
124-
See [here][reason] as to why Gunicorn is to be used instead of
125-
just [Uvicorn][uvicorn]. TLDR: Gunicorn is needed to spin up
126-
multiple processes/workers to handle more requests i.e. better
127-
for the sake of production needs.
128-
129117
=== "Windows PowerShell"
130118

131119
```powershell
132120
conda activate {{cookiecutter.repo_name}}
133-
uvicorn {{cookiecutter.src_package_name}}_fastapi.main:APP --app-dir src
121+
uvicorn {{cookiecutter.src_package_name}}_fastapi.main:APP `
122+
--host 0.0.0.0 --port 8080 --workers 2 --app-dir src
134123
```
135124

136125
And with that, our document site for our server is viewable through
@@ -169,7 +158,6 @@ With the returned JSON object, we have successfully submitted a request
169158
to the FastAPI server and it returned predictions as part of the
170159
response.
171160

172-
[reason]: https://fastapi.tiangolo.com/deployment/server-workers/
173161
[uvicorn]: https://www.uvicorn.org/
174162

175163
### Pydantic Settings

‎{{cookiecutter.repo_name}}/aisg-context/guide-site/docs/runai/07c-deployment.md

Lines changed: 5 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -124,17 +124,16 @@ that exists within the repository.
124124

125125
### Running the API Server
126126

127-
Run the FastAPI server using [Gunicorn](https://gunicorn.org):
127+
Run the FastAPI server using [`uvicorn`][uvicorn]:
128128

129129
=== "Coder Workspace Terminal"
130130

131131
```bash
132132
# Running in a working `{{cookiecutter.repo_name}}` repository
133133
conda activate {{cookiecutter.repo_name}}
134134
export MODEL_UUID=<MLFLOW_RUN_UUID>
135-
gunicorn {{cookiecutter.src_package_name}}_fastapi.main:APP \
136-
-k uvicorn.workers.UvicornWorker \
137-
-b 0.0.0.0:8080 -w 2 -t 90 --chdir src
135+
uvicorn {{cookiecutter.src_package_name}}_fastapi.main:APP \
136+
--host 0.0.0.0 --port 8080 --workers 2 --app-dir src
138137
```
139138

140139
And with that, the link to our document site for our server would
@@ -153,9 +152,8 @@ Run the FastAPI server using [Gunicorn](https://gunicorn.org):
153152
--existing-pvc claimname=<NAME_OF_DATA_SOURCE>,path=/<NAME_OF_DATA_SOURCE> \
154153
--cpu 2 --cpu-limit 2 --memory 4G --memory-limit 4G --backoff-limit 1 \
155154
--service-type external-url,port=8080:8080 \
156-
--command -- gunicorn {{cookiecutter.src_package_name}}_fastapi.main:APP \
157-
-k uvicorn.workers.UvicornWorker \
158-
-b 0.0.0.0:8080 -w 2 -t 90
155+
--command -- uvicorn {{cookiecutter.src_package_name}}_fastapi.main:APP \
156+
--host 0.0.0.0 --port 8080 --workers 2 --app-dir src
159157
```
160158

161159
And with that, the link to our document site for our server would
@@ -184,7 +182,6 @@ With the returned JSON object, we have successfully submitted a request
184182
to the FastAPI server and it returned predictions as part of the
185183
response.
186184

187-
[reason]: https://fastapi.tiangolo.com/deployment/server-workers/
188185
[uvicorn]: https://www.uvicorn.org/
189186

190187
### Pydantic Settings

‎{{cookiecutter.repo_name}}/problem-templates/cv/aisg-context/guide-site/docs/docker/07b-deployment.md.diff

Lines changed: 15 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -56,37 +56,37 @@
5656
[beginner tutorials]: https://fastapi.tiangolo.com/tutorial/
5757

5858
### Running the API Server
59-
@@ -145,8 +142,9 @@
59+
@@ -140,8 +137,9 @@
6060
docker run --rm \
6161
-p 8080:8080 \
6262
-v ./models:/home/aisg/{{cookiecutter.repo_name}}/models \
6363
-w /home/aisg/{{cookiecutter.repo_name}}/src \
6464
+ -e PRED_MODEL_PATH="../models/$PRED_MODEL_UUID/model/model.pt" \
6565
{{cookiecutter.registry_project_path}}/gpu:0.1.0 \
66-
gunicorn {{cookiecutter.src_package_name}}_fastapi.main:APP \
67-
-k uvicorn.workers.UvicornWorker \
68-
-b 0.0.0.0:8080 -w 2 -t 90
69-
@@ -158,8 +156,9 @@
66+
uvicorn {{cookiecutter.src_package_name}}_fastapi.main:APP \
67+
--host 0.0.0.0 --port 8080 --workers 2 --app-dir src
68+
```
69+
@@ -152,8 +150,9 @@
7070
docker run --rm \
7171
-p 8080:8080 \
7272
-v ./models:/home/aisg/{{cookiecutter.repo_name}}/models \
7373
-w /home/aisg/{{cookiecutter.repo_name}}/src \
7474
+ -e PRED_MODEL_PATH="../models/$PRED_MODEL_UUID/model/model.pt" \
7575
{{cookiecutter.registry_project_path}}/gpu:0.1.0 \
76-
gunicorn {{cookiecutter.src_package_name}}_fastapi.main:APP \
77-
-k uvicorn.workers.UvicornWorker \
78-
-b 0.0.0.0:8080 -w 2 -t 90
79-
@@ -179,8 +178,9 @@
76+
uvicorn {{cookiecutter.src_package_name}}_fastapi.main:APP \
77+
--host 0.0.0.0 --port 8080 --workers 2 --app-dir src
78+
```
79+
@@ -172,8 +171,9 @@
8080
docker run --rm `
8181
-p 8080:8080 `
8282
-v .\models:/home/aisg/{{cookiecutter.repo_name}}/models `
8383
-w /home/aisg/{{cookiecutter.repo_name}}/src `
8484
+ -e PRED_MODEL_PATH="../models/$Env:PRED_MODEL_UUID/model/model.pt" `
8585
{{cookiecutter.registry_project_path}}/gpu:0.1.0 `
86-
gunicorn {{cookiecutter.src_package_name}}_fastapi.main:APP `
87-
-k uvicorn.workers.UvicornWorker `
88-
-b 0.0.0.0:8080 -w 2 -t 90
89-
@@ -196,39 +196,39 @@
86+
uvicorn {{cookiecutter.src_package_name}}_fastapi.main:APP \
87+
--host 0.0.0.0 --port 8080 --workers 2 --app-dir src
88+
```
89+
@@ -188,39 +188,39 @@
9090

9191
=== "Linux"
9292

@@ -136,7 +136,7 @@
136136

137137
With the returned JSON object, we have successfully submitted a request
138138
to the FastAPI server and it returned predictions as part of the
139-
@@ -245,9 +245,10 @@
139+
@@ -236,9 +236,10 @@
140140
data and schema validation, as well as [settings management]. There's a
141141
class called `Settings` under the module
142142
`src/{{cookiecutter.src_package_name}}_fastapi/config.py`. This class
@@ -148,7 +148,7 @@
148148
`src/{{cookiecutter.src_package_name}}_fastapi/config.py`:
149149
```python
150150
...
151-
@@ -256,9 +257,12 @@
151+
@@ -247,9 +248,12 @@
152152
API_NAME: str = "{{cookiecutter.src_package_name}}_fastapi"
153153
API_V1_STR: str = "/api/v1"
154154
LOGGER_CONFIG_PATH: str = "../conf/logging.yaml"

‎{{cookiecutter.repo_name}}/problem-templates/cv/aisg-context/guide-site/docs/local/07a-deployment.md.diff

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,7 @@
5757
[beginner tutorials]: https://fastapi.tiangolo.com/tutorial/
5858

5959
### Running the API Server
60-
@@ -145,25 +160,25 @@
60+
@@ -134,25 +149,25 @@
6161

6262
```bash
6363
curl -X POST \
@@ -88,7 +88,7 @@
8888

8989
With the returned JSON object, we have successfully submitted a request
9090
to the FastAPI server and it returned predictions as part of the
91-
@@ -179,9 +194,10 @@
91+
@@ -167,9 +182,10 @@
9292
data and schema validation, as well as [settings management]. There's a
9393
class called `Settings` under the module
9494
`src/{{cookiecutter.src_package_name}}_fastapi/config.py`. This class
@@ -100,7 +100,7 @@
100100
`src/{{cookiecutter.src_package_name}}_fastapi/config.py`:
101101
```python
102102
...
103-
@@ -190,9 +206,12 @@
103+
@@ -178,9 +194,12 @@
104104
API_NAME: str = "{{cookiecutter.src_package_name}}_fastapi"
105105
API_V1_STR: str = "/api/v1"
106106
LOGGER_CONFIG_PATH: str = "../conf/logging.yaml"

‎{{cookiecutter.repo_name}}/problem-templates/cv/aisg-context/guide-site/docs/runai/07c-deployment.md.diff

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -58,11 +58,11 @@
5858
- export MODEL_UUID=<MLFLOW_RUN_UUID>
5959
+ export PRED_MODEL_UUID=<MLFLOW_RUN_UUID>
6060
+ export PRED_MODEL_PATH="<NAME_OF_DATA_SOURCE>/workspaces/<YOUR_HYPENATED_NAME>/models/$PRED_MODEL_UUID/model/model.pt"
61-
gunicorn {{cookiecutter.src_package_name}}_fastapi.main:APP \
62-
-k uvicorn.workers.UvicornWorker \
63-
-b 0.0.0.0:8080 -w 2 -t 90 --chdir src
61+
uvicorn {{cookiecutter.src_package_name}}_fastapi.main:APP \
62+
--host 0.0.0.0 --port 8080 --workers 2 --app-dir src
6463
```
65-
@@ -144,15 +148,16 @@
64+
65+
@@ -143,15 +147,16 @@
6666

6767
=== "Using Run:ai"
6868

@@ -77,10 +77,10 @@
7777
--cpu 2 --cpu-limit 2 --memory 4G --memory-limit 4G --backoff-limit 1 \
7878
+ -e PRED_MODEL_PATH=/<NAME_OF_DATA_SOURCE>/workspaces/<YOUR_HYPENATED_NAME>/models/$PRED_MODEL_UUID/model/model.pt \
7979
--service-type external-url,port=8080:8080 \
80-
--command -- gunicorn {{cookiecutter.src_package_name}}_fastapi.main:APP \
81-
-k uvicorn.workers.UvicornWorker \
82-
-b 0.0.0.0:8080 -w 2 -t 90
83-
@@ -169,16 +174,16 @@
80+
--command -- uvicorn {{cookiecutter.src_package_name}}_fastapi.main:APP \
81+
--host 0.0.0.0 --port 8080 --workers 2 --app-dir src
82+
```
83+
@@ -167,16 +172,16 @@
8484

8585
```bash
8686
curl -X POST \
@@ -100,7 +100,7 @@
100100

101101
With the returned JSON object, we have successfully submitted a request
102102
to the FastAPI server and it returned predictions as part of the
103-
@@ -194,9 +199,10 @@
103+
@@ -191,9 +196,10 @@
104104
data and schema validation, as well as [settings management]. There's a
105105
class called `Settings` under the module
106106
`src/{{cookiecutter.src_package_name}}_fastapi/config.py`. This class
@@ -112,7 +112,7 @@
112112
`src/{{cookiecutter.src_package_name}}_fastapi/config.py`:
113113
```python
114114
...
115-
@@ -205,9 +211,12 @@
115+
@@ -202,9 +208,12 @@
116116
API_NAME: str = "{{cookiecutter.src_package_name}}_fastapi"
117117
API_V1_STR: str = "/api/v1"
118118
LOGGER_CONFIG_PATH: str = "../conf/logging.yaml"

0 commit comments

Comments
 (0)