@@ -11,15 +11,239 @@ We will run through some examples as shown in the notebook `service-meshes/traef
11
11
* Traefik IngressRoute
12
12
* Traefik Middleware for adding a header
13
13
14
- ``` {literalinclude} ../../../../../../service-meshes/traefik/static/single-model.yaml
15
- :language: yaml
14
+ ``` yaml
15
+ apiVersion : v1
16
+ kind : Service
17
+ metadata :
18
+ name : myapps
19
+ namespace : seldon-mesh
20
+ spec :
21
+ ports :
22
+ - name : web
23
+ port : 80
24
+ protocol : TCP
25
+ selector :
26
+ app : traefik-ingress-lb
27
+ type : LoadBalancer
28
+ ---
29
+ apiVersion : mlops.seldon.io/v1alpha1
30
+ kind : Model
31
+ metadata :
32
+ name : iris
33
+ namespace : seldon-mesh
34
+ spec :
35
+ requirements :
36
+ - sklearn
37
+ storageUri : gs://seldon-models/mlserver/iris
38
+ ---
39
+ apiVersion : traefik.containo.us/v1alpha1
40
+ kind : IngressRoute
41
+ metadata :
42
+ name : iris
43
+ namespace : seldon-mesh
44
+ spec :
45
+ entryPoints :
46
+ - web
47
+ routes :
48
+ - kind : Rule
49
+ match : PathPrefix(`/`)
50
+ middlewares :
51
+ - name : iris-header
52
+ services :
53
+ - name : seldon-mesh
54
+ port : 80
55
+ scheme : h2c
56
+ ---
57
+ apiVersion : traefik.containo.us/v1alpha1
58
+ kind : Middleware
59
+ metadata :
60
+ name : iris-header
61
+ namespace : seldon-mesh
62
+ spec :
63
+ headers :
64
+ customRequestHeaders :
65
+ seldon-model : iris
16
66
` ` `
17
67
18
68
## Traffic Split
19
69
20
- ``` {warning}
21
- Traffic splitting does not presently work due to this [issue](https://github.com/emissary-ingress/emissary/issues/4062). We recommend you use a Seldon Experiment instead.
70
+ {% hint style="warning" %}
71
+ **Warning** Traffic splitting does not presently work due to this [issue](https://github.com/emissary-ingress/emissary/issues/4062). We recommend you use a Seldon Experiment instead.
72
+ {% endhint %}
73
+
74
+ ## Traefik Examples
75
+
76
+ Assumes
77
+
78
+ * You have installed Traefik as per their [docs](https://doc.traefik.io/traefik/getting-started/install-traefik/#use-the-helm-chart) into namespace traefik-v2
79
+
80
+ Tested with traefik-10.19.4
81
+
82
+
83
+
84
+ ` ` ` python
85
+ INGRESS_IP=!kubectl get svc traefik -n traefik-v2 -o jsonpath='{.status.loadBalancer.ingress[0].ip}'
86
+ INGRESS_IP=INGRESS_IP[0]
87
+ import os
88
+ os.environ['INGRESS_IP'] = INGRESS_IP
89
+ INGRESS_IP
22
90
```
23
91
24
- ``` {include} ../../../../../../service-meshes/traefik/README.md
92
+
93
+
94
+
95
+ '172.21.255.1'
96
+
97
+
98
+
99
+ ### Traefik Single Model Example
100
+
101
+
102
+ ``` python
103
+ ! kustomize build config/ single- model
25
104
```
105
+
106
+ apiVersion: v1
107
+ kind: Service
108
+ metadata:
109
+ name: myapps
110
+ namespace: seldon-mesh
111
+ spec:
112
+ ports:
113
+ - name: web
114
+ port: 80
115
+ protocol: TCP
116
+ selector:
117
+ app: traefik-ingress-lb
118
+ type: LoadBalancer
119
+ ---
120
+ apiVersion: mlops.seldon.io/v1alpha1
121
+ kind: Model
122
+ metadata:
123
+ name: iris
124
+ namespace: seldon-mesh
125
+ spec:
126
+ requirements:
127
+ - sklearn
128
+ storageUri: gs://seldon-models/mlserver/iris
129
+ ---
130
+ apiVersion: traefik.containo.us/v1alpha1
131
+ kind: IngressRoute
132
+ metadata:
133
+ name: iris
134
+ namespace: seldon-mesh
135
+ spec:
136
+ entryPoints:
137
+ - web
138
+ routes:
139
+ - kind: Rule
140
+ match: PathPrefix(`/`)
141
+ middlewares:
142
+ - name: iris-header
143
+ services:
144
+ - name: seldon-mesh
145
+ port: 80
146
+ scheme: h2c
147
+ ---
148
+ apiVersion: traefik.containo.us/v1alpha1
149
+ kind: Middleware
150
+ metadata:
151
+ name: iris-header
152
+ namespace: seldon-mesh
153
+ spec:
154
+ headers:
155
+ customRequestHeaders:
156
+ seldon-model: iris
157
+
158
+
159
+
160
+ ``` python
161
+ ! kustomize build config/ single- model | kubectl apply - f -
162
+ ```
163
+
164
+ service/myapps created
165
+ model.mlops.seldon.io/iris created
166
+ ingressroute.traefik.containo.us/iris created
167
+ middleware.traefik.containo.us/iris-header created
168
+
169
+
170
+
171
+ ``` python
172
+ ! kubectl wait -- for condition=ready -- timeout=300s model -- all - n seldon- mesh
173
+ ```
174
+
175
+ model.mlops.seldon.io/ iris condition met
176
+
177
+
178
+
179
+ ```python
180
+ ! curl - v http:// $ {INGRESS_IP }/ v2/ models/ iris/ infer - H " Content-Type: application/json" \
181
+ - d ' {"inputs": [{"name": "predict", "shape": [1, 4], "datatype": "FP32", "data": [[1, 2, 3, 4]]}]}'
182
+ ```
183
+
184
+ * Trying 172.21 .255.1...
185
+ * Connected to 172.21 .255.1 (172.21 .255.1) port 80 (# 0)
186
+ > POST / v2/ models/ iris/ infer HTTP / 1.1
187
+ > Host: 172.21 .255.1
188
+ > User- Agent: curl/ 7.47 .0
189
+ > Accept: */*
190
+ > Content- Type: application/ json
191
+ > Content- Length: 94
192
+ >
193
+ * upload completely sent off: 94 out of 94 bytes
194
+ < HTTP / 1.1 200 OK
195
+ < Content- Length: 196
196
+ < Content- Type: application/ json
197
+ < Date: Sat, 16 Apr 2022 15 :53 :27 GMT
198
+ < Seldon- Route: iris_1
199
+ < Server: envoy
200
+ < X- Envoy- Upstream- Service- Time: 895
201
+ <
202
+ * Connection # 0 to host 172.21.255.1 left intact
203
+ {" model_name" :" iris_1" ," model_version" :" 1" ," id" :" 0dccf477-78fa-4a11-92ff-4d7e4f1cdda8" ," parameters" :null," outputs" :[{" name" :" predict" ," shape" :[1 ]," datatype" :" INT64" ," parameters" :null," data" :[2 ]}]}
204
+
205
+
206
+ ```python
207
+ ! grpcurl - d ' {"model_name":"iris","inputs":[{"name":"input","contents":{"fp32_contents":[1,2,3,4]},"datatype":"FP32","shape":[1,4]}]}' \
208
+ - plaintext \
209
+ - import - path ../ ../ apis \
210
+ - proto ../ ../ apis/ mlops/ v2_dataplane/ v2_dataplane.proto \
211
+ $ {INGRESS_IP }:80 inference.GRPCInferenceService/ ModelInfer
212
+ ```
213
+
214
+ {
215
+ " modelName" : " iris_1" ,
216
+ " modelVersion" : " 1" ,
217
+ " outputs" : [
218
+ {
219
+ " name" : " predict" ,
220
+ " datatype" : " INT64" ,
221
+ " shape" : [
222
+ " 1"
223
+ ],
224
+ " contents" : {
225
+ " int64Contents" : [
226
+ " 2"
227
+ ]
228
+ }
229
+ }
230
+ ]
231
+ }
232
+
233
+
234
+
235
+ ```python
236
+ ! kustomize build config/ single- model | kubectl delete - f -
237
+ ```
238
+
239
+ service " myapps" deleted
240
+ model.mlops.seldon.io " iris" deleted
241
+ ingressroute.traefik.containo.us " iris" deleted
242
+ middleware.traefik.containo.us " iris-header" deleted
243
+
244
+
245
+
246
+ ```python
247
+
248
+ ```
249
+
0 commit comments