Skip to content

Commit 25af89c

Browse files
juliusvonkohoutakagami-harshgithub-advanced-security[bot]pschoen-itsc
authored
chore: Seaweedfs as Minio replacement (#11965)
add seaweedFS to KFP as default object store This change switches KFP's default objectstore deployment to Seaweedfs instead of Minio. Minio is still kept as an optional deployment to help users with migrating. CI is updated to accommodate testing for Minio and SeaweedFS. Multi-User testing is also introduced, which also includes namespace authorization testing in Seaweedfs. Some more work is needed to completely rid the KFP backend and frontend code of Minio specific code and labeling, which we accrue as tech debt as part of this change. Signed-off-by: juliusvonkohout <[email protected]> Signed-off-by: Julius von Kohout <[email protected]> Signed-off-by: Harshvir Potpose <[email protected]> Co-authored-by: Harshvir Potpose <[email protected]> Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> Co-authored-by: pschoen-itsc <[email protected]>
1 parent 8f3cddc commit 25af89c

File tree

87 files changed

+1900
-893
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

87 files changed

+1900
-893
lines changed

‎.github/actions/kfp-cluster/action.yml‎

Lines changed: 24 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,15 @@ inputs:
3030
image_registry:
3131
required: true
3232
description: "Image Registry address of the images"
33+
multi_user:
34+
description: "If KFP should be deployed in multi-user mode"
35+
required: false
36+
default: 'false'
37+
storage_backend:
38+
description: "Storage backend to use (minio or seaweedfs)"
39+
required: false
40+
default: 'seaweedfs'
41+
3342
runs:
3443
using: "composite"
3544
steps:
@@ -68,10 +77,22 @@ runs:
6877
6978
if [ "${{ inputs.proxy }}" = "true" ]; then
7079
ARGS="${ARGS} --proxy"
71-
elif [ "${{inputs.cache_enabled }}" = "false" ]; then
80+
fi
81+
82+
if [ "${{inputs.cache_enabled }}" = "false" ]; then
7283
ARGS="${ARGS} --cache-disabled"
73-
elif [ "${{inputs.pipeline_store }}" = "kubernetes" ]; then
74-
ARGS="${ARGS} --deploy-k8s-native"
84+
fi
85+
86+
if [ "${{inputs.pipeline_store }}" = "kubernetes" ]; then
87+
ARGS="${ARGS} --deploy-k8s-native"
88+
fi
89+
90+
if [ "${{ inputs.multi_user }}" = "true" ]; then
91+
ARGS="${ARGS} --multi-user"
92+
fi
93+
94+
if [ "${{ inputs.storage_backend }}" != "seaweedfs" ]; then
95+
ARGS="${ARGS} --storage ${{ inputs.storage_backend }}"
7596
fi
7697
7798
./.github/resources/scripts/deploy-kfp.sh $ARGS

‎.github/resources/manifests/argo/overlays/multi-user-minio/apiserver-env.yaml‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
apiVersion: apps/v1
2+
kind: Deployment
3+
metadata:
4+
name: ml-pipeline
5+
spec:
6+
template:
7+
spec:
8+
containers:
9+
- name: ml-pipeline-api-server
10+
env:
11+
- name: V2_DRIVER_IMAGE
12+
value: kind-registry:5000/driver
13+
- name: V2_LAUNCHER_IMAGE
14+
value: kind-registry:5000/launcher
15+
- name: LOG_LEVEL
16+
value: "debug"

‎.github/resources/manifests/argo/overlays/multi-user-minio/kustomization.yaml‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
apiVersion: kustomize.config.k8s.io/v1beta1
2+
kind: Kustomization
3+
4+
resources:
5+
- ../../../../../../manifests/kustomize/env/platform-agnostic-multi-user-minio
6+
7+
images:
8+
- name: ghcr.io/kubeflow/kfp-api-server
9+
newName: kind-registry:5000/apiserver
10+
newTag: latest
11+
- name: ghcr.io/kubeflow/kfp-persistence-agent
12+
newName: kind-registry:5000/persistenceagent
13+
newTag: latest
14+
- name: ghcr.io/kubeflow/kfp-scheduled-workflow-controller
15+
newName: kind-registry:5000/scheduledworkflow
16+
newTag: latest
17+
18+
patchesStrategicMerge:
19+
- apiserver-env.yaml

‎.github/resources/manifests/argo/overlays/multi-user/apiserver-env.yaml‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
apiVersion: apps/v1
2+
kind: Deployment
3+
metadata:
4+
name: ml-pipeline
5+
spec:
6+
template:
7+
spec:
8+
containers:
9+
- name: ml-pipeline-api-server
10+
env:
11+
- name: V2_DRIVER_IMAGE
12+
value: kind-registry:5000/driver
13+
- name: V2_LAUNCHER_IMAGE
14+
value: kind-registry:5000/launcher
15+
- name: LOG_LEVEL
16+
value: "debug"

‎.github/resources/manifests/argo/overlays/multi-user/kustomization.yaml‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
apiVersion: kustomize.config.k8s.io/v1beta1
2+
kind: Kustomization
3+
4+
resources:
5+
- ../../../../../../manifests/kustomize/env/platform-agnostic-multi-user
6+
7+
images:
8+
- name: ghcr.io/kubeflow/kfp-api-server
9+
newName: kind-registry:5000/apiserver
10+
newTag: latest
11+
- name: ghcr.io/kubeflow/kfp-persistence-agent
12+
newName: kind-registry:5000/persistenceagent
13+
newTag: latest
14+
- name: ghcr.io/kubeflow/kfp-scheduled-workflow-controller
15+
newName: kind-registry:5000/scheduledworkflow
16+
newTag: latest
17+
18+
patchesStrategicMerge:
19+
- apiserver-env.yaml

‎.github/resources/manifests/argo/overlays/no-proxy-minio/apiserver-env.yaml‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
apiVersion: apps/v1
2+
kind: Deployment
3+
metadata:
4+
name: ml-pipeline
5+
spec:
6+
template:
7+
spec:
8+
containers:
9+
- name: ml-pipeline-api-server
10+
env:
11+
- name: V2_DRIVER_IMAGE
12+
value: kind-registry:5000/driver
13+
- name: V2_LAUNCHER_IMAGE
14+
value: kind-registry:5000/launcher
15+
- name: LOG_LEVEL
16+
value: "debug"

‎.github/resources/manifests/argo/overlays/no-proxy-minio/kustomization.yaml‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
apiVersion: kustomize.config.k8s.io/v1beta1
2+
kind: Kustomization
3+
4+
resources:
5+
- ../../../../../../manifests/kustomize/env/platform-agnostic-minio
6+
7+
images:
8+
- name: ghcr.io/kubeflow/kfp-api-server
9+
newName: kind-registry:5000/apiserver
10+
newTag: latest
11+
- name: ghcr.io/kubeflow/kfp-persistence-agent
12+
newName: kind-registry:5000/persistenceagent
13+
newTag: latest
14+
- name: ghcr.io/kubeflow/kfp-scheduled-workflow-controller
15+
newName: kind-registry:5000/scheduledworkflow
16+
newTag: latest
17+
18+
patches:
19+
- path: apiserver-env.yaml

‎.github/resources/manifests/argo/overlays/no-proxy/kustomization.yaml‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,4 +17,3 @@ images:
1717

1818
patches:
1919
- path: apiserver-env.yaml
20-
- path: workflow-disable-logs-patch.yaml

‎.github/resources/manifests/argo/overlays/no-proxy/workflow-disable-logs-patch.yaml‎

Lines changed: 0 additions & 20 deletions
This file was deleted.

‎.github/resources/manifests/argo/overlays/proxy/proxy-env.yaml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,4 +13,6 @@ spec:
1313
- name: HTTPS_PROXY
1414
value: "http://squid.squid.svc.cluster.local:3128"
1515
- name: NO_PROXY
16-
value: "localhost,127.0.0.1,.svc.cluster.local,kubernetes.default.svc,metadata-grpc-service,0,1,2,3,4,5,6,7,8,9"
16+
value: "localhost,127.0.0.1,.svc.cluster.local,kubernetes.default.svc,minio-service.kubeflow,metadata-grpc-service,metadata-grpc-service.kubeflow,ml-pipeline.kubeflow"
17+
- name: OBJECTSTORECONFIG_HOST
18+
value: "minio-service.kubeflow.svc.cluster.local"

0 commit comments

Comments
 (0)