Common Component: kubeflow-knative

[rohank07]

Overview

EPIC: Kubeflow Upgrade Planning

Component	Local Manifests Path	Upstream	Initial Work
kubeflow-knative	common/kubeflow-knative	v1.3.1	StatCan/aaw#667

Adjustments

• Adjustments to config-domain for eventual serving workloads on the AAW domain
• Adjustments to config-istio for eventual serving workloads on the AAW domain

Kubeflow V2 Manifests

The following is the kustomize that was used in Kubeflow V2:

• https://github.com/StatCan/aaw-kubeflow-manifests/tree/master/kustomize/knative

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../.cache/manifests/manifests-1.2-branch/knative/installs/generic

AAW Dev / Prod Live Manifests

The difference is state is largely accounted for in these issue(s):

• Kserve integration and documentation aaw#667

Note: While most everything 95% would have been automated, stored as config and is using what is referenced above. I believe a few things could have been done as manual adjustments that we should make sure we are keeping. Largely any manual yaml adjustments would have been documented in high level GitHub issues or tracked in the YAML repository under the AAW group.

Kubeflow V3 Manifests

The following is the P.R. that will be merged into the main branch for Kubeflow V3:

• feat(component): Add common component knative #130

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- github.com/kubeflow/manifests/common/knative/knative-serving/base?ref=v1.3.1
- github.com/kubeflow/manifests/common/knative/knative-eventing/base?ref=v1.3.1

patchesStrategicMerge:
- config-domain.yaml

patches:
- config-istio.yaml

Note: I needed to override the whole config-istio object so used patches instead of patchesStrategicMerge.

Testing

Usually a good idea to make sure all of the overrides are working is to run the following command and verify all of the yaml output for the component is what you expect and all of the overrides are taken into account.

• Adjustments to config-domain were successful
• Adjustments to config-istio were successful

task stack:aaw:preview

Note: The command above will render all of the manifests into manifests top level folder with the name aaw.yaml. A trick to keep the yaml output small is under stacks/aaw/kustomization.yaml to only have the component you wish to test referenced.

[sylus]

I looked this over and updated it and also matched it with live environment.

The last things we need to handle and take into account for this issue is:

Istio Injection to Knative Serving Namespace

a) Add istio-injection label istio-injection=enabled

b) Edit the deployment and add sidecar.istio.io/istio-inject: false to the following:

• istio-webhook
• webhook

Note: I also made note in the AAW stack that we use the Istio Operator which does alot of the gateway handling for us so I didn't need to bring in that component which is mentioned in kubeflow/manifest as common/istio-1-9/cluster-local-gateway/base

[rohank07]

The kustomize output has the istio-injection label already. Do we want the label elsewhere as well?

And we want to add the sidecar label only to istio-webhook and webhook?

[sylus]

Oh great glad kubeflow has that now and yup we need those sidecar labels otherwise the API server can't reach them.