chore: test suite refactor #2289
Conversation
- Pulls logic for defaulting to active namespace (K8S) moved UP to CLI during flag default calculation. - Pushes logic of deciding between f.Namespace vs f.Deploy.Namespace down into implementations. - Updates some tests which needed to have their environment cleared. - Refactors Pipelines tests to use client API. - Removes namespaces as a state variable all structures, instead passing as an argument. - Moves FromTempDirectory to testing package for use outside cmd.
- Utilize the `make check-embedded-fs` target - Renames schema-check to check-schema to match other checks - Delegates runtime and builder matrix logic to test suite impl - Relocates converage reformatting to workflows
- Adds test-all target - integration and e2e tests clearly state they run unit tests too - integration and e2e tests clear cache - Adds a check-embedded-fs target - Adds several missing .PHONY lines
- Default builder and pusher set to embedded Host Builder/Pusher(oci) - Most tests clear environment - Environment defaults can be controlled via environment variables - Tests which require back-compat `git` binary actively check and skip when running with a cleared environment (both integration and unit). - Bugfixes for when run in tandem with E2E tests - Ignores go-created directories in the default home path (testdata)
- Updates docs and moves package doc to a README.md - Sets up MatrixRuntimes and MatrixBuilders as an opt-in filter - Implements delete - Bugfixes when run via make
|
Skipping CI for Draft Pull Request. |
knative-prow
bot
added
do-not-merge/work-in-progress
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: lkingland The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
knative-prow
bot
added
the
approved
knative-prow-robot
added
the
needs-rebase
knative-prow
bot
added
the
size/XXL
knative-prow-robot
removed
the
needs-rebase
knative-prow-robot
added
the
needs-rebase
|
PR needs rebase. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
|
||
| test-unit: |
There was a problem hiding this comment.
I would not recommend deleting these.
There was a problem hiding this comment.
Will make test-all run really all tests? What about coverage have you ensured that there are no conflicts?
| @@ -1,10 +1,10 @@ | |||
| name: Func Unit Test | |||
There was a problem hiding this comment.
I do not get this rename. This is foremost "unit test". It also runs "template test" but that is IMO secondary.
| go test -ldflags "$(LDFLAGS)" -cover -tags "e2e" -timeout 30m --coverprofile=coverage.txt ./... | ||
|
|
||
| .PHONY: test-e2e | ||
| test-all: func-instrumented ## Run unit+integration+E2E tests (cluster required) |
There was a problem hiding this comment.
Does this properly gather coverage from the instrumented func?
| .PHONY: test-e2e | ||
| test-all: func-instrumented ## Run unit+integration+E2E tests (cluster required) | ||
| go clean -testcache | ||
| go test -ldflags "$(LDFLAGS)" -cover -tags "integration e2e" -timeout 60m --coverprofile=coverage.txt ./... |
There was a problem hiding this comment.
does not this miss "on cluster build" tag?
|
This Pull Request is stale because it has been open for 90 days with |
github-actions
bot
added
the
lifecycle/stale
lkingland
removed
the
lifecycle/stale
This PR provides the general structure for an upgrade to the testing system (both integration and E2Es) relying on the newly created Host builder for core functionalit, along with general simplification.
Currently a WIP.
/kind cleanup