Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update docs for cri-tools #2191

Merged
merged 1 commit into from
Jun 11, 2018
Merged

Update docs for cri-tools #2191

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 20 additions & 15 deletions contributors/devel/cri-validation.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,26 @@

CRI validation testing provides a test framework and a suite of tests to validate that the Container Runtime Interface (CRI) server implementation meets all the requirements. This allows the CRI runtime developers to verify that their runtime conforms to CRI, without needing to set up Kubernetes components or run Kubernetes end-to-end tests.

CRI validation testing is currently Alpha and is hosted at the [cri-tools](https://github.com/kubernetes-incubator/cri-tools) repository. Performance benchmarking will be added in the future. We encourage the CRI developers to report bugs or help extend the test coverage by adding more tests.
CRI validation testing is GA since v1.11.0 and is hosted at the [cri-tools](https://github.com/kubernetes-incubator/cri-tools) repository. We encourage the CRI developers to report bugs or help extend the test coverage by adding more tests.

## Install

The test suites can be installed easily via `go get` command:
The test suites can be downloaded from cri-tools [release page](https://github.com/kubernetes-incubator/cri-tools/releases):

```sh
go get github.com/kubernetes-incubator/cri-tools/cmd/critest
VERSION="v1.11.0"
wget https://github.com/kubernetes-incubator/cri-tools/releases/download/$VERSION/critest-$VERSION-linux-amd64.tar.gz
sudo tar zxvf critest-$VERSION-linux-amd64.tar.gz -C /usr/local/bin
rm -f critest-$VERSION-linux-amd64.tar.gz
```

Then `critest` binary can be found in `$GOPATH/bin`.
critest requires [ginkgo](https://github.com/onsi/ginkgo) to run parallel tests. It could be installed by

*Note: ensure GO is installed and GOPATH is set before installing critest.*
```sh
go get -u github.com/onsi/ginkgo/ginkgo
```

*Note: ensure GO is installed and GOPATH is set before installing ginkgo.*

## Running tests

Expand All @@ -25,7 +32,7 @@ Before running the test, you need to _ensure that the CRI server under test is r
### Run

```sh
critest validation
critest
```

This will
Expand All @@ -34,15 +41,13 @@ This will
- Run the tests using `ginkgo`
- Output the test results to STDOUT

critest connects to `/var/run/dockershim.sock` by default. For other runtimes, the endpoint can be set in two ways:

- By setting flags `--runtime-endpoint` and `--image-endpoint`
- By setting environment variables `CRI_RUNTIME_ENDPOINT` and `CRI_IMAGE_ENDPOINT`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Were the environment variables removed, and why?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, those environment variables are introduced with lib github.com/urfave/cli. But with this lib, critest requires Go installed whenever running validation tests. So we moved to go test -c for building critest binary and it doesn't require Go for running critest.

critest connects to `unix:///var/run/dockershim.sock` by default. For other runtimes, the endpoint can be set by flags `-runtime-endpoint` and `-image-endpoint`.

## Additional options

- `--focus`, `-f`: Only run the tests that match the regular expression.
- -`-ginkgo-flags`, `-g`: Space-separated list of arguments to pass to Ginkgo test runner.
- `--image-endpoint`, `-i`: Set the endpoint of image service. Same with runtime-endpoint if not specified.
- `--runtime-endpoint`, `-r`: Set the endpoint of runtime service. Default to `/var/run/dockershim.sock`.
- `--skip`, `-s`: Skip the tests that match the regular expression.
- `-ginkgo.focus`: Only run the tests that match the regular expression.
- `-image-endpoint`: Set the endpoint of image service. Same with runtime-endpoint if not specified.
- `-runtime-endpoint`: Set the endpoint of runtime service. Default to `unix:///var/run/dockershim.sock`.
- `-ginkgo.skip`: Skip the tests that match the regular expression.
- `-parallel`: The number of parallel test nodes to run (default 1). ginkgo must be installed to run parallel tests.
- `-h`: Show help and all supported options.