This INSTALL guide will detail the minimum requirements needed for cnf-testsuite while then providing installation with configuration steps to run the cnf-testsuite binary from both a binary installation and source installation method.
This will detail the required minimum requirements needed in order to support cnf-testsuite.
- Kubernetes multi-node cluster (2 schedulable nodes minimum as a few tests require this. See supported K8s and installation details on installation.)
- containerd runtime - for K8s cluster running the CNF to be tested
- kubectl (run commands against K8s clusters, see installing kubectl for more details.)
- curl
- *helm 3.8.2 or newer (cnf-testsuite installs if not found locally)
- docker (needed for the cni_compatibility test)
Everything detailed in the minimum requirements and the following:
- git (used to check out code from github)
- crystal-lang version >=1.6.0 (to compile the source and build the binary, see crystal installation)
- shards (dependency manager for crystal-lang)
Click here to drop down details
- Access to a working Certified K8s multi-node cluster via KUBECONFIG environment variable. (See K8s Getting started guide for options)
- Follow the optional instructions below if you don't already have a K8s cluster setup
- Minimum of 2 schedulable nodes as some tests will require more than one node to run.
- Follow the kind install instructions to setup a cluster in kind.
- You can clone the CNF-Testbed project if you have an account at Equinix Metal (formerly Packet.net). Get the code by running the following:
git clone https://github.com/cncf/cnf-testbed.git
- Clone the k8s-infra repo then follow the prerequisites for deploying a K8s cluster for a Equinix Metal host.
- If you already have IP addresses for your provider, and you want to manually install a K8s cluster, you can use k8s-infra to do this within your cnf-testbed repo clone.
cd tools/ && git clone https://github.com/crosscloudci/k8s-infra.git
- Now follow the k8s-infra quick start for instructions on how to install.
We support the following methods of installing the cnf-testsuite:
- Curl installation (via latest binary release)
- Latest Binary (manual download)
- From Source on github.
- Air Gapped
There are two methods to install via curl, we prefer the first method (the others including the manual and source install are optional):
- This first command using curl will download, install, and export the path automatically (recommended method):
source <(curl -s https://raw.githubusercontent.com/cnti-testcatalog/testsuite/main/curl_install.sh)
Click here for the alternate curl and manual install method
- The other curl method to download and install requires you to export the PATH to the location of the executable:
curl -s https://raw.githubusercontent.com/cnti-testcatalog/testsuite/main/curl_install.sh | bash
- The Latest Binary (or you can select a previous release if desired) can be pulled down with wget, curl or you're own preferred method. Once downloaded you'll need to make the binary executable and manually add to your path:
wget https://github.com/cnti-testcatalog/testsuite/releases/download/latest/latest.tar.gz
tar xzf latest.tar.gz
cd cnf-testsuite
chmod +x cnf-testsuite
export OLDPATH=$PATH; export PATH=$PATH:$(pwd)
This is a brief summary for source installations and does have requirements in order to compile a binary from source. To read more on source installation, see the SOURCE-INSTALL document.
Click here for brief source install details
Follow these steps to checkout the source from github and compile a cnf-testsuite binary:
git clone https://github.com/cnti-testcatalog/testsuite.git
cd cnf-testsuite/
shards install
crystal build src/cnf-testsuite.cr
This should build a cnf-testsuite binary in the root directory of the git repo clone.
Now that you have cnf-testsuite installed, we need to prepare the suite.
First make sure your K8s cluster is accessible (part of the minimum pre-requisites). You can run the following to verify the cluster:
kubectl cluster-info
And it should print a running kubernetes master in the output. Common kubectl errors and issues might relate to your KUBECONFIG variable. You can export to your K8s config by doing the following:
export KUBECONFIG=path/to/mycluster.config
Note: We recommend running cnf-testsuite on a non-production cluster.
The next step is to run the setup which prepares the cnf-testsuite. This runs pre-reqs to verify you have everything needed in order to run the suite, simply run the following:
cnf-testsuite setup
The test suite by default will pull docker images from https://docker.io. You can set your own username and password with local environment variables by doing the following:
export DOCKERHUB_USERNAME=<USERNAME>
export DOCKERHUB_PASSWORD=<PASSWORD>
Please refer to the CNF_TESTSUITE_YML_USAGE.md for details on using a private registry.
Install Tab Completion for cnf-testsuite (Optional)
Check out our (experimental) support for tab completion!
NOTE: also compatible with the installation styles from kubectl completion install if you prefer https://kubernetes.io/docs/tasks/tools/install-kubectl/#enable-kubectl-autocompletion
cnf-testsuite completion -l error > test.sh
source test.sh
Now cnf-testsuite is setup, we're ready to configure it to point at a CNF to test.
- If you want to use an example CNF, you can download our CoreDNS example CNF by doing the following:
wget -O cnf-testsuite.yml https://raw.githubusercontent.com/cnti-testcatalog/testsuite/main/example-cnfs/coredns/cnf-testsuite.yml
- The wget gets a working config file, now tell cnf-testsuite to use it by doing the following:
cnf-testsuite cnf_setup cnf-config=./cnf-testsuite.yml
- There are other examples in the examples-cnfs directory that can be used for testing as well.
If you've brought your own CNF to test, review the CNF_TESTSUITE_YML_USAGE.md document on formatting and other requirements.
If you've followed the CNF_TESTSUITE_YML_USAGE.md guide and have your cnf-testsuite.yml ready, you can run the same command we ran for the example CNF to set it up:
cnf-testsuite cnf_setup cnf-config=./cnf-testsuite.yml
If you want to run all tests, do the following (this is assuming your cnf_setup ran without errors in the configuration steps:)
For complete usage, see the USAGE.md doc.
cnf-testsuite all
The following will run only workload tests:
cnf-testsuite workload
The following would run only the platform tests:
cnf-testsuite platform
In the console where the test suite runs:
- PASSED or FAILED will be displayed for the tests
A test log file, eg. cnf-testsuite-results-20201216.txt, will be created which lists PASS or FAIL for every test based on the date.
For more details on points, see our POINTS.md documentation.
Run the following to cleanup the specific cnf-testsuite test (this is assuming you installed the cnf-testsuite.yml in your present working directory):
cnf-testsuite cnf_cleanup cnf-config=./cnf-testsuite.yml
You can also run cleanall and cnf-testsuite will attempt to cleanup everything.
NOTE: Cleanup does not handle manually deployed CNFs