Skip to content
This repository has been archived by the owner on Apr 6, 2023. It is now read-only.

Commit

Permalink
Merge pull request #227 from mozilla-services/reorganize-test-enginee…
Browse files Browse the repository at this point in the history
…ring-files
  • Loading branch information
mythmon authored Nov 24, 2021
2 parents 86addf7 + 62c20a9 commit d5c0375
Show file tree
Hide file tree
Showing 14 changed files with 21 additions and 16 deletions.
14 changes: 7 additions & 7 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,8 @@ workflows:
tags:
only: /.+/

- integration-tests:
name: integration-tests
- contract-tests:
name: contract-tests
requires:
- checks
- build-and-test
Expand All @@ -46,7 +46,7 @@ workflows:
- checks
- build-and-test
- docker-image-build
- integration-tests
- contract-tests
filters:
branches:
only: main
Expand Down Expand Up @@ -130,7 +130,7 @@ jobs:
- store_artifacts:
path: /tmp/artifacts

integration-tests:
contract-tests:
machine:
docker_layer_caching: true
image: ubuntu-2004:202101-01 # Ubuntu 20.04, Docker v20.10.2, Docker Compose v1.28.2
Expand All @@ -144,11 +144,11 @@ jobs:
name: Load Docker image from workspace
command: docker load -i workspace/merino.tar.gz
- run:
name: Run Integration tests
name: Run contract tests
command: |
docker-compose --version
docker-compose -f test-engineering/docker-compose.yml build client
docker-compose -f test-engineering/docker-compose.yml up --abort-on-container-exit
docker-compose -f test-engineering/contract-tests/docker-compose.yml build client
docker-compose -f test-engineering/contract-tests/docker-compose.yml up --abort-on-container-exit
docker-image-publish:
docker:
Expand Down
14 changes: 9 additions & 5 deletions docs/dev/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,13 +55,17 @@ async fn lbheartbeat_works() {

For more details, see the documentation of the `merino-integration-tests` crate.

## System tests
## Contract tests

The tests in the `test-engineering` directory are system tests that consume
Merino's APIs using more opaque techniques. They cannot configure the server per
test, and are more concerned with external contracts and behavior.
The tests in the `test-engineering/contract-tests` directory are contract tests
that consume Merino's APIs using more opaque techniques. These tests run against
a Docker container of the service, specify settings via environment variables,
and operate on the HTTP API layer only and as such are more concerned with
external contracts and behavior. The contract tests cannot configure the server
per test.

For more details see the README.md file in the `test-engineering` directory.
For more details see the README.md file in the `test-engineering/contract-tests`
directory.

## Load tests

Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# client

This directory contains a Python-based test framework for the integration tests.
This directory contains a Python-based test framework for the contract tests.

The HTTP client used in the framework requests suggestions from Merino and
performs checks against the responses. The framework implements response models
for the Merino API.
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,13 @@ services:
merino:
# See `docker-image-build` job in `.circleci/config.yml`
image: app:runtime
build: ..
build: ../..
container_name: merino
environment:
# The configuration preset to use
MERINO_ENV: "ci"
volumes:
- ../dev:/tmp/dev
- ../../dev:/tmp/dev

client:
image: client
Expand All @@ -19,7 +19,7 @@ services:
- merino
volumes:
- ./volumes/client:/tmp/client
- ../dev/wait-for-it.sh:/wait-for-it.sh
- ../../dev/wait-for-it.sh:/wait-for-it.sh
environment:
MERINO_URL: http://merino:8000
SCENARIOS_FILE: /tmp/client/scenarios.yml
Expand Down

0 comments on commit d5c0375

Please sign in to comment.