Skip to content
This repository has been archived by the owner on Jul 26, 2024. It is now read-only.

Commit

Permalink
test: Merge contile-integration-test repository into contile
Browse files Browse the repository at this point in the history
test: Merge contile-integration-test repository into contile
* move contract-tests to new test-engineering directory
* integrate contile-integration-test repository

Co-authored-by: Ankita Shrivastava <ashrivastava@mozilla.com>
Co-authored-by: Ankita Shrivastava <62558650+ashrivastava-qa@users.noreply.github.com>
Co-authored-by: Raphael Pierzina <raphael@hackebrot.de>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: JR Conlin <jconlin+git@mozilla.com>
Co-authored-by: jrconlin <jr+git@mozilla.com>
Co-authored-by: Nan Jiang <njiang028@gmail.com>
  • Loading branch information
8 people authored Jun 27, 2022
1 parent 8210016 commit 885c978
Show file tree
Hide file tree
Showing 56 changed files with 2,847 additions and 17 deletions.
53 changes: 41 additions & 12 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -190,7 +190,31 @@ jobs:
else
echo "Not pushing to dockerhub for tag=${CIRCLE_TAG} branch=${CIRCLE_BRANCH}"
fi
contile-contract-tests:
contract-test-checks:
docker:
- image: python:3.8
working_directory: "~/contile/test-engineering/contract-tests"
steps:
- checkout:
path: ~/contile/
- run:
name: Setup tox
command: |
python -m pip install --upgrade pip
python -m pip install tox
- run:
name: Run tox checks for client
working_directory: client
command: |
tox
- run:
name: Run tox checks for partner
working_directory: partner
command: |
tox
contract-tests:
machine:
docker_layer_caching: true
image: ubuntu-2004:202101-01 # Ubuntu 20.04, Docker v20.10.2, Docker Compose v1.28.2
Expand All @@ -207,28 +231,32 @@ jobs:
command: |
echo "$DOCKER_PASS" | docker login -u "$DOCKER_USER" --password-stdin
- run:
name: Run contract tests
name: Build client and partner images
command: |
docker-compose --version
docker-compose \
-f contract-tests/docker-compose.yml \
-f test-engineering/contract-tests/docker-compose.yml \
build client partner
- run:
name: Run contract tests
command: |
docker-compose \
-f test-engineering/contract-tests/docker-compose.yml \
up --abort-on-container-exit --force-recreate
- run:
name: 'Run "204" contract tests'
command: |
docker-compose --version
docker-compose \
-f contract-tests/docker-compose.yml \
-f contract-tests/docker-compose.204.yml \
-f test-engineering/contract-tests/docker-compose.yml \
-f test-engineering/contract-tests/docker-compose.204.yml \
up --abort-on-container-exit --force-recreate
- run:
name: 'Run "init_error" contract tests'
command: |
docker-compose --version
set +e # We need this so that the run doesn't exit after docker-compose
docker-compose \
-f contract-tests/docker-compose.yml \
-f contract-tests/docker-compose.init_error.yml \
-f test-engineering/contract-tests/docker-compose.yml \
-f test-engineering/contract-tests/docker-compose.init_error.yml \
up --abort-on-container-exit --exit-code contile --force-recreate
contile_exit_code=$?
if [ "${contile_exit_code}" -eq 0 ]; then
Expand Down Expand Up @@ -257,15 +285,16 @@ workflows:
only: /.*/
requires:
- build
- contile-contract-tests:
name: contile-contract-tests
- contract-test-checks
- contract-tests:
requires:
- build
- contract-test-checks
- deploy:
requires:
- build
- test
- contile-contract-tests
- contract-tests
filters:
tags:
only: /.*/
2 changes: 2 additions & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
* @mozilla-services/context-services
test-engineering/* @mozilla-services/context-services-test-eng
20 changes: 20 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
version: 2
updates:
- package-ecosystem: "pip"
directory: "test-engineering/contract-tests/partner/"
schedule:
interval: "weekly"
day: "friday"
labels:
- "dependencies"
- "python"
open-pull-requests-limit: 5
- package-ecosystem: "pip"
directory: "test-engineering/contract-tests/client/"
schedule:
interval: "weekly"
day: "friday"
labels:
- "dependencies"
- "python"
open-pull-requests-limit: 5
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,8 @@ GOOGLE_APPLICATION_CREDENTIALS={path to your credential.json file} \
#### Contract tests

Contract tests are currently run using Docker images. This is so that they can be run as
part of our automated continuous integration (CI) testing. Running these tests outside of
Docker is possible, but requires some additional work. See [Contile Integration Tests](https://github.com/mozilla-services/contile-integration-tests) for details.
part of our automated continuous integration (CI) testing.
See the dedicated [contract-tests README](test-engineering/contract-tests/README.md) for details.

## Why "Contile"?

Expand Down
135 changes: 135 additions & 0 deletions test-engineering/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
pip-wheel-metadata/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
target/

# Jupyter Notebook
.ipynb_checkpoints

# IPython
profile_default/
ipython_config.py

# pyenv
.python-version

# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock

# PEP 582; used by e.g. github.com/David-OConnor/pyflow
__pypackages__/

# Celery stuff
celerybeat-schedule
celerybeat.pid

# SageMath parsed files
*.sage.py

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/
.dmypy.json
dmypy.json

# Pyre type checker
.pyre/

# VS Code
.vscode/

# JetBrains
.idea/
13 changes: 13 additions & 0 deletions test-engineering/contract-tests/.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
**/*.py[cod]
**/__pycache__
**/.pytest_cache
*.egg
*.egg-info
*.log
.cache
.git
.tox
.vscode
build
dist
pip-log.txt
59 changes: 59 additions & 0 deletions test-engineering/contract-tests/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# contile-contract-tests

This directory contains the automated contract test suite for the Mozilla Tile
Service (MTS). Passing contract tests are a prerequisite for moving to the next
phase in the rollout plan. The contract test framework was originally developed
in isolation, see [contile-integration-tests][contract-test-repo].

## Overview

The contract test suite is designed to be set up as a `docker-compose` CI
workflow. The following sections as well as the sequence diagram below describe
the individual components of the suite.

![Sequence diagram of the integration tests][sequence_diagram]

To run the contract tests locally, execute the following from the repository root:

```text
docker compose \
-f test-engineering\contract-tests\docker-compose.yml \
up --abort-on-container-exit --build
```

### partner

The `partner` directory contains a Python-based web service. The HTTP API of
this service implements the API specification of the partner API that MTS
connects to when requesting tiles to pass along to Firefox for display.

When a client sends a request to the MTS, information about the client's form
factor and OS family are parsed from the `User-Agent` header. Then, when the MTS
sends a request to the partner API the form factor and OS family information is
included in the query parameters. We leverage this behavior to map requests from
a client to specific responses from the partner API. We can control not only the
response content, but also the response status code, response headers and even
delay the response for a period of time, which allows us to effectively test the
MTS.

### client

The `client` directory contains a Python-based test framework for the
contract tests. The HTTP client used in the framework requests tiles from the
MTS and performs checks against the responses. The framework implements response
models for the MTS API.

### volumes

The `volumes` directory contains subdirectories which will be mounted as
volumes into the Docker containers used in the contract test suite:

- the `volumes/partner` directory contains a YML file which defines every
response that the API returns keyed by form-factor and then os-family
- the `volumes/contile` directory contains files that need to be provided to a
MTS Docker container such as a partner settings file
- the `volumes/client` directory contains a YML file which defines every test
scenario that the contract test suite will run

[contract-test-repo]: https://github.com/mozilla-services/contile-integration-tests
[sequence_diagram]: sequence_diagram.png
17 changes: 17 additions & 0 deletions test-engineering/contract-tests/client/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
FROM python:3.8-slim-buster

LABEL maintainer "Raphael Pierzina <raphael@hackebrot.de>"

ENV PYTHON_VENV=/venv
RUN python -m venv ${PYTHON_VENV}
ENV PATH="${PYTHON_VENV}/bin:${PATH}"

RUN python -m pip install --upgrade pip

COPY requirements.txt /tmp/requirements.txt
RUN python -m pip install -r /tmp/requirements.txt

COPY . /usr/src/client
WORKDIR /usr/src/client

ENTRYPOINT [ "pytest" ]
Loading

0 comments on commit 885c978

Please sign in to comment.