-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
0df2a99
commit 7a79c14
Showing
65 changed files
with
2,078 additions
and
155 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
LOG_LEVEL=INFO | ||
|
||
DB_HOST=NONE | ||
DB_PORT=NONE | ||
DB_DB=NONE | ||
DB_USER=NONE | ||
DB_PASS=NONE |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
LOG_LEVEL=INFO | ||
|
||
DB_HOST=NONE | ||
DB_PORT=NONE | ||
DB_DB=NONE | ||
DB_USER=NONE | ||
DB_PASS=NONE |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
[flake8] | ||
ignore = E203, E266, E501, W503, F403, F401 | ||
max-line-length = 88 | ||
max-complexity = 18 | ||
select = B,C,E,F,W,T4,B9 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,49 @@ | ||
# # This workflow will install Python dependencies, run tests and lint with a single version of Python | ||
# # For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions | ||
|
||
name: build-docker | ||
|
||
on: | ||
workflow_run: | ||
workflows: | ||
- python-testing | ||
branches: | ||
- main | ||
- develop | ||
types: | ||
- completed | ||
|
||
jobs: | ||
docker-build: | ||
|
||
runs-on: [SI, standard] | ||
|
||
steps: | ||
- uses: actions/checkout@v2 | ||
|
||
- name: Set up Python 3.10 | ||
uses: actions/setup-python@v2 | ||
with: | ||
python-version: '3.10' | ||
|
||
# - name: Docker Login | ||
# uses: docker/login-action@v1.10.0 | ||
# with: | ||
# registry: harbor.gradiant.org/si-medeva-pr-01432 | ||
# username: ${{ secrets.HarborUser }} | ||
# password: ${{ secrets.HarborPass }} | ||
# # Log out from the Docker registry at the end of a job | ||
# logout: true | ||
|
||
# - name: Extract Branch name | ||
# run: echo "BRANCH=${GITHUB_REF##*/}" >> $GITHUB_ENV | ||
|
||
# - name: Build image | ||
# uses: docker/build-push-action@v2 | ||
# with: | ||
# context: . | ||
# file: Dockerfile | ||
# push: true | ||
# tags: harbor.gradiant.org/si-medeva-pr-01432/SchemaMatchingModule:${{ env.BRANCH }} | ||
# build-args: | | ||
# BRANCH=${{ env.BRANCH }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,34 @@ | ||
# This workflow will install Python dependencies, run tests and lint with a single version of Python | ||
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions | ||
|
||
name: python-testing | ||
|
||
on: [push, pull_request, workflow_dispatch] | ||
|
||
jobs: | ||
python-testing: | ||
|
||
runs-on: [SI, standard] | ||
container: | ||
image: python:3.10-slim | ||
|
||
steps: | ||
- uses: actions/checkout@v2 | ||
|
||
- name: Install dependencies | ||
run: | | ||
python -m pip install --upgrade pip | ||
pip install "poetry-core>=1.1.0a6" "poetry>=1.2.0b3" | ||
poetry install | ||
- name: Lint with flake8 | ||
run: | | ||
# stop the build if there are Python syntax errors or undefined names | ||
poetry run flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics | ||
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide | ||
poetry run flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics | ||
- name: Test with pytest | ||
run: | | ||
export $(grep -v '#.*' .env.test | xargs) | ||
poetry run pytest matcher/tests/unit |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
repos: | ||
- repo: https://github.com/ambv/black | ||
rev: 22.3.0 | ||
hooks: | ||
- id: black | ||
language_version: python3.10 | ||
exclude: ^lx-importer/ | ||
- repo: https://gitlab.com/pycqa/flake8 | ||
rev: 3.9.2 | ||
hooks: | ||
- id: flake8 | ||
exclude: ^lx-importer/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
# Build image | ||
FROM python:3.10-slim | ||
RUN pip install "poetry" | ||
|
||
RUN mkdir /app | ||
WORKDIR /app | ||
ADD mymodule ./mymodule | ||
|
||
ADD setup.py . | ||
ADD pyproject.toml . | ||
|
||
RUN set -ex && \ | ||
poetry install --no-root &&\ | ||
poetry build | ||
|
||
|
||
# Final image | ||
FROM python:3.10-slim | ||
|
||
RUN mkdir /app | ||
WORKDIR /app | ||
COPY --from=0 /app/dist/mymodule-0.1.0-py3-none-any.whl ./ | ||
RUN pip install ./mymodule-0.1.0-py3-none-any.whl | ||
# this is the entrypoint that is set inside the setup.py file | ||
ENTRYPOINT ["mymodule"] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,44 @@ | ||
BASE_PROJECT_NAME := myproject | ||
PROJECT_NAME := mymodule | ||
DOCKER_IMAGE_TAG := $(BASE_PROJECT_NAME)/$(PROJECT_NAME) | ||
DOCKER_CONTAINER_NAME := $(BASE_PROJECT_NAME)-$(PROJECT_NAME) | ||
|
||
install-requirements: ## pip install requirements for app | ||
poetry install --no-root | ||
|
||
install-dev-requirements: ## pip install requirements for tests | ||
poetry install --with dev | ||
|
||
install-test-requirements: ## pip install requirements for tests | ||
poetry install --with test | ||
|
||
install-all: ## pip install requirements for app and tests | ||
poetry install | ||
|
||
test: ## run tests | ||
PYTHONPATH=. python -m pytest -sv ./matcher/tests | ||
|
||
test-cov: ## run tests with coverage reports (for Jenkins) | ||
PYTHONPATH=. python -m pytest mymodule\ | ||
-o junit_family=xunit2 -o cache_dir=/tmp \ | ||
--cov-report xml:./reports/coverage.xml --junitxml=./reports/junit-result.xml \ | ||
-sv ./mymodule/tests | ||
|
||
run: ## Run on host | ||
python -m mymodule.cli match---help | ||
|
||
docker-build: ## run docker build to create docker image | ||
docker build . -t $(DOCKER_IMAGE_TAG) | ||
|
||
docker-run-dev: ## run docker image in dev mode (with network=host and using the local .env) | ||
docker run --rm --net=host --name $(DOCKER_CONTAINER_NAME) -t $(DOCKER_IMAGE_TAG) . | ||
|
||
docker-run-test: ## run tests on docker image in dev mode (with network=host and using the local .env) | ||
docker run --rm --net=host --env-file=.env --name $(DOCKER_CONTAINER_NAME) -t $(DOCKER_IMAGE_TAG) \ | ||
python -m pytest -sv ./mymodule/tests | ||
|
||
docker-clean: ## remove docker image | ||
docker rmi $(DOCKER_IMAGE_TAG) || exit 0 | ||
|
||
help: ## This help | ||
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,119 @@ | ||
# SI-PR-01432-SchemaMatchig | ||
|
||
Intro | ||
|
||
## <a name="overview">Overview</a> | ||
Description | ||
|
||
|
||
## <a name="requirements">Requirements</a> | ||
|
||
This project has been developed using Python 3.10 and Poetry. | ||
So, `Python 3` and `pip3` are required. | ||
|
||
Dependencies are managed through the tool poetry that you can install as follows | ||
|
||
*Note: you need at least this versions to be able to use depencies groups inside pyproject.toml* | ||
|
||
``` | ||
pip install "poetry-core>=1.1.0a6" "poetry>=1.2.0b3" | ||
``` | ||
|
||
When you use poetry, you can use all the commans listed in this files following the pattern | ||
``` | ||
poetry run [tool/programa/aplication] | ||
``` | ||
If you run the commands using poetry, it will create a virtualenvironment that poetry will manage trasnparently. But the commands in this file are written taking into account that you are already using virtualenv or conda environments, so poetry doesn't need to create a managed virtualenvironment when it's run inside one of this. Then you can run the commands inside this file without the need to write `poetry run` | ||
|
||
|
||
|
||
|
||
## <a name="hooks">Pre-commit hooks</a> | ||
|
||
This project uses pre-commit hooks to check code quality and format before it s finally committed to the repository. To configure the pre-commit hooks to evaluate your code, follow this steps: | ||
|
||
1. Install pre-commit utility: `pip install pre-commit` | ||
2. Run inside the root directory the following command: `pre-commit install` | ||
3. Done | ||
|
||
After this setup, black and flake should run before each commit command | ||
|
||
``` | ||
git commit -m "example commit" | ||
[INFO] Installing environment for https://github.com/psf/black. | ||
[INFO] Once installed this environment will be reused. | ||
[INFO] This may take a few minutes... | ||
black....................................................................Passed | ||
flake8...................................................................Passed | ||
[master 12a838b] example commit | ||
``` | ||
|
||
If you don't follow the style guide, this tools will show warning and they will abort your commit | ||
|
||
``` | ||
git commit -m "example commit" | ||
black....................................................................Failed | ||
- hook id: black | ||
- files were modified by this hook | ||
reformatted example_package/common/infraestructure/timescale_engine.py | ||
All done! ✨ 🍰 ✨ | ||
1 file reformatted, 3 files left unchanged. | ||
flake8...................................................................Failed | ||
- hook id: flake8 | ||
- exit code: 1 | ||
example_package/cli.py:12:80: E501 line too long (88 > 79 characters) | ||
example_package/common/infraestructure/timescale_engine.py:10:80: E501 line too long (88 > 79 characters) | ||
``` | ||
|
||
## <a name="installation">Installation</a> | ||
|
||
The Python version used is Python 3.10. | ||
|
||
All the other required python packages may be installed using the command: | ||
|
||
```bash | ||
poetry install | ||
``` | ||
|
||
A new requirement must be added using the command `poetry add dependency` or editing de file `pyproject.toml` file. | ||
|
||
## <a name="configuration">Configuration</a> | ||
|
||
All the configuration variables are included on **.env** files. For | ||
further information read the [related documentation](https://pypi.org/project/python-dotenv/). An example **.env** file is provided `.env.example` you can use it as a base **.env** file if you rename it to `.env` | ||
|
||
|
||
|
||
## <a name="tests">Tests</a> | ||
|
||
```bash | ||
# All tests | ||
PYTHONPATH=. pytest ./src/tests | ||
|
||
# All tests verbose mode (not encouraged use logging module instead) | ||
pytest -s --log-cli-level=INFO | ||
|
||
# Unit tests | ||
pytest -s --log-cli-level=INFO src/tests/unit | ||
|
||
# Integration tests | ||
pytest -s --log-cli-level=INFO src/tests/integration | ||
|
||
# Validation tests | ||
pytest -s --log-cli-level=INFO src/tests/validation | ||
``` | ||
|
||
## <a name="run">Run</a> | ||
|
||
To run the command line without installing it, use the following command: `python3 -m src.cli --help` | ||
|
||
## Scaffolding | ||
|
||
- **src**: contiene el desarrollo realizado | ||
- **.github/workflows**: contiene los tareas de CI/CD definidas por Gradiant durante la realización del proyecto | ||
- **tools**: herramienta y utilidades de cara a ejecutar el servicio |
File renamed without changes.
This file was deleted.
Oops, something went wrong.
File renamed without changes.
File renamed without changes.
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
|
||
from typing import List | ||
from mymodule.common.infraestructure.postgres_engine import PostgresEngine | ||
from mymodule.analytics.infraestructure.clients_repository import ClientsRepository | ||
from mymodule.analytics.usecases.list_clients import ListClientUseCase | ||
|
||
class AnalyticsController: | ||
def __init__(self) -> None: | ||
self._engine = PostgresEngine() | ||
self._clients_repository = ClientsRepository(self._engine) | ||
|
||
def list_clients(self) -> List[str]: | ||
uc = ListClientUseCase(self._clients_repository) | ||
uc.execute() | ||
|
||
return uc.result |
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
from dataclasses import dataclass, field | ||
|
||
|
||
@dataclass(frozen=True) | ||
class Client: | ||
id: str = field(init=True) | ||
active: int | ||
prediction_horizon: str | ||
airport: bool | ||
weather: bool | ||
lead_time: int |
File renamed without changes.
File renamed without changes.
Oops, something went wrong.