Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🔥 Remove redis, websockets, sse, and webhooks services #1102

Merged
merged 39 commits into from
Oct 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
e2d2f08
:construction_worker: :fire: Remove redis from docker-compose and manage
ff137 Oct 9, 2024
c08ce4a
:art: rewrite comments for nats events
ff137 Oct 9, 2024
830c62d
:fire: delete redis-expire script
ff137 Oct 9, 2024
63ea92d
:heavy_minus_sign: remove redis dependency from endorser
ff137 Oct 9, 2024
70f753c
:construction_worker: replace use of `codenvy/che-ip` image
ff137 Oct 9, 2024
69f8feb
:construction_worker: :art: Replace redis_events reference with nats_…
ff137 Oct 9, 2024
248b929
:construction_worker: :fire: :art: Remove references to webhooks service
ff137 Oct 9, 2024
7a54122
:fire: delete webhooks module
ff137 Oct 9, 2024
6b1832a
:fire: remove webhooks endpoint, service and tests
ff137 Oct 9, 2024
e988381
:fire: remove webhooks dockerfile and mise tasks
ff137 Oct 9, 2024
82bab3a
:fire: remove redis service
ff137 Oct 9, 2024
aeceb90
:fire: delete websocket service and endpoint
ff137 Oct 9, 2024
386bb52
:heavy_minus_sign: remove fastapi_websocket_pubsub
ff137 Oct 9, 2024
01080e7
:heavy_minus_sign: remove extra packages from uvicorn install
ff137 Oct 9, 2024
c3e18d1
:wrench: :fire: delete redis/webhooks specific config files
ff137 Oct 9, 2024
663118e
:fire: delete webhooks e2e test
ff137 Oct 9, 2024
7d6034f
:art: update readme docs
ff137 Oct 9, 2024
ce536f2
:art: redis -> nats
ff137 Oct 9, 2024
d8332bd
:art: remove webhooks config
ff137 Oct 9, 2024
f0d87de
:art: remove lifespan check in test
ff137 Oct 9, 2024
011f901
:art: remove webhooks related env vars
ff137 Oct 9, 2024
d6f5b8c
:fire::art: remove SseListern's wait_for_state method
ff137 Oct 9, 2024
16e4715
:fire::art: remove sse, websocket endpoints from app main, and update…
ff137 Oct 9, 2024
663a64e
:arrow_up: Update lock files
ff137 Oct 9, 2024
c0d0995
:art: remove unused `look_back` arg
ff137 Oct 9, 2024
057c73e
:art:
ff137 Oct 9, 2024
0cc97f0
:art: remove billing service env vars
ff137 Oct 9, 2024
a15e8e2
:art: update project version
ff137 Oct 9, 2024
48f266d
:construction_worker: add Waypoint module to dependabot
ff137 Oct 9, 2024
50a6292
:construction: replace SSE URL to use Waypoint
ff137 Oct 9, 2024
4688c0f
:memo: Update openapi docs
ff137 Oct 10, 2024
32691dd
:art: remove unused WEBHOOK_TOPIC_ALL
ff137 Oct 10, 2024
2856b0c
:art: remove unused import
ff137 Oct 10, 2024
e59fed4
:fire::art: re-add SSE routes, and only remove endpoints that used We…
ff137 Oct 10, 2024
1e64c60
:rewind: revert k6 changes
ff137 Oct 10, 2024
b966d9f
:memo: Update openapi specs to include the one sse endpoint
ff137 Oct 10, 2024
9455773
:art: add init modules
ff137 Oct 10, 2024
50e85e3
:art: remove `look_back` references
ff137 Oct 10, 2024
50b3e8d
:art:
ff137 Oct 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@
!endorser/
!trustregistry/
!waypoint/
!webhooks/
!scripts/
!configuration/

Expand Down
1 change: 0 additions & 1 deletion .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -14,5 +14,4 @@ export TENANT_ADMIN_FASTAPI_ENDPOINT=http://cloudapi.127.0.0.1.nip.io/tenant-adm
export TENANT_FASTAPI_ENDPOINT=http://cloudapi.127.0.0.1.nip.io/tenant
export TRUST_REGISTRY_FASTAPI_ENDPOINT=http://cloudapi.127.0.0.1.nip.io/public
export TRUST_REGISTRY_URL=http://trust-registry.cloudapi.127.0.0.1.nip.io
export WEBHOOKS_URL=http://webhooks.cloudapi.127.0.0.1.nip.io
export WAYPOINT_URL=http://waypoint.cloudapi.127.0.0.1.nip.io
4 changes: 2 additions & 2 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,9 @@ updates:
day: saturday
time: "07:00"

# Maintain dependencies for Webhooks Service submodule
# Maintain dependencies for the Waypoint submodule
- package-ecosystem: pip
directory: /webhooks
directory: /waypoint
schedule:
interval: weekly
day: saturday
Expand Down
1 change: 0 additions & 1 deletion .github/workflows/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ PROJECTS=(
governance-ga-agent
governance-trust-registry
governance-multitenant-web
governance-webhooks-web
governance-multitenant-agent
governance-endorser
)
Expand Down
7 changes: 1 addition & 6 deletions .github/workflows/continuous-deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,6 @@ jobs:
governance-ga-web,
governance-tenant-web,
governance-public-web,
governance-webhooks-web,
governance-multitenant-agent,
governance-endorser,
pytest,
Expand Down Expand Up @@ -99,10 +98,6 @@ jobs:
context: .
file: dockerfiles/fastapi/Dockerfile
platforms: linux/amd64,linux/arm64
- image: governance-webhooks-web
context: .
file: dockerfiles/webhooks/Dockerfile
platforms: linux/amd64,linux/arm64
- image: governance-multitenant-agent
context: .
file: dockerfiles/agents/Dockerfile.author.agent
Expand Down Expand Up @@ -196,7 +191,7 @@ jobs:
run: mise run poetry:install:all
- name: Run Pylint
run: |
poetry run pylint app/ endorser/ shared/ trustregistry/ waypoint/ webhooks/ --rcfile=.pylintrc -r n --msg-template="{path}:{line}: [{msg_id}({symbol}), {obj}] {msg}" --exit-zero > pylintreport.txt
poetry run pylint app/ endorser/ shared/ trustregistry/ waypoint/ --rcfile=.pylintrc -r n --msg-template="{path}:{line}: [{msg_id}({symbol}), {obj}] {msg}" --exit-zero > pylintreport.txt

test:
if: github.event.pull_request.draft == false
Expand Down
7 changes: 0 additions & 7 deletions .mise/tasks/poetry/install/webhooks

This file was deleted.

7 changes: 0 additions & 7 deletions .mise/tasks/poetry/update/webhooks

This file was deleted.

48 changes: 6 additions & 42 deletions app/main.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
import io
import os
import traceback
from contextlib import asynccontextmanager

import pydantic
import yaml
Expand All @@ -22,15 +21,12 @@
sse,
trust_registry,
verifier,
webhooks,
websocket_endpoint,
)
from app.routes.admin import tenants
from app.routes.wallet import credentials as wallet_credentials
from app.routes.wallet import dids as wallet_dids
from app.routes.wallet import jws as wallet_jws
from app.routes.wallet import sd_jws as wallet_sd_jws
from app.services.event_handling.websocket_manager import WebsocketManager
from app.util.extract_validation_error import extract_validation_error_msg
from shared.constants import PROJECT_VERSION
from shared.exceptions import CloudApiValueError
Expand All @@ -41,31 +37,10 @@
ROOT_PATH = os.getenv("ROOT_PATH", "")

cloud_api_docs_description = """
Welcome to the Aries CloudAPI Python project.

In addition to the traditional HTTP-based endpoints described below, we also offer WebSocket endpoints for
real-time interfacing with webhook events.

WebSocket endpoints are authenticated. This means that only users with valid authentication tokens can establish
a WebSocket connection, and they can only subscribe to their own wallet's events. However, Admin users have the
ability to subscribe by topic, or to any wallet in their group.

Our WebSocket endpoints are as follows:

1. `/v1/ws/`: This endpoint allows admins to receive all webhook events for their group.

2. `/v1/ws/{wallet_id}`: This endpoint allows admins (or authenticated users holding this wallet) to receive webhook
events for a specific wallet ID.

3. `/v1/ws/{wallet_id}/{topic}`: Similar to above, but subscribing to a specific topic.
Welcome to the Aries CloudAPI Python project!

4. `/v1/ws/topic/{topic}`: This endpoint allows admins to receive all webhook events on a specific topic (e.g.
`connections`, `credentials`, `proofs`, `endorsements`).

For authentication, the WebSocket headers should include `x-api-key`: `<your key>`.

Please refer to our API documentation for more details about our authentication mechanism, as well as for information
about the available topics.
For detailed guidance on using the API, please visit our official documentation:
https://www.didx.co.za/ssi-dev-portal/docs/Welcome.
"""

default_docs_description = """
Expand All @@ -77,19 +52,8 @@
debug = not prod


@asynccontextmanager
async def lifespan(_: FastAPI):
# Startup logic occurs before yield
yield
# Shutdown logic occurs after yield
logger.info("Calling WebsocketManager shutdown")
await WebsocketManager.disconnect_all()


webhook_routes = [webhooks, sse, websocket_endpoint]

trust_registry_routes = [trust_registry]
tenant_admin_routes = [tenants] + webhook_routes
tenant_admin_routes = [tenants, sse]
tenant_routes = [
connections,
definitions,
Expand All @@ -102,7 +66,8 @@ async def lifespan(_: FastAPI):
wallet_dids,
wallet_jws,
wallet_sd_jws,
] + webhook_routes
sse,
]


def routes_for_role(role: str) -> list:
Expand Down Expand Up @@ -131,7 +96,6 @@ def create_app() -> FastAPI:
title=OPENAPI_NAME,
version=PROJECT_VERSION,
description=cloud_api_description(ROLE),
lifespan=lifespan,
debug=debug,
redoc_url=None,
docs_url=None,
Expand Down
Loading
Loading