Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: digital credential queue service tests #2330

Merged
merged 81 commits into from
Nov 27, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
81 commits
Select commit Hold shift + click to select a range
58e63e2
feat: devcontainer configuraton for vscode
amanji Sep 7, 2023
107604e
feat: hard code digital business card schema
amanji Sep 7, 2023
3793db1
feat: hard code digital business card schema
amanji Sep 7, 2023
d0c51ce
feat: issue credentials through Traction tenant
amanji Sep 11, 2023
7b88982
refactor: app initialization workflow
amanji Sep 12, 2023
ad0fa01
feat: use out-of-band invitation for connecting
amanji Sep 20, 2023
2e75d5f
feat: use v2.0 for issuing credential
amanji Sep 21, 2023
79a2631
feat: web socket implmentation with flask-socketio
amanji Oct 11, 2023
fc8edc4
feat: db migration script to enable revocation
amanji Oct 16, 2023
4fe8406
feat: revocation endpoint
amanji Oct 18, 2023
aec27f2
feat: replace endpoints
amanji Oct 19, 2023
99e54fe
chore: fix linting errors
amanji Oct 19, 2023
88e3ccc
chore: update requirements
amanji Oct 19, 2023
fd96a6e
chore: update tests
amanji Oct 25, 2023
e397e04
feat: traction token exchanger
amanji Oct 26, 2023
1f2fe2b
chore: update workflow variables
amanji Oct 26, 2023
4090b35
chore: update workflow variables
amanji Oct 26, 2023
7f1f546
refactor: ws cors setting is a config option
amanji Oct 26, 2023
a259818
chore: fix linting errors
amanji Oct 26, 2023
080825f
refactor: clean up init in digital credential service
amanji Oct 27, 2023
6beb1b1
18284: digital credentials (#2260)
amanji Oct 27, 2023
87d0854
feat: endpoints to reset credential offers
amanji Oct 28, 2023
4042676
Merge remote-tracking branch 'upstream/feature-digital-credentials' i…
amanji Oct 30, 2023
26a01e7
feat: credential id lookup table
amanji Oct 30, 2023
32a7d7a
feat: add business roles
amanji Oct 30, 2023
205dd68
18284 Add pre-fork server hook to gunicorn config (#2285)
argush3 Oct 30, 2023
c55cebc
chore: fix tests and linting
amanji Oct 30, 2023
8ad8cb9
chore: fix tests
amanji Oct 31, 2023
65a9a4d
18284 feat: digital credentials (#2281)
amanji Oct 31, 2023
2fc5cb3
Merge remote-tracking branch 'upstream/feature-digital-credentials' i…
amanji Oct 31, 2023
d368673
refactor: remove records from Traction on deletion
amanji Nov 1, 2023
0b3b60b
Revert "feat: web socket implmentation with flask-socketio"
amanji Nov 1, 2023
9763a17
fix: port so it doesnt overlap with airplay server on OSX
amanji Nov 1, 2023
633e3d9
Revert "fix: port so it doesnt overlap with airplay server on OSX"
amanji Nov 1, 2023
2976d11
feat: digital credentials (#2287)
amanji Nov 1, 2023
b22e419
Merge remote-tracking branch 'upstream/feature-digital-credentials' i…
amanji Nov 2, 2023
fc6c34a
Merge branch 'feature-digital-credentials'
amanji Nov 3, 2023
40f3af2
Merge remote-tracking branch 'upstream/main'
amanji Nov 3, 2023
2697cd6
Merge remote-tracking branch 'upstream/main'
amanji Nov 3, 2023
0224805
feat: base and scaffolding queue
amanji Oct 24, 2023
f9e6795
feat: add placeholders for events to capture
amanji Oct 24, 2023
808b58e
feat: add processor scaffolding and revocation reasons
amanji Oct 25, 2023
a4ceff6
chore: clean up code based on review comments
amanji Nov 1, 2023
18b4831
fix: 404 errors when attempting to remove invitations and credentials
amanji Nov 6, 2023
b0f66b5
refactor: update digital credential helpers
amanji Nov 7, 2023
27f4d19
feat: add init file for helpers module
amanji Nov 7, 2023
4fce169
refactor: move methods into digital credential service
amanji Nov 7, 2023
4eeebd9
feat: add query methods
amanji Nov 7, 2023
6951179
refactor: more code re-organization
amanji Nov 7, 2023
48e638c
feat: complete queue event processors
amanji Nov 8, 2023
95e54b6
feat: add manual revocation processor
amanji Nov 8, 2023
0f4834e
Merge remote-tracking branch 'upstream/main' into feature-digital-cre…
amanji Nov 8, 2023
b840527
fix: improved token validation in traction auth decorator
amanji Nov 15, 2023
18d2aaa
Merge remote-tracking branch 'upstream/main'
amanji Nov 16, 2023
822f35d
Merge branch 'main' into feature-digital-credentials
amanji Nov 16, 2023
d1a2463
chore: address first set of code review comments
amanji Nov 16, 2023
b3bbad1
chore: fix linting
amanji Nov 16, 2023
9524ecc
chore: address second set of code reivew comments
amanji Nov 16, 2023
4eae367
chore: address third set of code reivew comments
amanji Nov 16, 2023
764bdf1
Merge remote-tracking branch 'upstream/main'
amanji Nov 20, 2023
5a83dd7
Merge branch 'main' into feature-digital-credentials
amanji Nov 20, 2023
f359f9c
chore: clean up config file for edc queue
amanji Nov 20, 2023
dae32ad
feat: add DevOps config files
amanji Nov 20, 2023
ff89db3
chore: address code review comments
amanji Nov 20, 2023
48c1213
chore: fix linting errors
amanji Nov 20, 2023
1a732a9
refactor: relax error logging level
amanji Nov 20, 2023
81f8abc
chore: add k8s files
amanji Nov 20, 2023
b79f19b
Merge remote-tracking branch 'upstream/main'
amanji Nov 24, 2023
328e566
Merge branch 'main' into feature-digital-credentials
amanji Nov 24, 2023
390b009
chore: fix linting errors
amanji Nov 26, 2023
0d281f8
refactor: rename admin revoke processor
amanji Nov 26, 2023
5ec0e8d
feat: digital credential queue worker tests
amanji Nov 26, 2023
f2729b2
fix: digital credential queue worker code cleanup
amanji Nov 26, 2023
4b579c2
refactor: move fixtures to conftest
amanji Nov 26, 2023
0f720fa
chore: fix linting errors
amanji Nov 26, 2023
63ea59e
feat: add digital credential queue processor tests
amanji Nov 26, 2023
058637c
fix: update q_cli for admin filing types
amanji Nov 27, 2023
e14e872
fix: q_cli formatting
amanji Nov 27, 2023
e4aef45
fix: add another check to test dissolution
amanji Nov 27, 2023
d59384c
fix: code smells
amanji Nov 27, 2023
f3c1a00
fix: code smells
amanji Nov 27, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions queue_services/entity-digital-credentials/q_cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -87,20 +87,26 @@ def subscription_options():

payload = {
'specversion': '1.x-wip',
'type': f'bc.registry.business.{filing_type}',
'source': f'/businesses/{identifier}',
'id': str(uuid.uuid4()),
'time': datetime.utcfromtimestamp(time.time()).replace(tzinfo=timezone.utc).isoformat(),
'datacontenttype': 'application/json',
'identifier': identifier,
'data': {
'filing': {
'header': {'filingId': filing_id},
'business': {'identifier': identifier}
}
}
}

if filing_type == 'admin.revoke':
payload['type'] = 'bc.registry.admin.revoke'
else:
payload['type'] = f'bc.registry.business.{filing_type}'

if filing_id is not None:
payload['data']['filing']['header'] = {'filingId': filing_id}

await sc.publish(subject=subscription_options().get('subject'),
payload=json.dumps(payload).encode('utf-8'))

Expand All @@ -114,6 +120,7 @@ def subscription_options():
except getopt.GetoptError:
print('q_cli.py -i <identifier> -f <filing_id> -t <filing_type>')
sys.exit(2)
filing_id = None
for opt, arg in opts:
if opt == '-h':
print('q_cli.py -i <identifier> -f <filing_id> -t <filing_type>')
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ class _Config(): # pylint: disable=too-few-public-methods
DB_NAME = os.getenv('ENTITY_DATABASE_NAME', '')
DB_HOST = os.getenv('ENTITY_DATABASE_HOST', '')
DB_PORT = os.getenv('ENTITY_DATABASE_PORT', '5432')
# pylint: disable=consider-using-f-string
SQLALCHEMY_DATABASE_URI = 'postgresql://{user}:{password}@{host}:{port}/{name}'.format(
user=DB_USER,
password=DB_PASSWORD,
Expand Down Expand Up @@ -141,6 +142,7 @@ class TestConfig(_Config): # pylint: disable=too-few-public-methods
DB_HOST = os.getenv('DATABASE_TEST_HOST', '')
DB_PORT = os.getenv('DATABASE_TEST_PORT', '5432')
DEPLOYMENT_ENV = 'testing'
# pylint: disable=consider-using-f-string
SQLALCHEMY_DATABASE_URI = 'postgresql://{user}:{password}@{host}:{port}/{name}'.format(
user=DB_USER,
password=DB_PASSWORD,
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Copyright © 2023 Province of British Columbia
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""The Entity Digital Credentials service.

This module contains processors for issuing and revoking digital credentials for entity related events.
"""
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Processing manual actions."""
"""Processing admin revocation actions."""

from entity_queue_common.service_utils import logger
from legal_api.models import Business, DCRevocationReason
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ async def process(business: Business, filing_sub_type: str):
logger.warning('No issued credentials found for business: %s', business.identifier)
return None

if filing_sub_type == 'voluntary':
if filing_sub_type == 'voluntary': # pylint: disable=no-else-return
reason = DCRevocationReason.VOLUNTARY_DISSOLUTION
return replace_issued_digital_credential(business=business,
issued_credential=issued_credentials[0],
Expand All @@ -43,4 +43,4 @@ async def process(business: Business, filing_sub_type: str):
issued_credential=issued_credentials[0],
reason=reason)
else:
raise Exception('Invalid filing sub type.')
raise Exception('Invalid filing sub type.') # pylint: disable=broad-exception-raised
Original file line number Diff line number Diff line change
Expand Up @@ -31,13 +31,15 @@ def get_issued_digital_credentials(business: Business):
try:
# pylint: disable=superfluous-parens
if not (connection := DCConnection.find_active_by(business_id=business.id)):
raise Exception(f'{Business.identifier} active connection not found.')
# pylint: disable=broad-exception-raised
raise Exception(f'{business.identifier} active connection not found.')

# pylint: disable=superfluous-parens
if not (issued_credentials := DCIssuedCredential.find_by(dc_connection_id=connection.id)):
return []

return issued_credentials
# pylint: disable=broad-exception-raised
except Exception as err: # noqa: B902
raise err

Expand All @@ -48,11 +50,13 @@ def issue_digital_credential(business: Business, user: User, credential_type: DC
if not (definition := DCDefinition.find_by(DCDefinition.CredentialType[credential_type],
digital_credentials.business_schema_id,
digital_credentials.business_cred_def_id)):
# pylint: disable=broad-exception-raised
raise Exception(f'Definition not found for credential type: {credential_type}')

# pylint: disable=superfluous-parens
if not (connection := DCConnection.find_active_by(business_id=business.id)):
raise Exception(f'{Business.identifier} active connection not found.')
# pylint: disable=broad-exception-raised
raise Exception(f'{business.identifier} active connection not found.')

credential_data = DigitalCredentialsHelpers.get_digital_credential_data(business,
user,
Expand All @@ -62,7 +66,7 @@ def issue_digital_credential(business: Business, user: User, credential_type: DC
if not (response := digital_credentials.issue_credential(connection_id=connection.connection_id,
definition=definition,
data=credential_data)):
raise Exception('Failed to issue credential.')
raise Exception('Failed to issue credential.') # pylint: disable=broad-exception-raised

issued_credential = DCIssuedCredential(
dc_definition_id=definition.id,
Expand All @@ -73,6 +77,7 @@ def issue_digital_credential(business: Business, user: User, credential_type: DC
issued_credential.save()

return issued_credential
# pylint: disable=broad-exception-raised
except Exception as err: # noqa: B902
raise err

Expand All @@ -83,22 +88,25 @@ def revoke_issued_digital_credential(business: Business,
"""Revoke an issued digital credential for a business."""
try:
if not issued_credential.is_issued or issued_credential.is_revoked:
# pylint: disable=broad-exception-raised
raise Exception('Credential is not issued yet or is revoked already.')

# pylint: disable=superfluous-parens
if not (connection := DCConnection.find_active_by(business_id=business.id)):
raise Exception(f'{Business.identifier} active connection not found.')
# pylint: disable=broad-exception-raised
raise Exception(f'{business.identifier} active connection not found.')

if (revoked := digital_credentials.revoke_credential(connection.connection_id,
issued_credential.credential_revocation_id,
issued_credential.revocation_registry_id,
reason) is None):
raise Exception('Failed to revoke credential.')
raise Exception('Failed to revoke credential.') # pylint: disable=broad-exception-raised

issued_credential.is_revoked = True
issued_credential.save()

return revoked
# pylint: disable=broad-exception-raised
except Exception as err: # noqa: B902
raise err

Expand All @@ -116,17 +124,20 @@ def replace_issued_digital_credential(business: Business,
issued_credential.credential_exchange_id) is not None and
digital_credentials.remove_credential_exchange_record(
issued_credential.credential_exchange_id) is None):
raise Exception('Failed to remove credential exchange record.')
raise Exception('Failed to remove credential exchange record.') # pylint: disable=broad-exception-raised

if not (issued_business_user_credential := DCIssuedBusinessUserCredential.find_by_id(
dc_issued_business_user_id=issued_credential.credential_id)):
# pylint: disable=broad-exception-raised
raise Exception('Unable to find buisness user for issued credential.')

if not (user := User.find_by_id(issued_business_user_credential.user_id)): # pylint: disable=superfluous-parens
# pylint: disable=broad-exception-raised
raise Exception('Unable to find user for issued business user credential.')

issued_credential.delete()

return issue_digital_credential(business, user, credential_type)
# pylint: disable=broad-exception-raised
except Exception as err: # noqa: B902
raise err
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
"""
import json
import os
from enum import Enum

import nats
from entity_queue_common.service import QueueServiceManager
Expand All @@ -40,10 +41,10 @@

from entity_digital_credentials import config
from entity_digital_credentials.digital_credentials_processors import (
admin_revoke,
business_number,
change_of_registration,
dissolution,
manual,
put_back_on,
)

Expand All @@ -62,15 +63,30 @@
flags.init_app(FLASK_APP)


class AdminMessage(Enum):
"""Entity Digital Credential admin message type."""

REVOKE = 'bc.registry.admin.revoke'


class BusinessMessage(Enum):
"""Entity Digital Credential business message type."""

BN = 'bc.registry.business.bn'
CHANGE_OF_REGISTRATION = f'bc.registry.business.{FilingCore.FilingTypes.CHANGEOFREGISTRATION.value}'
DISSOLUTION = f'bc.registry.business.{FilingCore.FilingTypes.DISSOLUTION.value}'
PUT_BACK_ON = f'bc.registry.business.{FilingCore.FilingTypes.PUTBACKON.value}'


async def process_digital_credential(dc_msg: dict, flask_app: Flask):
# pylint: disable=too-many-branches, too-many-statements
"""Process any digital credential messages in queue."""
if not dc_msg or dc_msg.get('type') not in [
f'bc.registry.business.{FilingCore.FilingTypes.CHANGEOFREGISTRATION.value}',
f'bc.registry.business.{FilingCore.FilingTypes.DISSOLUTION.value}',
f'bc.registry.business.{FilingCore.FilingTypes.PUTBACKON.value}',
'bc.registry.admin.bn',
'bc.registry.admin.manual'
BusinessMessage.CHANGE_OF_REGISTRATION.value,
BusinessMessage.DISSOLUTION.value,
BusinessMessage.PUT_BACK_ON.value,
BusinessMessage.BN.value,
AdminMessage.REVOKE.value
]:
return None

Expand All @@ -80,26 +96,27 @@ async def process_digital_credential(dc_msg: dict, flask_app: Flask):
with flask_app.app_context():
logger.debug('Attempting to process digital credential message: %s', dc_msg)

if dc_msg['type'] in ('bc.registry.business.bn', 'bc.registry.business.manual'):
if dc_msg['type'] in (BusinessMessage.BN.value, AdminMessage.REVOKE.value):
# When a BN is added or changed or there is a manuak administrative update the queue message does not have
# a data object. We queue the business information using the identifier and revoke/reissue the credential
# immediately.
if dc_msg['identifier'] is None:
if dc_msg.get('identifier') is None:
raise QueueException('Digital credential message is missing identifier')

identifier = dc_msg['identifier']
if not (business := Business.find_by_identifier(identifier)): # pylint: disable=superfluous-parens
# pylint: disable=broad-exception-raised
raise Exception(f'Business with identifier: {identifier} not found.')

if dc_msg['type'] == 'bc.registry.business.bn':
if dc_msg['type'] == BusinessMessage.BN.value:
await business_number.process(business)
elif dc_msg['type'] == 'bc.registry.business.manual':
await manual.process(business)
elif dc_msg['type'] == AdminMessage.REVOKE.value:
await admin_revoke.process(business)
else:
if dc_msg['data'] is None \
or dc_msg['data']['filing'] is None \
or dc_msg['data']['filing']['header'] is None \
or dc_msg['data']['filing']['header']['filingId'] is None:
if dc_msg.get('data') is None \
or dc_msg.get('data').get('filing') is None \
or dc_msg.get('data').get('filing').get('header') is None \
or dc_msg.get('data').get('filing').get('header').get('filingId') is None:
raise QueueException('Digital credential message is missing data.')

filing_id = dc_msg['data']['filing']['header']['filingId']
Expand All @@ -115,6 +132,7 @@ async def process_digital_credential(dc_msg: dict, flask_app: Flask):

business_id = filing.business_id
if not (business := Business.find_by_internal_id(business_id)): # pylint: disable=superfluous-parens
# pylint: disable=broad-exception-raised
raise Exception(f'Business with internal id: {business_id} not found.')

# Process individual filing events
Expand All @@ -140,6 +158,6 @@ async def cb_subscription_handler(msg: nats.aio.client.Msg):
logger.error('Queue Blocked - Database Issue: %s',
json.dumps(dc_msg), exc_info=True)
raise err # We don't want to handle the error, as a DB down would drain the queue
except (QueueException, Exception) as err: # noqa B902; pylint: disable=W0703;
except (QueueException, Exception) as err: # noqa B902; pylint: disable=W0703, disable=unused-variable
# Catch Exception so that any error is still caught and the message is removed from the queue
logger.error('Queue Error: %s', json.dumps(dc_msg), exc_info=True)
14 changes: 14 additions & 0 deletions queue_services/entity-digital-credentials/tests/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Copyright © 2023 Province of British Columbia
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""The Test Suites to ensure that the service is built and operating correctly."""
Loading
Loading