build(docker): Create Docker Compose file for local setup WITHOUT usi…

…ng Postgres/LogTo [DEV-2681] (#250)

* Add RUN command for migrating.

* Update package-lock.json

* Create docker-compose.yml file for running locally
without using Postgres/LogTo.

* Add a new command using for docker-compose.

* Update Dockerfile and docker-compose.yml.

* Add start and stop docker compose scripts.

* Update start-compose.sh.

* Move Dockerfile and docker-compose.yml to "docker"
directory.

* Remove unused environment variables.

* Update docker-compose.yml.

* Remove start-compose and stop-compose scripts.

* Update .env.example.

* Update README.md.

* Update build.yml.

* Update README.md.

* Fix typo in README.md.

* Update .env.example and docker-compose.yml files.

* Update README.md.

* Update docker-compose.yml.

* Update Dockerfile and docker-compose.yml.

* Update Dockerfile.

* Remove unused ENV variables from Dockerfile.

* Update README.md.

* Add 2 profiles option(credential-service and credential-service-with-external-db)
to docker-compose.yml.

* Update README.md.

* Change docker image in docker-compose.yml.

* Change Docker service port from "8787" to "3000".

* bump deps

* Delete custom_button.js

* Move Swagger file

* Update package-lock.json

* Update Dockerfile

* No DB

* checkin compose

* Update docker-compose-no-db.yml

* more compose

* Fix problem with an invalid path to swagger.json.

* Fix bug with an incorrect logic of
ENABLE_EXTERNAL_DB toggle.

* Update .env.no-db file.

* Change the app image to an existing image.

* Update package.json.

* Update comment in docker-compose-no-db.yml.

* Rename environment variables

* Move Swagger back to top level

* Remove unnecessary NPM commands

* Update app.ts

* Set image to latest

* Rename with DB file

* Update .env.no-db

* Change guards

* Bump package

* Change env file name

* Rename example ENV

* Update .gitignore

* Fix problem with an invalid path to swagger.json.

* Update .dockerignore

* Update build.yml

* Downgrade did-provider-cheqd package

* Update package-lock.json

* downgrade did-provider-cheqd again

* Set default for ENABLE variables

* Update package.json

* Update package-lock.json

* Move files

* Update package-lock.json

* Checkin new folder structure

* Update package.json and package-lock.json files.

* Merge branch 'develop' into DEV-2681

* Bump package

* Fix paths

* Move files around

* Fix folder includes in tsconfig

* Move files around

* convert to typescript

* Fix TypeScript paths

* Update package.json

* Remove includes

* Update Dockerfile

* Revert "Update Dockerfile"

This reverts commit 481149c.

* Fix missing env

* Update README.md

---------

Co-authored-by: DaevMithran <61043607+DaevMithran@users.noreply.github.com>
Co-authored-by: Ankur Banerjee <ankurdotb@users.noreply.github.com>

.dockerignore

@@ -3,11 +3,10 @@
 **/.github
 **/node_modules
 **/dist
+**/docker
 
 # Skip unnecessary files
 **/*.md
-Dockerfile
-docker-compose.yml
 
 # Exceptions
 !LICENSE

.github/workflows/build.yml

@@ -47,6 +47,7 @@ jobs:
         uses: docker/metadata-action@v4
         with:
           images: |
+            ${{ env.IMAGE_NAME }}
             ghcr.io/${{ env.IMAGE_NAME }}
             registry.digitalocean.com/${{ env.IMAGE_NAME }}
           tags: |
@@ -58,7 +59,7 @@ jobs:
         uses: docker/build-push-action@v4
         with:
           context: .
-          file: Dockerfile
+          file: docker/Dockerfile
           platforms: linux/amd64
           load: true
           target: runner

.gitignore

@@ -1,11 +1,15 @@
 ### APP-SPECIFIC EXCLUSIONS ###
+
+### Folders ###
 tmp
-database.sqlite
-tsconfig.tsbuildinfo
-worker/
-transpiled
+
+### Files ###
 .env
 .snyk
+**/docker-compose-no-db-local.yml
+database.sqlite
+**/no-db-local.env
+tsconfig.tsbuildinfo
 
 ### GENERAL EXCLUSIONS ###
 

README.md

@@ -10,37 +10,14 @@
 
 The purpose of this service is to issue and verify credentials. This service by itself does not take care of storing the credentials. If you'd like to store credentials, you would have to pair this service with [secret-box-service](https://github.com/cheqd/secret-box-service.git). This service is also dependent on [auth0-service](https://github.com/cheqd/auth0-service)
 
-## 📖 Endpoints
-
-### Issue a credential
-
-- **Endpoint** POST `/credentials/issue`
-- **Accepts**: `application/json`
-- **Request Body**: JSON object with following fields
-  - `attributes` - A json object with all the credential attributes
-  - `subjectDid` - DID of the holder of the credential
-  - `type` - A string representation of the credential type e.g. "PERSON" (optional)
-  - `@context` - context of the issued credential (optional)
-  - `expirationDate` - Date of expiration of the JWT (optional)
-- **Success Response Code**: 200
-- **Invalid Request Response Code** - 400
-- **Internal Error Response Code** - 500
-
-### Verify a Credential
-
-- **Endpoint** POST `/credentials/verify`
-- **Accepts**: `application/json`
-- **Request Body**: JSON object with following fields:
-  - `credential` - A verifiable credential or the JWT string
-- **Success Response Code** - 200
-- **Invalid Request Response Code**:
-  - 400: Bad request body
-  - 405: Wrong content type
-- **Internal Error Response Code** - 500
-
-### Health Check
-
-- **Endpoint**: `/` (This endpoint redirects to the swagger api docs)
+## 📖 Usage
+
+We run hosted endpoints for this package (in case you don't want to run it yourself) which have Swagger / OpenAPI definition endpoints that list all of the APIs and how they work.
+
+The Swagger API definition pages are:
+
+- [Production / Stable Release APIs](https://credential-service.cheqd.net/swagger/)
+- [Staging / Development Release APIs](https://credential-service-staging.cheqd.net/swagger/)
 
 ## 🔧 Configuration
 
@@ -50,34 +27,35 @@ The application allows configuring the following parameters using environment va
 
 #### Network API endpoints
 
-1. `MAINNET_RPC_URL`: RPC endpoint for cheqd mainnet. (Default: `https://rpc.cheqd.net:443`)
-2. `TESTNET_RPC_URL`: RPC endpoint for cheqd testnet. (`https://rpc.cheqd.network:443`)
+1. `MAINNET_RPC_URL`: RPC endpoint for cheqd mainnet (Default: `https://rpc.cheqd.net:443`).
+2. `TESTNET_RPC_URL`: RPC endpoint for cheqd testnet (`https://rpc.cheqd.network:443`).
 3. `RESOLVER_URL`: API endpoint for a [DID Resolver](https://github.com/cheqd/did-resolver) endpoint that supports `did:cheqd`.
-4. `APPLICATION_BASE_URL`: URL of the application (external domain name)
+4. `APPLICATION_BASE_URL`: URL of the application (external domain name).
+5. `ALLOWED_ORIGINS`: CORS allowed origins used in the app.
 
 #### Veramo KMS Database
 
 The application supports two modes in which keys are managed: either just storing them in-memory while a container is running, or persisting them in a PostgresSQL database with Veramo SDK. Using an external Postgres database allows for "custodian" mode where identity and cheqd/Cosmos keys can be offloaded by client applications to be stored in the database.
 
-1. `DB_CONNECTION_URL`: Postgres database connection URL, e.g. `postgres://<user>:<password>@<host>:<port>/<database>`
-2. `DB_ENCRYPTION_KEY`: Secret key used to encrypt the Veramo key-specific database tables. This adds a layer of protection by not storing the database in plaintext.
-3. `DB_CERTIFICATE`: Custom CA certificate required to connect to the database (optional).
+1. `ENABLE_EXTERNAL_DB`: Turns external database on/off (Default: `false`). If `ENABLE_EXTERNAL_DB=true`, then define below environment variables in `.env` file:
+    - `EXTERNAL_DB_CONNECTION_URL`: Postgres database connection URL, e.g. `postgres://<user>:<password>@<host>:<port>/<database>`.
+    - `EXTERNAL_DB_ENCRYPTION_KEY`: Secret key used to encrypt the Veramo key-specific database tables. This adds a layer of protection by not storing the database in plaintext.
+    - `EXTERNAL_DB_CERTIFICATE`: Custom CA certificate required to connect to the database (optional).
 
 #### API Authentication using LogTo
 
 By default, the application has API authentication disabled (which can be changed in configuration). If, however, you'd like to run the app with API authentication features, the following variables need to be configured.
 
 We use a self-hosted version of [LogTo](https://logto.io/), which supports OpenID Connect. Theoretically, these values could also be replaced with [LogTo Cloud](http://cloud.logto.io/) or any other OpenID Connect identity provider.
 
-1. `ENABLE_AUTHENTICATION`: Turns API authentication guards on/off. (Default: `false`)
+1. `ENABLE_AUTHENTICATION`: Turns API authentication guards on/off (Default: `false`). If `ENABLE_AUTHENTICATION=false`, then define below environment variable in `.env` file:
+    - `DEFAULT_CUSTOMER_ID`: Customer/user in LogTo to use for unauthenticated users.
 2. `LOGTO_ENDPOINT`: API endpoint for LogTo server
 3. `LOGTO_RESOURCE_URL`: API resource associated with application
 4. `LOGTO_APP_ID`: Application ID from LogTo. For now, Application is supposed to be a TraditionalWeb
 5. `LOGTO_APP_SECRET`: Application secret. Also should encrypted in deployment
-6. `ALLOWED_ORIGINS`: CORS allowed origins used in the app
-7. `DEFAULT_CUSTOMER_ID`: Customer/user in LogTo to use for unauthenticated users
-8. `ALL_SCOPES`: List of all scopes. Should be a string with scopes divided by whitespace, like `account:create account:read did:create`
-9. `COOKIE_SECRET`: Secret for cookie encryption.
+6. `ALL_SCOPES`: List of all scopes. Should be a string with scopes divided by whitespace, like `account:create account:read did:create`
+7. `COOKIE_SECRET`: Secret for cookie encryption.
 
 ### 3rd Party Connectors
 
@@ -87,46 +65,49 @@ The app supports 3rd party connectors for credential storage and delivery.
 
 The app's [Verida Network](https://www.verida.network/) connector can be enabled to deliver generated credentials to Verida Wallet.
 
-1. `ENABLE_VERIDA_CONNECTOR`: Turns Verida connector on/off. (Default: `false`)
-2. `VERIDA_NETWORK`: Verida Network type to connect to. (Default: `testnet`)
-3. `VERIDA_PRIVATE_KEY`: Secret key for Verida Network API.
-4. `POLYGON_RPC_URL`: Polygon Network RPC URL for connections.
-5. `POLYGON_PRIVATE_KEY`: Secret key for Polygon Network.
+1. `ENABLE_VERIDA_CONNECTOR`: Turns Verida connector on/off (Default: `false`). If `ENABLE_VERIDA_CONNECTOR=true`, then define below environment variables in `.env` file:
+    - `VERIDA_NETWORK`: Verida Network type to connect to. (Default: `testnet`)
+    - `VERIDA_PRIVATE_KEY`: Secret key for Verida Network API.
+    - `POLYGON_PRIVATE_KEY`: Secret key for Polygon Network.
+
+## 🧑‍💻🛠 Developer Guide
 
-### Run the application
+### Run as standalone application using Docker Compose
 
-Initiate a Postgres database, in case you're using an external database.
+If you want to run the application without any external databases or dependent services, we provide [a Docker Compose file](docker/no-external-db/docker-compose-no-db.yml) to spin up a standalone service.
 
 ```bash
-docker pull postgres
-docker run --name some-postgres -e POSTGRES_PASSWORD=mysecretpassword -d postgres
+docker compose -f docker/no-external-db/docker-compose-no-db.yml up --detach
 ```
 
-Construct the postgres url and configure the env variables mentioned above
+This standalone service uses an in-memory database with no persistence, and therefore is recommended only if you're managing key/secret storage separately.
 
-Once configured, the app can be run using NPM:
+The [`no-db.env` file](docker/no-external-db/no-db.env) in the same folder contains all the environment variables necessary to configure the service. (See section *Configuration* above.)
 
-```bash
-npm start
-```
+### Run with external Key Management System (KMS) and/or authentication service using Docker Compose
 
-## 🧑‍💻🛠 Developer Guide
+Construct the postgres URL and configure the env variables mentioned above.
 
-### Build using NPM
+Spinning up a Docker container from the [pre-built credential-service Docker image on Github](https://github.com/cheqd/credential-service/pkgs/container/credential-service) is as simple as the command below:
 
-Dependencies can be installed using NPM or any other node package manager.
+- Running credential-service using Docker with external database:
+  - Set `POSTGRES_USER`, `POSTGRES_PASSWORD`, `POSTGRES_DB` environment variables in `docker/.env`:
+    - `POSTGRES_USER`: Postgres database username using in Docker database service.
+    - `POSTGRES_PASSWORD`: Postgres database password using in Docker database service.
+    - `POSTGRES_DB`: Postgres database name using in Docker database service.
+
+Run credential-service with external database:
 
 ```bash
-npm install
-npm run build
+docker compose -f docker/docker-compose.yml --profile credential-service-with-external-db up --detach
 ```
 
 ### Build using Docker
 
-To build and run in Docker, use the [Dockerfile](Dockerfile) provided.
+To build your own image using Docker, use the [Dockerfile](docker/Dockerfile) provided.
 
 ```bash
-docker build -t credential-service .
+docker build --file docker/Dockerfile --target runner . --tag credential-service:local
 ```
 
 ## 🐞 Bug reports & 🤔 feature requests
@@ -142,4 +123,3 @@ Please reach out to us there for discussions, help, and feedback on the project.
 ## 🙋 Find us elsewhere
 
 [![Telegram](https://img.shields.io/badge/Telegram-2CA5E0?style=for-the-badge\&logo=telegram\&logoColor=white)](https://t.me/cheqd) [![Discord](https://img.shields.io/badge/Discord-7289DA?style=for-the-badge\&logo=discord\&logoColor=white)](http://cheqd.link/discord-github) [![Twitter](https://img.shields.io/badge/Twitter-1DA1F2?style=for-the-badge\&logo=twitter\&logoColor=white)](https://twitter.com/intent/follow?screen\_name=cheqd\_io) [![LinkedIn](https://img.shields.io/badge/LinkedIn-0077B5?style=for-the-badge\&logo=linkedin\&logoColor=white)](http://cheqd.link/linkedin) [![Slack](https://img.shields.io/badge/Slack-4A154B?style=for-the-badge\&logo=slack\&logoColor=white)](http://cheqd.link/join-cheqd-slack) [![Medium](https://img.shields.io/badge/Medium-12100E?style=for-the-badge\&logo=medium\&logoColor=white)](https://blog.cheqd.io) [![YouTube](https://img.shields.io/badge/YouTube-FF0000?style=for-the-badge\&logo=youtube\&logoColor=white)](https://www.youtube.com/channel/UCBUGvvH6t3BAYo5u41hJPzw/)
-

Dockerfile → docker/Dockerfile

@@ -50,9 +50,10 @@ ARG TESTNET_RPC_URL=https://rpc.cheqd.network:443
 ARG RESOLVER_URL=https://resolver.cheqd.net/1.0/identifiers/
 
 # Veramo Database configuration: build-time
-ARG DB_CONNECTION_URL
-ARG DB_ENCRYPTION_KEY
-ARG DB_CERTIFICATE
+ARG ENABLE_EXTERNAL_DB=false
+ARG EXTERNAL_DB_CONNECTION_URL
+ARG EXTERNAL_DB_ENCRYPTION_KEY
+ARG EXTERNAL_DB_CERT
 
 #  LogTo: build-time
 ARG ENABLE_AUTHENTICATION=false
@@ -83,9 +84,10 @@ ENV RESOLVER_URL ${RESOLVER_URL}
 ENV APPLICATION_BASE_URL ${APPLICATION_BASE_URL}
 
 # Environment variables: Veramo Database configuration
-ENV DB_CONNECTION_URL ${DB_CONNECTION_URL}
-ENV DB_ENCRYPTION_KEY ${DB_ENCRYPTION_KEY}
-ENV DB_CERTIFICATE ${DB_CERTIFICATE}
+ENV ENABLE_EXTERNAL_DB ${ENABLE_EXTERNAL_DB}
+ENV EXTERNAL_DB_CONNECTION_URL ${EXTERNAL_DB_CONNECTION_URL}
+ENV EXTERNAL_DB_ENCRYPTION_KEY ${EXTERNAL_DB_ENCRYPTION_KEY}
+ENV EXTERNAL_DB_CERT ${EXTERNAL_DB_CERT}
 
 # Environment variables: LogTo
 ENV ENABLE_AUTHENTICATION ${ENABLE_AUTHENTICATION}

docker/no-external-db/docker-compose-no-db.yml

@@ -0,0 +1,13 @@
+version: '3.8'
+
+# CAUTION: Please ensure you edit necessary values in .env.no-db before using this Docker Compose file.
+
+# SYNTAX: docker compose -f docker/docker-compose-no-db.yml up --detach
+
+services:
+  app:
+    image: ghcr.io/cheqd/credential-service:latest
+    ports:
+      - 3000:3000
+    env_file:
+      - no-db.env

docker/no-external-db/no-db.env

@@ -0,0 +1,23 @@
+MAINNET_RPC_URL="https://rpc.cheqd.net:443"
+TESTNET_RPC_URL="https://rpc.cheqd.network:443"
+RESOLVER_URL="https://resolver.cheqd.net/1.0/identifiers/"
+APPLICATION_BASE_URL="http://localhost:3000"
+ALLOWED_ORIGINS="http://localhost:3000"
+
+# Database
+ENABLE_EXTERNAL_DB="false"
+
+# Authentication
+ENABLE_AUTHENTICATION="false"
+
+# verida
+ENABLE_VERIDA_CONNECTOR="false"
+VERIDA_PRIVATE_KEY="akjvncanv....avoa"
+POLYGON_PRIVATE_KEY="alnvca...dvncioa"
+VERIDA_NETWORK="testnet"
+
+# Standalone install (no external DB for KMS)
+ISSUER_PUBLIC_KEY_HEX="alnvca...dvncioa"
+ISSUER_PRIVATE_KEY_HEX="akjvncanv....avoa"
+ISSUER_DID="did:cheqd:testnet:afcnoa...adv"
+DEFAULT_FEE_PAYER_MNEMONIC="sketch mountain ....."

docker/with-external-db/docker-compose-with-db.yml

@@ -0,0 +1,67 @@
+version: '3.8'
+
+# CAUTION: Please ensure you edit necessary values in .env before using this Docker Compose file.
+
+# SYNTAX: docker compose -f docker/docker-compose.yml up --detach
+
+services:
+  app:
+    image: ghcr.io/cheqd/credential-service:latest
+    depends_on:
+      postgres:
+        condition: service_healthy
+    ports:
+      - 3000:3000
+    env_file:
+      - .env.with-db
+
+  app-migrations:
+    image: ghcr.io/cheqd/credential-service:latest
+    depends_on:
+      postgres:
+        condition: service_healthy
+    ports:
+      - 3000:3000
+    env_file:
+      - .env.with-db
+    entrypoint: [ "npm", "run", "migrate", "&&", "npm", "run", "start" ]
+    profiles:
+      - setup
+
+  logto:
+    image: ghcr.io/cheqd/creds-auth:latest
+    ports:
+      - 3001:3001
+      - 3002:3002
+    env_file:
+      - .env
+    profiles:
+      - logto
+
+  logto-migrations:
+    image: ghcr.io/cheqd/creds-auth:latest
+    ports:
+      - 3001:3001
+      - 3002:3002
+    env_file:
+      - .env
+    entrypoint: [ "npm", "run", "migrate", "&&", "npm", "run", "seed" ]
+    profiles:
+      - setup
+      - logto
+
+  postgres:
+    image: postgres
+    user: postgres
+    ports:
+      - 5432:5432
+    env_file:
+      - .env
+    healthcheck:
+      test: [ "CMD-SHELL", "pg_isready" ]
+      interval: 10s
+      timeout: 5s
+      retries: 5
+    profiles:
+      - external-kms
+      - logto