feat(SATP-Hermes): add gateway coordinator WIP

Signed-off-by: Rafael Belchior <rafael.belchior@tecnico.ulisboa.pt> feat(bungee): add skeleton for bungee Signed-off-by: André Augusto <andre.augusto@tecnico.ulisboa.pt> refactor(plugin-satp-hermes): update messages formats SATP core version 2.0.0 OpenAPI * created new messages in openAPI definition feat(bungee): fix bungee dependencies Signed-off-by: André Augusto <andre.augusto@tecnico.ulisboa.pt> feat(SATP-Hermes): gateway coordinator feat(SATP-Hermes): gateway coordinator feat(SATP-Hermes): gateway coordinator WIP Signed-off-by: Rafael Belchior <rafael.belchior@tecnico.ulisboa.pt> test(SATP-Hermes): enable decorators needed for gateway coordinator checks Signed-off-by: Rafael Belchior <rafael.belchior@tecnico.ulisboa.pt> docs(SATP-Hermes): update package.json Signed-off-by: Rafael Belchior <rafael.belchior@tecnico.ulisboa.pt> refactor(SATP-Hermes): re-estructure project Signed-off-by: Rafael Belchior <rafael.belchior@tecnico.ulisboa.pt> refactor(SATP-Hermes): re-estructure project Signed-off-by: Rafael Belchior <rafael.belchior@tecnico.ulisboa.pt> refactor(SATP-Hermes): re-estructure project Signed-off-by: Rafael Belchior <rafael.belchior@tecnico.ulisboa.pt> refactor(SATP-Hermes): remove kotlin sdk, add protobuffer Signed-off-by: Rafael Belchior <rafael.belchior@tecnico.ulisboa.pt> feat(SATP-Hermes): protobuf structure Signed-off-by: Rafael Belchior <rafael.belchior@tecnico.ulisboa.pt> feat(SATP-Hermes): protobuf structure Signed-off-by: Rafael Belchior <rafael.belchior@tecnico.ulisboa.pt> refactor(SATP-Hermes): refactor naming Signed-off-by: Rafael Belchior <rafael.belchior@tecnico.ulisboa.pt> feat(SATP-Hermes): protobuf structure improvements Signed-off-by: Rafael Belchior <rafael.belchior@tecnico.ulisboa.pt> feat(SATP-Hermes): update yarn.lock and clean up Signed-off-by: André Augusto <andre.augusto@tecnico.ulisboa.pt> refactor(test-tooling): fix types of streams: use NodeJS.ReadableStream 1. The container management library that we use in the test infrastructure (called dockerode) is expecting streams that are defined in the global namespace of the `@types/node` library, e.g. the standard library of NodeJS itself. 2. Previously we were using the "streams" package to provide type information to the streams that we were passing around to dockerode and it was working fine, but after some changes that seem unrelated this has broken the compilation process. 3. The mentioned changes are not yet on the main branch, but we expect them to be there soon and so this change is laying the groundwork for that by pre-emptively fixing the broken build's root cause which is that the test-tooling package does not declare it's typings related dependencies correctly: It implicitly uses the NodeJS standard library's types but so far had not declared them on the package level. 4. This change is therefore to rectify the issue of the `@types/node` dependency missing from the test-tooling package and also the refactoring of some of the test ledger classes which were relying on the `streams` builtin package instead of correctly using the NodeJS.ReadableStream global. 5. Earlier the reasoning for this was that we try to avoid pulling in types from the global scope because we try to avoid any sort of dependency on the global scope in general. Once we have proof though that this is causing issues with the build, then we must give up the principle for practical reasons (and only in the minimum viable scope, e.g. this does not change the fact that everywhere else in the codebase we should still do our best to avoid using the global scoped classes, types, functions, etc..). Thank you to @AndreAugusto11 and @RafaelAPB for pointing out this issue through the pull request of his that is currently being worked on at the time of this writing: RafaelAPB#72 Related to but does not address hyperledger-cacti#2811 Signed-off-by: Peter Somogyvari <peter.somogyvari@accenture.com> refactor(SATP-Hermes): remove unused packages * updated openapi version * updated bungee plugin version to 2.0.0-alpha.2 Co-authored-by: Peter Somogyvari <peter.somogyvari@accenture.com> Signed-off-by: André Augusto <andre.augusto@tecnico.ulisboa.pt> Signed-off-by: Peter Somogyvari <peter.somogyvari@accenture.com>
petermetz · Mar 10, 2024 · efd25de · efd25de
1 parent a3c7209
commit efd25de
Show file tree

Hide file tree

Showing 18 changed files with 156 additions and 121 deletions.
diff --git a/packages/cactus-plugin-bungee/src/main/typescript/plugin-bungee-benchmark.ts b/packages/cactus-plugin-bungee/src/main/typescript/plugin-bungee-benchmark.ts
@@ -81,18 +81,18 @@ export class PluginBUNGEEBenchmark {
   private level: LogLevelDesc;
   private logger: Logger;
   public pluginRegistry: PluginRegistry;
-  private tsStartGetAllAssetsKey: number;
-  private tsEndGetAllAssetsKey: number;
-  private tsStartGetAllTxByKey: number;
-  private tsEndGetAllTxByKey: number;
-  private tsStartfabricGetTxReceiptByTxIDV1: number;
-  private tsEndfabricGetTxReceiptByTxIDV1: number;
-  private tsStartgenerateLedgerStates: number;
-  private tsEndgenerateLedgerStates: number;
-  private tsStartgenerateSnapshot: number;
-  private tsEndgenerateSnapshot: number;
-  private tsStartGenerateView: number;
-  private tsEndGenerateView: number;
+  private tIgetAllAssetsKey: number;
+  private tFgetAllAssetsKey: number;
+  private tIgetAllTxByKey: number;
+  private tFgetAllTxByKey: number;
+  private tIfabricGetTxReceiptByTxIDV1: number;
+  private tFfabricGetTxReceiptByTxIDV1: number;
+  private tIgenerateLedgerStates: number;
+  private tFgenerateLedgerStates: number;
+  private tIgenerateSnapshot: number;
+  private tFgenerateSnapshot: number;
+  private tIgenerateView: number;
+  private tFgenerateView: number;
   private fileNameDate = Date.now();
 
   constructor(public readonly options: IPluginBUNGEEOptions) {
@@ -278,7 +278,7 @@ export class PluginBUNGEEBenchmark {
    * @param snapshot - Ledger Snapshot
    * */
   public generateView(snapshot: Snapshot): string {
-    this.tsStartGenerateView = performance.now();
+    this.tIgenerateView = performance.now();
     const crypto = require("crypto");
 
     this.logger.warn(this.pubKeyBungee);
@@ -302,7 +302,7 @@ export class PluginBUNGEEBenchmark {
       JSON.stringify(signedView, null, 2),
     );
 
-    this.tsEndGenerateView = performance.now();
+    this.tFgenerateView = performance.now();
     return JSON.stringify(signedView);
   }
 
@@ -313,7 +313,7 @@ export class PluginBUNGEEBenchmark {
    * @param transactionId - Transaction id to return the receipt
    * */
   async fabricGetTxReceiptByTxIDV1(transactionId: string): Promise<string> {
-    this.tsStartfabricGetTxReceiptByTxIDV1 = performance.now();
+    this.tIfabricGetTxReceiptByTxIDV1 = performance.now();
 
     const receiptLockRes = await this.fabricApi?.getTransactionReceiptByTxIDV1({
       signingCredential: this.fabricSigningCredential,
@@ -324,7 +324,7 @@ export class PluginBUNGEEBenchmark {
       params: [this.fabricChannelName, transactionId],
     } as FabricRunTransactionRequest);
 
-    this.tsEndfabricGetTxReceiptByTxIDV1 = performance.now();
+    this.tFfabricGetTxReceiptByTxIDV1 = performance.now();
     return JSON.stringify(receiptLockRes?.data);
   }
 
@@ -333,7 +333,7 @@ export class PluginBUNGEEBenchmark {
    * @abstract Returns all assets key found in the world state.
    * */
   async getAllAssetsKey(): Promise<string> {
-    this.tsStartGetAllAssetsKey = performance.now();
+    this.tIgetAllAssetsKey = performance.now();
     const response = await this.fabricApi?.runTransactionV1({
       signingCredential: this.fabricSigningCredential,
       channelName: this.fabricChannelName,
@@ -344,7 +344,7 @@ export class PluginBUNGEEBenchmark {
     } as FabricRunTransactionRequest);
 
     if (response != undefined) {
-      this.tsEndGetAllAssetsKey = performance.now();
+      this.tFgetAllAssetsKey = performance.now();
       return response.data.functionOutput;
     }
 
@@ -358,7 +358,7 @@ export class PluginBUNGEEBenchmark {
    * @param key - Key used to get correspondent transactions
    * */
   async getAllTxByKey(key: string): Promise<Transaction[]> {
-    this.tsStartGetAllTxByKey = performance.now();
+    this.tIgetAllTxByKey = performance.now();
     const response = await this.fabricApi?.runTransactionV1({
       signingCredential: this.fabricSigningCredential,
       channelName: this.fabricChannelName,
@@ -369,7 +369,7 @@ export class PluginBUNGEEBenchmark {
     } as FabricRunTransactionRequest);
 
     if (response != undefined) {
-      this.tsEndGetAllTxByKey = performance.now();
+      this.tFgetAllTxByKey = performance.now();
       return Utils.txsStringToTxs(response.data.functionOutput);
     }
 
@@ -401,29 +401,29 @@ export class PluginBUNGEEBenchmark {
 
   public generateBenchmarkReport(
     numberOfTransactions: number,
-    tsStartTotalTime: number,
-    tsEndTotalTime: number,
-    tsStartTransactionsTime: number,
-    tsEndTransactionsTime: number,
+    tItotalTime: number,
+    tFtotalTime: number,
+    tItransactionsTime: number,
+    tFtransactionsTime: number,
   ): void {
     const report = {
-      tsStartGetAllAssetsKey: this.tsStartGetAllAssetsKey,
-      tsEndGetAllAssetsKey: this.tsEndGetAllAssetsKey,
-      tsStartGetAllTxByKey: this.tsStartGetAllTxByKey,
-      tsEndGetAllTxByKey: this.tsEndGetAllTxByKey,
-      tsStartfabricGetTxReceiptByTxIDV1: this.tsStartfabricGetTxReceiptByTxIDV1,
+      tIgetAllAssetsKey: this.tIgetAllAssetsKey,
+      tFgetAllAssetsKey: this.tFgetAllAssetsKey,
+      tIgetAllTxByKey: this.tIgetAllTxByKey,
+      tFgetAllTxByKey: this.tFgetAllTxByKey,
+      tIfabricGetTxReceiptByTxIDV1: this.tIfabricGetTxReceiptByTxIDV1,
       numberOfTransactions: numberOfTransactions,
-      tsEndfabricGetTxReceiptByTxIDV1: this.tsEndfabricGetTxReceiptByTxIDV1,
-      tsStartgenerateLedgerStates: this.tsStartgenerateLedgerStates,
-      tsEndgenerateLedgerStates: this.tsEndgenerateLedgerStates,
-      tsStartgenerateSnapshot: this.tsStartgenerateSnapshot,
-      tsEndgenerateSnapshot: this.tsEndgenerateSnapshot,
-      tsStartGenerateView: this.tsStartGenerateView,
-      tsEndGenerateView: this.tsEndGenerateView,
-      tsStartTotalTime: tsStartTotalTime,
-      tsEndTotalTime: tsEndTotalTime,
-      tsStartTransactionsTime: tsStartTransactionsTime,
-      tsEndTransactionsTime: tsEndTransactionsTime,
+      tFfabricGetTxReceiptByTxIDV1: this.tFfabricGetTxReceiptByTxIDV1,
+      tIgenerateLedgerStates: this.tIgenerateLedgerStates,
+      tFgenerateLedgerStates: this.tFgenerateLedgerStates,
+      tIgenerateSnapshot: this.tIgenerateSnapshot,
+      tFgenerateSnapshot: this.tFgenerateSnapshot,
+      tIgenerateView: this.tIgenerateView,
+      tFgenerateView: this.tFgenerateView,
+      tItotalTime: tItotalTime,
+      tFtotalTime: tFtotalTime,
+      tItransactionsTime: tItransactionsTime,
+      tFtransactionsTime: tFtransactionsTime,
     };
     const reportString = JSON.stringify(report, null, 2);
     this.saveToFile(

diff --git a/packages/cactus-plugin-bungee/src/main/typescript/plugin-bungee.ts b/packages/cactus-plugin-bungee/src/main/typescript/plugin-bungee.ts
@@ -191,8 +191,7 @@ export class PluginBUNGEE implements ICactusPlugin, IPluginWebService {
   }
 
   async onCreateView(request: CreateViewRequest): Promise<CreateViewResponse> {
-    const fnTag = `${this.className}#onCreateView()`;
-    this.log.info(`${fnTag}, request received, ${request}`);
+    //const fnTag = `${this.className}#onCreateView()`;
     const response = this.generateView(this.generateSnapshot());
     return {
       y: response,

diff --git a/packages/cactus-plugin-satp-hermes/package.json b/packages/cactus-plugin-satp-hermes/package.json
@@ -45,7 +45,7 @@
   "scripts": {
     "build": "run-p tsc codegen",
     "build-proto": "buf build --path src/main/proto --verbose",
-    "build:dev:backend:postbuild": "mkdir -p ./dist/lib/knex && cp -r ./src/knex/* ./dist/lib/knex",
+    "build:dev:backend:postbuild": "mkdir -p ./dist/lib/main/knex && cp -r ./knex/* ./dist/lib/main/knex",
     "codegen": "run-p 'codegen:*'",
     "codegen:openapi": "npm run generate-sdk",
     "codegen:proto": "npm run generate-proto",

diff --git a/packages/cactus-plugin-satp-hermes/src/main/typescript/core/stage-services/stage3.ts b/packages/cactus-plugin-satp-hermes/src/main/typescript/core/stage-services/stage3.ts
@@ -0,0 +1 @@
+
diff --git a/packages/cactus-plugin-satp-hermes/src/main/typescript/core/stage-services/test-services.ts b/packages/cactus-plugin-satp-hermes/src/main/typescript/core/stage-services/test-services.ts
@@ -1,5 +1,5 @@
 import { ConnectRouter } from "@connectrpc/connect";
-// import { Message } from "../../generated/proto/test/message_pb";
+import { Message } from "../../generated/proto/test/message_pb";
 import { TestService } from "../../generated/proto/test/message_connect";
 
 export const testRouter = (router: ConnectRouter) =>
@@ -8,7 +8,8 @@ export const testRouter = (router: ConnectRouter) =>
     // implements rpc Say
     async sendMessage(req) {
       return {
-        sentence: `You said: ${req}`,
-      };
+        sentence: `You said: ${req}`
+      }
     },
   });
+
diff --git a/packages/cactus-plugin-satp-hermes/src/main/typescript/core/types.ts b/packages/cactus-plugin-satp-hermes/src/main/typescript/core/types.ts
@@ -1,4 +1,12 @@
-import { Secp256k1Keys, LogLevelDesc } from "@hyperledger/cactus-common";
+import {
+  Secp256k1Keys,
+  Logger,
+  Checks,
+  LoggerProvider,
+  JsObjectSigner,
+  IJsObjectSignerOptions,
+  LogLevelDesc,
+} from "@hyperledger/cactus-common";
 import { ValidatorOptions } from "class-validator";
 
 export enum CurrentDrafts {

diff --git a/...tus-plugin-satp-hermes/src/main/typescript/factory/plugin-factory-gateway-orchestrator.ts b/...tus-plugin-satp-hermes/src/main/typescript/factory/plugin-factory-gateway-orchestrator.ts
@@ -2,15 +2,20 @@ import {
   IPluginFactoryOptions,
   PluginFactory,
 } from "@hyperledger/cactus-core-api";
-import { SATPGateway, SATPGatewayConfig } from "../gateway-refactor";
+import {
+  SATPGateway,
+  SATPGatewayConfig,
+} from "../gateway-refactor";
 import { validateOrReject } from "class-validator";
 
 export class PluginFactorySATPGateway extends PluginFactory<
   SATPGateway,
   SATPGatewayConfig,
   IPluginFactoryOptions
 > {
-  async create(pluginOptions: SATPGatewayConfig): Promise<SATPGateway> {
+  async create(
+    pluginOptions: SATPGatewayConfig,
+  ): Promise<SATPGateway> {
     const coordinator = new SATPGateway(pluginOptions);
 
     try {

diff --git a/packages/cactus-plugin-satp-hermes/src/main/typescript/gateway-refactor.ts b/packages/cactus-plugin-satp-hermes/src/main/typescript/gateway-refactor.ts
@@ -3,29 +3,53 @@ import {
   Logger,
   Checks,
   LoggerProvider,
+  JsObjectSigner,
+  IJsObjectSignerOptions,
   ILoggerOptions,
 } from "@hyperledger/cactus-common";
 import { v4 as uuidv4 } from "uuid";
 
 import {
+  ICactusPlugin,
+  IPluginWebService,
+  IWebServiceEndpoint,
+  Configuration,
+} from "@hyperledger/cactus-core-api";
+
+import {
+  MinLength,
+  MaxLength,
+  IsNotEmpty,
+  ValidateNested,
   IsDefined,
   IsNotEmptyObject,
   IsObject,
   IsString,
   Contains,
 } from "class-validator";
+import { Type } from "class-transformer";
 
+import fs from "fs";
 import path from "path";
-
+import swaggerUi = require("swagger-ui-express");
 import {
+  IPluginSatpGatewayConstructorOptions,
+  PluginSATPGateway,
+} from "./plugin-satp-gateway";
+import { Server } from "node:http";
+import {
+  CurrentDrafts,
+  DraftVersions,
   SATPGatewayConfig,
   GatewayIdentity,
   ShutdownHook,
   SupportedGatewayImplementations,
 } from "./core/types";
+import { pass } from "jest-extended";
 import { GatewayOrchestrator } from "./gol/gateway-orchestrator";
+import { log } from "console";
 export { SATPGatewayConfig };
-import express from "express";
+import express, { Express, Request, Response } from 'express';
 import { expressConnectMiddleware } from "@connectrpc/connect-express";
 import http from "http";
 import { configureRoutes } from "./web-services/router";
@@ -52,7 +76,7 @@ export class SATPGateway {
   private server: any | undefined;
   // TODO!: add logic to manage sessions (parallelization, user input, freeze, unfreeze, rollback, recovery)
   // private sessions: Map<string, Session> = new Map();
-
+  
   constructor(public readonly options: SATPGatewayConfig) {
     const fnTag = `${this.label}#constructor()`;
     Checks.truthy(options, `${fnTag} arg options`);
@@ -86,25 +110,27 @@ export class SATPGateway {
       this.server = express();
       this.server.use(expressConnectMiddleware({ routes: configureRoutes }));
       http.createServer(this.server).listen(this.options.gid?.port);
-    } else {
+
+    } else  {
       this.logger.warn("Server already running");
     }
   }
-  async shutdownServer(): Promise<void> {
-    const fnTag = `${this.label}#shutdown()`;
-    this.logger.debug(`Entering ${fnTag}`);
-    if (this.server) {
-      try {
-        this.server.close();
-        this.server = undefined;
-        this.logger.info("Server shut down");
-      } catch (error) {
-        this.logger.error(`Error shutting down the server: ${error}`);
-      }
-    } else {
-      this.logger.warn("Server is not running.");
+async shutdownServer(): Promise<void> {
+  const fnTag = `${this.label}#shutdown()`;
+  this.logger.debug(`Entering ${fnTag}`);
+  if (this.server) {
+    try {
+      this.server.close();
+      this.server = undefined;
+      this.logger.info("Server shut down");
+    } catch (error) {
+      this.logger.error(`Error shutting down the server: ${error}`);
     }
+  } else {
+    this.logger.warn("Server is not running.");
   }
+}
+
 
   async addGateways(IDs: string[]): Promise<void> {
     const fnTag = `${this.label}#connectToGateway()`;
@@ -180,7 +206,7 @@ export class SATPGateway {
     const specPath = path.join(__dirname, "../../", "/json", "openapi.json");
     this.logger.debug(`Loading OpenAPI specification from ${specPath}`);
 
-    //const OpenAPISpec = JSON.parse(fs.readFileSync(specPath).toString());
+    const OpenAPISpec = JSON.parse(fs.readFileSync(specPath).toString());
     this.logger.info(
       `OpenAPI docs and documentation set up at 📖: \n ${this.config.gid?.address}:${this.config.gid?.port}/api-docs`,
     );
@@ -286,8 +312,9 @@ export class SATPGateway {
     return this.config.gid!;
   }
 
-  async shutdown(): Promise<number> {
+  async shutdown(): Promise<number>   {
     this.logger.info("Shutting down Gateway Coordinator");
     return await this.gatewayConnectionManager.disconnectAll();
+
   }
 }