Client deployment to use with the sdk #698

daveroga · 2022-06-02T00:00:48Z

Build client to be able to test with the SDK for testnet (goerli) and production (mainnet).
Based in docker-compose and script to start client.

Updates in nightfall-client and worker to download files from the public S3 bucket of the contracts abi in json and output of circuits based on some parameters.
Updated README in nightfall-client that was outdated and added information about to configure and run the client.

To test the client run:

Configure your client.env in the root folder of nightfall-client
Run the script ./start-client -d in the root folder of nightfall-client

Run the test in client folder with correct values and connect:

TRANSFER_VALUE=1 USER1_MNEMONIC="xxx xxx xxx xxx xxx xxx xxx xxx xxx xxx xxx xxx" 
USER1_KEY="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" TRANSACTIONS_PER_BLOCK=32 
BLOCKCHAIN_WS_HOST="your goerli wss provider url" npm run test-erc20-cli

Now the client is ready to persist the data in the volume and when you start again with ./start-client -d it syncs again but without removing the data from the volumes.
If you want to remove the volumes first you can run ./start-client -d -r with the -r for removing the volumes first.

Tasks

Build client service based on docker-compose
Run some tests with the SDK to test client in testnet environment
Fix persistent database error duplicated key when syncing

druiz0992

add README how to deploy client
I’d move start-client inside client
I think you need a client.env where one can configure parameters as Web3Url and proposerUrl. Also, i would add the location of s3 bucket in a variable in case it changes.

daveroga · 2022-06-02T07:53:34Z

Added suggested changes

druiz0992 · 2022-06-02T07:57:21Z

nightfall-client/README.md

@@ -8,7 +8,7 @@

 # nightfall-client

-This code generates a containerised application that can be used to interact with Nightfall_2 via
+This code generates a containerised application that can be used to interact with Nightfall_3 via


Let's call this Polygon Nightfall from now on

druiz0992 · 2022-06-02T07:58:41Z

nightfall-client/README.md

@@ -19,79 +19,39 @@ nightfall-client requires a number of services to be present for it to work. The
 instructions explain how to run all of these up, in a similar 'developer' mode with local file


This README is intended for outside people trying to use the SDK. So, it needs to be very clear. Wha'ts developer mode?

druiz0992 · 2022-06-02T07:58:59Z

nightfall-client/README.md

-[nightfall-scripts](https://github.com/EYBlockchain/nightfall-scripts), which does a similar job but
-uses docker images that do not rely on your local file system and therefore requires no other code
-repositories to be downloaded.
+If you just want to run Nightfall_3 then an easier way is to use


Polygon Nightfall

druiz0992 · 2022-06-02T08:01:21Z

nightfall-client/client.env

@@ -0,0 +1,4 @@
+ETH_NETWORK=goerli
+BLOCKCHAIN_URL=your web3 url provider to access the blockchain


Can you provide an example to understand format

nightfall-client/client.env

druiz0992 · 2022-06-02T08:05:12Z

docker-compose.client.yml

@@ -0,0 +1,72 @@
+version: '3.5'


There are some requirements on the docker-compose version required to be installed. Can you clarify?

I choose the same version and format of the docker-compose files of the root folder for other purposes.

What i meant is that there should be a requirements section in the README where we explain what sw is needed to run this. For example, we need nodeJS v xxx, docker-compose v.xxx,... Again, this README needs to be enough for a user to spin a client without the need to go to the main README. All information required should be here.

druiz0992 · 2022-06-02T08:08:02Z

nightfall-client/client.env

@@ -0,0 +1,4 @@
+ETH_NETWORK=goerli


Call this file something else (client-example.env for example) . Add client.env to .gitignore. In the instructions, then specify that one should copy client-example.env to client.env and add their own information

druiz0992 · 2022-06-02T08:09:49Z

nightfall-client/README.md

-
-```sh
-$ npm i
+ETH_NETWORK=goerli or mainnet


It needs to be clear that this client is to connect to existing testnet and mainnet deployments. Also, can you comment on how to deploy 2+ clients? And if one wants to launch a client vs a local environment. How would this be done?

Some comments in README

druiz0992

Left my comments in the review

druiz0992 · 2022-06-02T08:20:46Z

nightfall-client/README.md

-```sh
-$ docker-compose build --build-arg GPR_TOKEN=<paste your token here>
+```
+./start-client -c


How do you stop a client?

With ctrl+c as start-nightfall. It shuts down the containers because the trap in the start-client script catch the event and runs a docker-compose down.

druiz0992 · 2022-06-02T15:00:40Z

docker-compose.client.yml

+# bindings.  See the readme for more information.
+services:
+  client1:
+    image: ghcr.io/eyblockchain/nightfall3-client:latest


i think you should change the image name.... its strange that its called eyblockchain

Yes, but this applies to all images that we are building now in the Dockerfiles in the root folder and not only this PR.
So we have to do it in all images I think.

ghcr.io/eyblockchain/nightfall3-deployer:latest

ghcr.io/eyblockchain/nightfall3-worker:latest

ghcr.io/eyblockchain/nightfall3-client:latest

ghcr.io/eyblockchain/nightfall3-optimist:latest

ghcr.io/eyblockchain/nightfall3-proposer:latest

ghcr.io/eyblockchain/nightfall3-user-local:latest

druiz0992 · 2022-06-02T15:09:33Z

nightfall-client/start-client

+usage()
+{
+  echo "Usage:"
+  echo "  -c or --client; for client service"


Why do we need this flag (-c)? Does it have sense to call this script without this flag?

True. Changed to always run this part by default.

druiz0992 · 2022-06-02T15:10:37Z

nightfall-client/start-client

+}
+
+# select a Geth or Ganache client
+if [ -z "$1" ]; then


What is this for?

Removed. Expected at least 1 argument. With -c by default and after removal then it has no sense.

daveroga · 2022-06-03T08:56:54Z

Added TRANSFER_VALUE in config to be able to pass the amount to be transferred in tests.

transferValue: process.env.TRANSFER_VALUE || 10,

druiz0992 · 2022-06-02T15:11:05Z

nightfall-client/start-client

+fi
+
+# delete env file
+rm -f ${ENV_FILE}


Whats this ENV_FILE?

This is from copy paste. Removed.

druiz0992 · 2022-06-02T15:14:08Z

nightfall-client/start-client

+  exit 1
+fi
+
+VOLUME_LIST=$(docker volume ls -q)


How does docker volume get initialized? Doesn't this require to have run setup-nightfall?

The docker volume gets initialized with start-client in the docker-compose docker-compose.client.yml

israelboudoux

Hello @daveroga, I think this needs some changes. Let me know if you wanna discuss them.

config/default.js

docker-compose.client.dev.yml

docker-compose.client.yml

israelboudoux · 2022-06-17T16:55:45Z

nightfall-client/src/services/state-sync.mjs

@@ -44,14 +53,73 @@ const syncState = async (fromBlock = 'earliest', toBlock = 'latest', eventFilter
  }
 };

+const downloadFile = async (fileUrl, outputLocationPath) => {


It would be good to have validations for these parameters (e.g. valid URL, path exists, etc).

createWriteStream creates the file passed as outputLocationPath. It doesn't exists in principle. If the URL is not valid will thrown an error that should be captured in the parent. Is this what you mean?

nightfall-client/src/services/state-sync.mjs

test/client/erc20.test.mjs

israelboudoux · 2022-06-17T17:47:40Z

test/client/erc20.test.mjs

+      logger.debug(after, before, before + remainingBalance - transferValue);
+      expect(after).to.be.lessThan(before);
+      expect(after).to.satisfy(balance => {
+        return balance < before || balance === before + remainingBalance - transferValue;


You're relying on that the sequence of the tests will be preserved by the testing library when running them, I'm not sure if this is the best approach, what if someone changes the ordering of the test methods?

What do you mean by changing the ordering of the test methods?

zokrates-worker/src/index.mjs

israelboudoux

Hey @daveroga, I think you've missed some comments, please take a look on them.

daveroga · 2022-06-20T15:38:42Z

Hey @daveroga, I think you've missed some comments, please take a look on them.

Reviewed and updated.

israelboudoux

Looks better now, thanks!

daveroga

Reviewed changes and suggestions

config/default.js

docker-compose.client.dev.yml

docker-compose.client.yml

nightfall-client/src/services/state-sync.mjs

zokrates-worker/src/index.mjs

daveroga · 2022-06-20T14:07:13Z

nightfall-client/src/services/state-sync.mjs

@@ -44,14 +53,73 @@ const syncState = async (fromBlock = 'earliest', toBlock = 'latest', eventFilter
  }
 };

+const downloadFile = async (fileUrl, outputLocationPath) => {


createWriteStream creates the file passed as outputLocationPath. It doesn't exists in principle. If the URL is not valid will thrown an error that should be captured in the parent. Is this what you mean?

nightfall-client/src/services/state-sync.mjs

test/utils.mjs

test/client/erc20.test.mjs

israelboudoux

Looks good to me!

druiz0992 suggested changes Jun 2, 2022

View reviewed changes

daveroga requested a review from druiz0992 June 2, 2022 07:53

druiz0992 reviewed Jun 2, 2022

View reviewed changes

nightfall-client/client.env Outdated Show resolved Hide resolved

druiz0992 reviewed Jun 2, 2022

View reviewed changes

druiz0992 suggested changes Jun 2, 2022

View reviewed changes

druiz0992 reviewed Jun 2, 2022

View reviewed changes

daveroga requested a review from druiz0992 June 2, 2022 08:30

druiz0992 reviewed Jun 2, 2022

View reviewed changes

daveroga requested a review from druiz0992 June 3, 2022 08:57

imagobea added the SDK To be reflected on the SDK repo label Jun 3, 2022

daveroga marked this pull request as ready for review June 3, 2022 12:18

imagobea assigned daveroga Jun 3, 2022

daveroga force-pushed the daveroga/client-deployment branch from 10902c6 to 7a0fc37 Compare June 8, 2022 11:31

daveroga added 7 commits June 14, 2022 10:49

feat: client deployment to use with the sdk

345fb24

fix: update README and client.env file

3b8ce61

fix: readme add some information about deployment

0faa442

fix: config variables url files default values

ac6201e

fix: add test for the client

425760e

fix: remove -c param in start-client

9c33b23

fix: logs in debug

0836bb4

daveroga added 6 commits June 14, 2022 10:49

fix: changes balances

ffe6c44

fix: remove -c param from readme

3f68a01

fix: avoid saving transactions and timber while syncing

48ea3f3

fix: rebase errors

5d3b272

fix: utils error

1d54580

fix: count deposit utils

6dcf638

daveroga force-pushed the daveroga/client-deployment branch from c7da625 to 6dcf638 Compare June 14, 2022 08:51

druiz0992 approved these changes Jun 15, 2022

View reviewed changes

fix: readme add requirements

2f818f1

druiz0992 added the One more approval needed One reviewer has approved this PR but another is needed label Jun 17, 2022

israelboudoux suggested changes Jun 17, 2022

View reviewed changes

daveroga added 2 commits June 20, 2022 12:42

fix: suggestions renaming

7950b58

fix: reuse function with httputils

cd44fdc

daveroga requested a review from israelboudoux June 20, 2022 13:38

fix: ignore unused exports httputils

3aaf633

israelboudoux suggested changes Jun 20, 2022

View reviewed changes

daveroga and others added 2 commits June 20, 2022 16:00

Merge branch 'master' into daveroga/client-deployment

d2adf95

fix: removing assignment

a5ca923

fix: waitForTime utils and other changes

a2b1944

daveroga requested a review from israelboudoux June 21, 2022 07:33

fix: waitForTime utils and other changes

6ea7798

israelboudoux reviewed Jun 21, 2022

View reviewed changes

daveroga commented Jun 21, 2022

View reviewed changes

daveroga requested a review from israelboudoux June 21, 2022 10:53

israelboudoux approved these changes Jun 21, 2022

View reviewed changes

daveroga removed the One more approval needed One reviewer has approved this PR but another is needed label Jun 21, 2022

fix: package-lock.json

3b8bdf5

daveroga merged commit 1cd5266 into master Jun 21, 2022

imagobea mentioned this pull request Aug 22, 2022

Deploy nightfall client maticnetwork/nightfall-sdk#18

Closed

		@@ -19,79 +19,39 @@ nightfall-client requires a number of services to be present for it to work. The
		instructions explain how to run all of these up, in a similar 'developer' mode with local file

		@@ -0,0 +1,4 @@
		ETH_NETWORK=goerli
		BLOCKCHAIN_URL=your web3 url provider to access the blockchain

Client deployment to use with the sdk #698

Client deployment to use with the sdk #698

Conversation

daveroga commented Jun 2, 2022 • edited Loading

druiz0992 left a comment

Choose a reason for hiding this comment

daveroga commented Jun 2, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

druiz0992 Jun 2, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

druiz0992 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

daveroga commented Jun 3, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

israelboudoux left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

israelboudoux left a comment

Choose a reason for hiding this comment

daveroga commented Jun 20, 2022

israelboudoux left a comment

Choose a reason for hiding this comment

daveroga left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

israelboudoux left a comment

Choose a reason for hiding this comment

daveroga commented Jun 2, 2022 •

edited

Loading

druiz0992 Jun 2, 2022 •

edited

Loading