Add Orbit & DDB basics #94

laurentsenta · 2018-05-09T10:02:33Z

Description

Resolves DDB: Create Profile Operation #84: Add the basics for DDB in Dapp: orbitdb, ipfs and User Profiles
Resolves DDB: Integration testing with IPFS & OrbitDB #83: Add the corresponding integration testing.

Deps

New dependencies:

ipfs : Storage used by orbit-db.
orbit-db: Decentralized database

rdig

This is still very rough. It looks to me like it's still a work-in-progress.

Please see my comments and align this PR with our rules in the codebase.

Please use the included prettier and eslint rules as those will help you avoid most things I've outlined.

👍

rdig · 2018-05-16T10:57:40Z

integration-testing/data/factory.js

@@ -0,0 +1,134 @@
+import rimraf from 'rimraf';


I would put this file under ./integration-testing/utils to not mix it up with the tests

rdig · 2018-05-16T11:00:45Z

jest-integration.conf.json

@@ -3,6 +3,9 @@
    "rootDir": "integration-testing",
    "globalSetup": "./utils/integration-testing-setup.js",
    "globalTeardown": "./utils/integration-testing-teardown.js",
+    "setupFiles": [


This is going do generate a merge conflict, as I've also added a bunch of files in #93

rdig · 2018-05-16T11:01:44Z

src/data/index.js

+
+  subscribe(f) {
+    this._store.events.on('replicated', () => {
+      // console.log(


Is this commented out code needed, or just a leftover ?

I tend to keep these around to quickly uncomment and track my errors, I'll remove it.

Do we have a preferred logging library that'd let me log debug / info / warn / alert with namespaced loggers that you can enable and disable with a piece of config?

rdig · 2018-05-16T11:02:15Z

src/data/index.js

+  }
+
+  async getUserProfile(key: PublicKey): Promise<UserProfile> {
+    // console.log('Build User Profile Store with key=', key);


Same here, are these needed, or can they be removed ?

rdig · 2018-05-16T11:08:50Z

integration-testing/data/UserProfile.test.js

+
+    let update = {};
+    p2.subscribe(x => {
+      console.log('Subscription triggered with x=', x);


There's a lot of console.log statements all over the tests. Do we need this ?

As they seem to me to be more for a debugging purpose.

rdig · 2018-05-16T11:42:06Z

integration-testing/data/UserProfile.test.js

+import { retryUntilValue } from '../utils/tools';
+
+let factory = null;
+let pinner = null;


This variable is not used in this file.

rdig · 2018-05-16T11:42:21Z

integration-testing/data/UserProfile.test.js

+  await factory.clear();
+}, Factory.TIMEOUT);
+
+


Please remove the extra line here.

rdig · 2018-05-16T11:42:39Z

integration-testing/data/UserProfile.test.js

+
+  test('Create my user profile and set its name', async () => {
+    const p1 = await data2.getMyUserProfile();
+    let name = factory.name('Kanye West');


name is never reassigned, please use const

Don't we have a linter precommit hook? That'd be a good thing to set up.

I passed yarn lint before pushing.
We need to add integration-testing/.

Yep, good point 👍

Although, curiously, my editor's linter (which takes it's rules from the local project's .eslintrc) works...

rdig · 2018-05-16T11:43:30Z

integration-testing/data/UserProfile.test.js

+    expect(p1.getName()).toBe(name);
+  });
+
+  test('Create my user profile and set its name sync with another', async () => {


This line exceeds the max length of 80.

Please use the included prettier and eslint rules.

rdig · 2018-05-16T11:43:53Z

integration-testing/data/UserProfile.test.js

+
+    expect(p1.isEmpty()).toBeFalsy();
+    expect(await retryUntilValue(() => p2.isEmpty(), { value: false })).toBeFalsy();
+    expect(await retryUntilValue(() => p2.getName(), { value: name })).toBe(name);


These two lines exceed the max length of 80.

Please use the included prettier and eslint rules.

laurentsenta · 2018-05-21T10:07:11Z

@rdig I yarn lint before pushing (with integration-testing/ folders)
but that doesn't seem enough.

And yarn prettier --write ./integration-testing/**/*.js

doesn't follow our coding style (it puts " quotes everywhere for example).

what's your prettier setup?

rdig · 2018-05-21T12:17:23Z

what's your prettier setup?

prettier is built in our lint setup, so you should just have to run yarn lint and that will do the trick. Although I think you're right, as that one doesn't take into account the integration-tests/ folder.

I'll add it in and fix it in #112

rdig · 2018-05-21T13:16:40Z

Added in #112

There's a manual command to lint integration tests files: yarn test:integration.

But everything is wired up to check these at commit time.

laurentsenta · 2018-05-22T09:43:10Z

@rdig thanks for the thorough review, my editors are now configured with our complete coding style and I fixed all your comments ;)

Could you take another look?

Also, do you Squash & Rebase or just Merge? I kept small & focused commits to make the review easier for now.

rdig · 2018-05-22T11:20:04Z

Could you take another look?

I'll try to make some time today

Also, do you Squash & Rebase or just Merge? I kept small & focused commits to make the review easier for now.

As long as the commits are made with common sense we don't squash, just merge. I we do ever decide to squash, it's going to be brought up in the review.

rdig

Yep, this is good to go now.

Tests output it much cleaner:

There's still some eslint errors left, but these have to do with rule config, rather than code: (this is fixed in #112 so just ignore it for now)

PS: Please rebase on the latest master and re-run tests with the updated versions of the submodules to make sure they still pass

Nice work Laurent 💯

laurentsenta · 2018-05-23T18:24:25Z

@rdig Arr, the PR is ready to merge, but something broke with the Colony Client apparently:
https://circleci.com/gh/JoinColony/colonyDapp/506?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link

rdig · 2018-05-23T18:31:28Z

There's a lot of things that seem to be failing:

colony client (i assume this is due to jest)
linting on the main circle workflow, most likely we forgot something that eslint picked up

I'll take a look tommorow at it and possibly fix it.

rdig · 2018-05-24T09:56:30Z

I've fixed CI tests, but I still get this failure when running integration tests:

Can you please investigate ?

PS: I've added a new commit, so make sure to rebase if you're working locally 👍

laurentsenta · 2018-05-24T10:27:40Z

Yea, there's a bug in the libp2p suite,

The tests show this error but succeed anyway?
Could we merge this and progress on the DAPP while ipfs-pubsub-room fix the issue?

rdig · 2018-05-24T10:30:12Z

The tests show this error but succeed anyway?

Yes because the console method call fails, not an expect() assertion, so from jest's point of view everything works fine.

Could we merge this and progress on the DAPP while ipfs-pubsub-room fix the issue?

Yes, but add a @TODO comment to the code block that fails, so we can keep track of it.

- add initial integration testing code - add ipfs and orbit nodes - add pinner mock to fix data ipfs tests - add testing with orbitdb - add a testing factory for ipfs / orbit & cleanup tests

- set name and retrieve - tests - subscriptions & load store

laurentsenta changed the title ~~Feature/83 orbit in dapp~~ Add Orbit & DDB basics May 9, 2018

laurentsenta mentioned this pull request May 9, 2018

💣 WIP: DDB User Profiles #92

Closed

4 tasks

laurentsenta force-pushed the feature/83-orbit-in-dapp branch 5 times, most recently from f5cfa63 to 390b4c5 Compare May 15, 2018 08:47

laurentsenta added the Needs code review label May 15, 2018

laurentsenta assigned coyotespike and unassigned coyotespike May 15, 2018

laurentsenta requested review from coyotespike and rdig May 15, 2018 09:03

rdig added under-review and removed Needs code review labels May 16, 2018

rdig assigned laurentsenta May 16, 2018

rdig requested changes May 16, 2018

View reviewed changes

laurentsenta force-pushed the feature/83-orbit-in-dapp branch from 42d84ff to 1d3ef14 Compare May 21, 2018 10:06

laurentsenta force-pushed the feature/83-orbit-in-dapp branch from 1d3ef14 to 4b82688 Compare May 21, 2018 12:37

laurentsenta force-pushed the feature/83-orbit-in-dapp branch from 4b82688 to 7f95222 Compare May 22, 2018 09:36

rdig approved these changes May 22, 2018

View reviewed changes

rdig removed the under-review label May 22, 2018

laurentsenta force-pushed the feature/83-orbit-in-dapp branch from 7f95222 to de2bfdd Compare May 23, 2018 18:03

rdig force-pushed the feature/83-orbit-in-dapp branch from b922674 to 5b7f15d Compare May 24, 2018 09:29

rdig force-pushed the feature/83-orbit-in-dapp branch 2 times, most recently from bf78f62 to ed14f4d Compare May 24, 2018 09:44

laurentsenta force-pushed the feature/83-orbit-in-dapp branch from ed14f4d to 58094b4 Compare May 28, 2018 09:44

laurentsenta added 18 commits May 28, 2018 14:56

Add: DDB orbitdb, ipfs & testing

b1e7b42

- add initial integration testing code - add ipfs and orbit nodes - add pinner mock to fix data ipfs tests - add testing with orbitdb - add a testing factory for ipfs / orbit & cleanup tests

Add: DDB UserProfiles

ea1b605

- set name and retrieve - tests - subscriptions & load store

integrate with integration-testing work

91982a6

Fix: add flow types in data/

87ecf0b

Fix: pass lint in data/

efe84c9

Fix: remove network-based tests in data/__test__

02ac719

Ignore coverage for data (integration tested)

bca72d4

tests: add note related to crypto polyfill

af23836

Lint: remove dead logs

3312bb9

Fix: move ddb's testing factory to utils/

7beea09

Lint: use @todos in ddb code

c512379

Lint: remove dead code

10dd828

Fix: remove useless else

bae309a

Fix: retryUntilValue style

558edd2

Linting testings & src

4be1207

Fix: missing flow annotations

c5701df

Add TODO related to ddb errors and libp2p

d88de62

Lint: ColonyCreation logs and Fields

9513849

laurentsenta force-pushed the feature/83-orbit-in-dapp branch from 5c6ab92 to 9513849 Compare May 28, 2018 12:57

rdig merged commit b801772 into master May 28, 2018

rdig deleted the feature/83-orbit-in-dapp branch May 28, 2018 13:07

laurentsenta added this to the Sprint 2 milestone May 28, 2018

collinvine unassigned laurentsenta Nov 25, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Orbit & DDB basics #94

Add Orbit & DDB basics #94

laurentsenta commented May 9, 2018

rdig left a comment

rdig May 16, 2018

rdig May 16, 2018

rdig May 16, 2018

laurentsenta May 16, 2018

rdig May 16, 2018

rdig May 16, 2018

rdig May 16, 2018

rdig May 16, 2018

rdig May 16, 2018

chmanie May 16, 2018

rdig May 16, 2018

laurentsenta May 21, 2018 •

edited

Loading

rdig May 21, 2018

rdig May 16, 2018

rdig May 16, 2018

laurentsenta commented May 21, 2018 •

edited

Loading

rdig commented May 21, 2018

rdig commented May 21, 2018

laurentsenta commented May 22, 2018

rdig commented May 22, 2018

rdig left a comment •

edited

Loading

laurentsenta commented May 23, 2018

rdig commented May 23, 2018

rdig commented May 24, 2018

laurentsenta commented May 24, 2018

rdig commented May 24, 2018 •

edited

Loading

Add Orbit & DDB basics #94

Add Orbit & DDB basics #94

Conversation

laurentsenta commented May 9, 2018

Description

Deps

rdig left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

laurentsenta May 21, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

laurentsenta commented May 21, 2018 • edited Loading

rdig commented May 21, 2018

rdig commented May 21, 2018

laurentsenta commented May 22, 2018

rdig commented May 22, 2018

rdig left a comment • edited Loading

Choose a reason for hiding this comment

laurentsenta commented May 23, 2018

rdig commented May 23, 2018

rdig commented May 24, 2018

laurentsenta commented May 24, 2018

rdig commented May 24, 2018 • edited Loading

laurentsenta May 21, 2018 •

edited

Loading

laurentsenta commented May 21, 2018 •

edited

Loading

rdig left a comment •

edited

Loading

rdig commented May 24, 2018 •

edited

Loading