Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Grid syft #54

Merged
merged 52 commits into from
Sep 22, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
6420b5d
Grid and syft communicating
cereallarceny Aug 2, 2019
5076c5a
Upgrading dependencies
cereallarceny Aug 2, 2019
74b501f
Upgraded examples dependencies
cereallarceny Aug 2, 2019
9b8a473
Updated Serde to be the latest version of the PySyft Serde library
cereallarceny Aug 5, 2019
e30961f
Finished all tests for logger
cereallarceny Aug 5, 2019
3496619
Finished testing the observer
cereallarceny Aug 5, 2019
8dc5f93
Sockets tests done but not really testing anything
cereallarceny Aug 5, 2019
fe7eec7
Getting of protocols, creating of scope, and everything except the pa…
cereallarceny Aug 6, 2019
c2ee3bb
Syft is properly doing everything with grid and both the creator and …
cereallarceny Aug 7, 2019
e76b01f
Local webrtc working without an additional socket server
cereallarceny Aug 13, 2019
f6b1a38
WebRTC multi-user messaging is somewhat working
cereallarceny Aug 14, 2019
4ef23c1
Substituting for better message constants
cereallarceny Aug 15, 2019
dc1eb97
Working
cereallarceny Aug 15, 2019
92a59f2
Not working, but better documented
cereallarceny Aug 15, 2019
914a8c4
Working again
cereallarceny Aug 15, 2019
3320da5
Finished off WebRTC
cereallarceny Aug 16, 2019
ac1da97
Better demo
cereallarceny Aug 16, 2019
2a48f1b
Figured out timeout issue
cereallarceny Aug 17, 2019
f319909
An attempt at new workflow
cereallarceny Aug 19, 2019
3b6810b
Forgot await
cereallarceny Aug 19, 2019
770faba
Correct URL
cereallarceny Aug 19, 2019
92ce800
Refactor complete
cereallarceny Aug 19, 2019
508cec3
Correct scope in demo
cereallarceny Aug 19, 2019
452bded
Removing need for uuid and creating instanceId on grid.js
cereallarceny Aug 19, 2019
c63ce1f
Working on with-grid watching issues
cereallarceny Aug 19, 2019
299caf2
With grid working
cereallarceny Aug 19, 2019
e2ee631
Better deploy script
cereallarceny Aug 19, 2019
a2579e7
A bunch of small organization changes
cereallarceny Aug 21, 2019
7971465
Migrating serde, event observer, logger, constants, custom types, and…
cereallarceny Aug 26, 2019
34f583e
Switching back to NPM
cereallarceny Aug 30, 2019
5ad6a39
add sockets.js tests and minor fixes
Sep 2, 2019
d0c0ecb
replace non-finishing test
Sep 2, 2019
758b6f1
Removing unnecessary lines
cereallarceny Sep 2, 2019
176785f
Merge branch 'grid-syft' into socket-tests
Sep 2, 2019
556ffbf
fix error in webrtc test
Sep 2, 2019
5e2b2dd
Clearing up example
cereallarceny Sep 2, 2019
b464427
Fixed issues with Redis
cereallarceny Sep 2, 2019
da31141
Changing small bit of logger functionality
cereallarceny Sep 3, 2019
5d0c98b
Merge branch 'grid-syft' into socket-tests
cereallarceny Sep 3, 2019
8dfb8d3
fix code style
Sep 4, 2019
df31983
revert data shallow copy
Sep 4, 2019
9346128
Update sockets.js
cereallarceny Sep 4, 2019
533ab48
Merge pull request #50 from vvmnnnkv/socket-tests
cereallarceny Sep 4, 2019
fe0fbbe
Cleaning up tests with a few comments for @Vova
cereallarceny Sep 4, 2019
d3ef518
Not spying on mocks
cereallarceny Sep 5, 2019
e443b7c
remove unused variables
Sep 6, 2019
ed60769
apply prettier formatting
Sep 6, 2019
d319b84
Merge pull request #51 from vvmnnnkv/socket-tests-fix
cereallarceny Sep 6, 2019
cf07613
Add unit tests for webrtc.js
Sep 17, 2019
31e5e0f
add more tests, changes are PR review
Sep 19, 2019
cb7116f
apply prettier
Sep 19, 2019
60c43bf
Merge pull request #52 from vvmnnnkv/webrtc-tests
cereallarceny Sep 19, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
node_modules
coverage
dist
tmp
.DS_Store
yarn.lock
.idea
7 changes: 4 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,21 +20,22 @@ PySyft acts as the parent node, instructing child nodes \(Syft.js instances runn
If you're using a package manage like NPM:

```text
npm install --save syft.js @tensorflow/tfjs
npm install --save syft.js @tensorflow/tfjs webrtc-adapter
```

Or if Yarn is your cup of tea:

```text
yarn add syft.js @tensorflow/tfjs
yarn add syft.js @tensorflow/tfjs webrtc-adapter
```

When using a package manager, TensorFlow.js will be automatically installed.
When using a package manager, [TensorFlow.js](https://www.tensorflow.org/js) and the [WebRTC adapter](https://github.com/webrtc/adapter) will be automatically installed.

If you're not using a package manager, you can also include Syft.js within a `<script>` tag:

```html
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@1.2.5/dist/tf.min.js"></script>
<script src="https://webrtc.github.io/adapter/adapter-latest.js"></script>
<script src="https://unpkg.com/syft.js@latest"></script>
```

Expand Down
11 changes: 11 additions & 0 deletions copy-examples.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/usr/bin/env bash
rm -rf ./tmp
mkdir tmp

for directory in $(find ./examples -type d -mindepth 1 -maxdepth 1);
do
directory="$directory/dist/"
name="$(cut -d'/' -f3 <<<$directory)"

rsync -avzh "$directory" "./tmp/$name"
done
50 changes: 0 additions & 50 deletions examples/basic-example/index.js

This file was deleted.

23 changes: 0 additions & 23 deletions examples/basic-example/webpack.config.js

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# syft.js Example
# syft.js/grid.js Example

In order to get this running, you simply need to run the following:

Expand Down
35 changes: 35 additions & 0 deletions examples/with-grid/_helpers.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
export const getQueryVariable = variable => {
const query = window.location.search.substring(1),
vars = query.split('&');

for (let i = 0; i < vars.length; i++) {
const pair = vars[i].split('=');

if (decodeURIComponent(pair[0]) === variable) {
return decodeURIComponent(pair[1]);
}
}

return null;
};

export const writeLinksToDOM = links => {
document.getElementById('participant-links').innerHTML = `
<ul>
${links
.map(
(link, i) =>
`
<li>
<a href="${link}" target="_blank">Link for participant ${i + 1}</a>
</li>
`
)
.join('')}
</ul>
`;
};

export const writeIdentityToDOM = text => {
document.getElementById('identity').innerHTML = text;
};
84 changes: 84 additions & 0 deletions examples/with-grid/index.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<link
rel="stylesheet"
type="text/css"
href="https://cdnjs.cloudflare.com/ajax/libs/normalize/8.0.1/normalize.min.css"
/>

<style type="text/css">
body {
padding: 20px;
}

#logo {
width: 100px;
margin-bottom: 20px;
}

h1 {
margin-top: 0px;
}

a {
text-decoration: none;
}

#participant-links ul {
margin: 0px;
padding: 0px;
list-style-type: none;
}

#participant-links li {
padding: 5px 0px;
}
</style>

<title>syft.js Example</title>
</head>
<body>
<img
src="https://www.openmined.org/static/media/logo-square-color.2b2fb62b.svg"
id="logo"
alt="OpenMined"
/>
<h1>Millionaire's problem</h1>
<p id="explanation">
<i
>This is a demo using
<a href="https://github.com/OpenMined/syft.js" target="_blank"
>syft.js from OpenMined</a
>
to display secure multi-party computation in the browser. Please open
your Javascript console to see what's going on (instructions for
<a
href="https://developers.google.com/web/tools/chrome-devtools/open#console"
target="_blank"
>Chrome</a
>
and
<a
href="https://developer.mozilla.org/en-US/docs/Tools/Web_Console/Opening_the_Web_Console"
target="_blank"
>Firefox</a
>)</i
>.
</p>
<input type="text" id="grid-instance" value="ws://localhost:3000" />
<button id="connect">Connect to Grid.js instance</button>
<div id="app">
<button id="disconnect">Disconnect</button>
<p id="identity"></p>
<div id="participant-links"></div>
<textarea
id="message"
placeholder="Type some message in here..."
></textarea
><br />
<button id="message-send">Send message</button>
</div>
</body>
</html>
111 changes: 111 additions & 0 deletions examples/with-grid/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
/*
The following is a step-by-step explanation of what's going on below:

1. Initiate syft.js and connect to grid.js
2. Get the list of plans that this user is supposed to work on
- If there is no instanceId, grid.js will generate one for us
- If there is no scopeId, grid.js will generate a scope and make this user the creator
- Altogether, grid.js will send back this user's information, their list of plans, and the instanceId's of the other participants
3. Links are created whereby other participants may join
- These links are to be shared with the other participants
- Note that each user will need to request their list of plans from the grid... they won't have access to another user's list of plans
4. Create a direct peer-to-peer connection with the other participants
- This is done using WebRTC under the hood using a mesh network by which every peer has a private data connection to every other peer
- This is an asynchronous action, meaning that peers may come and go at any point
- The syft.js library is capable of handling connections, disconnections, and reconnections without issue
*/

// In the real world: import syft from 'syft.js';
import syft from '../../src';
import {
getQueryVariable,
writeIdentityToDOM,
writeLinksToDOM
} from './_helpers';

const gridInstance = document.getElementById('grid-instance');
const connectButton = document.getElementById('connect');
const disconnectButton = document.getElementById('disconnect');
const appContainer = document.getElementById('app');
const textarea = document.getElementById('message');
const submitButton = document.getElementById('message-send');

appContainer.style.display = 'none';

connectButton.onclick = () => {
appContainer.style.display = 'block';
gridInstance.style.display = 'none';
connectButton.style.display = 'none';

startSyft(gridInstance.value);
};

const startSyft = url => {
const instanceId = getQueryVariable('instance_id');
const scopeId = getQueryVariable('scope_id');

// 1. Initiate syft.js and create socket connection
const mySyft = new syft({
verbose: true,
url,
instanceId,
scopeId,
protocolId: 'millionaire-problem'
});

submitButton.onclick = () => {
mySyft.sendToParticipants(textarea.value);

textarea.value = '';
};

disconnectButton.onclick = () => {
mySyft.disconnectFromParticipants();
mySyft.disconnectFromGrid();

appContainer.style.display = 'none';
gridInstance.style.display = 'block';
connectButton.style.display = 'block';
};

mySyft.onSocketStatus(async ({ connected }) => {
if (connected) {
// 2. Get the plans that are assigned to me
const plans = await mySyft.getPlans();

console.log('PLANS', plans);

// Write my identity to the screen - not required
writeIdentityToDOM(
`You are ${mySyft.role} "${mySyft.instanceId}" in scope "${mySyft.scopeId}"`
);

// Push the instanceId and scopeId onto the current URL if they aren't already there
// This isn't strictly necessary, but if a user is a creator of a scope (instead of a participant),
// then they won't be able to refresh and rejoin the scope they created
if (!instanceId && !scopeId) {
window.history.pushState(
{},
null,
`?instance_id=${mySyft.instanceId}&scope_id=${mySyft.scopeId}`
);
}

// 3. Create links for the other participants
if (mySyft.role === 'creator') {
writeLinksToDOM(
mySyft.participants.map(
id =>
`${window.location.origin +
window.location.pathname}?instance_id=${id}&scope_id=${
mySyft.scopeId
}`
)
);
}

// 4. Create a direct P2P connection with the other participants
mySyft.connectToParticipants();
}
});
};
Loading