Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

letter shop tutorial #1

Merged
merged 12 commits into from
Oct 4, 2017
1 change: 1 addition & 0 deletions _config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
theme: jekyll-theme-merlot
232 changes: 232 additions & 0 deletions letter-shop.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,232 @@
# Letter Shop Tutorial

## What you need before you start:

* a laptop with an internet connection and NodeJS installed
* basic knowledge of the command line terminal
* basic knowledge of JavaScript, including Promises and Buffers
* (optional) the concept of one-way hash functions

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also git if you're going to have people clone the repo

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So far I just added the scripts and a package.json, but the text still describe cut-and-pasting the files one-by-one.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you have the code there it seems better to just ask people to clone the repo instead of copying and pasting

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like git is now a prerequisite


## What you'll learn:

* the concept Hash Time Lock Agreements (HTLAs)
* the Ledger Plugin Interface (LPI)
* how to build a paid web app using Interledger!

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like that you don't have to copy and paste the code anymore but it almost feels too quick now, like it's missing the actual code walkthrough. What do you think about the tutorial style of something like React where they show small snippets next to explanation and then also provide the full code of the example? (They also link to codepen, which is cool cause there's no need to download it to play around with it, but then again it is a frontend framework)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about the tutorial style of something like React where they show small snippets next to explanation and then also provide the full code of the example?

That's my preferred style of tutorial. Snippets inside code and then full code at the end.

The Kotlin Tutorials have a very cool way of showing code snippets that you can expand and execute: https://kotlinlang.org/docs/reference/basic-syntax.html

Not sure what library they use.


## Step 1:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe make this heading more descriptive like Step 1: Creating the Letter Shop


Save the following JavaScript as `shop.js`:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about making a repo that contains all of the example code that you just clone?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, while I was writing #2 I ended up adding the code in this repo, I'll add this tutorial's files to this PR.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should start with the instructions for cloning and npm installing the repo instead

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. An advantage of that would be that we can then discuss shorter snippets of the code and not have to paste entire files - especially where shop.js and shop2.js and shop3.js from the next tutorial have a lot in common.


```js
const http = require('http')

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about using something like koa instead? I think it's pretty uncommon in the Node ecosystem to use the http module directly

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm, interesting argument. I don't think I've ever met a NodeJS developer who was unfamiliar with the http module though, have you? And I definitely know there are people the other way around, for instant backend developers (including myself) would be more familiar with the http module than with koa. Also, in the second tutorial I need to flush the headers, so I think it wouldn't work there.

const crypto = require('crypto')
const Plugin = require('ilp-plugin-xrp-escrow')
function base64 (buf) { return buf.toString('base64').replace(/\+/g, '-').replace(/\//g, '_').replace(/=+$/, '') }

let fulfillments = {}
let letters = {}

const plugin = new Plugin({
secret: 'ssGjGT4sz4rp2xahcDj87P71rTYXo',
account: 'rrhnXcox5bEmZfJCHzPxajUtwdt772zrCW',
server: 'wss://s.altnet.rippletest.net:51233',
prefix: 'test.crypto.xrp.'
})

plugin.connect().then(function () {
plugin.on('incoming_prepare', function (transfer) {
plugin.fulfillCondition(transfer.id, fulfillments[transfer.executionCondition]).catch(function () {})

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an insecure way to handle fulfilling payments (because you're not checking the amount). We should not encourage people to do this themselves because they are likely to make mistakes like this and lose money. The example should use the ilp module instead. If neither PSK nor IPR is suitable for this we should discuss why they aren't, come up with an alternative, and include it in that module.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll check the amount (the second tutorial does check the transfer amount against the IPP amount). Checking the expiry is already done by the ledger. I don't think we need to include a module for that?

The payment request here is a human-readable phrase. Further on, the proxy actually checks if the first word on the page is 'Please', and only pays if it is. :) I'm starting with this simplistic flow on purpose; using IPR would just hide the under-the-hood semantics from sight.

I agree about setting a price per letter though, and then checking that. It's currently a 'pay what you want' shop and that's just silly. :)

})

http.createServer(function (req, res) {
if (letters[req.url.substring(1)]) {
res.end('Your letter: ' + letters[req.url.substring(1)])
} else {
const secret = crypto.randomBytes(32)
const fulfillment = base64(secret)
const condition = base64(crypto.createHash('sha256').update(secret).digest())

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: put some of the function calls on different lines or make the code area wider so the whole line fits without scrolling

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, the code area is annoyingly narrow, but then also having to use linebreaks everywhere hurts the readability. I've gone back and forth on that. Maybe I can make the code area wider somehow.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed it to Modernist now, which gives 23 more chars width (up from 61 to 84). We could get another 8 chars by switching to Cayman, and that theme may fit better in the Interledger house style, but probably looks less fancy for tutorials.

const letter = ('ABCDEFGHIJKLMNOPQRSTUVWXYZ').split('')[(Math.floor(Math.random() * 26))]
fulfillments[condition] = fulfillment
letters[fulfillment] = letter
console.log('Generated letter for visitor on ', req.url, { secret, fulfillment, condition, letter })
res.end('Please send an Interledger payment to ' + plugin.getAccount() + ' with condition ' + condition)
}
}).listen(8000)
})
```

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm trying to read this from the perspective of someone who hasn't seen our code before. It feels like this code could use more explanation of what it's doing. What do you think about either:

  1. Walking people through building up this example, explaining what each step is doing or
  2. Breaking down the code segment by segment after you show the full example or
  3. Adding more comments in the code about what's going on?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, if we tell people to clone the repo then we have more freedom to break down the snippets into smaller ones.


Set up a Letter Shop website on http://localhost:8000, by running:

```sh
npm install michielbdejong/ilp-plugin-xrp-escrow#3fadeb4

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't think this is necessary now that it's in the package.json

node shop.js
```
In the code, you see that an 'ilp-plugin-xrp-escrow' Plugin is being configured with secret, account, server, and prefix.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are very dense paragraphs. It would be good to break them up into much shorter sections and maybe put some headings above them to help when the reader is skimming

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably need to explain that plugins are abstractions over the functionality of different ledgers before going to Interledger details

The first three come from the [XRP Testnet Faucet](https://ripple.com/build/xrp-test-net/). The prefix is an Interledger prefix, which is like an [IP subnet](https://en.wikipedia.org/wiki/Subnetwork). In this case, `test.` indicates that we are connecting to the Interledger testnet-of-testnet. The next part, `crypto.` indicates that we will be referring to a crypto currency's ledger. And finally, `xrp.` indicates that this ledger is the XRP testnet ledger. If you know the ledger prefix and the account, you can put them together to get the [Interledger Address](https://interledger.org/rfcs/0015-ilp-addresses/draft-1.html). In this case, the Interledger address of our Letter Shop is `test.crypto.xrp.rrhnXcox5bEmZfJCHzPxajUtwdt772zrCW`.

The plugin used here is specific to XRP, and to the use of on-ledger escrow. Escrow for XRP is described [here](https://ripple.com/build/rippleapi/#transaction-types). Escrow transfers differ from normal transfer in that the recipient doesn't automatically receive the amount of the transfer in their account; the need to produce something in order to claim the funds. During the time between the sender's action of preparing the transfer (creating the escrow), and the time the recipient produces the fulfillment for the transfer's condition, the money is on hold on the ledger. If the recipient doesn't produce the fulfillment in time, the transaction is canceled, and the money goes back into the sender's account.

In the case of Interledger, the fulfillment is always a 32-byte string, and the condition is the sha256 hash of that string.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This description kind of reads like you expect the reader to know what condition and fulfillment mean, but those terms are somewhat specific to Interledger

SHA256 is a one-way hash function, so if you know `fulfillment`, then it's easy and quick to calculate `condition = sha256(fulfillment)`, but if you only know the condition, if you only have a million years or less, it's near-impossible to find (i.e., guess) a fulfillment for which `condition = sha256(fulfillment)` would hold. And in practice, we tend to use rollback timeouts that are of course much shorter! :) Because the transfer is *locked* until the recipient produces the correct fulfillment, the condition of the transfer is a *hash* of its fulfillment, and the transfer will *time* out after a while if the recipient doesn't produce the fulfillment to claim the funds, we call this type of conditional transfer a *Hash Time Lock Contract (HTLC)*. We call this a contract between sender an recipient, that is enforced by the ledger. If the sender and the recipient would exchange the condition and the fulfillment directly, that would be an off-ledger transaction, and instead of a Hash Time Lock Contract, we would use the more general term Hash Time Lock Agreement, to indicate that the interaction happens without having the ledger as an arbitor.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe link to the HTLA explanation doc in case people want (a lot) more details


## Paying for your letter

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add "Step 2:"

Visit http://localhost:8000. You'll see something like:
```txt
Please send an Interledger payment to test.crypto.xrp.rrhnXcox5bEmZfJCHzPxajUtwdt772zrCW with condition A6-zI1uIEtjOXMTDoZLtML1xj6YPxBA6yxIQyVh4qhE
```

Save the following script as `pay.js`:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not necessary

```js
const Plugin = require('ilp-plugin-xrp-escrow')
const uuid = require('uuid/v4')

const plugin = new Plugin({
secret: 'sndb5JDdyWiHZia9zv44zSr2itRy1',
account: 'rGtqDAJNTDMLaNNfq1RVYgPT8onFMj19Aj',
server: 'wss://s.altnet.rippletest.net:51233',
prefix: 'test.crypto.xrp.'
})

function sendTransfer (obj) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's more confusing for there to be a separate function called sendTransfer. Why not just put these parameters into the call below?

obj.id = uuid()
obj.from = plugin.getAccount()
// to
obj.ledger = plugin.getInfo().prefix
// amount
obj.ilp = 'AA'

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the example should use proper ILP packets

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I introduce those in #2. I guess setting it to 'AA' is a violation of the LPI though... I could set it to a proper ILP packet, and add a comment saying this field will be discussed in next tutorial.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds good. I think it's better to do it properly and put a note saying "I'll explain this later"

obj.noteToSelf = {}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't needed anymore

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

draft 5 still has it as a required field? We mentioned making it optional in interledger/rfcs#289 but never actioned that.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm talking about the noteToSelf and maybe the custom fields. There used to be a bug where ilp-plugin-xrp-escrow would throw an error if those weren't set but I think that's fixed now

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regardless of the behavior of this plugin, and regardless of what we may agree as common sense, I'm reading the current version of IL-RFC-4 (draft 5) doesn't say they are optional, so therefore it would be an incorrect use of the LPI if this tutorial were to leave them out when calling plugin.sendTransfer.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess the difference is, you're looking at what the plugin does, I'm looking at what the spec says. :)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

interledger/rfcs#304

It wasn't explicit in the field documentation but this example shows sending a transfer without those fields set.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IL-RFC-4 (draft 5)

Draft numbers finally come in useful! 🍾

// executionCondition
obj.expiresAt = new Date(new Date().getTime() + 1000000).toISOString()
obj.custom = {}
return plugin.sendTransfer(obj)
}

plugin.connect().then(function () {
plugin.on('outgoing_fulfill', function (transferId, fulfillment) {
console.log('Got the fulfillment, you paid for your letter! Go get it at http://localhost:8000/' + fulfillment)
plugin.disconnect()
process.exit()
})

sendTransfer({
to: process.argv[2],
amount: '1',
executionCondition: process.argv[3]

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: The fact that it's pulling these options from the command line is a little bit hard to find since they're buried down here. Maybe put them up at the top in a variable so it's the first thing you see (after the imports)

}).then(function () {
console.log('transfer prepared, waiting for fulfillment...')
}, function (err) {
console.error(err.message)
})
})
```

Now run, something like the following, (put the condition from your own shop's local website as the second argument):
```sh
$ node ./pay.js test.crypto.xrp.rrhnXcox5bEmZfJCHzPxajUtwdt772zrCW A6-zI1uIEtjOXMTDoZLtML1xj6YPxBA6yxIQyVh4qhE
```

Now wait for about 30 seconds, until you see something like:
```txt
Got the fulfillment, you paid for your letter! Go get it at http://localhost:8000/RWtoGF_sOVoIH3Casd-nmApQ-Thzl03lH-cInXume_g
```

As the instructions say, visit that URL to get your letter! :)

## Paying proxy

It's of course very cumbersome to cut and paste the condition from your browser to your command-line terminal each time you need to pay for something online, and then to cut and past back the fulfillment from your terminal to your browser once you paid. Therefore, the following paying proxy is useful, which parses the shop's payment instructions, executes them, retreives the paid content, and serves it on port 8001. Save it as `proxy.js`:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also don't need to save this anymore

```js
const Plugin = require('ilp-plugin-xrp-escrow')
const http = require('http')
const fetch = require('node-fetch')
const uuid = require('uuid/v4')

const plugin = new Plugin({
secret: 'sndb5JDdyWiHZia9zv44zSr2itRy1',
account: 'rGtqDAJNTDMLaNNfq1RVYgPT8onFMj19Aj',
server: 'wss://s.altnet.rippletest.net:51233',
prefix: 'test.crypto.xrp.'
})

const pendingRes = {}

function sendTransfer (obj) {
obj.id = uuid()
obj.from = plugin.getAccount()
// to
obj.ledger = plugin.getInfo().prefix
// amount

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, where's the amount?

Copy link
Contributor Author

@michielbdejong michielbdejong Sep 26, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All this function does is add the "standard" fields. You still need to set the "interesting" fields yourself, see for instance how it gets called here.

obj.ilp = 'AA'
obj.noteToSelf = {}
// executionCondition
obj.expiresAt = new Date(new Date().getTime() + 1000000).toISOString()
obj.custom = {}
return plugin.sendTransfer(obj).then(function () {
return obj.id
})
}

plugin.connect().then(function () {
plugin.on('outgoing_fulfill', function (transfer, fulfillment) {
console.log('outgoing fulfill', transfer, fulfillment, 'http://localhost:8000/' + fulfillment)
fetch('http://localhost:8000/' + fulfillment).then(function (inRes) {
return inRes.text()
}).then(function (body) {
pendingRes[transfer.id].end(body)
})
})

http.createServer(function (req, outRes) {
fetch('http://localhost:8000' + req.url).then(function (inRes) {
return inRes.text()
}).then(function (body) {
const parts = body.split(' ')
if (parts[0] === 'Please') {
sendTransfer({
to: parts[6],
amount: '1',
executionCondition: parts[9]
}).then(function (transferId) {
console.log('transfer sent', transferId)
pendingRes[transferId] = outRes

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be cleaner to add an event listener on the plugin here instead of storing the responses in that global object

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one listener per transferId? and then remove it once called, using arguments.callee, I guess?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, using a named function. ok, that works! :)

}, function (err) {
console.error(err.message)
})
} else {
outRes.end(parts.join(' '))
}
})
}).listen(8001)
})
```

Now run:
```sh
npm install node-fetch

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not necessary

node ./shop.js # unless your shop was still running from before
node ./proxy.js
```
and instead of visiting http://localhost:8000/, visit http://localhost:8001/ - you'll see that viewing the letter shop through the paying proxy is slower, but more convenient! You'll get a new letter each time you refresh the page.

## Concepts learned

The plugin used in all three scripts exposes the [Ledger Plugin Interface (LPI)](https://interledger.org/rfcs/0004-ledger-plugin-interface/draft-5.html), and of that, this script uses the following methods and events:
* `sendTransfer` method (in `pay.js` and `proxy.js`, prepares a transfer to some other account on the same ledger)
* `getInfo` method (used in `pay.js` and `proxy.js` to fill in the `ledger` field to pass to `sendTransfer`)
* `getAccount` method (used in `pay.js` and `proxy.js` to fill in the `from` field to pass to `sendTransfer`)
* `fulfillCondition` method (in `shop.js`, fulfills the condition of an incoming transfer)
* `incoming_prepare` event (in `shop.js`, is triggered when someone else sends you a conditional transfer)
* `outgoing_fulfill` event (in `pay.js` and `proxy.js`, is triggered when someone else fulfills your conditional transfer)

If you read the paragraphs above, you will have seen the following new words; see the [glossary](https://interledger.org/rfcs/0019-glossary/) as a reference if you
forget some of them.

* transfer
* condition
* fulfillment
* HTLA
* ledger prefix
* Interledger address
* Ledger Plugin Interface, and some of its methods