From 4e742ae54dcd738b67a4a541f9070837a3c16e98 Mon Sep 17 00:00:00 2001 From: Kincaid O'Neil Date: Wed, 19 Jun 2019 23:44:45 -0400 Subject: [PATCH 01/32] docs: add settlement engine RFC --- .../0000-settlement-engine.md | 353 ++++++++++++++++++ 1 file changed, 353 insertions(+) create mode 100644 0000-settlement-engine/0000-settlement-engine.md diff --git a/0000-settlement-engine/0000-settlement-engine.md b/0000-settlement-engine/0000-settlement-engine.md new file mode 100644 index 00000000..2a0a13ce --- /dev/null +++ b/0000-settlement-engine/0000-settlement-engine.md @@ -0,0 +1,353 @@ +--- +title: Settlement Engine +draft: 1 +--- + +# Settlement Engine + +## Conventions + +The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC 2119](https://tools.ietf.org/html/rfc2119). + +## Definitions + +**Peers** are two participants that share an accounting relationship and have a medium to communicate with one another. + +A **transport** is a communication layer for two peers to send messages between one another. + +An **account** is a record of accrued obligations between two peers as they transact with one another, denominated in a particular fungible asset. + +An **accounting system** tracks a participant's accounts and the net balance owed between peers. + +A **settlement** is a reconciliation of a liability. + +A **settlement system** is a medium for exchanging value. + +## Overview + +This specification codifies a **settlement engine**, a common interface between an accounting system and heterogeneous settlement systems. + +Compatible settlement engine implementations enable peers to mutually reconcile one's liabilities through integrating with a shared settlement system. + +The purpose of a settlement engine is to perform two primary operations: + +1. Execute outgoing settlements to a peer +2. Acknowledge incoming settlements from a peer + +Settlements may differ depending on the settlement engine implementation, such as transferring value on the underlying system or ledger, and/or using a custom protocol through which the peers mutually agree to reconcile a liability. + +The core test case for a settlement engine implementation is **the sum of amounts an instance is instructed to settle should eventually equal the sum of amounts the peer's instance acknowledges receipt of.** + +Varying implementations, arrangements, and conditions (such as connectivity to the ledger) may delay consistency of instructed and acknowledged settlements by an indeterminate amount of time. + +## Example Settlement Arrangements + +### Simple value transfers + +Suppose two peers, Alice and Bob, are operating the same or compatible settlement engine implementations, which settle using a ledger which both Alice and Bob maintain accounts on. + +1. Alice's accounting system instructs her settlement engine to perform a settlement of 5 units to Bob. +2. Subsequently, Alice's settlement engine sends a value transfer of 5 units from Alice's ledger account to Bob's ledger account. + - To know where to send the payment, Alice may have configured her settlement engine with Bob's ledger identifier, or the settlement engine may automatically send a message to Bob's settlement engine to request Bob's ledger identifier. + - The settlement engine implementation should ensure that an incoming payment is only credited to the peer's account that sent it. +3. Bob's settlement engine, always watching the ledger, recognizes the payment, correlates that Alice sent it, and notifies its accounting system of the incoming settlement for 5 units. + +### Payment channels + +Payment channels are special bilateral or multilateral settlement systems backed by escrows on an underlying shared ledger. They enable peers to adjust the allocation of these escrows among themselves. A participant can sign claims—messages with digital signatures entitling its peer to some greater portion of the escrow—which the peer can validate immediately without informing the underlying ledger. When the peers are done transacting, the most recent of many such claims is enforceable by the underlying ledger so the escrowed funds can be released and distributed accordingly. + +Payment channels offer very fast and cheap settlement, which is beneficial to peers which don't extend much credit to one another. + +Suppose two peers, Alice and Bob, are operating the same or compatible settlement engine implementations, which settle using a payment channel on a shared ledger. + +1. Alice instructs her settlement engine to create a payment channel to Bob and fund it for 20 units. + - Alice's settlement engine should escrow 20 units in the payment channel and inform Bob's settlement engine that a payment channel was created. + - Note, however, that the funds are still in Alice's custody, and are still only accessible by Alice. +2. Alice's accounting system instructs her settlement engine to settle 5 units to Bob. +3. Alice's settlement engine signs a claim entitling Bob to 5 units from Alice's escrowed funds, and sends it to Bob's settlement engine. + - If Bob already had some claim, the new claim should entitle Bob to 5 _additional_ units. + - If Alice has already entitled Bob to all the available escrowed funds, the settlement engine may track that it owes Bob 5 units without sending a claim, until there's sufficient capacity in the payment channel. +4. Bob's settlement engine, upon receiving the claim, should verify the message is valid and its digital signature is authentic, and determine how much additional funds it's entitled to (in this case, 5 units). Then, it should it notify the accounting system that it received the given amount, 5 units. +5. Alice and Bob may repeat this process many times, as long as the party sending the payment channel claim has funds that haven't already been allocated to the peer. If additional funds need to be escrowed, the settlement engine may offer this functionality through a custom instruction. +6. At a later time, Alice or Bob may instruct their settlement engines to close the payment channel and release the allotted funds back to their respective ledger accounts. + - Depending upon the payment channel implementation, there may be both cooperative and non-cooperative mechanisms for this, with varying time delays until the funds can be dispersed. + - Settlement engines using payment channels may also have functionality to watch the underlying ledger to ensure the peer doesn't try to submit an old message that would incorrectly distribute the escrowed funds. + +### Sender collects a fee + +Suppose Alice and Bob's settlement arrangement is such that, in order the Alice to settle her liabilities to Bob, Alice requires that first Bob must pay a fee. + +1. The settlement engine implementations that Alice and Bob are operating might negotiate the fee that Alice will collect (suppose a fee of 3 units). + - The configuration of the settlement engines could ensure the fee is reasonable, or Bob could instruct his settlement engine of some limit for the fee. +2. After it's negotiated, Bob's settlement engine preemptively acknowledges it received an incoming settlement for 3 units. + - Note that even though no settlement on the shared ledger or system was received, Alice and Bob did mutually agree to reconcile a future liability: that is, the fee charged by Alice on Bob's behalf. +3. As Alice and Bob transact with one another, the liabilities that Alice owes Bob accrue, and Alice's accounting system instructs her settlement engine to settle them. +4. Alice's settlement engine, instead of sending a settlement on the shared settlement system, should track the amount owed to Bob, until it accrues to 3 units, the value of the agreed fee. +5. After the value of the fee is accounted for through accrued settlements, if Alice's accounting system instructs her settlement engine to settle, it should send these additional settlements on the shared system or ledger. + - For example, after negotiating the fee, suppose Alice's accounting system tells her settlement engine to settle 2 units. Nothing should happen, since it's accruing owed balance to cover the fee of 3 units. + - If the settlement engine is later instructed to settle another 2 units, it should send a payment on the shared settlement system for 1 unit, since the other unit was taken out as apart of the fee. + - Any further instructions to settle should trigger payments on the shared system as normal. + +### Recipient collects a fee + +Suppose Alice and Bob's settlement arrangement is such that, before Bob can begin acknowledging incoming settlements from Alice, Bob must collect some initiation fee related to the settlement system. + +1. Alice sends a custom instruction to her settlement engine to settle the fee. Note that she does not use the instruction reserved for the accounting system. +2. Alice's settlement engine automatically negotiates the fee with Bob's settlement engine (likely subject to some limit). +3. Alice's settlement engine sends a payment on the shared ledger or system to Bob's settlement engine for the value of the fee. +4. Bob's settlement engine, instead of informing the accounting system of the incoming settlement, does nothing, but internally tracks that the negotiated fee was paid. +5. If Alice's accounting system triggers any subsequent settlements to Bob, Bob's settlement engine should acknowledge them to the accounting system as normal. + +## Specification + +Operating a settlement engine requires three components each exposing a particular HTTP API: a settlement engine, an accounting system, and a transport. + +API endpoints exposed by these systems may be classified in two ways: + +1. Endpoints that may be invoked by one of these other two components, and MUST NOT be invoked manually. At a minimum, all settlement engine implementations MUST support these endpoints. +2. Endpoints that may be invoked manually by an operator. + +This document outlines the first class of endpoints: those only essential to automated interoperability between these systems, and not manual operation. + +### Accounts and identifiers + +Each account MUST be identified by a unique, URL-safe string. + +#### Accounts vs ledger identities + +The settlement engine is RECOMMENDED to be responsible for correlating an account identifier to the peer's identity on the shared ledger or settlement system, if required. + +For separation of concerns between the accounting system and the settlement system, the accounting system is NOT RECOMMENDED to have knowledge of the peer's identity on the shared settlement system, but rather only the account identifier described here. + +#### Multiple accounts + +Settlement engine implementations SHOULD support settling with multiple accounts. + +#### Correlating messages + +The transport MUST correlate incoming messages from the peer to the correct account identifier, and MUST correlate outgoing messages from an account identifier to the correct peer to send them to. + +### Units and quantities + +#### `Quantity` type + +A quantity represents an amount denominated in some unit of a particular fungible asset. In uses of this type, the asset should be implicit. + +##### Attributes + +- **`amount`** string + - Number of the given unit of the asset between 0 and the maximum 64-bit unsigned integer, 18,446,744,073,709,551,615 (inclusive). + - The amount is encoded as a string to ensure no precision is lost on platforms that don't natively support full precision 64-bit integers. +- **`scale`** number + - Number of orders of magnitude smaller this unit is as compared to the unit of exchange for the asset. + - All units MUST be at least as precise as the unit of exchange. For example, if the asset is U.S. dollars, quantities cannot be denominated in $100 bills, since that's less precise than quantities denominated in multiples of $1. Thus, the scale MUST NOT be a negative integer. + +##### Example + +To represent $2.54 in units of cents, where the amount is multiple of $0.01: + +```json +{ + "amount": "254", + "scale": 2 +} +``` + +#### Choosing scales + +The unit or scale used for quantities by the settlement engine is RECOMMENDED to be the smallest denomination of the asset on the settlement system. + +The accounting system and settlement engine are RECOMMENDED to use the same unit or scale. + +#### Scale conversions + +Since the systems MAY use different scales, when interfacing with one another, care must be taken to ensure the system with additional precision tracks the amount leftover by the other system so it may accumulate and eventually be correctly accounted for. + +If one of the systems receives a request with a **[`Quantity`](#quantity-type)** denominated in a unit more precise than its unit, it SHOULD convert the quantity into its native unit. If so, the resulting amount MUST be rounded down before fulfilling the request. + +### Settlement Engine HTTP API + +#### Initiate an account + +Inform the settlement engine that a new accounting relationship was instantiated. The settlement engine MAY perform tasks as prerequisite to settle the new account. For example, a settlement engine implementation might send messages to the peer to exchange ledger identifiers or to negotiate terms of the settlement arrangement. + +The transport MUST be the only entity invoking this instruction to ensure the account identifier can be correlated with the correct peer to send and receive messages. Note that the transport also MUST have a mechanism to notify the accounting system that a new account should be instantiated (out-of-scope of this specification). + +##### Request + +```http +POST /accounts/:id HTTP/1.1 +``` + +##### Response + +```http +HTTP/1.1 201 CREATED +``` + +#### Perform outgoing settlement + +Reconcile a liability owed to the peer for the given amount. + +Note that the settlement engine MAY accrue owed settlements without settling immediately due to varying arrangements, implementations, or conditions. The settlement engine SHOULD persist amounts owed to the peer that have yet to be settled so they can be settled later. + +The accounting system MUST be the only entity invoking this instruction and MUST preemptively debit this amount from the accounts payable, or some liability account tracking a balance owed to the peer, to ensure accurate accounting. + +##### Request + +```http +POST /accounts/:id/settle HTTP/1.1 +Accept: application/json +Content-Type: application/json +``` + +> **[`Quantity`](#quantity-type)** to settle +> +> - If the amount is `0`, the settlement engine MAY retry failed or queued settlements. + +##### Response + +```http +HTTP/1.1 200 OK +Content-Type: application/json +``` + +> **[`Quantity`](#quantity-type)** enqueued to settle +> +> - This response does not guarantee that a settlement was executed. +> - If the settlement engine uses a unit less precise than the accounting system's unit, in order to prevent the two systems getting out-of-sync, it must indicate the amount it queued so the accounting system can correctly track the amount leftover. + +#### Handle incoming request + +Respond to an incoming message from the given peer's settlement engine. The transport MUST be the only entity invoking this instruction. + +##### Request + +```http +POST /accounts/:id/handleMessage HTTP/1.1 +Accept: application/octet-stream +Content-Type: application/octet-stream +``` + +> _<raw bytes of message>_ + +##### Response + +```http +HTTP/1.1 200 OK +Content-Type: application/octet-stream +``` + +> _<raw bytes of response>_ + +#### Complete specification + +
+ The full settlement engine HTTP API, including all response codes, error codes, and additional endpoints for manual management are available in this Swagger definition. + +``` +TODO Embed the definition here +``` + +
+ +Settlement engine implementations are RECOMMENDED to implement the standardized endpoints for manual management noted above. + +#### Additional RPCs + +Settlement engine implementations MAY expose additional, non-standard endpoints for manual operations or configuration. + +For example, settlement engines that use payment channels may expose functionality to manually open, fund, and close channels. + +### Accounting System HTTP API + +#### Credit incoming settlement + +Notify the accounting system of receipt of an incoming settlement. Note that the settlement engine MAY accrue incoming settlement acknowledgements without immediately informing the accounting system. + +The settlement engine MUST be the only entity invoking this message and MUST credit this amount to the accounts receivable, or some asset account tracking the balance owed by the peer, to ensure accurate accounting. + +##### Request + +```http +POST /accounts/:id/receipt HTTP/1.1 +Accept: application/json +Content-Type: application/json +``` + +> **[`Quantity`](#quantity-type)** that should be credited to the account as an incoming settlement + +##### Response + +```http +HTTP/1.1 200 OK +Content-Type: application/json +``` + +> **[`Quantity`](#quantity-type)** credited to the account +> +> - If the accounting system uses a unit less precise than the settlement engine's unit in order to prevent the two systems getting out-of-sync, it must indicate the amount it acknowledged receipt of so the settlement engine can track the amount leftover. + +### Transport HTTP API + +#### Send outgoing request + +Send a message to given peer's settlement engine and return its response. The settlement engine MUST be the only entity invoking this instruction. + +##### Request + +```http +POST /accounts/:id/sendMessage HTTP/1.1 +Accept: application/octet-stream +Content-Type: application/octet-stream +``` + +> _<raw bytes of message>_ + +##### Response + +```http +HTTP/1.1 200 OK +Content-Type: application/octet-stream +``` + +> _<raw bytes of response>_ + +### Idempotency + +_(The following section was adapted from the [Stripe API documentation](https://stripe.com/docs/api/idempotent_requests).)_ + +Each API MUST support [idempotency](https://en.wikipedia.org/wiki/Idempotence) for safely retrying requests without accidentally performing the same operation twice. This is useful when an API call is disrupted in transit and you do not receive a response. For example, if a request to send a settlement does not respond due to a network connection error, you can retry the request with the same idempotency key to guarantee that no more than one charge is created. + +To perform an idempotent request, provide an additional `Idempotency-Key: ` header to the request. + +Idempotency works by saving the resulting status code and body of the first request made for any given idempotency key, regardless of whether it succeeded or failed. + +Subsequent requests with the same key return the same result, including `500` errors. + +An idempotency key is a unique value generated by the client which the server uses to recognize subsequent retries of the same request. How you create unique keys is up to you, but we suggest using V4 UUIDs, or another random string with enough entropy to avoid collisions. + +Keys expire after 24 hours, so a new request is generated if a key is reused outside of that time frame. The idempotency layer compares incoming parameters to those of the original request and errors unless they're the same to prevent accidental misuse. + +Results are only saved if an API endpoint started executing. If incoming parameters failed validation, or the request conflicted with another that was executing concurrently, no idempotent result is saved because no API endpoint began execution. It is safe to retry these requests. + +All `POST` requests accept idempotency keys. Sending idempotency keys in `GET` and `DELETE` requests has no effect and should be avoided, as these requests are idempotent by definition. + +## Usage in Interledger + +Peers using the [Interledger protocol](https://github.com/interledger/rfcs/tree/master/0001-interledger-architecture) clear and fulfill conditional IOUs which affect their mutual accounting relationship. + +Since Interledger merely provides a credit network, a peer's liabilities may accrue to the point at which they must reconcile them. Settlement engines provide a mechanism for peers in Interledger to settle their liabilities and continue transacting. + +An Interledger connector MAY operate the accounting system and transport system that interfaces with the settlement engine. + +### Motivation + +The [Ledger Plugin Interface (LPIv2)](https://github.com/interledger/rfcs/blob/master/0024-ledger-plugin-interface-2/0024-ledger-plugin-interface-2.md) was originally designed as an abstraction for ledger integrations using Interledger. However, it introduced several problems: + +1. Multi-account plugins must include logic for handling ILP packets, increasing implementation complexity. +2. Plugins bundle settlement and bilateral communication functionality, limiting composability with other transports. +3. The JavaScript specific interface prevents connector implementations in other programming languages from using existing plugins. +4. Plugins operate in the same process as the connector, which limits scaling the two systems independently. + +This specification aims to provide an abstraction that rectifies these issues. From 5ea7eddaa084d7dfc1aa2af56340037771227491 Mon Sep 17 00:00:00 2001 From: Kincaid O'Neil Date: Thu, 11 Jul 2019 00:16:54 +0200 Subject: [PATCH 02/32] fix: initial edit batch, accounting explanation --- .../0000-settlement-engine.md | 230 ++++++++++-------- 1 file changed, 134 insertions(+), 96 deletions(-) diff --git a/0000-settlement-engine/0000-settlement-engine.md b/0000-settlement-engine/0000-settlement-engine.md index 2a0a13ce..37973457 100644 --- a/0000-settlement-engine/0000-settlement-engine.md +++ b/0000-settlement-engine/0000-settlement-engine.md @@ -9,105 +9,97 @@ draft: 1 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC 2119](https://tools.ietf.org/html/rfc2119). -## Definitions +## Overview -**Peers** are two participants that share an accounting relationship and have a medium to communicate with one another. +This specification codifies a common interface for a **settlement engine**. Settlement engines are services which perform two primary operations: -A **transport** is a communication layer for two peers to send messages between one another. +1. **Send payments**: execute outgoing settlements +2. **Credit payments**: acknowledge incoming settlements -An **account** is a record of accrued obligations between two peers as they transact with one another, denominated in a particular fungible asset. +**Peers**, or entities that transact with one another, may operate compatible settlement engines to settle their liabilities. Different implementations may utilize different settlement systems or types of settlements, such as: -An **accounting system** tracks a participant's accounts and the net balance owed between peers. +- **Ledger transfers:** adjusting balances on an underlying shared ledger +- **Physical settlement:** delivering an asset to the counterparty +- **Payment channels:** sending a claim entitling the peer to an escrow +- Using a custom protocol to mututally agree to discharge a liability -A **settlement** is a reconciliation of a liability. +## Usage in Interledger -A **settlement system** is a medium for exchanging value. +Connectors using the [Interledger protocol](https://github.com/interledger/rfcs/tree/master/0001-interledger-architecture) clear and fulfill ILP packets with peers, which represent conditional IOUs that affect their mutual accounting relationships. -## Overview +Connectors may extend some particular peer a limited line of credit (or none, if they're untrusted). As a connector receives incoming ILP PREPARE packets from a peer, forwards them, and returns corresponding ILP FULFILL packets, the peer's liabilities accrue. If the peer's liabilities to the connector exceeds the credit limit assigned to it, the connector may reject and decline to forward additional packets. -This specification codifies a **settlement engine**, a common interface between an accounting system and heterogeneous settlement systems. +In order to continue transacting, the peer must settle their liabilities. In most cases, this is accomplished through sending a payment on a settlement system that both peers have in common. The connector should acknowledge the incoming payment—irrevocably discharging some liability the peer owed it—and enabling it to clear and forward subsequent packets on credit. -Compatible settlement engine implementations enable peers to mutually reconcile one's liabilities through integrating with a shared settlement system. +Settlement engines provide a standardized mechanism for Interledger connectors to coordinate and reconcile these settlements. -The purpose of a settlement engine is to perform two primary operations: +### Accounting -1. Execute outgoing settlements to a peer -2. Acknowledge incoming settlements from a peer +An **account** represents a record of transactions between two peers, denominated in some fungible asset. Each account may be comprised of multiple **account balances**, which each represent the net difference between amounts received (credits) and amounts owed (debits) for some subset of the transactions. -Settlements may differ depending on the settlement engine implementation, such as transferring value on the underlying system or ledger, and/or using a custom protocol through which the peers mutually agree to reconcile a liability. +Interledger connectors are RECOMMENDED to operate an **accounting system** which keeps a record of two balances for each account: -The core test case for a settlement engine implementation is **the sum of amounts an instance is instructed to settle should eventually equal the sum of amounts the peer's instance acknowledges receipt of.** +- **Accounts payable**, the amount owed by the operator to the counterparty for outgoing obligations its counterparty has fulfilled. + - Positive amount indicates the operator is indebted to its counterparty (a _liability_ for the operator). + - Negative amount indicates the operator has prefunded. +- **Accounts receivable**, the amount owed to the operator by its counterparty for incoming obligations the operator has fulfilled. + - Positive amount indicates its counterparty is indebted to the operator (an _asset_ to the operator). + - Negative amount indicates its counterparty has prefunded. -Varying implementations, arrangements, and conditions (such as connectivity to the ledger) may delay consistency of instructed and acknowledged settlements by an indeterminate amount of time. +(Note: in context of "accounts payable" or "accounts receivable," "accounts" actually refers to a single account balance, not multiple accounts.) -## Example Settlement Arrangements +Thus, the connector's accounts payable with its peer should mirror its peer's accounts receivable with the connector, and respectively, the connector's accounts receivable should equal its peer's accounts payable. -### Simple value transfers +The interface in this specification provides a mechanism to reconcile settled cash balances, representing _value_, with the aforementioned account balances tracked by the accounting system, representing _credit_. -Suppose two peers, Alice and Bob, are operating the same or compatible settlement engine implementations, which settle using a ledger which both Alice and Bob maintain accounts on. +Together, a settlement engine and an accounting system interface with one another to perform double-entry bookkeeping. -1. Alice's accounting system instructs her settlement engine to perform a settlement of 5 units to Bob. -2. Subsequently, Alice's settlement engine sends a value transfer of 5 units from Alice's ledger account to Bob's ledger account. - - To know where to send the payment, Alice may have configured her settlement engine with Bob's ledger identifier, or the settlement engine may automatically send a message to Bob's settlement engine to request Bob's ledger identifier. - - The settlement engine implementation should ensure that an incoming payment is only credited to the peer's account that sent it. -3. Bob's settlement engine, always watching the ledger, recognizes the payment, correlates that Alice sent it, and notifies its accounting system of the incoming settlement for 5 units. +### Settlement -### Payment channels +A **settlement** is the full and irrevocable discharge of an obligation. Settlements may be performed by using a **settlement system**, or medium for exchanging value. -Payment channels are special bilateral or multilateral settlement systems backed by escrows on an underlying shared ledger. They enable peers to adjust the allocation of these escrows among themselves. A participant can sign claims—messages with digital signatures entitling its peer to some greater portion of the escrow—which the peer can validate immediately without informing the underlying ledger. When the peers are done transacting, the most recent of many such claims is enforceable by the underlying ledger so the escrowed funds can be released and distributed accordingly. +A **ledger** is a registry of account balances and/or transactions. (Although not all settlement systems are ledgers, here, the terms are sometimes used interchangeably.) -Payment channels offer very fast and cheap settlement, which is beneficial to peers which don't extend much credit to one another. +TODO explain the below better -Suppose two peers, Alice and Bob, are operating the same or compatible settlement engine implementations, which settle using a payment channel on a shared ledger. +To ensure correct double-entry bookkeeping, several guarantees must be upheld: +- If the accounting system instructs the settlement engine to settle some amount: + 1. The accounting system MUST subtract the amount from the accounts payable. + 2. The settlement engine MUST eventually settle the amount. +- If the settlement engine receives an incoming settlement: + 1. The settlement engine MUST acknowledge the settlement to the accounting system. + 2. The accounting system MUST add the amount to the accounts receivable. -1. Alice instructs her settlement engine to create a payment channel to Bob and fund it for 20 units. - - Alice's settlement engine should escrow 20 units in the payment channel and inform Bob's settlement engine that a payment channel was created. - - Note, however, that the funds are still in Alice's custody, and are still only accessible by Alice. -2. Alice's accounting system instructs her settlement engine to settle 5 units to Bob. -3. Alice's settlement engine signs a claim entitling Bob to 5 units from Alice's escrowed funds, and sends it to Bob's settlement engine. - - If Bob already had some claim, the new claim should entitle Bob to 5 _additional_ units. - - If Alice has already entitled Bob to all the available escrowed funds, the settlement engine may track that it owes Bob 5 units without sending a claim, until there's sufficient capacity in the payment channel. -4. Bob's settlement engine, upon receiving the claim, should verify the message is valid and its digital signature is authentic, and determine how much additional funds it's entitled to (in this case, 5 units). Then, it should it notify the accounting system that it received the given amount, 5 units. -5. Alice and Bob may repeat this process many times, as long as the party sending the payment channel claim has funds that haven't already been allocated to the peer. If additional funds need to be escrowed, the settlement engine may offer this functionality through a custom instruction. -6. At a later time, Alice or Bob may instruct their settlement engines to close the payment channel and release the allotted funds back to their respective ledger accounts. - - Depending upon the payment channel implementation, there may be both cooperative and non-cooperative mechanisms for this, with varying time delays until the funds can be dispersed. - - Settlement engines using payment channels may also have functionality to watch the underlying ledger to ensure the peer doesn't try to submit an old message that would incorrectly distribute the escrowed funds. +### Bilateral communication -### Sender collects a fee +Interledger connectors use a transport, such as HTTP or WebSockets, to send and receive ILP packets with peers. -Suppose Alice and Bob's settlement arrangement is such that, in order the Alice to settle her liabilities to Bob, Alice requires that first Bob must pay a fee. +To send messages or requests to the peer's settlement engine instance, settlement engine implementations SHOULD proxy all requests through a service operated by the connector. The connector will forward messages to the peer's connector using the shared transport, which will then send them to its settlement engine for processing. The peer settlement engine's response to the request will then be sent back, across the two connectors, to the origin settlement engine. -1. The settlement engine implementations that Alice and Bob are operating might negotiate the fee that Alice will collect (suppose a fee of 3 units). - - The configuration of the settlement engines could ensure the fee is reasonable, or Bob could instruct his settlement engine of some limit for the fee. -2. After it's negotiated, Bob's settlement engine preemptively acknowledges it received an incoming settlement for 3 units. - - Note that even though no settlement on the shared ledger or system was received, Alice and Bob did mutually agree to reconcile a future liability: that is, the fee charged by Alice on Bob's behalf. -3. As Alice and Bob transact with one another, the liabilities that Alice owes Bob accrue, and Alice's accounting system instructs her settlement engine to settle them. -4. Alice's settlement engine, instead of sending a settlement on the shared settlement system, should track the amount owed to Bob, until it accrues to 3 units, the value of the agreed fee. -5. After the value of the fee is accounted for through accrued settlements, if Alice's accounting system instructs her settlement engine to settle, it should send these additional settlements on the shared system or ledger. - - For example, after negotiating the fee, suppose Alice's accounting system tells her settlement engine to settle 2 units. Nothing should happen, since it's accruing owed balance to cover the fee of 3 units. - - If the settlement engine is later instructed to settle another 2 units, it should send a payment on the shared settlement system for 1 unit, since the other unit was taken out as apart of the fee. - - Any further instructions to settle should trigger payments on the shared system as normal. +#### Usage with [ILP-over-HTTP](https://github.com/interledger/rfcs/blob/master/0035-ilp-over-http/0035-ilp-over-http.md) -### Recipient collects a fee +To send the message to the peer, the raw message should be encoded in the `data` field of an ILP packet and addressed to `peer.settle`. The connector that receives the packet should identify the peer and send it to the settlement engine instanced associated with that account. -Suppose Alice and Bob's settlement arrangement is such that, before Bob can begin acknowledging incoming settlements from Alice, Bob must collect some initiation fee related to the settlement system. +ILP packets intended for the peer to proxy to its settlement engine should be addressed to `peer.settle` and MUST NOT be forwarded to any subsequent connectors. -1. Alice sends a custom instruction to her settlement engine to settle the fee. Note that she does not use the instruction reserved for the accounting system. -2. Alice's settlement engine automatically negotiates the fee with Bob's settlement engine (likely subject to some limit). -3. Alice's settlement engine sends a payment on the shared ledger or system to Bob's settlement engine for the value of the fee. -4. Bob's settlement engine, instead of informing the accounting system of the incoming settlement, does nothing, but internally tracks that the negotiated fee was paid. -5. If Alice's accounting system triggers any subsequent settlements to Bob, Bob's settlement engine should acknowledge them to the accounting system as normal. +Connectors MUST track which settlement engine a peer's account is associated with and ensure their messages are delivered to that particular settlement engine. -## Specification +### Motivation + +Settlement engines supercede the [Ledger Plugin Interface (LPIv2)](https://github.com/interledger/rfcs/blob/master/0024-ledger-plugin-interface-2/0024-ledger-plugin-interface-2.md), an earlier abstraction for settlement integrations with Interledger. The new model addresses these issues: -Operating a settlement engine requires three components each exposing a particular HTTP API: a settlement engine, an accounting system, and a transport. +1. Multi-account plugins required logic for handling ILP packets, increasing implementation complexity. +2. Plugins bundled settlement and bilateral communication functionality, limiting composability with other transports. +3. The JavaScript specific interface prevented connector implementations in other programming languages from using existing plugins. +4. Plugins operated in the same process as the connector, which limited scaling the two systems independently. -API endpoints exposed by these systems may be classified in two ways: +## Specification + +### Core Functionality -1. Endpoints that may be invoked by one of these other two components, and MUST NOT be invoked manually. At a minimum, all settlement engine implementations MUST support these endpoints. -2. Endpoints that may be invoked manually by an operator. +All settlement engine implementations MUST functionally guarantee that **the sum of amounts a settlement engine instance is instructed to settle should eventually equal the sum of amounts the peer's settlement engine instance acknowledges receipt of.** -This document outlines the first class of endpoints: those only essential to automated interoperability between these systems, and not manual operation. +Varying implementations, arrangements, and conditions (such as connectivity to the settlement system) may delay consistency of instructed and acknowledged settlements by an indeterminate amount of time. ### Accounts and identifiers @@ -115,23 +107,23 @@ Each account MUST be identified by a unique, URL-safe string. #### Accounts vs ledger identities -The settlement engine is RECOMMENDED to be responsible for correlating an account identifier to the peer's identity on the shared ledger or settlement system, if required. +The settlement engine MUST be responsible for correlating an account identifier to the peer's identity on the shared ledger or settlement system, if required. For separation of concerns between the accounting system and the settlement system, the accounting system is NOT RECOMMENDED to have knowledge of the peer's identity on the shared settlement system, but rather only the account identifier described here. -#### Multiple accounts +### Units and quantities -Settlement engine implementations SHOULD support settling with multiple accounts. +A **monetary unit** is the standard unit of value of a particular currency or asset, such \$1 in the case of USD, or 1 BTC in the case of Bitcoin. -#### Correlating messages +A **fractional monetary unit** is some unit smaller than the standard monetary unit with greater precision: for example, one cent (\$0.01) in the case of USD, or 1 satoshi (0.00000001 BTC) in the case of Bitcoin. -The transport MUST correlate incoming messages from the peer to the correct account identifier, and MUST correlate outgoing messages from an account identifier to the correct peer to send them to. +An **asset scale** for some fractional monetary unit is the difference in orders of magnitude between that unit and the standard monetary unit. More formally, a non-negative integer (0, 1, 2, …) such that one monetary unit equals `10^(-scale)` of the given fractional monetary unit, or 0 if the unit is the standard monetary unit. -### Units and quantities +For example, one cent represents an asset scale of 2 in the case of USD, whereas 1 satoshi represents an asset scale of 8 in the case of Bitcoin. -#### `Quantity` type +#### `Quantity` JSON type -A quantity represents an amount denominated in some unit of a particular fungible asset. In uses of this type, the asset should be implicit. +A quantity represents an amount denominated in some unit of a particular fungible asset. In uses of this type, the type of asset should be implicit. ##### Attributes @@ -139,12 +131,11 @@ A quantity represents an amount denominated in some unit of a particular fungibl - Number of the given unit of the asset between 0 and the maximum 64-bit unsigned integer, 18,446,744,073,709,551,615 (inclusive). - The amount is encoded as a string to ensure no precision is lost on platforms that don't natively support full precision 64-bit integers. - **`scale`** number - - Number of orders of magnitude smaller this unit is as compared to the unit of exchange for the asset. - - All units MUST be at least as precise as the unit of exchange. For example, if the asset is U.S. dollars, quantities cannot be denominated in $100 bills, since that's less precise than quantities denominated in multiples of $1. Thus, the scale MUST NOT be a negative integer. + - Represents an _asset scale_ as defined in the previous section. ##### Example -To represent $2.54 in units of cents, where the amount is multiple of $0.01: +To represent $2.54 in units of cents, where the amount is a multiple of $0.01: ```json { @@ -155,15 +146,17 @@ To represent $2.54 in units of cents, where the amount is multiple of $0.01: #### Choosing scales -The unit or scale used for quantities by the settlement engine is RECOMMENDED to be the smallest denomination of the asset on the settlement system. +The native unit or scale of the settlement engine is RECOMMENDED to be the same unit its settlements are denominated in, which is typically the smallest denomination of the asset on the settlement system. -The accounting system and settlement engine are RECOMMENDED to use the same unit or scale. +The accounting system and settlement engine are RECOMMENDED to use the same unit or scale to minimize conversion inconsistencies. #### Scale conversions -Since the systems MAY use different scales, when interfacing with one another, care must be taken to ensure the system with additional precision tracks the amount leftover by the other system so it may accumulate and eventually be correctly accounted for. +If the accounting system or settlement engine receives a request with a **[`Quantity`](#quantity-type)** denominated in a unit more precise than its unit, it MUST convert the quantity into its native unit. If so, the resulting amount MUST be rounded down before fulfilling the request. + +The response to the request MUST include the converted, rounded **[`Quantity`](#quantity-type)** used to fulfill the request, which MUST be less than or equal to the amount sent in the original request. -If one of the systems receives a request with a **[`Quantity`](#quantity-type)** denominated in a unit more precise than its unit, it SHOULD convert the quantity into its native unit. If so, the resulting amount MUST be rounded down before fulfilling the request. +Then, the system with the additional precision initiating the request MUST track the sum leftover by subtracting the two the other so it may accumulate and be added to the amount in subsequent requests. ### Settlement Engine HTTP API @@ -203,19 +196,20 @@ Content-Type: application/json > **[`Quantity`](#quantity-type)** to settle > -> - If the amount is `0`, the settlement engine MAY retry failed or queued settlements. +> - If the amount is `0`, the settlement engine MAY retry failed settlements. ##### Response ```http -HTTP/1.1 200 OK +HTTP/1.1 202 ACCEPTED Content-Type: application/json ``` > **[`Quantity`](#quantity-type)** enqueued to settle > -> - This response does not guarantee that a settlement was executed. -> - If the settlement engine uses a unit less precise than the accounting system's unit, in order to prevent the two systems getting out-of-sync, it must indicate the amount it queued so the accounting system can correctly track the amount leftover. +> - This response indicates the given amount will eventually be settled, but does not guarantee that a settlement was executed. +> - The amount enqueued to settle MUST always be less than or equal to the quantity in the original request. +> - If the quantity enqueued to settle is less than the quantity of the original request, the accounting system MUST credit the leftover amount back to the accounts payable, or the same liability account tracking a balance owed to the peer. This is to prevent the systems getting out-of-sync if the settlement engine uses a unit less precise than the accounting system's unit. #### Handle incoming request @@ -259,6 +253,8 @@ Settlement engine implementations MAY expose additional, non-standard endpoints For example, settlement engines that use payment channels may expose functionality to manually open, fund, and close channels. +TODO Rename to "Settlement engine callback API" ? + ### Accounting System HTTP API #### Credit incoming settlement @@ -267,6 +263,8 @@ Notify the accounting system of receipt of an incoming settlement. Note that the The settlement engine MUST be the only entity invoking this message and MUST credit this amount to the accounts receivable, or some asset account tracking the balance owed by the peer, to ensure accurate accounting. +TODO "to properly reconcile the settlement" + ##### Request ```http @@ -286,7 +284,8 @@ Content-Type: application/json > **[`Quantity`](#quantity-type)** credited to the account > -> - If the accounting system uses a unit less precise than the settlement engine's unit in order to prevent the two systems getting out-of-sync, it must indicate the amount it acknowledged receipt of so the settlement engine can track the amount leftover. +> - The amount credited MUST always be less than or equal to the quantity in the original request. +> - If the quantity credited is less than the quantity of the original request, the settlement engine MUST track the leftover amount so it may accumulate and be added to subsequent notifications to prevent the two systems getting out-of-sync. ### Transport HTTP API @@ -333,21 +332,60 @@ Results are only saved if an API endpoint started executing. If incoming paramet All `POST` requests accept idempotency keys. Sending idempotency keys in `GET` and `DELETE` requests has no effect and should be avoided, as these requests are idempotent by definition. -## Usage in Interledger +## Example Implementations & Arrangements -Peers using the [Interledger protocol](https://github.com/interledger/rfcs/tree/master/0001-interledger-architecture) clear and fulfill conditional IOUs which affect their mutual accounting relationship. +### Simple value transfers -Since Interledger merely provides a credit network, a peer's liabilities may accrue to the point at which they must reconcile them. Settlement engines provide a mechanism for peers in Interledger to settle their liabilities and continue transacting. +Suppose two peers, Alice and Bob, are operating the same or compatible settlement engine implementations, which settle using a ledger which both Alice and Bob maintain accounts on. -An Interledger connector MAY operate the accounting system and transport system that interfaces with the settlement engine. +1. Alice's accounting system instructs her settlement engine to perform a settlement of 5 units to Bob. +2. Subsequently, Alice's settlement engine sends a value transfer of 5 units from Alice's ledger account to Bob's ledger account. + - To know where to send the payment, Alice may have configured her settlement engine with Bob's ledger identifier, or the settlement engine may automatically send a message to Bob's settlement engine to request Bob's ledger identifier. + - The settlement engine implementation MUST ensure that an incoming payment is only credited to the peer's account that sent it. +3. Bob's settlement engine, always watching the ledger, recognizes the payment, correlates that Alice sent it, and notifies its accounting system of the incoming settlement for 5 units. -### Motivation +### Payment channels + +Payment channels are special bilateral or multilateral settlement systems backed by escrows on an underlying shared ledger. They enable peers to adjust the allocation of these escrows among themselves. A participant can sign claims—messages with digital signatures entitling its peer to some greater portion of the escrow—which the peer can validate immediately without informing the underlying ledger. When the peers are done transacting, the most recent of many such claims is enforceable by the underlying ledger so the escrowed funds can be released and distributed accordingly. + +Payment channels offer very fast and cheap settlement, which is beneficial to peers which don't extend much credit to one another. + +Suppose two peers, Alice and Bob, are operating the same or compatible settlement engine implementations, which settle using a payment channel on a shared ledger. + +1. Alice instructs her settlement engine to create a payment channel to Bob and fund it for 20 units. + - Alice's settlement engine should escrow 20 units in the payment channel and inform Bob's settlement engine that a payment channel was created. + - Note, however, that the funds are still in Alice's custody, and are still only accessible by Alice. +2. Alice's accounting system instructs her settlement engine to settle 5 units to Bob. +3. Alice's settlement engine signs a claim entitling Bob to 5 units from Alice's escrowed funds, and sends it to Bob's settlement engine. + - If Bob already had some claim, the new claim should entitle Bob to 5 _additional_ units. + - If Alice has already entitled Bob to all the available escrowed funds, the settlement engine may track that it owes Bob 5 units without sending a claim, until there's sufficient capacity in the payment channel. +4. Bob's settlement engine, upon receiving the claim, should verify the message is valid and its digital signature is authentic, and determine how much additional funds it's entitled to (in this case, 5 units). Then, it should it notify the accounting system that it received the given amount, 5 units. +5. Alice and Bob may repeat this process many times, as long as the party sending the payment channel claim has funds that haven't already been allocated to the peer. If additional funds need to be escrowed, the settlement engine may offer this functionality through a custom instruction. +6. At a later time, Alice or Bob may instruct their settlement engines to close the payment channel and release the allotted funds back to their respective ledger accounts. + - Depending upon the payment channel implementation, there may be both cooperative and non-cooperative mechanisms for this, with varying time delays until the funds can be dispersed. + - Settlement engines using payment channels may also have functionality to watch the underlying ledger to ensure the peer doesn't try to submit an old message that would incorrectly distribute the escrowed funds. -The [Ledger Plugin Interface (LPIv2)](https://github.com/interledger/rfcs/blob/master/0024-ledger-plugin-interface-2/0024-ledger-plugin-interface-2.md) was originally designed as an abstraction for ledger integrations using Interledger. However, it introduced several problems: +### Recipient collects a fee -1. Multi-account plugins must include logic for handling ILP packets, increasing implementation complexity. -2. Plugins bundle settlement and bilateral communication functionality, limiting composability with other transports. -3. The JavaScript specific interface prevents connector implementations in other programming languages from using existing plugins. -4. Plugins operate in the same process as the connector, which limits scaling the two systems independently. +Suppose Alice and Bob's settlement arrangement is such that, before Bob can begin acknowledging incoming settlements from Alice, Bob must collect some initiation fee related to the settlement system. + +1. Alice sends a custom instruction to her settlement engine to settle the fee. Note that she does not use the instruction reserved for the accounting system. +2. Alice's settlement engine automatically negotiates the fee with Bob's settlement engine (likely subject to some limit). +3. Alice's settlement engine sends a payment on the shared ledger or system to Bob's settlement engine for the value of the fee. +4. Bob's settlement engine, instead of informing the accounting system of the incoming settlement, does nothing, but internally tracks that the negotiated fee was paid. +5. If Alice's accounting system triggers any subsequent settlements to Bob, Bob's settlement engine should acknowledge them to the accounting system as normal. -This specification aims to provide an abstraction that rectifies these issues. +### Sender collects a fee + +Suppose Alice and Bob's settlement arrangement is such that, in order for Alice to settle her liabilities to Bob, Alice requires that first Bob must pay a fee. + +1. The settlement engine implementations that Alice and Bob are operating might negotiate the fee that Alice will collect (suppose a fee of 3 units). + - The configuration of the settlement engines could ensure the fee is reasonable, or Bob could instruct his settlement engine of some limit for the fee. +2. After it's negotiated, Bob's settlement engine preemptively acknowledges it received an incoming settlement for 3 units. + - Note that even though no settlement on the shared ledger or system was received, Alice and Bob did mutually agree to reconcile a future liability: that is, the fee charged to Bob by Alice. +3. As Alice and Bob transact with one another, the liabilities that Alice owes Bob accrue, and Alice's accounting system instructs her settlement engine to settle them. +4. Alice's settlement engine, instead of sending a settlement on the shared settlement system, should track the amount owed to Bob, until it accrues to 3 units, the value of the agreed fee. +5. After the value of the fee is accounted for through accrued settlements, if Alice's accounting system instructs her settlement engine to settle, it should send these additional settlements on the shared system or ledger. + - For example, after negotiating the fee, suppose Alice's accounting system tells her settlement engine to settle 2 units. Nothing should happen, since it's accruing owed balance to cover the fee of 3 units. + - If the settlement engine is later instructed to settle another 2 units, it should send a payment on the shared settlement system for 1 unit, since the other unit was taken out as apart of the fee. + - Any further instructions to settle should trigger payments on the shared system as normal. From 42361713d12c00f52650f3f60c69ea76e967f5b9 Mon Sep 17 00:00:00 2001 From: Kincaid O'Neil Date: Fri, 2 Aug 2019 02:29:35 -0400 Subject: [PATCH 03/32] docs: flesh out bookkeeping and peer messaging - numerous verbiage changes for clarity - explain necessary guarantees for double-entry bookkeeping - more explicit spec for use with ILP-over-HTTP - rename transport/acc sys apis to "settlement engine callback api" - rename API endpoints to match consensus/implementations - remove confusing examples (more relevant for a tutorial) - remove old idempotency section, new version is WIP --- .../0000-settlement-engine.md | 307 +++++++++--------- 1 file changed, 161 insertions(+), 146 deletions(-) diff --git a/0000-settlement-engine/0000-settlement-engine.md b/0000-settlement-engine/0000-settlement-engine.md index 37973457..26f2e4b2 100644 --- a/0000-settlement-engine/0000-settlement-engine.md +++ b/0000-settlement-engine/0000-settlement-engine.md @@ -13,117 +13,183 @@ The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "S This specification codifies a common interface for a **settlement engine**. Settlement engines are services which perform two primary operations: -1. **Send payments**: execute outgoing settlements -2. **Credit payments**: acknowledge incoming settlements +1. Send payments: execute outgoing settlements +2. Receive payments: acknowledge incoming settlements -**Peers**, or entities that transact with one another, may operate compatible settlement engines to settle their liabilities. Different implementations may utilize different settlement systems or types of settlements, such as: +Counterparties may operate compatible settlement engines to settle their liabilities between one another. Different implementations may utilize different settlement systems or types of settlements, such as: -- **Ledger transfers:** adjusting balances on an underlying shared ledger -- **Physical settlement:** delivering an asset to the counterparty -- **Payment channels:** sending a claim entitling the peer to an escrow -- Using a custom protocol to mututally agree to discharge a liability +- Sending money or assets to the counterparty +- Transferring value on a shared ledger +- Signing and exchanging payment channel claims +- Performing a task for the counterparty with a mutually ascribed value ## Usage in Interledger -Connectors using the [Interledger protocol](https://github.com/interledger/rfcs/tree/master/0001-interledger-architecture) clear and fulfill ILP packets with peers, which represent conditional IOUs that affect their mutual accounting relationships. +### Interledger packet flow -Connectors may extend some particular peer a limited line of credit (or none, if they're untrusted). As a connector receives incoming ILP PREPARE packets from a peer, forwards them, and returns corresponding ILP FULFILL packets, the peer's liabilities accrue. If the peer's liabilities to the connector exceeds the credit limit assigned to it, the connector may reject and decline to forward additional packets. +In the [Interledger protocol](../0001-interledger-architecture/0001-interledger-architecture.md), a **peer** is a connector who is a counterparty to another connector. Connectors clear and fulfill ILP packets with their peers, which represent conditional IOUs that affect their mutual accounting relationships. -In order to continue transacting, the peer must settle their liabilities. In most cases, this is accomplished through sending a payment on a settlement system that both peers have in common. The connector should acknowledge the incoming payment—irrevocably discharging some liability the peer owed it—and enabling it to clear and forward subsequent packets on credit. +A connector may extend a given peer a limited line of credit, or none at all, depending upon their trustworthiness. As the connector receives incoming ILP PREPARE packets from a peer, forwards them, and returns corresponding ILP FULFILL packets, that peer's liabilities to the connector accrue. If the peer's liabilities exceed the credit limit assigned to it, the connector may reject and decline to forward additional packets. + +In order to continue transacting, the peer must settle their liabilities. In most cases, this is accomplished through sending a payment on a settlement system that both peers have agreed to use. The connector should acknowledge the incoming payment—irrevocably discharging some liability the peer owed to it—and enabling it to clear and forward subsequent packets from the peer on credit. Settlement engines provide a standardized mechanism for Interledger connectors to coordinate and reconcile these settlements. ### Accounting -An **account** represents a record of transactions between two peers, denominated in some fungible asset. Each account may be comprised of multiple **account balances**, which each represent the net difference between amounts received (credits) and amounts owed (debits) for some subset of the transactions. +An **account** represents a record of transactions between counterparties, denominated in some fungible asset. Each account may be comprised of multiple account balances, which each represent the net difference between amounts received (credits) and amounts owed (debits) for some subset of the transactions. Interledger connectors are RECOMMENDED to operate an **accounting system** which keeps a record of two balances for each account: -- **Accounts payable**, the amount owed by the operator to the counterparty for outgoing obligations its counterparty has fulfilled. - - Positive amount indicates the operator is indebted to its counterparty (a _liability_ for the operator). - - Negative amount indicates the operator has prefunded. -- **Accounts receivable**, the amount owed to the operator by its counterparty for incoming obligations the operator has fulfilled. - - Positive amount indicates its counterparty is indebted to the operator (an _asset_ to the operator). - - Negative amount indicates its counterparty has prefunded. +- **Accounts payable**, the amount owed by the connector to its peer for packets its peer has fulfilled. + - Positive amount indicates the connector is indebted to its peer (a _liability_ for the connector). + - Negative amount indicates the connector has prefunded. +- **Accounts receivable**, the amount owed to the connector by its peer for packets the connector has fulfilled. + - Positive amount indicates its peer is indebted to the connector (an _asset_ to the connector). + - Negative amount indicates its peer has prefunded. (Note: in context of "accounts payable" or "accounts receivable," "accounts" actually refers to a single account balance, not multiple accounts.) Thus, the connector's accounts payable with its peer should mirror its peer's accounts receivable with the connector, and respectively, the connector's accounts receivable should equal its peer's accounts payable. +### Settlement + +A **settlement** is the irrevocable discharge of a liability by providing something of value to the party to whom the liability is owed. + +Settlements may be performed by using a **settlement system**, or medium for exchanging value. Some settlements may transfer funds on a **ledger**, or registry of account balances and/or transactions, which is a type of settlement system. (Although not all settlement systems are ledgers, here, the terms are sometimes used interchangeably.) + +Settlement systems include, but are not limited to: + +- Cryptocurrencies/distributed ledgers +- Payment channel networks built on blockchains +- Bank clearing houses +- Credit card processors +- Money transfer services + +TODO Flesh out examples of settlements/settlement systmes + +TODO Explain that settlement is triggered by the accounting system + +TODO Should the expected netting behavior be included? + +### Double-entry bookkeeping + The interface in this specification provides a mechanism to reconcile settled cash balances, representing _value_, with the aforementioned account balances tracked by the accounting system, representing _credit_. Together, a settlement engine and an accounting system interface with one another to perform double-entry bookkeeping. -### Settlement +To ensure accurate double-entry bookkeeping, settlement engine and accounting system implementations MUST enforce several core guarantees. + +#### Accounting System Guarantees + +##### Account for outgoing settlements + +If the accounting system opts to trigger a settlement: + +1. The accounting system MUST debit the accounts payable, subtracing the amount of the settlement. +2. Then, the accounting system MUST send a request to the settlement engine to settle the amount. + +If the settlement engine responds that it only queued a partial amount for settlement (due to lesser precision), the accounting system MUST credit back the accounts payable, adding the leftover amount. -A **settlement** is the full and irrevocable discharge of an obligation. Settlements may be performed by using a **settlement system**, or medium for exchanging value. +If request retries fail per the [idempotency rules](#idempotency), the accounting system MUST credit back the accounts payable, adding the amount of the failed settlement. -A **ledger** is a registry of account balances and/or transactions. (Although not all settlement systems are ledgers, here, the terms are sometimes used interchangeably.) +##### Account for incoming settlements -TODO explain the below better +If the settlement engine instructs the accounting system a settlement was received, the accounting system MUST credit the accounts receivable, subtracting the amount of the settlement. -To ensure correct double-entry bookkeeping, several guarantees must be upheld: -- If the accounting system instructs the settlement engine to settle some amount: - 1. The accounting system MUST subtract the amount from the accounts payable. - 2. The settlement engine MUST eventually settle the amount. -- If the settlement engine receives an incoming settlement: - 1. The settlement engine MUST acknowledge the settlement to the accounting system. - 2. The accounting system MUST add the amount to the accounts receivable. +#### Settlement Engine Guarantees + +##### Retry crediting incoming settlements + +After the settlement engine requests the accounting system to credit an incoming settlement, if the accounting system responds that it only credited a partial amount (due to lesser precision), the settlement engine MUST track the leftover amount to retry later. + +If the request fails after several attempts per the [idempotency rules](#idempotency), the settlement engine MUST track the failed amount to retry later. + +TODO clarify "retry later" means "when subsequent settlements are received, prompting another notification" + +##### Settlement symmetry + +Given two non-malicious peers transacting under normal conditions, compatible settlement engine implementations MUST ensure outgoing settlements from one peer are eventually acknowledged as incoming settlements by the other. + +More formally, the sum of amounts one settlement engine instance is instructed to settle MUST eventually equal the sum of amounts the peer's settlement engine instance instructs its accounting system to credit as incoming settlements. + +Varying external conditions, such as connectivity between peers or settlement system latency, MAY delay or prevent consistency of these instructed and acknowledged settlements. + +#### Trade-offs of aynchronous settlement + +TODO Explain the rationale for the async balance model/why it's okay if the settlement engine fails ### Bilateral communication -Interledger connectors use a transport, such as HTTP or WebSockets, to send and receive ILP packets with peers. +In order to settle or receive settlements with a peer, a settlement engine may first need to retrieve or communicate information with the peer's settlement engine. Two peered settlement engine instances may send and receive settlement-related messages among themselves, such as idenifiers for their ledger accounts, payment channel claims, or to negotiate fees. -To send messages or requests to the peer's settlement engine instance, settlement engine implementations SHOULD proxy all requests through a service operated by the connector. The connector will forward messages to the peer's connector using the shared transport, which will then send them to its settlement engine for processing. The peer settlement engine's response to the request will then be sent back, across the two connectors, to the origin settlement engine. +Interledger connectors use a transport, such as HTTP or WebSockets, to send and receive data with peers. Settlement engine implementations SHOULD proxy all messages through services operated by its Interledger connector leveraging its existing transport to forward settlement messages to its peer's connector and peer's settlement engine. -#### Usage with [ILP-over-HTTP](https://github.com/interledger/rfcs/blob/master/0035-ilp-over-http/0035-ilp-over-http.md) +#### The Flow -To send the message to the peer, the raw message should be encoded in the `data` field of an ILP packet and addressed to `peer.settle`. The connector that receives the packet should identify the peer and send it to the settlement engine instanced associated with that account. +1. Origin settlement engine sends a callback request to its connector with the settlement-related message to forward. +2. Origin connector forwards the message to the peer's connector using its existing transport. +3. Peer connector receives the message, identifies which settlement engine instance the account is associated with, and sends a request to its settlement engine to handle the message. +4. Peer settlement engine processes the message and responds with its own message. +5. Peer connector sends the response message back across the transport channel to the origin connector. +6. Origin connector sends the response message back to the origin settlement engine. -ILP packets intended for the peer to proxy to its settlement engine should be addressed to `peer.settle` and MUST NOT be forwarded to any subsequent connectors. +#### Usage with [ILP-over-HTTP](../0035-ilp-over-http/0035-ilp-over-http.md) -Connectors MUST track which settlement engine a peer's account is associated with and ensure their messages are delivered to that particular settlement engine. +When a connector receives a request to send a message to the peer, the raw message from the settlement engine MUST be encoded within an ILP PREPARE packet as described below. Then, the ILP PREPARE should be sent to the peer's connector associated with the account using ILP-over-HTTP. -### Motivation +The peer's connector MUST extract the message data and forward it to its settlement engine for processing, and MUST NOT forward the ILP packet to any other connectors. When the settlement engine responds with a response message, the connector MUST encode the raw message from the settlement engine within an ILP FULFILL or ILP REJECT, depending upon the code of the response. If the connector was unable to process the request, it MUST respond with an ILP REJECT. -Settlement engines supercede the [Ledger Plugin Interface (LPIv2)](https://github.com/interledger/rfcs/blob/master/0024-ledger-plugin-interface-2/0024-ledger-plugin-interface-2.md), an earlier abstraction for settlement integrations with Interledger. The new model addresses these issues: +##### PREPARE -1. Multi-account plugins required logic for handling ILP packets, increasing implementation complexity. -2. Plugins bundled settlement and bilateral communication functionality, limiting composability with other transports. -3. The JavaScript specific interface prevented connector implementations in other programming languages from using existing plugins. -4. Plugins operated in the same process as the connector, which limited scaling the two systems independently. +- `amount`: `0` +- `expiresAt`: _Determined by connector_ +- `executionCondition`: `e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855` +- `destination`: `peer.settle` +- `data`: _Request message from sender settlement engine_ -## Specification +##### FULFILL -### Core Functionality +- `fulfillment`: `0000000000000000000000000000000000000000000000000000000000000000` +- `data`: _Response message from recipient settlement engine_ -All settlement engine implementations MUST functionally guarantee that **the sum of amounts a settlement engine instance is instructed to settle should eventually equal the sum of amounts the peer's settlement engine instance acknowledges receipt of.** +##### REJECT -Varying implementations, arrangements, and conditions (such as connectivity to the settlement system) may delay consistency of instructed and acknowledged settlements by an indeterminate amount of time. +- `code`: _Determined by connector from HTTP status of forwarded request_ +- `triggeredBy`: `peer.settle` +- `message`: _Determined by connector_ +- `data`: _Response message from recipient settlement engine, or empty if antecedent failure_ -### Accounts and identifiers +### Motivation + +Settlement engines supercede the [Ledger Plugin Interface (LPIv2)](../deprecated/0024-ledger-plugin-interface-2/0024-ledger-plugin-interface-2.md), an earlier abstraction for settlement integrations with Interledger. This new model addresses these issues: -Each account MUST be identified by a unique, URL-safe string. +1. Multi-account plugins required logic for handling ILP packets, increasing implementation complexity. +2. Plugins bundled settlement and bilateral communication functionality together, limiting composability. +3. JavaScript plugins limited interoperability with non-JavaScript connector implementations. +4. Plugins operated in the same process as the connector, which limited scaling the two services independently. -#### Accounts vs ledger identities +## Specification -The settlement engine MUST be responsible for correlating an account identifier to the peer's identity on the shared ledger or settlement system, if required. +### Accounts and identifiers -For separation of concerns between the accounting system and the settlement system, the accounting system is NOT RECOMMENDED to have knowledge of the peer's identity on the shared settlement system, but rather only the account identifier described here. +Each account MUST be identified by a unique, [URL-safe](https://tools.ietf.org/html/rfc3986#section-2.3) string. + +The settlement engine MUST be responsible for correlating an account identifier to the peer's identity on the shared ledger or settlement system, if required. For separation of concerns between clearing and settlement, the accounting system is NOT RECOMMENDED to have knowledge of the peer's identity on the shared settlement system. ### Units and quantities -A **monetary unit** is the standard unit of value of a particular currency or asset, such \$1 in the case of USD, or 1 BTC in the case of Bitcoin. +An amount of a given asset or currency may be represented using different denominations or units of the asset. For example, one U.S. dollar may be represented as \$1 or 100 cents, and one Bitcoin may be represented as 1 BTC or 100000000 satoshis. -A **fractional monetary unit** is some unit smaller than the standard monetary unit with greater precision: for example, one cent (\$0.01) in the case of USD, or 1 satoshi (0.00000001 BTC) in the case of Bitcoin. +A **monetary unit** is the standard unit of value of a particular currency or asset, such as \$1 in the case of U.S. dollars, or 1 BTC in the case of Bitcoin. -An **asset scale** for some fractional monetary unit is the difference in orders of magnitude between that unit and the standard monetary unit. More formally, a non-negative integer (0, 1, 2, …) such that one monetary unit equals `10^(-scale)` of the given fractional monetary unit, or 0 if the unit is the standard monetary unit. +A **fractional monetary unit** is some unit smaller than the standard monetary unit with greater precision. Examples of fractional monetary units include one cent (\$0.01) in the case of USD, or 1 satoshi (0.00000001 BTC) in the case of Bitcoin. -For example, one cent represents an asset scale of 2 in the case of USD, whereas 1 satoshi represents an asset scale of 8 in the case of Bitcoin. +An **asset scale** for some fractional monetary unit is the difference in orders of magnitude between that unit and the standard monetary unit. More formally, a non-negative integer (0, 1, 2, …) such that one monetary unit equals `10^(-scale)` of the given fractional monetary unit, or 0 if the unit is the standard monetary unit. For example, one cent represents an asset scale of 2 in the case of USD, whereas 1 satoshi represents an asset scale of 8 in the case of Bitcoin. #### `Quantity` JSON type -A quantity represents an amount denominated in some unit of a particular fungible asset. In uses of this type, the type of asset should be implicit. +A quantity represents an amount denominated in some unit of a particular fungible asset. (Since account balances may only be denominated in a single asset, the type of asset should be implicit). ##### Attributes @@ -144,34 +210,41 @@ To represent $2.54 in units of cents, where the amount is a multiple of $0.01: } ``` -#### Choosing scales - -The native unit or scale of the settlement engine is RECOMMENDED to be the same unit its settlements are denominated in, which is typically the smallest denomination of the asset on the settlement system. - -The accounting system and settlement engine are RECOMMENDED to use the same unit or scale to minimize conversion inconsistencies. - #### Scale conversions If the accounting system or settlement engine receives a request with a **[`Quantity`](#quantity-type)** denominated in a unit more precise than its unit, it MUST convert the quantity into its native unit. If so, the resulting amount MUST be rounded down before fulfilling the request. The response to the request MUST include the converted, rounded **[`Quantity`](#quantity-type)** used to fulfill the request, which MUST be less than or equal to the amount sent in the original request. -Then, the system with the additional precision initiating the request MUST track the sum leftover by subtracting the two the other so it may accumulate and be added to the amount in subsequent requests. +Then, the system with the additional precision initiating the request MUST track the leftover sum so it may accumulate and be retried in subsequent requests. + +#### Choosing scales + +All quantities the settlement engine includes in its responses or callback requests MUST use the same unit its settlements are denominated in, which is typically the smallest denomination of the asset on the settlement system. + +The accounting system is RECOMMENDED to use the same unit or scale as the settlement engine to minimize conversion inconsistencies. ### Settlement Engine HTTP API -#### Initiate an account +Settlement engines MUST implement these endpoints. -Inform the settlement engine that a new accounting relationship was instantiated. The settlement engine MAY perform tasks as prerequisite to settle the new account. For example, a settlement engine implementation might send messages to the peer to exchange ledger identifiers or to negotiate terms of the settlement arrangement. +#### Initiate an account -The transport MUST be the only entity invoking this instruction to ensure the account identifier can be correlated with the correct peer to send and receive messages. Note that the transport also MUST have a mechanism to notify the accounting system that a new account should be instantiated (out-of-scope of this specification). +Inform the settlement engine that a new accounting relationship was instantiated. The settlement engine MAY perform tasks as a prerequisite to settle the new account. For example, a settlement engine implementation might send messages to the peer to exchange ledger identifiers or to negotiate terms of the settlement arrangement. ##### Request ```http -POST /accounts/:id HTTP/1.1 +POST /accounts HTTP/1.1 +Content-Type: application/json ``` +> ```json +> { +> id: +> } +> ``` + ##### Response ```http @@ -180,7 +253,7 @@ HTTP/1.1 201 CREATED #### Perform outgoing settlement -Reconcile a liability owed to the peer for the given amount. +Settle a liability owed to the peer for the given amount. Note that the settlement engine MAY accrue owed settlements without settling immediately due to varying arrangements, implementations, or conditions. The settlement engine SHOULD persist amounts owed to the peer that have yet to be settled so they can be settled later. @@ -189,7 +262,7 @@ The accounting system MUST be the only entity invoking this instruction and MUST ##### Request ```http -POST /accounts/:id/settle HTTP/1.1 +POST /accounts/:id/settlements HTTP/1.1 Accept: application/json Content-Type: application/json ``` @@ -218,7 +291,7 @@ Respond to an incoming message from the given peer's settlement engine. The tran ##### Request ```http -POST /accounts/:id/handleMessage HTTP/1.1 +POST /accounts/:id/messages HTTP/1.1 Accept: application/octet-stream Content-Type: application/octet-stream ``` @@ -253,22 +326,18 @@ Settlement engine implementations MAY expose additional, non-standard endpoints For example, settlement engines that use payment channels may expose functionality to manually open, fund, and close channels. -TODO Rename to "Settlement engine callback API" ? - -### Accounting System HTTP API +### Settlement Engine Callback HTTP API -#### Credit incoming settlement +Settlement engine implementations MUST implement the following callbacks. -Notify the accounting system of receipt of an incoming settlement. Note that the settlement engine MAY accrue incoming settlement acknowledgements without immediately informing the accounting system. +#### Receipt of incoming settlement -The settlement engine MUST be the only entity invoking this message and MUST credit this amount to the accounts receivable, or some asset account tracking the balance owed by the peer, to ensure accurate accounting. - -TODO "to properly reconcile the settlement" +Settlement engine implementations SHOULD send a callback request to this endpoint when incoming settlements are received. The endpoint handling the call, such as an accounting system, SHOULD credit the amount of the settlement to an account balance to ensure accurate double-entry bookkeeping. ##### Request ```http -POST /accounts/:id/receipt HTTP/1.1 +POST /accounts/:id/settlements HTTP/1.1 Accept: application/json Content-Type: application/json ``` @@ -278,7 +347,7 @@ Content-Type: application/json ##### Response ```http -HTTP/1.1 200 OK +HTTP/1.1 201 CREATED Content-Type: application/json ``` @@ -287,16 +356,14 @@ Content-Type: application/json > - The amount credited MUST always be less than or equal to the quantity in the original request. > - If the quantity credited is less than the quantity of the original request, the settlement engine MUST track the leftover amount so it may accumulate and be added to subsequent notifications to prevent the two systems getting out-of-sync. -### Transport HTTP API - #### Send outgoing request -Send a message to given peer's settlement engine and return its response. The settlement engine MUST be the only entity invoking this instruction. +Send a message to given peer's settlement engine and return its response. ##### Request ```http -POST /accounts/:id/sendMessage HTTP/1.1 +POST /accounts/:id/messages HTTP/1.1 Accept: application/octet-stream Content-Type: application/octet-stream ``` @@ -314,78 +381,26 @@ Content-Type: application/octet-stream ### Idempotency -_(The following section was adapted from the [Stripe API documentation](https://stripe.com/docs/api/idempotent_requests).)_ - -Each API MUST support [idempotency](https://en.wikipedia.org/wiki/Idempotence) for safely retrying requests without accidentally performing the same operation twice. This is useful when an API call is disrupted in transit and you do not receive a response. For example, if a request to send a settlement does not respond due to a network connection error, you can retry the request with the same idempotency key to guarantee that no more than one charge is created. - -To perform an idempotent request, provide an additional `Idempotency-Key: ` header to the request. - -Idempotency works by saving the resulting status code and body of the first request made for any given idempotency key, regardless of whether it succeeded or failed. - -Subsequent requests with the same key return the same result, including `500` errors. - -An idempotency key is a unique value generated by the client which the server uses to recognize subsequent retries of the same request. How you create unique keys is up to you, but we suggest using V4 UUIDs, or another random string with enough entropy to avoid collisions. - -Keys expire after 24 hours, so a new request is generated if a key is reused outside of that time frame. The idempotency layer compares incoming parameters to those of the original request and errors unless they're the same to prevent accidental misuse. - -Results are only saved if an API endpoint started executing. If incoming parameters failed validation, or the request conflicted with another that was executing concurrently, no idempotent result is saved because no API endpoint began execution. It is safe to retry these requests. - -All `POST` requests accept idempotency keys. Sending idempotency keys in `GET` and `DELETE` requests has no effect and should be avoided, as these requests are idempotent by definition. - -## Example Implementations & Arrangements - -### Simple value transfers - -Suppose two peers, Alice and Bob, are operating the same or compatible settlement engine implementations, which settle using a ledger which both Alice and Bob maintain accounts on. - -1. Alice's accounting system instructs her settlement engine to perform a settlement of 5 units to Bob. -2. Subsequently, Alice's settlement engine sends a value transfer of 5 units from Alice's ledger account to Bob's ledger account. - - To know where to send the payment, Alice may have configured her settlement engine with Bob's ledger identifier, or the settlement engine may automatically send a message to Bob's settlement engine to request Bob's ledger identifier. - - The settlement engine implementation MUST ensure that an incoming payment is only credited to the peer's account that sent it. -3. Bob's settlement engine, always watching the ledger, recognizes the payment, correlates that Alice sent it, and notifies its accounting system of the incoming settlement for 5 units. +TODO Good reference: https://stripe.com/blog/idempotency -### Payment channels +TODO "fault tolerant" -Payment channels are special bilateral or multilateral settlement systems backed by escrows on an underlying shared ledger. They enable peers to adjust the allocation of these escrows among themselves. A participant can sign claims—messages with digital signatures entitling its peer to some greater portion of the escrow—which the peer can validate immediately without informing the underlying ledger. When the peers are done transacting, the most recent of many such claims is enforceable by the underlying ledger so the escrowed funds can be released and distributed accordingly. +Implementations MUST support [idempotency](https://en.wikipedia.org/wiki/Idempotence) for safely retrying requests without accidentally performing the same operation multiple times. This is useful when an API call is disrupted in transit and you do not receive a response. For example, if a request to send a settlement does not respond due to a network connection error, you can retry the request with the same idempotency key to guarantee that no more than one settlement is performed. -Payment channels offer very fast and cheap settlement, which is beneficial to peers which don't extend much credit to one another. +#### Performing idempotent requests -Suppose two peers, Alice and Bob, are operating the same or compatible settlement engine implementations, which settle using a payment channel on a shared ledger. +All `POST` requests MUST supply idempotency keys by providing an additional `Idempotency-Key: ` header with the request. -1. Alice instructs her settlement engine to create a payment channel to Bob and fund it for 20 units. - - Alice's settlement engine should escrow 20 units in the payment channel and inform Bob's settlement engine that a payment channel was created. - - Note, however, that the funds are still in Alice's custody, and are still only accessible by Alice. -2. Alice's accounting system instructs her settlement engine to settle 5 units to Bob. -3. Alice's settlement engine signs a claim entitling Bob to 5 units from Alice's escrowed funds, and sends it to Bob's settlement engine. - - If Bob already had some claim, the new claim should entitle Bob to 5 _additional_ units. - - If Alice has already entitled Bob to all the available escrowed funds, the settlement engine may track that it owes Bob 5 units without sending a claim, until there's sufficient capacity in the payment channel. -4. Bob's settlement engine, upon receiving the claim, should verify the message is valid and its digital signature is authentic, and determine how much additional funds it's entitled to (in this case, 5 units). Then, it should it notify the accounting system that it received the given amount, 5 units. -5. Alice and Bob may repeat this process many times, as long as the party sending the payment channel claim has funds that haven't already been allocated to the peer. If additional funds need to be escrowed, the settlement engine may offer this functionality through a custom instruction. -6. At a later time, Alice or Bob may instruct their settlement engines to close the payment channel and release the allotted funds back to their respective ledger accounts. - - Depending upon the payment channel implementation, there may be both cooperative and non-cooperative mechanisms for this, with varying time delays until the funds can be dispersed. - - Settlement engines using payment channels may also have functionality to watch the underlying ledger to ensure the peer doesn't try to submit an old message that would incorrectly distribute the escrowed funds. +For each unique request, such as an individual settlement, a unique key should be generated. The key MUST generated with a cryptographically secure pseudorandom number generator to avoid collisions. Implementations are RECOMMENDED to use a v4 UUID. -### Recipient collects a fee +TODO -Suppose Alice and Bob's settlement arrangement is such that, before Bob can begin acknowledging incoming settlements from Alice, Bob must collect some initiation fee related to the settlement system. +#### Recommended retry behavior -1. Alice sends a custom instruction to her settlement engine to settle the fee. Note that she does not use the instruction reserved for the accounting system. -2. Alice's settlement engine automatically negotiates the fee with Bob's settlement engine (likely subject to some limit). -3. Alice's settlement engine sends a payment on the shared ledger or system to Bob's settlement engine for the value of the fee. -4. Bob's settlement engine, instead of informing the accounting system of the incoming settlement, does nothing, but internally tracks that the negotiated fee was paid. -5. If Alice's accounting system triggers any subsequent settlements to Bob, Bob's settlement engine should acknowledge them to the accounting system as normal. +TODO -### Sender collects a fee +#### Handling idempotent requests -Suppose Alice and Bob's settlement arrangement is such that, in order for Alice to settle her liabilities to Bob, Alice requires that first Bob must pay a fee. +All `POST` endpoints MUST support idempotency keys. -1. The settlement engine implementations that Alice and Bob are operating might negotiate the fee that Alice will collect (suppose a fee of 3 units). - - The configuration of the settlement engines could ensure the fee is reasonable, or Bob could instruct his settlement engine of some limit for the fee. -2. After it's negotiated, Bob's settlement engine preemptively acknowledges it received an incoming settlement for 3 units. - - Note that even though no settlement on the shared ledger or system was received, Alice and Bob did mutually agree to reconcile a future liability: that is, the fee charged to Bob by Alice. -3. As Alice and Bob transact with one another, the liabilities that Alice owes Bob accrue, and Alice's accounting system instructs her settlement engine to settle them. -4. Alice's settlement engine, instead of sending a settlement on the shared settlement system, should track the amount owed to Bob, until it accrues to 3 units, the value of the agreed fee. -5. After the value of the fee is accounted for through accrued settlements, if Alice's accounting system instructs her settlement engine to settle, it should send these additional settlements on the shared system or ledger. - - For example, after negotiating the fee, suppose Alice's accounting system tells her settlement engine to settle 2 units. Nothing should happen, since it's accruing owed balance to cover the fee of 3 units. - - If the settlement engine is later instructed to settle another 2 units, it should send a payment on the shared settlement system for 1 unit, since the other unit was taken out as apart of the fee. - - Any further instructions to settle should trigger payments on the shared system as normal. +TODO From 8ecad82f091e480e6ae130b47702ffc089569e9d Mon Sep 17 00:00:00 2001 From: Kincaid O'Neil Date: Fri, 2 Aug 2019 11:41:31 -0400 Subject: [PATCH 04/32] docs: highlight settlement engine spec, hide LPIv2 --- .../0001-interledger-architecture.md | 14 ++++-------- README.md | 22 +++++++------------ .../0024-ledger-plugin-interface-2.md | 0 3 files changed, 12 insertions(+), 24 deletions(-) rename {0024-ledger-plugin-interface-2 => deprecated/0024-ledger-plugin-interface-2}/0024-ledger-plugin-interface-2.md (100%) diff --git a/0001-interledger-architecture/0001-interledger-architecture.md b/0001-interledger-architecture/0001-interledger-architecture.md index dc2c018e..ee9a5458 100644 --- a/0001-interledger-architecture/0001-interledger-architecture.md +++ b/0001-interledger-architecture/0001-interledger-architecture.md @@ -2,6 +2,7 @@ title: Interledger Architecture draft: 6 --- + # Interledger Architecture Interledger provides for secure payments across multiple assets on different ledgers. The architecture consists of a conceptual model for interledger payments, a mechanism for securing payments, and a suite of protocols that implement this design. @@ -58,8 +59,7 @@ If settlement of one account in the Interledger is contingent on the status of a Nodes can also choose never to settle their obligations. This configuration may be useful when several nodes representing different pieces of software or devices are all owned by the same person or business, and all their traffic with the outside world goes through a single "home router" connector. This is the model of [moneyd](https://github.com/interledgerjs/moneyd), one of the current implementations of Interledger. -Most implementations of Interledger use a plugin architecture to settle obligations automatically while abstracting the differences between different ledger layer protocols. For an example of this, see [IL-RFC-24: JavaScript Ledger Plugin Interface version 2](../0024-ledger-plugin-interface-2/0024-ledger-plugin-interface-2.md). - +Most implementations of Interledger use [settlement engines](../0000-settlement-engine/0000-settlement-engine.md) to settle obligations automatically while abstracting the differences between different ledger layer protocols. #### Link Protocols @@ -72,7 +72,6 @@ Peers in the Interledger Protocol require a way to communicate securely with one The implementation of a Link protocol may be incorporated into a ledger plugin, since the Link protocol has to communicate settlements that occur in the underlying ledger. - #### Interledger Protocol The Interledger Protocol version 4 (ILPv4) is the core protocol of the entire Interledger Protocol suite. This protocol's packets pass through all participants in the chain: from the sender, through one or more connectors, to the receiver. This protocol is compatible with any variety of currencies and underlying ledger systems. @@ -81,7 +80,6 @@ This level is concerned with currency amounts, routing, and whether each step in This layer abstracts the layers above and below it from one another, so there can be only one protocol at this layer. Other protocols, including older versions of the Interledger Protocol, are incompatible. The current protocol is defined by [IL-RFC-27: Interledger Protocol version 4](../0027-interledger-protocol-4/0027-interledger-protocol-4.md). - #### Transport Protocols Transport layer protocols are used for **end-to-end communication** between sender and receiver; connectors are not expected to be involved. This layer is responsible for: @@ -94,7 +92,6 @@ Transport layer protocols are used for **end-to-end communication** between send For an example, see the STREAM protocol, defined by [IL-RFC-29: STREAM](../0029-stream/0029-stream.md). STREAM creates a bidirectional connection between a sender and receiver that consists of many individual Interledger packets. - #### Application Protocols Protocols at the application level communicate details outside of the minimum information that is technically necessary to complete a payment. For example, application protocols may check that participants are interested in conducting a transaction and are legally allowed to do so. @@ -110,7 +107,6 @@ An example of an application layer protocol is the [Simple Payment Setup Protoco Many different messaging protocols can be defined on and above the Application level of the Interledger clearing system. - #### Comparison to Traditional Financial Infrastructure The layers of Interledger are similar to the different layers of traditional inter-bank systems: @@ -120,7 +116,6 @@ The layers of Interledger are similar to the different layers of traditional int - Interledger's _Link protocols_ don't have a direct banking equivalent, but they provide authenticated messaging to enable the Interledger Protocol layer, and they also associate settlement events in the underlying ledgers to balances in the Interledger Protocol layer. - The underlying _Ledger_ systems are equivalent of _settlement_ in banking terms. - ### Interledger Protocol Flow Interledger moves money by relaying _packets_. In the Interledger Protocol, a "prepare" packet represents a possible movement of some money and comes with a condition for releasing it. As the packet moves forward through the chain of connectors, the sender and connectors prepare balance changes for the accounts between them. The connectors also adjust the amount for any currency conversions and fees subtracted. @@ -145,7 +140,6 @@ A packet does not have to represent the full amount of a real-world payment. _Tr The Interledger Protocol does not have a specific definition of "small", nor a size limit on packets. Each connector can choose minimum and maximum packet sizes they are willing to relay; as a result, any path's maximum packet size is the smallest maximum packet size among the connectors in that path. To be compatible with as much of the network as possible, one should choose packet sizes that fit between the minimum and maximum values of as many connectors as possible. - ### Addresses _Interledger addresses_ (also called _ILP addresses_) provide a universal way to address senders, receivers and connectors. These addresses are used in several different protocol layers, but their most important feature is to enable routing on the Interleder Protocol layer. Interledger addresses are hierarchical, dot-separated strings where the left-most segment is most significant. An example address might look like: @@ -157,9 +151,9 @@ If two parties in the Interledger have a "parent/child" connection rather than a ## Interledger Security -**Interledger uses *conditional transfers* to secure payments across multiple hops and even through untrusted connectors.** Everyone only needs to trust their direct peers, no matter how many connectors are involved in forwarding a given packet. Connectors take some risk, but this risk can be managed and is primarily based upon the connector's chosen peers. +**Interledger uses _conditional transfers_ to secure payments across multiple hops and even through untrusted connectors.** Everyone only needs to trust their direct peers, no matter how many connectors are involved in forwarding a given packet. Connectors take some risk, but this risk can be managed and is primarily based upon the connector's chosen peers. -> **Hint:** Conditional transfers or *authorization holds* are the financial equivalent of a [two-phase commit](http://foldoc.org/two-phase%20commit). +> **Hint:** Conditional transfers or _authorization holds_ are the financial equivalent of a [two-phase commit](http://foldoc.org/two-phase%20commit). Because each party is isolated from risks beyond their immediate peers, longer paths are not inherently more risky than shorter paths. This enables longer paths to compete with shorter paths to convey money from any given sender to any given receiver, while reducing the risk to the sender. diff --git a/README.md b/README.md index 73a0f61e..681a4ff4 100644 --- a/README.md +++ b/README.md @@ -23,42 +23,36 @@ The default encoding rules for Interledger protocols are the Canonical Octet Enc ## Interledger Overview and Explanatory Docs -* **[1: Interledger Architecture](0001-interledger-architecture/0001-interledger-architecture.md)** +- **[1: Interledger Architecture](0001-interledger-architecture/0001-interledger-architecture.md)** Overview of the Interledger architecture. -* **[19: Glossary](./0019-glossary/0019-glossary.md)** +- **[19: Glossary](./0019-glossary/0019-glossary.md)** Definitions of Interledger terminology. ## Core Interledger Protocol Specs - -* **[27: Interledger Protocol V4 (ILPv4)](0027-interledger-protocol-4/0027-interledger-protocol-4.md)** +- **[27: Interledger Protocol V4 (ILPv4)](0027-interledger-protocol-4/0027-interledger-protocol-4.md)** Specifies the Interledger Protocol and Interledger Packet, which are used for sending payment instructions across different ledgers and connectors. This is the core protocol in the Interledger stack. -* **[15: ILP Addresses](0015-ilp-addresses/0015-ilp-addresses.md)** +- **[15: ILP Addresses](0015-ilp-addresses/0015-ilp-addresses.md)** Specifies the Interledger Address format for ledgers and accounts. ## Protocols Built Upon ILP -* **[9: Simple Payment Setup Protocol (SPSP)](0009-simple-payment-setup-protocol/0009-simple-payment-setup-protocol.md)** +- **[9: Simple Payment Setup Protocol (SPSP)](0009-simple-payment-setup-protocol/0009-simple-payment-setup-protocol.md)** A basic Application Layer protocol that uses HTTPS to exchange details needed to set up an Interledger payment. -* **[29: STREAM](0029-stream/0029-stream.md)** +- **[29: STREAM](0029-stream/0029-stream.md)** The recommended Transport Layer protocol for most use cases, which handles quoting, individual payments, chunked payments, and streaming payments using a shared secret between the sender and receiver. ## Ledger Layer -* **[24: Ledger Plugin Interface V2](0024-ledger-plugin-interface-2/0024-ledger-plugin-interface-2.md)** - - Ledger abstraction used in the JavaScript implementation. This can be used as a model for defining such plugins in other languages. - -* **[23: Bilateral Transfer Protocol (BTP)](0023-bilateral-transfer-protocol/0023-bilateral-transfer-protocol.md)** - - Recommended API for trustlines and payment channels. +- **[00: Settlement Engines](0000-settlement-engine/0000-settlement-engine.md)** + Specifies an interface to send and receive payments across different settlement systems and ledgers. diff --git a/0024-ledger-plugin-interface-2/0024-ledger-plugin-interface-2.md b/deprecated/0024-ledger-plugin-interface-2/0024-ledger-plugin-interface-2.md similarity index 100% rename from 0024-ledger-plugin-interface-2/0024-ledger-plugin-interface-2.md rename to deprecated/0024-ledger-plugin-interface-2/0024-ledger-plugin-interface-2.md From 536932dcd0b984ef7c38390de0437e5664f16c24 Mon Sep 17 00:00:00 2001 From: Kincaid O'Neil Date: Mon, 5 Aug 2019 15:34:32 -0400 Subject: [PATCH 05/32] fix: more general asset/unit definitions Co-Authored-By: David Fuelling --- 0000-settlement-engine/0000-settlement-engine.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/0000-settlement-engine/0000-settlement-engine.md b/0000-settlement-engine/0000-settlement-engine.md index 26f2e4b2..ee7da39d 100644 --- a/0000-settlement-engine/0000-settlement-engine.md +++ b/0000-settlement-engine/0000-settlement-engine.md @@ -181,11 +181,13 @@ The settlement engine MUST be responsible for correlating an account identifier An amount of a given asset or currency may be represented using different denominations or units of the asset. For example, one U.S. dollar may be represented as \$1 or 100 cents, and one Bitcoin may be represented as 1 BTC or 100000000 satoshis. -A **monetary unit** is the standard unit of value of a particular currency or asset, such as \$1 in the case of U.S. dollars, or 1 BTC in the case of Bitcoin. +A **standard unit** is the typical unit of value for a particular asset, such as \$1 in the case of U.S. dollars, or 1 BTC in the case of Bitcoin. -A **fractional monetary unit** is some unit smaller than the standard monetary unit with greater precision. Examples of fractional monetary units include one cent (\$0.01) in the case of USD, or 1 satoshi (0.00000001 BTC) in the case of Bitcoin. +A **fractional unit** represents some unit smaller than the standard unit, but with greater precision. Examples of fractional monetary units include one cent (\$0.01 USD), or 1 satoshi (0.00000001 BTC). -An **asset scale** for some fractional monetary unit is the difference in orders of magnitude between that unit and the standard monetary unit. More formally, a non-negative integer (0, 1, 2, …) such that one monetary unit equals `10^(-scale)` of the given fractional monetary unit, or 0 if the unit is the standard monetary unit. For example, one cent represents an asset scale of 2 in the case of USD, whereas 1 satoshi represents an asset scale of 8 in the case of Bitcoin. +An **asset scale** is the difference in orders of magnitude between a **standard unit** and a corresponding **fractional unit**. More formally, the asset scale is a non-negative integer (0, 1, 2, …) such that one **standard unit** equals `10^(-scale)` of a corresponding **fractional unit**. If the fractional unit equals the standard unit, then the asset scale is 0. + +For example, one cent represents an asset scale of 2 in the case of USD, whereas 1 satoshi represents an asset scale of 8 in the case of Bitcoin. #### `Quantity` JSON type From 2da9cbd513cb7dfac5f61b15ae23451cd42fd4bf Mon Sep 17 00:00:00 2001 From: Kincaid O'Neil Date: Tue, 6 Aug 2019 10:31:08 -0400 Subject: [PATCH 06/32] fix: misc edits, async settlement design - fix: miscellaneous verbiage edits for clarity - add section explaining rationale for async settlement design - simpler expalantion for exchanging messages between SEs - fix: no u64 limit to quantity - add delete account endpoint - fix anchor links to Quantity section --- .../0000-settlement-engine.md | 155 +++++++++--------- 1 file changed, 81 insertions(+), 74 deletions(-) diff --git a/0000-settlement-engine/0000-settlement-engine.md b/0000-settlement-engine/0000-settlement-engine.md index ee7da39d..927b89c8 100644 --- a/0000-settlement-engine/0000-settlement-engine.md +++ b/0000-settlement-engine/0000-settlement-engine.md @@ -37,7 +37,7 @@ Settlement engines provide a standardized mechanism for Interledger connectors t ### Accounting -An **account** represents a record of transactions between counterparties, denominated in some fungible asset. Each account may be comprised of multiple account balances, which each represent the net difference between amounts received (credits) and amounts owed (debits) for some subset of the transactions. +An **account** represents a record of transactions between counterparties, denominated in some fungible asset. Each account may be comprised of multiple account balances, which each represent the net difference between amounts received (credits) and amounts owed (debits) for some subset of their transactions. Interledger connectors are RECOMMENDED to operate an **accounting system** which keeps a record of two balances for each account: @@ -60,13 +60,12 @@ Settlements may be performed by using a **settlement system**, or medium for exc Settlement systems include, but are not limited to: -- Cryptocurrencies/distributed ledgers -- Payment channel networks built on blockchains +- Cryptocurrencies, blockchains, and distributed ledgers +- Payment channels and layer 2 networks - Bank clearing houses - Credit card processors - Money transfer services - -TODO Flesh out examples of settlements/settlement systmes +- Cash or physical delivery of assets TODO Explain that settlement is triggered by the accounting system @@ -80,7 +79,25 @@ Together, a settlement engine and an accounting system interface with one anothe To ensure accurate double-entry bookkeeping, settlement engine and accounting system implementations MUST enforce several core guarantees. -#### Accounting System Guarantees +#### Settlement engine guarantees + +##### Settlement symmetry + +Given two non-malicious peers transacting under normal conditions, compatible settlement engine implementations MUST ensure outgoing settlements from one peer are eventually acknowledged as incoming settlements by the other. + +More formally, the sum of amounts one settlement engine instance is instructed to settle MUST eventually equal the sum of amounts the peer's settlement engine instance instructs its accounting system to credit as incoming settlements. + +Varying external conditions, such as connectivity between peers or settlement system latency, MAY delay or prevent consistency of these instructed and acknowledged settlements. + +##### Track uncredited incoming settlements + +After the settlement engine requests the accounting system to credit an incoming settlement, if the accounting system responds that it only credited a partial amount (due to lesser precision), the settlement engine MUST track the uncredited leftover amount. + +If the request fails after retrying per the [idempotency rules](#idempotency), the settlement engine MUST track the uncredited amount to retry later. + +When a subsequent settlement is received, the settlement engine MUST request the accounting system to credit a new incoming settlement for the total amount yet to be credited, including the leftover amount. + +#### Accounting system guarantees ##### Account for outgoing settlements @@ -97,35 +114,27 @@ If request retries fail per the [idempotency rules](#idempotency), the accountin If the settlement engine instructs the accounting system a settlement was received, the accounting system MUST credit the accounts receivable, subtracting the amount of the settlement. -#### Settlement Engine Guarantees - -##### Retry crediting incoming settlements +#### Asynchronous design -After the settlement engine requests the accounting system to credit an incoming settlement, if the accounting system responds that it only credited a partial amount (due to lesser precision), the settlement engine MUST track the leftover amount to retry later. +To trigger a settlement, the accounting system optimistically debits the accounts payable before the settlement has been initiated. For a period of time, the accounts payable of the peer sending the settlement is inconsistent with the accounts receivable of the peer who will later receive the settlement. -If the request fails after several attempts per the [idempotency rules](#idempotency), the settlement engine MUST track the failed amount to retry later. +Nonetheless, the bookkeeping is correct because the settlement engine tracks the owed balance if the settlement fails: it's merely the representations of the balances within the two peers' accounting systems that are temporarily inconsistent. -TODO clarify "retry later" means "when subsequent settlements are received, prompting another notification" +This asynchronicity was determined to be an acceptable trade-off instead of refunding failed settlements back to the accounts payable. -##### Settlement symmetry - -Given two non-malicious peers transacting under normal conditions, compatible settlement engine implementations MUST ensure outgoing settlements from one peer are eventually acknowledged as incoming settlements by the other. - -More formally, the sum of amounts one settlement engine instance is instructed to settle MUST eventually equal the sum of amounts the peer's settlement engine instance instructs its accounting system to credit as incoming settlements. - -Varying external conditions, such as connectivity between peers or settlement system latency, MAY delay or prevent consistency of these instructed and acknowledged settlements. +Since the settlement engine tracks the amounts of settlements that fail, if the conditions change so the settlement engine is later able to settle, a new settlement attempt may safely begin immediately. By contrast, if the failed settlement was credited back to the accounts payable, enabling the settlement engine to safely trigger new settlements would necessitate a more complex API. -#### Trade-offs of aynchronous settlement +Refunding failed settlements does enable those amounts to be netted against packets fulfilled in the opposite direction. However, if settlement continues to fail, the credit limit will eventually be breached and prevent the peers from transacting, negating this utility. -TODO Explain the rationale for the async balance model/why it's okay if the settlement engine fails +### Exchanging messages -### Bilateral communication +In order to settle or receive settlements with a peer, a settlement engine may first need to retrieve or communicate information with the peer's settlement engine. Two peered settlement engine instances may send and receive settlement-related messages among themselves, such as identifiers for their ledger accounts. -In order to settle or receive settlements with a peer, a settlement engine may first need to retrieve or communicate information with the peer's settlement engine. Two peered settlement engine instances may send and receive settlement-related messages among themselves, such as idenifiers for their ledger accounts, payment channel claims, or to negotiate fees. +To support multiple interoperable settlement engine implementations for a particular settlement system, implementators may standardize the schema and type of messages their settlement engines use to communicate with one another. This work is out-of-scope of this RFC. -Interledger connectors use a transport, such as HTTP or WebSockets, to send and receive data with peers. Settlement engine implementations SHOULD proxy all messages through services operated by its Interledger connector leveraging its existing transport to forward settlement messages to its peer's connector and peer's settlement engine. +#### Usage with [ILP-over-HTTP](../0035-ilp-over-http/0035-ilp-over-http.md) -#### The Flow +Interledger connectors use a transport, such as HTTP or WebSockets, to send and receive data with peers. Settlement engine implementations SHOULD proxy all messages its Interledger connector's existing transport like so: 1. Origin settlement engine sends a callback request to its connector with the settlement-related message to forward. 2. Origin connector forwards the message to the peer's connector using its existing transport. @@ -134,8 +143,6 @@ Interledger connectors use a transport, such as HTTP or WebSockets, to send and 5. Peer connector sends the response message back across the transport channel to the origin connector. 6. Origin connector sends the response message back to the origin settlement engine. -#### Usage with [ILP-over-HTTP](../0035-ilp-over-http/0035-ilp-over-http.md) - When a connector receives a request to send a message to the peer, the raw message from the settlement engine MUST be encoded within an ILP PREPARE packet as described below. Then, the ILP PREPARE should be sent to the peer's connector associated with the account using ILP-over-HTTP. The peer's connector MUST extract the message data and forward it to its settlement engine for processing, and MUST NOT forward the ILP packet to any other connectors. When the settlement engine responds with a response message, the connector MUST encode the raw message from the settlement engine within an ILP FULFILL or ILP REJECT, depending upon the code of the response. If the connector was unable to process the request, it MUST respond with an ILP REJECT. @@ -179,7 +186,7 @@ The settlement engine MUST be responsible for correlating an account identifier ### Units and quantities -An amount of a given asset or currency may be represented using different denominations or units of the asset. For example, one U.S. dollar may be represented as \$1 or 100 cents, and one Bitcoin may be represented as 1 BTC or 100000000 satoshis. +Asset amounts may be represented using different denominations. For example, one U.S. dollar may be represented as \$1 or 100 cents, each of which is an equivalent unit of value. Likewise, one Bitcoin may be represented as 1 BTC or 100,000,000 satoshis. A **standard unit** is the typical unit of value for a particular asset, such as \$1 in the case of U.S. dollars, or 1 BTC in the case of Bitcoin. @@ -196,10 +203,10 @@ A quantity represents an amount denominated in some unit of a particular fungibl ##### Attributes - **`amount`** string - - Number of the given unit of the asset between 0 and the maximum 64-bit unsigned integer, 18,446,744,073,709,551,615 (inclusive). - - The amount is encoded as a string to ensure no precision is lost on platforms that don't natively support full precision 64-bit integers. + - Quantity of the unit, which is a non-negative integer. + - This amount is encoded as a string to ensure no precision is lost on platforms that don't natively support arbitrary precision integers. - **`scale`** number - - Represents an _asset scale_ as defined in the previous section. + - Asset scale of the unit, between `0` and the maximum 8-bit unsigned integer, `255` (inclusive). ##### Example @@ -214,9 +221,9 @@ To represent $2.54 in units of cents, where the amount is a multiple of $0.01: #### Scale conversions -If the accounting system or settlement engine receives a request with a **[`Quantity`](#quantity-type)** denominated in a unit more precise than its unit, it MUST convert the quantity into its native unit. If so, the resulting amount MUST be rounded down before fulfilling the request. +If the accounting system or settlement engine receives a request with a **[`Quantity`](#quantity-json-type)** denominated in a unit more precise than its unit, it MUST convert the quantity into its native unit. If so, the resulting amount MUST be rounded down before fulfilling the request. -The response to the request MUST include the converted, rounded **[`Quantity`](#quantity-type)** used to fulfill the request, which MUST be less than or equal to the amount sent in the original request. +The response to the request MUST include the converted, rounded **[`Quantity`](#quantity-json-type)** used to fulfill the request, which MUST be less than or equal to the amount sent in the original request. (If the amount rounds down to 0, this this amount MAY be 0.) Then, the system with the additional precision initiating the request MUST track the leftover sum so it may accumulate and be retried in subsequent requests. @@ -232,25 +239,44 @@ Settlement engines MUST implement these endpoints. #### Initiate an account -Inform the settlement engine that a new accounting relationship was instantiated. The settlement engine MAY perform tasks as a prerequisite to settle the new account. For example, a settlement engine implementation might send messages to the peer to exchange ledger identifiers or to negotiate terms of the settlement arrangement. +Inform the settlement engine that a new accounting relationship was instantiated. The settlement engine MAY perform tasks as a prerequisite to settle the new account. For example, a settlement engine implementation might send messages to the peer to exchange ledger identifiers or to negotiate settlement-related fees. ##### Request ```http -POST /accounts HTTP/1.1 -Content-Type: application/json +PUT /accounts/:id HTTP/1.1 ``` -> ```json -> { -> id: -> } -> ``` +##### Success Response + +```http +HTTP/1.1 201 CREATED +``` + +TODO Should this be 204 No Content? + +#### Delete an account + +TODO Should deleting an account be an endpoint? + +##### Request + +```http +DELETE /accounts/:id HTTP/1.1 +``` ##### Response +###### Success + ```http -HTTP/1.1 201 CREATED +HTTP/1.1 204 No Content +``` + +###### Account doesn't exist + +```http +HTTP/1.1 404 Not Found ``` #### Perform outgoing settlement @@ -269,20 +295,18 @@ Accept: application/json Content-Type: application/json ``` -> **[`Quantity`](#quantity-type)** to settle -> -> - If the amount is `0`, the settlement engine MAY retry failed settlements. +> **[`Quantity`](#quantity-json-type)** to settle -##### Response +##### Success Response ```http HTTP/1.1 202 ACCEPTED Content-Type: application/json ``` -> **[`Quantity`](#quantity-type)** enqueued to settle +> **[`Quantity`](#quantity-json-type)** enqueued to settle > -> - This response indicates the given amount will eventually be settled, but does not guarantee that a settlement was executed. +> - This response should guarantee the given amount will eventually be settled, but not that a settlement was already executed. > - The amount enqueued to settle MUST always be less than or equal to the quantity in the original request. > - If the quantity enqueued to settle is less than the quantity of the original request, the accounting system MUST credit the leftover amount back to the accounts payable, or the same liability account tracking a balance owed to the peer. This is to prevent the systems getting out-of-sync if the settlement engine uses a unit less precise than the accounting system's unit. @@ -309,24 +333,9 @@ Content-Type: application/octet-stream > _<raw bytes of response>_ -#### Complete specification - -
- The full settlement engine HTTP API, including all response codes, error codes, and additional endpoints for manual management are available in this Swagger definition. - -``` -TODO Embed the definition here -``` - -
- -Settlement engine implementations are RECOMMENDED to implement the standardized endpoints for manual management noted above. - #### Additional RPCs -Settlement engine implementations MAY expose additional, non-standard endpoints for manual operations or configuration. - -For example, settlement engines that use payment channels may expose functionality to manually open, fund, and close channels. +Settlement engine implementations MAY expose additional, non-standard endpoints for manual operations or other configuration. ### Settlement Engine Callback HTTP API @@ -344,7 +353,7 @@ Accept: application/json Content-Type: application/json ``` -> **[`Quantity`](#quantity-type)** that should be credited to the account as an incoming settlement +> **[`Quantity`](#quantity-json-type)** to be credited to the account as an incoming settlement ##### Response @@ -353,7 +362,7 @@ HTTP/1.1 201 CREATED Content-Type: application/json ``` -> **[`Quantity`](#quantity-type)** credited to the account +> **[`Quantity`](#quantity-json-type)** credited to the account > > - The amount credited MUST always be less than or equal to the quantity in the original request. > - If the quantity credited is less than the quantity of the original request, the settlement engine MUST track the leftover amount so it may accumulate and be added to subsequent notifications to prevent the two systems getting out-of-sync. @@ -381,11 +390,13 @@ Content-Type: application/octet-stream > _<raw bytes of response>_ -### Idempotency +#### Additional callbacks -TODO Good reference: https://stripe.com/blog/idempotency +Settlement engine implementations MAY support additional, non-standard callbacks for manual operation or applications. -TODO "fault tolerant" +### Idempotency + +TODO Good reference: https://stripe.com/blog/idempotency, "fault-tolerant" Implementations MUST support [idempotency](https://en.wikipedia.org/wiki/Idempotence) for safely retrying requests without accidentally performing the same operation multiple times. This is useful when an API call is disrupted in transit and you do not receive a response. For example, if a request to send a settlement does not respond due to a network connection error, you can retry the request with the same idempotency key to guarantee that no more than one settlement is performed. @@ -395,14 +406,10 @@ All `POST` requests MUST supply idempotency keys by providing an additional `Ide For each unique request, such as an individual settlement, a unique key should be generated. The key MUST generated with a cryptographically secure pseudorandom number generator to avoid collisions. Implementations are RECOMMENDED to use a v4 UUID. -TODO +(TODO Is tx hash fine instead?) #### Recommended retry behavior -TODO - #### Handling idempotent requests All `POST` endpoints MUST support idempotency keys. - -TODO From 579e27cf5209cf3cda4bd19f615f647e3fd48776 Mon Sep 17 00:00:00 2001 From: Kincaid O'Neil Date: Tue, 6 Aug 2019 10:54:55 -0400 Subject: [PATCH 07/32] fix: increment interledger arch draft --- .../0001-interledger-architecture.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/0001-interledger-architecture/0001-interledger-architecture.md b/0001-interledger-architecture/0001-interledger-architecture.md index ee9a5458..d396f04a 100644 --- a/0001-interledger-architecture/0001-interledger-architecture.md +++ b/0001-interledger-architecture/0001-interledger-architecture.md @@ -1,6 +1,6 @@ --- title: Interledger Architecture -draft: 6 +draft: 7 --- # Interledger Architecture @@ -39,8 +39,6 @@ Like the Internet protocol stack that inspired it, the Interledger protocol suit > **Aside:** Not pictured in the diagram are configuration protocols including [IL-RFC-31: Interledger Dynamic Configuration Protocol](../0031-dynamic-configuration-protocol/0031-dynamic-configuration-protocol.md) and **Route Broadcasting Protocol**. These protocols are built on top of the Interledger Protocol layer and support it, but are not considered to be transport or application-layer protocols. - - The following sections describe the general functions of each layer in the protocol suite. For an alternate explanation with detailed depictions of the protocols' data formats, see [IL-RFC-33: Relationship Between Protocols](../0033-relationship-between-protocols/0033-relationship-between-protocols.md). #### Ledger Protocols @@ -59,7 +57,7 @@ If settlement of one account in the Interledger is contingent on the status of a Nodes can also choose never to settle their obligations. This configuration may be useful when several nodes representing different pieces of software or devices are all owned by the same person or business, and all their traffic with the outside world goes through a single "home router" connector. This is the model of [moneyd](https://github.com/interledgerjs/moneyd), one of the current implementations of Interledger. -Most implementations of Interledger use [settlement engines](../0000-settlement-engine/0000-settlement-engine.md) to settle obligations automatically while abstracting the differences between different ledger layer protocols. +Implementations of Interledger are recommended to use settlement engines as defined in [IL-RFC-00: Settlement Engine](../0000-settlement-engine/0000-settlement-engine.md) to settle obligations automatically while abstracting the differences between different settlement systems and ledgers. #### Link Protocols From 18f2eec9c89295778b158070d6ad3e4b79dafb3e Mon Sep 17 00:00:00 2001 From: Kincaid O'Neil Date: Tue, 6 Aug 2019 11:56:36 -0400 Subject: [PATCH 08/32] fix: explain rationale to support different scales --- .../0000-settlement-engine.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/0000-settlement-engine/0000-settlement-engine.md b/0000-settlement-engine/0000-settlement-engine.md index 927b89c8..75305db8 100644 --- a/0000-settlement-engine/0000-settlement-engine.md +++ b/0000-settlement-engine/0000-settlement-engine.md @@ -67,7 +67,7 @@ Settlement systems include, but are not limited to: - Money transfer services - Cash or physical delivery of assets -TODO Explain that settlement is triggered by the accounting system +TODO Explain that settlement is triggered by the accounting system? TODO Should the expected netting behavior be included? @@ -186,16 +186,22 @@ The settlement engine MUST be responsible for correlating an account identifier ### Units and quantities -Asset amounts may be represented using different denominations. For example, one U.S. dollar may be represented as \$1 or 100 cents, each of which is an equivalent unit of value. Likewise, one Bitcoin may be represented as 1 BTC or 100,000,000 satoshis. +Asset amounts may be represented using different denominations. For example, one U.S. dollar may be represented as \$1 or 100 cents, each of which is equivalent in value. Likewise, one Bitcoin may be represented as 1 BTC or 100,000,000 satoshis. A **standard unit** is the typical unit of value for a particular asset, such as \$1 in the case of U.S. dollars, or 1 BTC in the case of Bitcoin. A **fractional unit** represents some unit smaller than the standard unit, but with greater precision. Examples of fractional monetary units include one cent (\$0.01 USD), or 1 satoshi (0.00000001 BTC). -An **asset scale** is the difference in orders of magnitude between a **standard unit** and a corresponding **fractional unit**. More formally, the asset scale is a non-negative integer (0, 1, 2, …) such that one **standard unit** equals `10^(-scale)` of a corresponding **fractional unit**. If the fractional unit equals the standard unit, then the asset scale is 0. +An **asset scale** is the difference in orders of magnitude between a standard unit and a corresponding fractional unit. More formally, the asset scale is a non-negative integer (0, 1, 2, …) such that one standard unit equals `10^(-scale)` of a corresponding fractional unit. If the fractional unit equals the standard unit, then the asset scale is 0. For example, one cent represents an asset scale of 2 in the case of USD, whereas 1 satoshi represents an asset scale of 8 in the case of Bitcoin. +#### Selecting scales + +Settlement engines MUST use the denomination of its settlements, which is typically the smallest denomination of the asset on the settlement system, as the scale to perform or fulfill requests. + +Accounting systems and settlement engines are RECOMMENDED to use the same denomination or scale to minimize conversion inconsistencies, but MAY use different denominations. For example, micropayments may require more precision than can actually be settled, or database limitations of the accounting system may require less precision than the settlement system is capable of. + #### `Quantity` JSON type A quantity represents an amount denominated in some unit of a particular fungible asset. (Since account balances may only be denominated in a single asset, the type of asset should be implicit). @@ -227,12 +233,6 @@ The response to the request MUST include the converted, rounded **[`Quantity`](# Then, the system with the additional precision initiating the request MUST track the leftover sum so it may accumulate and be retried in subsequent requests. -#### Choosing scales - -All quantities the settlement engine includes in its responses or callback requests MUST use the same unit its settlements are denominated in, which is typically the smallest denomination of the asset on the settlement system. - -The accounting system is RECOMMENDED to use the same unit or scale as the settlement engine to minimize conversion inconsistencies. - ### Settlement Engine HTTP API Settlement engines MUST implement these endpoints. From 43cfa0695026acc9165805395b9b9e0b319f9001 Mon Sep 17 00:00:00 2001 From: Kincaid O'Neil Date: Tue, 6 Aug 2019 14:31:06 -0400 Subject: [PATCH 09/32] fix: clarify asset amount not tied to denomination Co-Authored-By: David Fuelling --- 0000-settlement-engine/0000-settlement-engine.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/0000-settlement-engine/0000-settlement-engine.md b/0000-settlement-engine/0000-settlement-engine.md index 75305db8..c4d9bd06 100644 --- a/0000-settlement-engine/0000-settlement-engine.md +++ b/0000-settlement-engine/0000-settlement-engine.md @@ -186,7 +186,7 @@ The settlement engine MUST be responsible for correlating an account identifier ### Units and quantities -Asset amounts may be represented using different denominations. For example, one U.S. dollar may be represented as \$1 or 100 cents, each of which is equivalent in value. Likewise, one Bitcoin may be represented as 1 BTC or 100,000,000 satoshis. +Asset amounts may be represented using any arbitrary denomination. For example, one U.S. dollar may be represented as \$1 or 100 cents, each of which is equivalent in value. Likewise, one Bitcoin may be represented as 1 BTC or 100,000,000 satoshis. A **standard unit** is the typical unit of value for a particular asset, such as \$1 in the case of U.S. dollars, or 1 BTC in the case of Bitcoin. From 93e476c46d367e16dbe1007c1586525cd3d07e51 Mon Sep 17 00:00:00 2001 From: Kincaid O'Neil Date: Tue, 6 Aug 2019 16:11:38 -0400 Subject: [PATCH 10/32] fix: clarify pre-payment, minor changes --- 0000-settlement-engine/0000-settlement-engine.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/0000-settlement-engine/0000-settlement-engine.md b/0000-settlement-engine/0000-settlement-engine.md index c4d9bd06..04c4db85 100644 --- a/0000-settlement-engine/0000-settlement-engine.md +++ b/0000-settlement-engine/0000-settlement-engine.md @@ -37,16 +37,16 @@ Settlement engines provide a standardized mechanism for Interledger connectors t ### Accounting -An **account** represents a record of transactions between counterparties, denominated in some fungible asset. Each account may be comprised of multiple account balances, which each represent the net difference between amounts received (credits) and amounts owed (debits) for some subset of their transactions. +An **account** represents a record of transactions between counterparties, denominated in a single, fungible asset. Each account may be comprised of multiple account balances, which each represent the net difference between amounts received (credits) and amounts owed (debits) for some subset of their transactions. Interledger connectors are RECOMMENDED to operate an **accounting system** which keeps a record of two balances for each account: - **Accounts payable**, the amount owed by the connector to its peer for packets its peer has fulfilled. - Positive amount indicates the connector is indebted to its peer (a _liability_ for the connector). - - Negative amount indicates the connector has prefunded. + - Negative amount indicates the connector has sent a pre-payment to its peer. - **Accounts receivable**, the amount owed to the connector by its peer for packets the connector has fulfilled. - Positive amount indicates its peer is indebted to the connector (an _asset_ to the connector). - - Negative amount indicates its peer has prefunded. + - Negative amount indicates its peer has sent a pre-payment to the connector. (Note: in context of "accounts payable" or "accounts receivable," "accounts" actually refers to a single account balance, not multiple accounts.) @@ -95,7 +95,7 @@ After the settlement engine requests the accounting system to credit an incoming If the request fails after retrying per the [idempotency rules](#idempotency), the settlement engine MUST track the uncredited amount to retry later. -When a subsequent settlement is received, the settlement engine MUST request the accounting system to credit a new incoming settlement for the total amount yet to be credited, including the leftover amount. +When a subsequent settlement is received, the settlement engine MUST request the accounting system to credit a new incoming settlement for the total amount yet to be credited, including the leftover amount(s). #### Accounting system guarantees @@ -134,7 +134,7 @@ To support multiple interoperable settlement engine implementations for a partic #### Usage with [ILP-over-HTTP](../0035-ilp-over-http/0035-ilp-over-http.md) -Interledger connectors use a transport, such as HTTP or WebSockets, to send and receive data with peers. Settlement engine implementations SHOULD proxy all messages its Interledger connector's existing transport like so: +Interledger connectors use a transport, such as HTTP or WebSockets, to send and receive data with peers. Settlement engine implementations SHOULD proxy all messages through its Interledger connector's existing transport like so: 1. Origin settlement engine sends a callback request to its connector with the settlement-related message to forward. 2. Origin connector forwards the message to the peer's connector using its existing transport. From fa37819a8ea609d6918058be5f1ba882fb61240c Mon Sep 17 00:00:00 2001 From: Kincaid O'Neil Date: Wed, 7 Aug 2019 12:19:52 -0400 Subject: [PATCH 11/32] fix: clarify account, improve settlement symmetry --- .../0000-settlement-engine.md | 38 ++++++++++++------- 1 file changed, 24 insertions(+), 14 deletions(-) diff --git a/0000-settlement-engine/0000-settlement-engine.md b/0000-settlement-engine/0000-settlement-engine.md index 04c4db85..82d94f76 100644 --- a/0000-settlement-engine/0000-settlement-engine.md +++ b/0000-settlement-engine/0000-settlement-engine.md @@ -27,7 +27,7 @@ Counterparties may operate compatible settlement engines to settle their liabili ### Interledger packet flow -In the [Interledger protocol](../0001-interledger-architecture/0001-interledger-architecture.md), a **peer** is a connector who is a counterparty to another connector. Connectors clear and fulfill ILP packets with their peers, which represent conditional IOUs that affect their mutual accounting relationships. +In the [Interledger protocol](../0001-interledger-architecture/0001-interledger-architecture.md), a **peer** is a connector who is a counterparty to another connector. An **account** is the connector's relationship with the counterparty, representing their arrangement to extend credit and transact with one another. Connectors clear and fulfill ILP packets with their peers, which represent conditional IOUs that affect these accounts. A connector may extend a given peer a limited line of credit, or none at all, depending upon their trustworthiness. As the connector receives incoming ILP PREPARE packets from a peer, forwards them, and returns corresponding ILP FULFILL packets, that peer's liabilities to the connector accrue. If the peer's liabilities exceed the credit limit assigned to it, the connector may reject and decline to forward additional packets. @@ -37,9 +37,11 @@ Settlement engines provide a standardized mechanism for Interledger connectors t ### Accounting -An **account** represents a record of transactions between counterparties, denominated in a single, fungible asset. Each account may be comprised of multiple account balances, which each represent the net difference between amounts received (credits) and amounts owed (debits) for some subset of their transactions. +Peers MUST record their transactions and/or the effects of their transactions through the process of accounting. -Interledger connectors are RECOMMENDED to operate an **accounting system** which keeps a record of two balances for each account: +In this accounting context, an **account** represents amounts received (credits) and amounts owed (debits) for a set of transactions between counterparties. The **balance** of an account is the net difference between these credits and debits. All the balances and transactions of an account are denominated in a single, fungible asset. + +Interledger connectors are RECOMMENDED to operate an **accounting system** which keeps a record of two accounts for each peer: - **Accounts payable**, the amount owed by the connector to its peer for packets its peer has fulfilled. - Positive amount indicates the connector is indebted to its peer (a _liability_ for the connector). @@ -48,7 +50,7 @@ Interledger connectors are RECOMMENDED to operate an **accounting system** which - Positive amount indicates its peer is indebted to the connector (an _asset_ to the connector). - Negative amount indicates its peer has sent a pre-payment to the connector. -(Note: in context of "accounts payable" or "accounts receivable," "accounts" actually refers to a single account balance, not multiple accounts.) +(Note that "accounts payable" and "accounts receivable" refer to accounts as _records_, distinct from accounts referring to the _arrangement_ between counterparties.) Thus, the connector's accounts payable with its peer should mirror its peer's accounts receivable with the connector, and respectively, the connector's accounts receivable should equal its peer's accounts payable. @@ -56,7 +58,9 @@ Thus, the connector's accounts payable with its peer should mirror its peer's ac A **settlement** is the irrevocable discharge of a liability by providing something of value to the party to whom the liability is owed. -Settlements may be performed by using a **settlement system**, or medium for exchanging value. Some settlements may transfer funds on a **ledger**, or registry of account balances and/or transactions, which is a type of settlement system. (Although not all settlement systems are ledgers, here, the terms are sometimes used interchangeably.) +The accounting system is responsible for discharging the liability, while the settlement is responsible for transferring value to the counterparty. + +Settlements may be performed by integrating with a **settlement system**, or medium for exchanging value. Some settlements may transfer funds on a **ledger**, or registry of account balances and/or transactions, which is a type of settlement system. (Although not all settlement systems are ledgers, here, the terms are sometimes used interchangeably.) Settlement systems include, but are not limited to: @@ -69,7 +73,9 @@ Settlement systems include, but are not limited to: TODO Explain that settlement is triggered by the accounting system? -TODO Should the expected netting behavior be included? +TODO Explain _how_ the value is provided to the other party -- some ways to do that, and why the spec is ultimately agnostic to it + +Individual settlement engine implementations define how these settlements are performed and how these settlements are received. ### Double-entry bookkeeping @@ -77,17 +83,21 @@ The interface in this specification provides a mechanism to reconcile settled ca Together, a settlement engine and an accounting system interface with one another to perform double-entry bookkeeping. -To ensure accurate double-entry bookkeeping, settlement engine and accounting system implementations MUST enforce several core guarantees. +To ensure accurate, balanced double-entry bookkeeping, settlement engine and accounting system implementations MUST enforce several invariants. -#### Settlement engine guarantees +#### Settlement engine invariants ##### Settlement symmetry -Given two non-malicious peers transacting under normal conditions, compatible settlement engine implementations MUST ensure outgoing settlements from one peer are eventually acknowledged as incoming settlements by the other. +The fundamental expected behavior of a settlement engine implementation is the sum of amounts one instance is instructed to settle eventually equals the sum of amounts the recipient instance instructs its accounting system to credit as incoming settlements. -More formally, the sum of amounts one settlement engine instance is instructed to settle MUST eventually equal the sum of amounts the peer's settlement engine instance instructs its accounting system to credit as incoming settlements. +Many factors may cause temporary or lasting inconsistencies between these instructed and acknowledged settlements: +- Time to finalize settlements on the underlying ledger +- Unintended bugs in the settlement engine implementation +- External conditions, such as poor network connectivity +- Intended settlement engine behavior, such as accruing owed balance to cover a fee -Varying external conditions, such as connectivity between peers or settlement system latency, MAY delay or prevent consistency of these instructed and acknowledged settlements. +As long as the instructed settlements do not equal the acknowledged settlements, the double-entry bookkeeping is out-of-balance. ##### Track uncredited incoming settlements @@ -97,7 +107,7 @@ If the request fails after retrying per the [idempotency rules](#idempotency), t When a subsequent settlement is received, the settlement engine MUST request the accounting system to credit a new incoming settlement for the total amount yet to be credited, including the leftover amount(s). -#### Accounting system guarantees +#### Accounting system invariants ##### Account for outgoing settlements @@ -198,9 +208,9 @@ For example, one cent represents an asset scale of 2 in the case of USD, whereas #### Selecting scales -Settlement engines MUST use the denomination of its settlements, which is typically the smallest denomination of the asset on the settlement system, as the scale to perform or fulfill requests. +Settlement engines are RECOMMENDED to perform or fulfill requests with amounts denominated in the same scale as its settlements. -Accounting systems and settlement engines are RECOMMENDED to use the same denomination or scale to minimize conversion inconsistencies, but MAY use different denominations. For example, micropayments may require more precision than can actually be settled, or database limitations of the accounting system may require less precision than the settlement system is capable of. +Account balances within the accounting systems are RECOMMENDED to be denominated in the same scale as their corresponding settlement engine, but MAY use different ones. For example, micropayments may require more precision than can actually be settled, or databases may limit precision to less than the settlement system is capable of. #### `Quantity` JSON type From 5b493134eb21a59d6585a61d4535bfda0015380d Mon Sep 17 00:00:00 2001 From: Kincaid O'Neil Date: Wed, 7 Aug 2019 15:12:47 -0400 Subject: [PATCH 12/32] fix: plural settlement engines - add WIP of webhook API --- .../0000-settlement-engines.md | 88 ++++++++++++------- .../0001-interledger-architecture.md | 2 +- README.md | 2 +- 3 files changed, 60 insertions(+), 32 deletions(-) rename 0000-settlement-engine/0000-settlement-engine.md => 0000-settlement-engines/0000-settlement-engines.md (92%) diff --git a/0000-settlement-engine/0000-settlement-engine.md b/0000-settlement-engines/0000-settlement-engines.md similarity index 92% rename from 0000-settlement-engine/0000-settlement-engine.md rename to 0000-settlement-engines/0000-settlement-engines.md index 82d94f76..d87eeb3d 100644 --- a/0000-settlement-engine/0000-settlement-engine.md +++ b/0000-settlement-engines/0000-settlement-engines.md @@ -1,9 +1,9 @@ --- -title: Settlement Engine +title: Settlement Engines draft: 1 --- -# Settlement Engine +# Settlement Engines ## Conventions @@ -11,7 +11,7 @@ The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "S ## Overview -This specification codifies a common interface for a **settlement engine**. Settlement engines are services which perform two primary operations: +This specification codifies a common interface for **settlement engines**. Settlement engines are services which perform two primary operations: 1. Send payments: execute outgoing settlements 2. Receive payments: acknowledge incoming settlements @@ -58,7 +58,7 @@ Thus, the connector's accounts payable with its peer should mirror its peer's ac A **settlement** is the irrevocable discharge of a liability by providing something of value to the party to whom the liability is owed. -The accounting system is responsible for discharging the liability, while the settlement is responsible for transferring value to the counterparty. +The accounting system is responsible for discharging the liability, while the settlement engine is responsible for transferring value to the counterparty. Settlements may be performed by integrating with a **settlement system**, or medium for exchanging value. Some settlements may transfer funds on a **ledger**, or registry of account balances and/or transactions, which is a type of settlement system. (Although not all settlement systems are ledgers, here, the terms are sometimes used interchangeably.) @@ -212,16 +212,18 @@ Settlement engines are RECOMMENDED to perform or fulfill requests with amounts d Account balances within the accounting systems are RECOMMENDED to be denominated in the same scale as their corresponding settlement engine, but MAY use different ones. For example, micropayments may require more precision than can actually be settled, or databases may limit precision to less than the settlement system is capable of. -#### `Quantity` JSON type +#### `Quantity` object + +TODO Reference that this is JSON? A quantity represents an amount denominated in some unit of a particular fungible asset. (Since account balances may only be denominated in a single asset, the type of asset should be implicit). ##### Attributes -- **`amount`** string +- **`amount`** — string - Quantity of the unit, which is a non-negative integer. - This amount is encoded as a string to ensure no precision is lost on platforms that don't natively support arbitrary precision integers. -- **`scale`** number +- **`scale`** — number - Asset scale of the unit, between `0` and the maximum 8-bit unsigned integer, `255` (inclusive). ##### Example @@ -247,6 +249,13 @@ Then, the system with the additional precision initiating the request MUST track Settlement engines MUST implement these endpoints. +#### `AccountData` object + +TODO Spec out these and ensure the naming is consistent + +- "sendMessageCallbackUrl" +- "creditSettlementCallbackUrl" + #### Initiate an account Inform the settlement engine that a new accounting relationship was instantiated. The settlement engine MAY perform tasks as a prerequisite to settle the new account. For example, a settlement engine implementation might send messages to the peer to exchange ledger identifiers or to negotiate settlement-related fees. @@ -257,36 +266,53 @@ Inform the settlement engine that a new accounting relationship was instantiated PUT /accounts/:id HTTP/1.1 ``` -##### Success Response +##### Response ```http HTTP/1.1 201 CREATED ``` -TODO Should this be 204 No Content? +#### Retrieve an account -#### Delete an account +##### Request -TODO Should deleting an account be an endpoint? +##### Response + +#### Update an account ##### Request ```http -DELETE /accounts/:id HTTP/1.1 +PUT /accounts/:id HTTP/1.1 +Accept: application/json +Content-Type: application/json ``` +TODO paylaod + ##### Response -###### Success +```http +HTTP/1.1 200 OK +Content-Type: application/json +``` + +TODO payload + +#### Delete an account + +TODO Should deleting an account be an endpoint? + +##### Request ```http -HTTP/1.1 204 No Content +DELETE /accounts/:id HTTP/1.1 ``` -###### Account doesn't exist +##### Response ```http -HTTP/1.1 404 Not Found +HTTP/1.1 204 No Content ``` #### Perform outgoing settlement @@ -307,7 +333,7 @@ Content-Type: application/json > **[`Quantity`](#quantity-json-type)** to settle -##### Success Response +##### Response ```http HTTP/1.1 202 ACCEPTED @@ -320,7 +346,7 @@ Content-Type: application/json > - The amount enqueued to settle MUST always be less than or equal to the quantity in the original request. > - If the quantity enqueued to settle is less than the quantity of the original request, the accounting system MUST credit the leftover amount back to the accounts payable, or the same liability account tracking a balance owed to the peer. This is to prevent the systems getting out-of-sync if the settlement engine uses a unit less precise than the accounting system's unit. -#### Handle incoming request +#### Handle incoming message Respond to an incoming message from the given peer's settlement engine. The transport MUST be the only entity invoking this instruction. @@ -347,18 +373,20 @@ Content-Type: application/octet-stream Settlement engine implementations MAY expose additional, non-standard endpoints for manual operations or other configuration. -### Settlement Engine Callback HTTP API +### Settlement Engine Webhooks Settlement engine implementations MUST implement the following callbacks. -#### Receipt of incoming settlement +#### Credit incoming settlement webhook + +Settlement engine implementations MUST send a callback request to an account's `creditSettlementCallbackUrl` after an incoming settlement is received. -Settlement engine implementations SHOULD send a callback request to this endpoint when incoming settlements are received. The endpoint handling the call, such as an accounting system, SHOULD credit the amount of the settlement to an account balance to ensure accurate double-entry bookkeeping. +To ensure accurate double-entry bookkeeping, the endpoint handling the call, such as an accounting system, is RECOMMENDED to [account for the incoming settlement](#account-for-incoming-settlements). ##### Request ```http -POST /accounts/:id/settlements HTTP/1.1 +POST HTTP/1.1 Accept: application/json Content-Type: application/json ``` @@ -372,19 +400,23 @@ HTTP/1.1 201 CREATED Content-Type: application/json ``` -> **[`Quantity`](#quantity-json-type)** credited to the account +> **[`Quantity`](#quantity-json-type)** credited to the account as an incoming settlement > > - The amount credited MUST always be less than or equal to the quantity in the original request. > - If the quantity credited is less than the quantity of the original request, the settlement engine MUST track the leftover amount so it may accumulate and be added to subsequent notifications to prevent the two systems getting out-of-sync. -#### Send outgoing request +#### Send outgoing message webhook + +Settlement engine implementations MUST send a callback request to an account's `sendMessageCallbackUrl` to send a settlement-related message to the peer's settlmenet engine. -Send a message to given peer's settlement engine and return its response. +To support messaging between peers, the endpoint handling the call SHOULD send the message to the peer's settlement engine and return its response. + +TODO link to #usage-with-ilp-over-http ##### Request ```http -POST /accounts/:id/messages HTTP/1.1 +POST HTTP/1.1 Accept: application/octet-stream Content-Type: application/octet-stream ``` @@ -400,10 +432,6 @@ Content-Type: application/octet-stream > _<raw bytes of response>_ -#### Additional callbacks - -Settlement engine implementations MAY support additional, non-standard callbacks for manual operation or applications. - ### Idempotency TODO Good reference: https://stripe.com/blog/idempotency, "fault-tolerant" diff --git a/0001-interledger-architecture/0001-interledger-architecture.md b/0001-interledger-architecture/0001-interledger-architecture.md index d396f04a..5489e804 100644 --- a/0001-interledger-architecture/0001-interledger-architecture.md +++ b/0001-interledger-architecture/0001-interledger-architecture.md @@ -57,7 +57,7 @@ If settlement of one account in the Interledger is contingent on the status of a Nodes can also choose never to settle their obligations. This configuration may be useful when several nodes representing different pieces of software or devices are all owned by the same person or business, and all their traffic with the outside world goes through a single "home router" connector. This is the model of [moneyd](https://github.com/interledgerjs/moneyd), one of the current implementations of Interledger. -Implementations of Interledger are recommended to use settlement engines as defined in [IL-RFC-00: Settlement Engine](../0000-settlement-engine/0000-settlement-engine.md) to settle obligations automatically while abstracting the differences between different settlement systems and ledgers. +Implementations of Interledger are recommended to use settlement engines as defined in [IL-RFC-00: Settlement Engines](../0000-settlement-engines/0000-settlement-engines.md) to settle obligations automatically while abstracting the differences between different settlement systems and ledgers. #### Link Protocols diff --git a/README.md b/README.md index 681a4ff4..33361192 100644 --- a/README.md +++ b/README.md @@ -53,6 +53,6 @@ The default encoding rules for Interledger protocols are the Canonical Octet Enc ## Ledger Layer -- **[00: Settlement Engines](0000-settlement-engine/0000-settlement-engine.md)** +- **[00: Settlement Engines](0000-settlement-engines/0000-settlement-engines.md)** Specifies an interface to send and receive payments across different settlement systems and ledgers. From 3a495b474463dda10c9ef28b573672777035d54f Mon Sep 17 00:00:00 2001 From: Kincaid O'Neil Date: Wed, 7 Aug 2019 17:46:11 -0400 Subject: [PATCH 13/32] docs: add idempotence behavior --- .../0000-settlement-engines.md | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/0000-settlement-engines/0000-settlement-engines.md b/0000-settlement-engines/0000-settlement-engines.md index d87eeb3d..1458ec50 100644 --- a/0000-settlement-engines/0000-settlement-engines.md +++ b/0000-settlement-engines/0000-settlement-engines.md @@ -432,22 +432,24 @@ Content-Type: application/octet-stream > _<raw bytes of response>_ -### Idempotency +### Idempotence -TODO Good reference: https://stripe.com/blog/idempotency, "fault-tolerant" - -Implementations MUST support [idempotency](https://en.wikipedia.org/wiki/Idempotence) for safely retrying requests without accidentally performing the same operation multiple times. This is useful when an API call is disrupted in transit and you do not receive a response. For example, if a request to send a settlement does not respond due to a network connection error, you can retry the request with the same idempotency key to guarantee that no more than one settlement is performed. +Idempotent requests ensure each side effect, such as performing a settlement, only happens once, even though the same request may be called multiple times. For example, if a request to send a settlement fails due to a network connection error, [idempotence](https://en.wikipedia.org/wiki/Idempotence) prevents a client from accidentally triggering multiple settlements when it retries the request. #### Performing idempotent requests -All `POST` requests MUST supply idempotency keys by providing an additional `Idempotency-Key: ` header with the request. +Each distinct operation, such as each request to settle, must have an idempotency key, or globally unique string. This key MUST be derived from a cryptographically secure source of randomness to avoid collisions, such as a v4 UUID. Clients MUST provide this key within an `Idempotency-Key: ` header in all POST requests. -For each unique request, such as an individual settlement, a unique key should be generated. The key MUST generated with a cryptographically secure pseudorandom number generator to avoid collisions. Implementations are RECOMMENDED to use a v4 UUID. +#### Recommended retry behavior -(TODO Is tx hash fine instead?) +If the client receives no response, it MUST retry the request with the same idempotency key. -#### Recommended retry behavior +To prevent overwhelming the server, the client SHOULD exponentially backoff each retry attempt and add random "jitter" to vary the retry interval. After several failed attempts, the client MUST rollback any side effects it performed, such as [refunding balances](#double-entry-bookkeeping). #### Handling idempotent requests All `POST` endpoints MUST support idempotency keys. + +Before an endpoint responds to a `POST` request with a new idempotency key (one it hasn't seen before), it should persist the idempotency key and the state of its response. If a subsequent request is sent with the same idempotency key, the server should use the state from the initial request to return the same response. + +Servers MUST persist idempotency keys and response state for at least 24 hours after the initial request was performed. From b5d88d1bf983a17a5b21ef7b615f4f3d5a4e768d Mon Sep 17 00:00:00 2001 From: Kincaid O'Neil Date: Thu, 8 Aug 2019 14:03:26 -0400 Subject: [PATCH 14/32] fix: no idempotence for sending messages --- 0000-settlement-engines/0000-settlement-engines.md | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/0000-settlement-engines/0000-settlement-engines.md b/0000-settlement-engines/0000-settlement-engines.md index 1458ec50..334d093e 100644 --- a/0000-settlement-engines/0000-settlement-engines.md +++ b/0000-settlement-engines/0000-settlement-engines.md @@ -434,11 +434,11 @@ Content-Type: application/octet-stream ### Idempotence -Idempotent requests ensure each side effect, such as performing a settlement, only happens once, even though the same request may be called multiple times. For example, if a request to send a settlement fails due to a network connection error, [idempotence](https://en.wikipedia.org/wiki/Idempotence) prevents a client from accidentally triggering multiple settlements when it retries the request. +Idempotent requests ensure each side effect only happens once, even though the same request may be called multiple times. For example, if a request to perform a settlement fails due to a network connection error, [idempotence](https://en.wikipedia.org/wiki/Idempotence) prevents a client from accidentally triggering multiple settlements when it retries the request. #### Performing idempotent requests -Each distinct operation, such as each request to settle, must have an idempotency key, or globally unique string. This key MUST be derived from a cryptographically secure source of randomness to avoid collisions, such as a v4 UUID. Clients MUST provide this key within an `Idempotency-Key: ` header in all POST requests. +Requests to settle or requests to credit incoming settlements MUST include an idempotency key, or globally unique string, within an `Idempotency-Key: ` header. To avoid collisions, this key MUST be derived from a cryptographically secure source of randomness, such as a v4 UUID. #### Recommended retry behavior @@ -448,8 +448,6 @@ To prevent overwhelming the server, the client SHOULD exponentially backoff each #### Handling idempotent requests -All `POST` endpoints MUST support idempotency keys. - -Before an endpoint responds to a `POST` request with a new idempotency key (one it hasn't seen before), it should persist the idempotency key and the state of its response. If a subsequent request is sent with the same idempotency key, the server should use the state from the initial request to return the same response. +Endpoints to settle and endpoints to credit incoming settlements MUST support idempotency keys. Before an endpoint responds to the request with a new idempotency key (one it hasn't seen before), it should persist the idempotency key and the state of its response. If a subsequent request is sent with the same idempotency key, the server should use the state from the initial request to return the same response. Servers MUST persist idempotency keys and response state for at least 24 hours after the initial request was performed. From f0349fcf78dd5dcbcaece4b848b6e6fda23fd403 Mon Sep 17 00:00:00 2001 From: Kincaid O'Neil Date: Thu, 8 Aug 2019 14:13:12 -0400 Subject: [PATCH 15/32] fix: minor edits --- .../0000-settlement-engines.md | 53 ++++++++----------- 1 file changed, 22 insertions(+), 31 deletions(-) diff --git a/0000-settlement-engines/0000-settlement-engines.md b/0000-settlement-engines/0000-settlement-engines.md index 334d093e..b458ede0 100644 --- a/0000-settlement-engines/0000-settlement-engines.md +++ b/0000-settlement-engines/0000-settlement-engines.md @@ -27,7 +27,7 @@ Counterparties may operate compatible settlement engines to settle their liabili ### Interledger packet flow -In the [Interledger protocol](../0001-interledger-architecture/0001-interledger-architecture.md), a **peer** is a connector who is a counterparty to another connector. An **account** is the connector's relationship with the counterparty, representing their arrangement to extend credit and transact with one another. Connectors clear and fulfill ILP packets with their peers, which represent conditional IOUs that affect these accounts. +In the [Interledger protocol](../0001-interledger-architecture/0001-interledger-architecture.md), a **peer** is a connector who is a counterparty to another connector. An **account** is the connector's relationship with the counterparty, representing their arrangement to transact with one another. Connectors clear and fulfill ILP packets with their peers, which represent conditional IOUs that affect these accounts. A connector may extend a given peer a limited line of credit, or none at all, depending upon their trustworthiness. As the connector receives incoming ILP PREPARE packets from a peer, forwards them, and returns corresponding ILP FULFILL packets, that peer's liabilities to the connector accrue. If the peer's liabilities exceed the credit limit assigned to it, the connector may reject and decline to forward additional packets. @@ -37,7 +37,7 @@ Settlement engines provide a standardized mechanism for Interledger connectors t ### Accounting -Peers MUST record their transactions and/or the effects of their transactions through the process of accounting. +Connectors MUST record their transactions with peers and/or the effects of their transactions with peers through the process of accounting. In this accounting context, an **account** represents amounts received (credits) and amounts owed (debits) for a set of transactions between counterparties. The **balance** of an account is the net difference between these credits and debits. All the balances and transactions of an account are denominated in a single, fungible asset. @@ -58,11 +58,9 @@ Thus, the connector's accounts payable with its peer should mirror its peer's ac A **settlement** is the irrevocable discharge of a liability by providing something of value to the party to whom the liability is owed. -The accounting system is responsible for discharging the liability, while the settlement engine is responsible for transferring value to the counterparty. +Settlements may occur on a **settlement system**, or medium for exchanging value. Some settlements may transfer funds on a **ledger**, or registry of account balances and/or transactions, which is a type of settlement system. (Although not all settlement systems are ledgers, here, the terms are sometimes used interchangeably.) -Settlements may be performed by integrating with a **settlement system**, or medium for exchanging value. Some settlements may transfer funds on a **ledger**, or registry of account balances and/or transactions, which is a type of settlement system. (Although not all settlement systems are ledgers, here, the terms are sometimes used interchangeably.) - -Settlement systems include, but are not limited to: +Examples of settlement systems include: - Cryptocurrencies, blockchains, and distributed ledgers - Payment channels and layer 2 networks @@ -71,11 +69,7 @@ Settlement systems include, but are not limited to: - Money transfer services - Cash or physical delivery of assets -TODO Explain that settlement is triggered by the accounting system? - -TODO Explain _how_ the value is provided to the other party -- some ways to do that, and why the spec is ultimately agnostic to it - -Individual settlement engine implementations define how these settlements are performed and how these settlements are received. +The accounting system is responsible for triggering settlements, which may be based on the TODO liabilities owed the peer. ### Double-entry bookkeeping @@ -89,13 +83,16 @@ To ensure accurate, balanced double-entry bookkeeping, settlement engine and acc ##### Settlement symmetry +Individual settlement engine implementations define how value is transferred to the counterparty: this specification merely defines settlements in terms of how settlement engines behave. TODO + The fundamental expected behavior of a settlement engine implementation is the sum of amounts one instance is instructed to settle eventually equals the sum of amounts the recipient instance instructs its accounting system to credit as incoming settlements. -Many factors may cause temporary or lasting inconsistencies between these instructed and acknowledged settlements: -- Time to finalize settlements on the underlying ledger +Many factors may cause temporary or lasting inconsistencies between these instructed and acknowledged settlements. For example: + +- External conditions, such as the time to finalize settlements on an underlying ledger - Unintended bugs in the settlement engine implementation -- External conditions, such as poor network connectivity - Intended settlement engine behavior, such as accruing owed balance to cover a fee +- Malicious peers or peers using incompatible settlement engines As long as the instructed settlements do not equal the acknowledged settlements, the double-entry bookkeeping is out-of-balance. @@ -103,7 +100,7 @@ As long as the instructed settlements do not equal the acknowledged settlements, After the settlement engine requests the accounting system to credit an incoming settlement, if the accounting system responds that it only credited a partial amount (due to lesser precision), the settlement engine MUST track the uncredited leftover amount. -If the request fails after retrying per the [idempotency rules](#idempotency), the settlement engine MUST track the uncredited amount to retry later. +If the request fails after retrying per the [idempotence rules](#idempotence), the settlement engine MUST track the uncredited amount to retry later. When a subsequent settlement is received, the settlement engine MUST request the accounting system to credit a new incoming settlement for the total amount yet to be credited, including the leftover amount(s). @@ -118,7 +115,7 @@ If the accounting system opts to trigger a settlement: If the settlement engine responds that it only queued a partial amount for settlement (due to lesser precision), the accounting system MUST credit back the accounts payable, adding the leftover amount. -If request retries fail per the [idempotency rules](#idempotency), the accounting system MUST credit back the accounts payable, adding the amount of the failed settlement. +If request retries fail per the [idempotence rules](#idempotence), the accounting system MUST credit back the accounts payable, adding the amount of the failed settlement. ##### Account for incoming settlements @@ -190,7 +187,7 @@ Settlement engines supercede the [Ledger Plugin Interface (LPIv2)](../deprecated ### Accounts and identifiers -Each account MUST be identified by a unique, [URL-safe](https://tools.ietf.org/html/rfc3986#section-2.3) string. +Each account MUST be identified by a unique, [URL-safe](https://tools.ietf.org/html/rfc3986#section-2.3) string, linked for the lifetime of the account. The settlement engine MUST be responsible for correlating an account identifier to the peer's identity on the shared ledger or settlement system, if required. For separation of concerns between clearing and settlement, the accounting system is NOT RECOMMENDED to have knowledge of the peer's identity on the shared settlement system. @@ -214,9 +211,7 @@ Account balances within the accounting systems are RECOMMENDED to be denominated #### `Quantity` object -TODO Reference that this is JSON? - -A quantity represents an amount denominated in some unit of a particular fungible asset. (Since account balances may only be denominated in a single asset, the type of asset should be implicit). +Represents an amount denominated in some unit of a particular fungible asset. (Since account balances may only be denominated in a single asset, the type of asset should be implicit). ##### Attributes @@ -317,34 +312,29 @@ HTTP/1.1 204 No Content #### Perform outgoing settlement -Settle a liability owed to the peer for the given amount. - -Note that the settlement engine MAY accrue owed settlements without settling immediately due to varying arrangements, implementations, or conditions. The settlement engine SHOULD persist amounts owed to the peer that have yet to be settled so they can be settled later. - -The accounting system MUST be the only entity invoking this instruction and MUST preemptively debit this amount from the accounts payable, or some liability account tracking a balance owed to the peer, to ensure accurate accounting. - ##### Request +The accounting system requests a settlement to occur with a particular account. The accounting system should [ensure accurate double-entry bookkeeping](#account-for-outgoing-settlements). + ```http POST /accounts/:id/settlements HTTP/1.1 Accept: application/json Content-Type: application/json +Idempotency-Key: ``` > **[`Quantity`](#quantity-json-type)** to settle ##### Response +The settlement engine should [asynchronously](#asynchronous-design) perform a [settlement](#settlement). + ```http HTTP/1.1 202 ACCEPTED Content-Type: application/json ``` -> **[`Quantity`](#quantity-json-type)** enqueued to settle -> -> - This response should guarantee the given amount will eventually be settled, but not that a settlement was already executed. -> - The amount enqueued to settle MUST always be less than or equal to the quantity in the original request. -> - If the quantity enqueued to settle is less than the quantity of the original request, the accounting system MUST credit the leftover amount back to the accounts payable, or the same liability account tracking a balance owed to the peer. This is to prevent the systems getting out-of-sync if the settlement engine uses a unit less precise than the accounting system's unit. +> **[`Quantity`](#quantity-json-type)** enqueued to settle, which is always less than or equal to the quantity in the original request #### Handle incoming message @@ -389,6 +379,7 @@ To ensure accurate double-entry bookkeeping, the endpoint handling the call, suc POST HTTP/1.1 Accept: application/json Content-Type: application/json +Idempotency-Key: ``` > **[`Quantity`](#quantity-json-type)** to be credited to the account as an incoming settlement From 14978680629c3e5a1ceb77ad46750fd6719a1a6d Mon Sep 17 00:00:00 2001 From: Kincaid O'Neil Date: Thu, 8 Aug 2019 16:01:15 -0400 Subject: [PATCH 16/32] docs(settlement): clean up api docs, webhooks --- .../0000-settlement-engines.md | 179 +++++++++++------- 1 file changed, 113 insertions(+), 66 deletions(-) diff --git a/0000-settlement-engines/0000-settlement-engines.md b/0000-settlement-engines/0000-settlement-engines.md index b458ede0..d76a13bd 100644 --- a/0000-settlement-engines/0000-settlement-engines.md +++ b/0000-settlement-engines/0000-settlement-engines.md @@ -13,13 +13,13 @@ The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "S This specification codifies a common interface for **settlement engines**. Settlement engines are services which perform two primary operations: -1. Send payments: execute outgoing settlements -2. Receive payments: acknowledge incoming settlements +1. Execute outgoing settlements +2. Acknowledge incoming settlements Counterparties may operate compatible settlement engines to settle their liabilities between one another. Different implementations may utilize different settlement systems or types of settlements, such as: -- Sending money or assets to the counterparty -- Transferring value on a shared ledger +- Transferring money or assets to the counterparty +- Sending payments on a shared ledger - Signing and exchanging payment channel claims - Performing a task for the counterparty with a mutually ascribed value @@ -62,14 +62,14 @@ Settlements may occur on a **settlement system**, or medium for exchanging value Examples of settlement systems include: -- Cryptocurrencies, blockchains, and distributed ledgers -- Payment channels and layer 2 networks - Bank clearing houses - Credit card processors - Money transfer services +- Cryptocurrencies, blockchains, and distributed ledgers +- Payment channels and layer 2 networks - Cash or physical delivery of assets -The accounting system is responsible for triggering settlements, which may be based on the TODO liabilities owed the peer. +The accounting system is responsible for triggering outgoing settlements. For example, when the accounts payable reaches a particular threshold, the accounting system could trigger a settlement to reduce the liabilities owed to the peer to a predefined, lesser amount. ### Double-entry bookkeeping @@ -83,7 +83,7 @@ To ensure accurate, balanced double-entry bookkeeping, settlement engine and acc ##### Settlement symmetry -Individual settlement engine implementations define how value is transferred to the counterparty: this specification merely defines settlements in terms of how settlement engines behave. TODO +Individual settlement engine implementations define how value is transferred to the counterparty: this specification merely defines settlements in terms of how settlement engines and accounting systems must behave. The fundamental expected behavior of a settlement engine implementation is the sum of amounts one instance is instructed to settle eventually equals the sum of amounts the recipient instance instructs its accounting system to credit as incoming settlements. @@ -147,7 +147,7 @@ Interledger connectors use a transport, such as HTTP or WebSockets, to send and 2. Origin connector forwards the message to the peer's connector using its existing transport. 3. Peer connector receives the message, identifies which settlement engine instance the account is associated with, and sends a request to its settlement engine to handle the message. 4. Peer settlement engine processes the message and responds with its own message. -5. Peer connector sends the response message back across the transport channel to the origin connector. +5. Peer connector sends the response message back across the transport to the origin connector. 6. Origin connector sends the response message back to the origin settlement engine. When a connector receives a request to send a message to the peer, the raw message from the settlement engine MUST be encoded within an ILP PREPARE packet as described below. Then, the ILP PREPARE should be sent to the peer's connector associated with the account using ILP-over-HTTP. @@ -211,7 +211,7 @@ Account balances within the accounting systems are RECOMMENDED to be denominated #### `Quantity` object -Represents an amount denominated in some unit of a particular fungible asset. (Since account balances may only be denominated in a single asset, the type of asset should be implicit). +An amount denominated in some unit of a single, fungible asset. (Since each account is denominated in a single asset, the type of asset is implied.) ##### Attributes @@ -240,17 +240,10 @@ The response to the request MUST include the converted, rounded **[`Quantity`](# Then, the system with the additional precision initiating the request MUST track the leftover sum so it may accumulate and be retried in subsequent requests. -### Settlement Engine HTTP API +### Settlement Engine HTTP API Endpoints Settlement engines MUST implement these endpoints. -#### `AccountData` object - -TODO Spec out these and ensure the naming is consistent - -- "sendMessageCallbackUrl" -- "creditSettlementCallbackUrl" - #### Initiate an account Inform the settlement engine that a new accounting relationship was instantiated. The settlement engine MAY perform tasks as a prerequisite to settle the new account. For example, a settlement engine implementation might send messages to the peer to exchange ledger identifiers or to negotiate settlement-related fees. @@ -258,128 +251,189 @@ Inform the settlement engine that a new accounting relationship was instantiated ##### Request ```http -PUT /accounts/:id HTTP/1.1 +POST /accounts/:id HTTP/1.1 ``` ##### Response ```http -HTTP/1.1 201 CREATED +HTTP/1.1 200 OK ``` -#### Retrieve an account +#### Delete an account + +Instructs the connector that an accounting relationship was terminated. ##### Request +```http +DELETE /accounts/:id HTTP/1.1 +``` + ##### Response -#### Update an account +```http +HTTP/1.1 200 OK +``` + +#### Perform outgoing settlement + +The accounting system requests a settlement to occur with a particular account. The accounting system should [account for the outgoing settlement](#account-for-outgoing-settlements). + +The settlement engine serving the request should [asynchronously](#asynchronous-design) perform a [settlement](#settlement). ##### Request ```http -PUT /accounts/:id HTTP/1.1 +POST /accounts/:id/settlements HTTP/1.1 Accept: application/json Content-Type: application/json +Idempotency-Key: ``` -TODO paylaod +> **[`Quantity`](#quantity-json-type)** to settle ##### Response ```http -HTTP/1.1 200 OK +HTTP/1.1 202 ACCEPTED Content-Type: application/json ``` -TODO payload +> **[`Quantity`](#quantity-json-type)** enqueued to settle, which MUST be less than or equal to the quantity in the original request -#### Delete an account +#### Handle incoming message -TODO Should deleting an account be an endpoint? +Incoming messages from the peer should prompt requests to this endpoint so the settlement engine may handle them. The settlement engine can respond with its own message, which should be proxied back to the peer it originated from. ##### Request ```http -DELETE /accounts/:id HTTP/1.1 +POST /accounts/:id/messages HTTP/1.1 +Accept: application/octet-stream +Content-Type: application/octet-stream ``` +> _<raw bytes of message>_ + ##### Response ```http -HTTP/1.1 204 No Content +HTTP/1.1 200 OK +Content-Type: application/octet-stream ``` -#### Perform outgoing settlement +> _<raw bytes of response>_ -##### Request +#### Create or update incoming settlement webhook -The accounting system requests a settlement to occur with a particular account. The accounting system should [ensure accurate double-entry bookkeeping](#account-for-outgoing-settlements). +##### Request ```http -POST /accounts/:id/settlements HTTP/1.1 +POST /accounts/:id/webhooks/settlements Accept: application/json Content-Type: application/json -Idempotency-Key: ``` -> **[`Quantity`](#quantity-json-type)** to settle +```json +{ + "url": +} +``` ##### Response -The settlement engine should [asynchronously](#asynchronous-design) perform a [settlement](#settlement). - ```http -HTTP/1.1 202 ACCEPTED +HTTP/1.1 201 CREATED Content-Type: application/json ``` -> **[`Quantity`](#quantity-json-type)** enqueued to settle, which is always less than or equal to the quantity in the original request +```json +{ + "url": +} +``` -#### Handle incoming message +#### Delete incoming settlement webhook + +##### Request + +```http +DELETE /accounts/:id/webhooks/settlements +``` + +##### Response + +```http +HTTP/1.1 200 OK +``` -Respond to an incoming message from the given peer's settlement engine. The transport MUST be the only entity invoking this instruction. +#### Create or update outgoing message webhook ##### Request ```http -POST /accounts/:id/messages HTTP/1.1 -Accept: application/octet-stream -Content-Type: application/octet-stream +POST /accounts/:id/webhooks/messages +Accept: application/json +Content-Type: application/json ``` -> _<raw bytes of message>_ +```json +{ + "url": +} +``` ##### Response ```http -HTTP/1.1 200 OK -Content-Type: application/octet-stream +HTTP/1.1 201 CREATED +Content-Type: application/json ``` -> _<raw bytes of response>_ +```json +{ + "url": +} +``` + +#### Delete outgoing message webhook -#### Additional RPCs +##### Request + +```http +POST /accounts/:id/webhooks/messages +``` + +##### Response + +```http +HTTP/1.1 200 OK +``` + +#### Additional endpoints Settlement engine implementations MAY expose additional, non-standard endpoints for manual operations or other configuration. -### Settlement Engine Webhooks +### Settlement Engine HTTP API Webhooks + +Settlement engine implementations MUST execute the following callback requests. Accounting system endpoints configured to handle the callbacks MUST respond accordingly. -Settlement engine implementations MUST implement the following callbacks. +Note that all webhook callback URLs MUST include the account identifier and type of event in the URL since the information is omitted from the request payload. #### Credit incoming settlement webhook -Settlement engine implementations MUST send a callback request to an account's `creditSettlementCallbackUrl` after an incoming settlement is received. +The settlement engine should send requests to this callback as it receives incoming settlements in order to ensure [settlement symmetry](#settlement-symmetry). -To ensure accurate double-entry bookkeeping, the endpoint handling the call, such as an accounting system, is RECOMMENDED to [account for the incoming settlement](#account-for-incoming-settlements). +The accounting system handling the request should [account for the incoming settlement](#account-for-incoming-settlements). If the accounting system only credits a partial amount, the settlement engine should [track the uncredited amounts](#track-uncredited-incoming-settlements). ##### Request ```http -POST HTTP/1.1 +POST HTTP/1.1 Accept: application/json Content-Type: application/json -Idempotency-Key: +Idempotency-Key: ``` > **[`Quantity`](#quantity-json-type)** to be credited to the account as an incoming settlement @@ -391,23 +445,16 @@ HTTP/1.1 201 CREATED Content-Type: application/json ``` -> **[`Quantity`](#quantity-json-type)** credited to the account as an incoming settlement -> -> - The amount credited MUST always be less than or equal to the quantity in the original request. -> - If the quantity credited is less than the quantity of the original request, the settlement engine MUST track the leftover amount so it may accumulate and be added to subsequent notifications to prevent the two systems getting out-of-sync. +> **[`Quantity`](#quantity-json-type)** credited to the account as an incoming settlement, which MUST be less than or equal to the quantity in the original request #### Send outgoing message webhook -Settlement engine implementations MUST send a callback request to an account's `sendMessageCallbackUrl` to send a settlement-related message to the peer's settlmenet engine. - -To support messaging between peers, the endpoint handling the call SHOULD send the message to the peer's settlement engine and return its response. - -TODO link to #usage-with-ilp-over-http +To send a message to a peer's settlement engine, settlement engines should send this callback request to the configured webhook. The configured endpoint should [proxy the message](#exchanging-messages) to the peer's settlement engine and return its response. ##### Request ```http -POST HTTP/1.1 +POST HTTP/1.1 Accept: application/octet-stream Content-Type: application/octet-stream ``` From cca172f4c7a2bd9c8cbd3bff9a440e49d6c56208 Mon Sep 17 00:00:00 2001 From: Kincaid O'Neil Date: Fri, 9 Aug 2019 10:19:06 -0400 Subject: [PATCH 17/32] fix: clarify rounding to 0 behavior Co-Authored-By: Evan Schwartz --- 0000-settlement-engines/0000-settlement-engines.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/0000-settlement-engines/0000-settlement-engines.md b/0000-settlement-engines/0000-settlement-engines.md index d76a13bd..e23b9e8b 100644 --- a/0000-settlement-engines/0000-settlement-engines.md +++ b/0000-settlement-engines/0000-settlement-engines.md @@ -236,7 +236,7 @@ To represent $2.54 in units of cents, where the amount is a multiple of $0.01: If the accounting system or settlement engine receives a request with a **[`Quantity`](#quantity-json-type)** denominated in a unit more precise than its unit, it MUST convert the quantity into its native unit. If so, the resulting amount MUST be rounded down before fulfilling the request. -The response to the request MUST include the converted, rounded **[`Quantity`](#quantity-json-type)** used to fulfill the request, which MUST be less than or equal to the amount sent in the original request. (If the amount rounds down to 0, this this amount MAY be 0.) +The response to the request MUST include the converted, rounded **[`Quantity`](#quantity-json-type)** used to fulfill the request, which MUST be less than or equal to the amount sent in the original request. (If the amount rounds down to 0, then the amount in the response would be 0.) Then, the system with the additional precision initiating the request MUST track the leftover sum so it may accumulate and be retried in subsequent requests. From 61baa23ad953f409799c343a24207a16ecb9d8dd Mon Sep 17 00:00:00 2001 From: Kincaid O'Neil Date: Fri, 9 Aug 2019 10:48:16 -0400 Subject: [PATCH 18/32] fix: minor edits, address comments --- .../0000-settlement-engines.md | 25 ++++++++++--------- 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/0000-settlement-engines/0000-settlement-engines.md b/0000-settlement-engines/0000-settlement-engines.md index e23b9e8b..b511afb8 100644 --- a/0000-settlement-engines/0000-settlement-engines.md +++ b/0000-settlement-engines/0000-settlement-engines.md @@ -13,15 +13,14 @@ The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "S This specification codifies a common interface for **settlement engines**. Settlement engines are services which perform two primary operations: -1. Execute outgoing settlements -2. Acknowledge incoming settlements +1. Send outgoing settlements +2. Receive incoming settlements Counterparties may operate compatible settlement engines to settle their liabilities between one another. Different implementations may utilize different settlement systems or types of settlements, such as: - Transferring money or assets to the counterparty - Sending payments on a shared ledger - Signing and exchanging payment channel claims -- Performing a task for the counterparty with a mutually ascribed value ## Usage in Interledger @@ -29,7 +28,7 @@ Counterparties may operate compatible settlement engines to settle their liabili In the [Interledger protocol](../0001-interledger-architecture/0001-interledger-architecture.md), a **peer** is a connector who is a counterparty to another connector. An **account** is the connector's relationship with the counterparty, representing their arrangement to transact with one another. Connectors clear and fulfill ILP packets with their peers, which represent conditional IOUs that affect these accounts. -A connector may extend a given peer a limited line of credit, or none at all, depending upon their trustworthiness. As the connector receives incoming ILP PREPARE packets from a peer, forwards them, and returns corresponding ILP FULFILL packets, that peer's liabilities to the connector accrue. If the peer's liabilities exceed the credit limit assigned to it, the connector may reject and decline to forward additional packets. +A connector may extend a given peer a limited line of credit, or none at all, depending upon their trustworthiness. As the connector receives incoming ILP Prepare packets from a peer, forwards them, and returns corresponding ILP Fulfill packets, that peer's liabilities to the connector accrue. If the peer's liabilities exceed the credit limit assigned to it, the connector may reject and decline to forward additional packets. In order to continue transacting, the peer must settle their liabilities. In most cases, this is accomplished through sending a payment on a settlement system that both peers have agreed to use. The connector should acknowledge the incoming payment—irrevocably discharging some liability the peer owed to it—and enabling it to clear and forward subsequent packets from the peer on credit. @@ -150,11 +149,11 @@ Interledger connectors use a transport, such as HTTP or WebSockets, to send and 5. Peer connector sends the response message back across the transport to the origin connector. 6. Origin connector sends the response message back to the origin settlement engine. -When a connector receives a request to send a message to the peer, the raw message from the settlement engine MUST be encoded within an ILP PREPARE packet as described below. Then, the ILP PREPARE should be sent to the peer's connector associated with the account using ILP-over-HTTP. +When a connector receives a request to send a message to the peer, the raw message from the settlement engine MUST be encoded within an ILP Prepare packet as described below. Then, the ILP Prepare should be sent to the peer's connector associated with the account using ILP-over-HTTP. -The peer's connector MUST extract the message data and forward it to its settlement engine for processing, and MUST NOT forward the ILP packet to any other connectors. When the settlement engine responds with a response message, the connector MUST encode the raw message from the settlement engine within an ILP FULFILL or ILP REJECT, depending upon the code of the response. If the connector was unable to process the request, it MUST respond with an ILP REJECT. +The peer's connector MUST extract the message data and forward it to its settlement engine for processing, and MUST NOT forward the ILP packet to any other connectors. When the settlement engine responds with a response message, the connector MUST encode the raw message from the settlement engine within an ILP Fulfill or ILP Reject, depending upon the code of the response. If the connector was unable to process the request, it MUST respond with an ILP Reject. -##### PREPARE +##### ILP Prepare - `amount`: `0` - `expiresAt`: _Determined by connector_ @@ -162,12 +161,12 @@ The peer's connector MUST extract the message data and forward it to its settlem - `destination`: `peer.settle` - `data`: _Request message from sender settlement engine_ -##### FULFILL +##### ILP Fulfill - `fulfillment`: `0000000000000000000000000000000000000000000000000000000000000000` - `data`: _Response message from recipient settlement engine_ -##### REJECT +##### ILP Reject - `code`: _Determined by connector from HTTP status of forwarded request_ - `triggeredBy`: `peer.settle` @@ -216,7 +215,7 @@ An amount denominated in some unit of a single, fungible asset. (Since each acco ##### Attributes - **`amount`** — string - - Quantity of the unit, which is a non-negative integer. + - Amount of the unit, which is a non-negative integer. - This amount is encoded as a string to ensure no precision is lost on platforms that don't natively support arbitrary precision integers. - **`scale`** — number - Asset scale of the unit, between `0` and the maximum 8-bit unsigned integer, `255` (inclusive). @@ -486,6 +485,8 @@ To prevent overwhelming the server, the client SHOULD exponentially backoff each #### Handling idempotent requests -Endpoints to settle and endpoints to credit incoming settlements MUST support idempotency keys. Before an endpoint responds to the request with a new idempotency key (one it hasn't seen before), it should persist the idempotency key and the state of its response. If a subsequent request is sent with the same idempotency key, the server should use the state from the initial request to return the same response. +Endpoints to settle and endpoints to credit incoming settlements MUST support idempotency keys. -Servers MUST persist idempotency keys and response state for at least 24 hours after the initial request was performed. +Before an endpoint responds to the request with a new idempotency key (one it hasn't seen before), it should persist the idempotency key and the state of its response. If a subsequent request is sent with the same idempotency key, the server should use the state from the initial request to return the same response. Servers MUST persist idempotency keys and response state for at least 24 hours after the initial request was performed. + +Idempotency keys may be any length, but at a minimum, servers MUST support idempotency keys up to 32 characters long. From 69dd834301245d096fb24f24a9291d544a5e57a4 Mon Sep 17 00:00:00 2001 From: Kincaid O'Neil Date: Mon, 12 Aug 2019 11:50:35 -0400 Subject: [PATCH 19/32] fix: simplify, address comments, add diagram --- .../0000-settlement-engines.md | 317 ++++++------------ shared/graphs/settlement-architecture.svg | 1 + 2 files changed, 111 insertions(+), 207 deletions(-) create mode 100644 shared/graphs/settlement-architecture.svg diff --git a/0000-settlement-engines/0000-settlement-engines.md b/0000-settlement-engines/0000-settlement-engines.md index b511afb8..2046f16d 100644 --- a/0000-settlement-engines/0000-settlement-engines.md +++ b/0000-settlement-engines/0000-settlement-engines.md @@ -22,19 +22,30 @@ Counterparties may operate compatible settlement engines to settle their liabili - Sending payments on a shared ledger - Signing and exchanging payment channel claims -## Usage in Interledger +## Motivation -### Interledger packet flow +Settlement engines supercede the [Ledger Plugin Interface (LPIv2)](../deprecated/0024-ledger-plugin-interface-2/0024-ledger-plugin-interface-2.md), an earlier abstraction for settlement integrations with Interledger. This new model addresses these issues: + +1. Multi-account plugins required logic for handling ILP packets, increasing implementation complexity. +2. Plugins bundled settlement and bilateral communication functionality together, limiting composability. +3. JavaScript plugins limited interoperability with non-JavaScript connector implementations. +4. Plugins operated in the same process as the connector, which limited scaling the two services independently. + +TODO Should I explicitly outline the "goals"? What is the aim of this specification? (seems to be some disagreement) + +## The Flow In the [Interledger protocol](../0001-interledger-architecture/0001-interledger-architecture.md), a **peer** is a connector who is a counterparty to another connector. An **account** is the connector's relationship with the counterparty, representing their arrangement to transact with one another. Connectors clear and fulfill ILP packets with their peers, which represent conditional IOUs that affect these accounts. A connector may extend a given peer a limited line of credit, or none at all, depending upon their trustworthiness. As the connector receives incoming ILP Prepare packets from a peer, forwards them, and returns corresponding ILP Fulfill packets, that peer's liabilities to the connector accrue. If the peer's liabilities exceed the credit limit assigned to it, the connector may reject and decline to forward additional packets. -In order to continue transacting, the peer must settle their liabilities. In most cases, this is accomplished through sending a payment on a settlement system that both peers have agreed to use. The connector should acknowledge the incoming payment—irrevocably discharging some liability the peer owed to it—and enabling it to clear and forward subsequent packets from the peer on credit. +In order to continue transacting, the peer must settle their liabilities. In most cases, this is accomplished through sending a payment on a settlement system that both peers have agreed to use. The connector should credit the incoming payment, irrevocably discharging a sum the peer owed to it, which enables clearing subsequent packets from the peer. Settlement engines provide a standardized mechanism for Interledger connectors to coordinate and reconcile these settlements. -### Accounting +![Settlement architecture](../shared/graphs/settlement-architecture.svg) + +## Accounting Connectors MUST record their transactions with peers and/or the effects of their transactions with peers through the process of accounting. @@ -53,14 +64,16 @@ Interledger connectors are RECOMMENDED to operate an **accounting system** which Thus, the connector's accounts payable with its peer should mirror its peer's accounts receivable with the connector, and respectively, the connector's accounts receivable should equal its peer's accounts payable. -### Settlement +## Settlement -A **settlement** is the irrevocable discharge of a liability by providing something of value to the party to whom the liability is owed. +A **settlement** is the irrevocable discharge of a liability by providing something of value to the party owed. Settlements may occur on a **settlement system**, or medium for exchanging value. Some settlements may transfer funds on a **ledger**, or registry of account balances and/or transactions, which is a type of settlement system. (Although not all settlement systems are ledgers, here, the terms are sometimes used interchangeably.) Examples of settlement systems include: +TODO Are these actually settlement systems? + - Bank clearing houses - Credit card processors - Money transfer services @@ -68,92 +81,85 @@ Examples of settlement systems include: - Payment channels and layer 2 networks - Cash or physical delivery of assets -The accounting system is responsible for triggering outgoing settlements. For example, when the accounts payable reaches a particular threshold, the accounting system could trigger a settlement to reduce the liabilities owed to the peer to a predefined, lesser amount. +## Double-entry bookkeeping -### Double-entry bookkeeping +TODO Evan: Is it correct to refer to it as "credit" if you were prefunding or settling things right away? (It might be, but then it would be worth clarifying the different uses of credit -- IOUs, "credits vs debits", and this usage) -The interface in this specification provides a mechanism to reconcile settled cash balances, representing _value_, with the aforementioned account balances tracked by the accounting system, representing _credit_. +TODO Explain clearing vs settlement? -Together, a settlement engine and an accounting system interface with one another to perform double-entry bookkeeping. +Together, a settlement engine and an accounting system interface with one another to perform double-entry bookkeeping. To ensure accurate, balanced double-entry bookkeeping, settlement engine and accounting system implementations MUST enforce several invariants. -To ensure accurate, balanced double-entry bookkeeping, settlement engine and accounting system implementations MUST enforce several invariants. +### Accounting system invariants -#### Settlement engine invariants +#### Account for outgoing settlements -##### Settlement symmetry +The accounting system is responsible for triggering outgoing settlements. For example, when the accounts payable reaches a particular threshold, the accounting system could trigger a settlement to reduce the amount owed to the peer to a predefined, lesser amount. -Individual settlement engine implementations define how value is transferred to the counterparty: this specification merely defines settlements in terms of how settlement engines and accounting systems must behave. +If the accounting system opts to trigger a settlement: -The fundamental expected behavior of a settlement engine implementation is the sum of amounts one instance is instructed to settle eventually equals the sum of amounts the recipient instance instructs its accounting system to credit as incoming settlements. +1. The accounting system MUST debit the accounts payable, subtracing the amount of the settlement. +2. Then, the accounting system MUST send a request to the settlement engine to settle the amount. -Many factors may cause temporary or lasting inconsistencies between these instructed and acknowledged settlements. For example: +If the settlement engine responds that it only queued a partial amount for settlement (due to lesser precision), the accounting system MUST credit back the accounts payable, adding the leftover amount. -- External conditions, such as the time to finalize settlements on an underlying ledger -- Unintended bugs in the settlement engine implementation -- Intended settlement engine behavior, such as accruing owed balance to cover a fee -- Malicious peers or peers using incompatible settlement engines +If request retries fail per the [idempotence rules](#idempotence), the accounting system MUST credit back the accounts payable, adding the amount of the failed settlement. -As long as the instructed settlements do not equal the acknowledged settlements, the double-entry bookkeeping is out-of-balance. +#### Account for incoming settlements -##### Track uncredited incoming settlements +If the settlement engine instructs the accounting system a settlement was received, the accounting system MUST credit the accounts receivable, subtracting the amount of the settlement. -After the settlement engine requests the accounting system to credit an incoming settlement, if the accounting system responds that it only credited a partial amount (due to lesser precision), the settlement engine MUST track the uncredited leftover amount. +The accounting system MUST respond with the amount it credited to the account. If it only credited a partial amount (due to lesser precision), the settlement engine tracks the leftover, uncredited amount. -If the request fails after retrying per the [idempotence rules](#idempotence), the settlement engine MUST track the uncredited amount to retry later. +### Settlement symmetry invariant -When a subsequent settlement is received, the settlement engine MUST request the accounting system to credit a new incoming settlement for the total amount yet to be credited, including the leftover amount(s). +The fundamental expected behavior of a settlement engine implementation is the sum of amounts one instance is instructed to settle eventually equals the sum of amounts the recipient instance instructs its accounting system to credit as incoming settlements. -#### Accounting system invariants +As long as the instructed settlements do not equal the acknowledged settlements, the double-entry bookkeeping is out-of-balance. Settlement engines MUST minimize the time that the bookkeeping is unbalanced. -##### Account for outgoing settlements +To trigger a settlement, the accounting system preemptively debits the accounts payable before the settlement has been initiated. For a period of time, the accounts payable of the peer sending the settlement is inconsistent with the accounts receivable of the peer who will later receive the settlement. -If the accounting system opts to trigger a settlement: +Many factors may result in these inconsistencies: -1. The accounting system MUST debit the accounts payable, subtracing the amount of the settlement. -2. Then, the accounting system MUST send a request to the settlement engine to settle the amount. +#### Settlement delay -If the settlement engine responds that it only queued a partial amount for settlement (due to lesser precision), the accounting system MUST credit back the accounts payable, adding the leftover amount. +All settlement engine implementations necessitate some settlement delay, or time until an instructed settlement is credited by the counterparty, due to network latency between peers or the time to finalize settlements on an underlying ledger or system. -If request retries fail per the [idempotence rules](#idempotence), the accounting system MUST credit back the accounts payable, adding the amount of the failed settlement. +#### Failed outgoing settlements -##### Account for incoming settlements +If an outgoing settlement fails, the settlement engine MUST track the amount of the settlement to retry it later. -If the settlement engine instructs the accounting system a settlement was received, the accounting system MUST credit the accounts receivable, subtracting the amount of the settlement. +An intentional design decision was to hide failures from the accounting system rather than refunding failed settlements back to the accounts payable. Since the settlement engine tracks these failed amounts, if the conditions change so the settlement engine is later able to settle, a new settlement attempt may safely begin immediately. -#### Asynchronous design +Refunding failed settlements would enable the peer's accounting system balances to appear synchronized. However, if settlement continues to fail, the credit limit would eventually be breached and prevent the peers from transacting, negating this utility. -To trigger a settlement, the accounting system optimistically debits the accounts payable before the settlement has been initiated. For a period of time, the accounts payable of the peer sending the settlement is inconsistent with the accounts receivable of the peer who will later receive the settlement. +#### Uncredited incoming settlements -Nonetheless, the bookkeeping is correct because the settlement engine tracks the owed balance if the settlement fails: it's merely the representations of the balances within the two peers' accounting systems that are temporarily inconsistent. +After the settlement engine requests the accounting system to credit an incoming settlement, if the accounting system responds that it only credited a partial amount (due to lesser precision), the settlement engine MUST track the uncredited leftover amount. + +If the request fails after retrying per the [idempotence rules](#idempotence), the settlement engine MUST track the uncredited amount to retry later. -This asynchronicity was determined to be an acceptable trade-off instead of refunding failed settlements back to the accounts payable. +When a subsequent settlement is received, the settlement engine MUST request the accounting system to credit a new incoming settlement for the total amount yet to be credited, including the leftover, uncredited amount(s). -Since the settlement engine tracks the amounts of settlements that fail, if the conditions change so the settlement engine is later able to settle, a new settlement attempt may safely begin immediately. By contrast, if the failed settlement was credited back to the accounts payable, enabling the settlement engine to safely trigger new settlements would necessitate a more complex API. +#### Operating inconsistences -Refunding failed settlements does enable those amounts to be netted against packets fulfilled in the opposite direction. However, if settlement continues to fail, the credit limit will eventually be breached and prevent the peers from transacting, negating this utility. +Operators and external factors outside the control of the settlement engine implementation may also cause inconsistences, such as network connectivity, peers operating incompatible settlement engines, or malicious peers. -### Exchanging messages +## Exchanging messages In order to settle or receive settlements with a peer, a settlement engine may first need to retrieve or communicate information with the peer's settlement engine. Two peered settlement engine instances may send and receive settlement-related messages among themselves, such as identifiers for their ledger accounts. To support multiple interoperable settlement engine implementations for a particular settlement system, implementators may standardize the schema and type of messages their settlement engines use to communicate with one another. This work is out-of-scope of this RFC. -#### Usage with [ILP-over-HTTP](../0035-ilp-over-http/0035-ilp-over-http.md) - Interledger connectors use a transport, such as HTTP or WebSockets, to send and receive data with peers. Settlement engine implementations SHOULD proxy all messages through its Interledger connector's existing transport like so: -1. Origin settlement engine sends a callback request to its connector with the settlement-related message to forward. -2. Origin connector forwards the message to the peer's connector using its existing transport. -3. Peer connector receives the message, identifies which settlement engine instance the account is associated with, and sends a request to its settlement engine to handle the message. +1. Origin settlement engine sends a request to its connector with the settlement-related message to forward. +2. Origin connector encodes the raw message within an ILP Prepare packet (described below), which is sent to the peer's connector using its existing transport. +3. Peer connector receives the message within the ILP Prepare, identifies which settlement engine instance the account is associated with, and sends a request to its settlement engine to handle the message. The peer connector MUST NOT forward the ILP Prepare to any other connectors. 4. Peer settlement engine processes the message and responds with its own message. -5. Peer connector sends the response message back across the transport to the origin connector. +5. Peer connector sends the response message back across the transport to the origin connector within an ILP Fulfill or ILP Reject, depending upon the code of the response (described below). If the peer, connector was unable to process the request, it MUST respond with an ILP Reject. 6. Origin connector sends the response message back to the origin settlement engine. -When a connector receives a request to send a message to the peer, the raw message from the settlement engine MUST be encoded within an ILP Prepare packet as described below. Then, the ILP Prepare should be sent to the peer's connector associated with the account using ILP-over-HTTP. - -The peer's connector MUST extract the message data and forward it to its settlement engine for processing, and MUST NOT forward the ILP packet to any other connectors. When the settlement engine responds with a response message, the connector MUST encode the raw message from the settlement engine within an ILP Fulfill or ILP Reject, depending upon the code of the response. If the connector was unable to process the request, it MUST respond with an ILP Reject. - -##### ILP Prepare +### ILP Prepare - `amount`: `0` - `expiresAt`: _Determined by connector_ @@ -161,36 +167,25 @@ The peer's connector MUST extract the message data and forward it to its settlem - `destination`: `peer.settle` - `data`: _Request message from sender settlement engine_ -##### ILP Fulfill +### ILP Fulfill - `fulfillment`: `0000000000000000000000000000000000000000000000000000000000000000` - `data`: _Response message from recipient settlement engine_ -##### ILP Reject +### ILP Reject - `code`: _Determined by connector from HTTP status of forwarded request_ - `triggeredBy`: `peer.settle` - `message`: _Determined by connector_ - `data`: _Response message from recipient settlement engine, or empty if antecedent failure_ -### Motivation +## Accounts and identifiers -Settlement engines supercede the [Ledger Plugin Interface (LPIv2)](../deprecated/0024-ledger-plugin-interface-2/0024-ledger-plugin-interface-2.md), an earlier abstraction for settlement integrations with Interledger. This new model addresses these issues: - -1. Multi-account plugins required logic for handling ILP packets, increasing implementation complexity. -2. Plugins bundled settlement and bilateral communication functionality together, limiting composability. -3. JavaScript plugins limited interoperability with non-JavaScript connector implementations. -4. Plugins operated in the same process as the connector, which limited scaling the two services independently. - -## Specification - -### Accounts and identifiers - -Each account MUST be identified by a unique, [URL-safe](https://tools.ietf.org/html/rfc3986#section-2.3) string, linked for the lifetime of the account. +Each account MUST be identified by a unique, [URL-safe](https://tools.ietf.org/html/rfc3986#section-2.3) string, immutable for the lifetime of the account. The settlement engine MUST be responsible for correlating an account identifier to the peer's identity on the shared ledger or settlement system, if required. For separation of concerns between clearing and settlement, the accounting system is NOT RECOMMENDED to have knowledge of the peer's identity on the shared settlement system. -### Units and quantities +## Units and quantities Asset amounts may be represented using any arbitrary denomination. For example, one U.S. dollar may be represented as \$1 or 100 cents, each of which is equivalent in value. Likewise, one Bitcoin may be represented as 1 BTC or 100,000,000 satoshis. @@ -198,21 +193,21 @@ A **standard unit** is the typical unit of value for a particular asset, such as A **fractional unit** represents some unit smaller than the standard unit, but with greater precision. Examples of fractional monetary units include one cent (\$0.01 USD), or 1 satoshi (0.00000001 BTC). -An **asset scale** is the difference in orders of magnitude between a standard unit and a corresponding fractional unit. More formally, the asset scale is a non-negative integer (0, 1, 2, …) such that one standard unit equals `10^(-scale)` of a corresponding fractional unit. If the fractional unit equals the standard unit, then the asset scale is 0. +An **asset scale** is the difference in orders of magnitude between the standard unit and a corresponding fractional unit. More formally, the asset scale is a non-negative integer (0, 1, 2, …) such that one standard unit equals the value of `10^(scale)` corresponding fractional units. If the fractional unit equals the standard unit, then the asset scale is 0. -For example, one cent represents an asset scale of 2 in the case of USD, whereas 1 satoshi represents an asset scale of 8 in the case of Bitcoin. +For example, one cent represents an asset scale of 2 in the case of USD, whereas one satoshi represents an asset scale of 8 in the case of Bitcoin. -#### Selecting scales +### Selecting scales Settlement engines are RECOMMENDED to perform or fulfill requests with amounts denominated in the same scale as its settlements. Account balances within the accounting systems are RECOMMENDED to be denominated in the same scale as their corresponding settlement engine, but MAY use different ones. For example, micropayments may require more precision than can actually be settled, or databases may limit precision to less than the settlement system is capable of. -#### `Quantity` object +### **`Quantity`** object An amount denominated in some unit of a single, fungible asset. (Since each account is denominated in a single asset, the type of asset is implied.) -##### Attributes +#### Attributes - **`amount`** — string - Amount of the unit, which is a non-negative integer. @@ -220,7 +215,7 @@ An amount denominated in some unit of a single, fungible asset. (Since each acco - **`scale`** — number - Asset scale of the unit, between `0` and the maximum 8-bit unsigned integer, `255` (inclusive). -##### Example +#### Example To represent $2.54 in units of cents, where the amount is a multiple of $0.01: @@ -231,7 +226,7 @@ To represent $2.54 in units of cents, where the amount is a multiple of $0.01: } ``` -#### Scale conversions +### Scale conversions If the accounting system or settlement engine receives a request with a **[`Quantity`](#quantity-json-type)** denominated in a unit more precise than its unit, it MUST convert the quantity into its native unit. If so, the resulting amount MUST be rounded down before fulfilling the request. @@ -239,49 +234,49 @@ The response to the request MUST include the converted, rounded **[`Quantity`](# Then, the system with the additional precision initiating the request MUST track the leftover sum so it may accumulate and be retried in subsequent requests. -### Settlement Engine HTTP API Endpoints +## Settlement Engine HTTP API -Settlement engines MUST implement these endpoints. +_Connector → settlement engine_ -#### Initiate an account +HTTP/2 is RECOMMENDED for performance reasons, although HTTP/1.1 MAY also be used. Implementations SHOULD support HTTP version negotiation via Application Protocol Negotiation (ALPN). -Inform the settlement engine that a new accounting relationship was instantiated. The settlement engine MAY perform tasks as a prerequisite to settle the new account. For example, a settlement engine implementation might send messages to the peer to exchange ledger identifiers or to negotiate settlement-related fees. +### Setup an account -##### Request +Informs the settlement engine that a new account was created within the accounting system using the given [account identifier](#accounts-and-identifiers). The settlement engine MAY perform tasks as a prerequisite to settle with the account. For example, a settlement engine implementation might send messages to the peer to exchange ledger identifiers or to negotiate settlement-related fees. + +#### Request ```http -POST /accounts/:id HTTP/1.1 +PUT /accounts/:id HTTP/1.1 ``` -##### Response +#### Response ```http HTTP/1.1 200 OK ``` -#### Delete an account +### Delete an account -Instructs the connector that an accounting relationship was terminated. +Instructs the settlement engine that an account was deleted. -##### Request +#### Request ```http DELETE /accounts/:id HTTP/1.1 ``` -##### Response +#### Response ```http HTTP/1.1 200 OK ``` -#### Perform outgoing settlement - -The accounting system requests a settlement to occur with a particular account. The accounting system should [account for the outgoing settlement](#account-for-outgoing-settlements). +### Perform outgoing settlement -The settlement engine serving the request should [asynchronously](#asynchronous-design) perform a [settlement](#settlement). +Asynchronously send an outgoing settlement. The accounting system sends this request and [accounts for outgoing settlements](#account-for-outgoing-settlements). -##### Request +#### Request ```http POST /accounts/:id/settlements HTTP/1.1 @@ -292,7 +287,7 @@ Idempotency-Key: > **[`Quantity`](#quantity-json-type)** to settle -##### Response +#### Response ```http HTTP/1.1 202 ACCEPTED @@ -301,11 +296,11 @@ Content-Type: application/json > **[`Quantity`](#quantity-json-type)** enqueued to settle, which MUST be less than or equal to the quantity in the original request -#### Handle incoming message +### Handle incoming message -Incoming messages from the peer should prompt requests to this endpoint so the settlement engine may handle them. The settlement engine can respond with its own message, which should be proxied back to the peer it originated from. +Process and respond to an incoming message from the peer's settlement engine. The connector sends this request when it receives it an incoming settlement message from the peer, and returns the response message back to the peer. -##### Request +#### Request ```http POST /accounts/:id/messages HTTP/1.1 @@ -315,7 +310,7 @@ Content-Type: application/octet-stream > _<raw bytes of message>_ -##### Response +#### Response ```http HTTP/1.1 200 OK @@ -324,120 +319,28 @@ Content-Type: application/octet-stream > _<raw bytes of response>_ -#### Create or update incoming settlement webhook +## Accounting System HTTP API -##### Request +_Settlement engine → connector_ -```http -POST /accounts/:id/webhooks/settlements -Accept: application/json -Content-Type: application/json -``` +HTTP/2 is RECOMMENDED for performance reasons, although HTTP/1.1 MAY also be used. Implementations SHOULD support HTTP version negotiation via Application Protocol Negotiation (ALPN). -```json -{ - "url": -} -``` +### Credit incoming settlement -##### Response +[Account for an incoming settlement](#account-for-incoming-settlements). The settlement engine sends this request as it receives incoming settlements in in order to ensure [settlement symmetry](#settlement-symmetry). If the accounting system responds that it only credited a partial amount, the settlement engine [tracks the uncredited amounts](#track-uncredited-incoming-settlements). -```http -HTTP/1.1 201 CREATED -Content-Type: application/json -``` - -```json -{ - "url": -} -``` - -#### Delete incoming settlement webhook - -##### Request - -```http -DELETE /accounts/:id/webhooks/settlements -``` - -##### Response +#### Request ```http -HTTP/1.1 200 OK -``` - -#### Create or update outgoing message webhook - -##### Request - -```http -POST /accounts/:id/webhooks/messages -Accept: application/json -Content-Type: application/json -``` - -```json -{ - "url": -} -``` - -##### Response - -```http -HTTP/1.1 201 CREATED -Content-Type: application/json -``` - -```json -{ - "url": -} -``` - -#### Delete outgoing message webhook - -##### Request - -```http -POST /accounts/:id/webhooks/messages -``` - -##### Response - -```http -HTTP/1.1 200 OK -``` - -#### Additional endpoints - -Settlement engine implementations MAY expose additional, non-standard endpoints for manual operations or other configuration. - -### Settlement Engine HTTP API Webhooks - -Settlement engine implementations MUST execute the following callback requests. Accounting system endpoints configured to handle the callbacks MUST respond accordingly. - -Note that all webhook callback URLs MUST include the account identifier and type of event in the URL since the information is omitted from the request payload. - -#### Credit incoming settlement webhook - -The settlement engine should send requests to this callback as it receives incoming settlements in order to ensure [settlement symmetry](#settlement-symmetry). - -The accounting system handling the request should [account for the incoming settlement](#account-for-incoming-settlements). If the accounting system only credits a partial amount, the settlement engine should [track the uncredited amounts](#track-uncredited-incoming-settlements). - -##### Request - -```http -POST HTTP/1.1 +POST /accounts/:id/settlements HTTP/1.1 Accept: application/json Content-Type: application/json -Idempotency-Key: +Idempotency-Key: ``` > **[`Quantity`](#quantity-json-type)** to be credited to the account as an incoming settlement -##### Response +#### Response ```http HTTP/1.1 201 CREATED @@ -446,21 +349,21 @@ Content-Type: application/json > **[`Quantity`](#quantity-json-type)** credited to the account as an incoming settlement, which MUST be less than or equal to the quantity in the original request -#### Send outgoing message webhook +### Send outgoing message -To send a message to a peer's settlement engine, settlement engines should send this callback request to the configured webhook. The configured endpoint should [proxy the message](#exchanging-messages) to the peer's settlement engine and return its response. +Send the message to the given peer's settlement engine and return its response. The connector handles [proxying the message](#exchanging-messages) through the peer's connector. -##### Request +#### Request ```http -POST HTTP/1.1 +POST /accounts/:id/messages HTTP/1.1 Accept: application/octet-stream Content-Type: application/octet-stream ``` > _<raw bytes of message>_ -##### Response +#### Response ```http HTTP/1.1 200 OK @@ -469,21 +372,21 @@ Content-Type: application/octet-stream > _<raw bytes of response>_ -### Idempotence +## Idempotence Idempotent requests ensure each side effect only happens once, even though the same request may be called multiple times. For example, if a request to perform a settlement fails due to a network connection error, [idempotence](https://en.wikipedia.org/wiki/Idempotence) prevents a client from accidentally triggering multiple settlements when it retries the request. -#### Performing idempotent requests +### Performing idempotent requests -Requests to settle or requests to credit incoming settlements MUST include an idempotency key, or globally unique string, within an `Idempotency-Key: ` header. To avoid collisions, this key MUST be derived from a cryptographically secure source of randomness, such as a v4 UUID. +Requests to settle or requests to credit incoming settlements MUST include an idempotency key, or globally unique string, within an `Idempotency-Key: ` header. To avoid collisions, this key MUST be derived from a cryptographically secure source of randomness. -#### Recommended retry behavior +### Recommended retry behavior If the client receives no response, it MUST retry the request with the same idempotency key. To prevent overwhelming the server, the client SHOULD exponentially backoff each retry attempt and add random "jitter" to vary the retry interval. After several failed attempts, the client MUST rollback any side effects it performed, such as [refunding balances](#double-entry-bookkeeping). -#### Handling idempotent requests +### Handling idempotent requests Endpoints to settle and endpoints to credit incoming settlements MUST support idempotency keys. diff --git a/shared/graphs/settlement-architecture.svg b/shared/graphs/settlement-architecture.svg new file mode 100644 index 00000000..2acb1a5a --- /dev/null +++ b/shared/graphs/settlement-architecture.svg @@ -0,0 +1 @@ + \ No newline at end of file From ace4c0c82b956af1ec7b404f57fe0994c2d32522 Mon Sep 17 00:00:00 2001 From: Kincaid O'Neil Date: Mon, 12 Aug 2019 12:41:29 -0400 Subject: [PATCH 20/32] fix: update flow diagram for readability --- shared/graphs/settlement-architecture.svg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/shared/graphs/settlement-architecture.svg b/shared/graphs/settlement-architecture.svg index 2acb1a5a..7dca1f36 100644 --- a/shared/graphs/settlement-architecture.svg +++ b/shared/graphs/settlement-architecture.svg @@ -1 +1 @@ - \ No newline at end of file + \ No newline at end of file From 6aea82c4e89d5f0ba35f9d971bcccd573420fbea Mon Sep 17 00:00:00 2001 From: Kincaid O'Neil Date: Mon, 12 Aug 2019 12:44:59 -0400 Subject: [PATCH 21/32] fix: smaller margins on flow diagram --- shared/graphs/settlement-architecture.svg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/shared/graphs/settlement-architecture.svg b/shared/graphs/settlement-architecture.svg index 7dca1f36..88b84360 100644 --- a/shared/graphs/settlement-architecture.svg +++ b/shared/graphs/settlement-architecture.svg @@ -1 +1 @@ - \ No newline at end of file + \ No newline at end of file From ef8d9741d2c6858009cc451d21d7a6f83fb38a61 Mon Sep 17 00:00:00 2001 From: Kincaid O'Neil Date: Mon, 12 Aug 2019 14:38:20 -0400 Subject: [PATCH 22/32] fix: even smaller diagram margins --- shared/graphs/settlement-architecture.svg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/shared/graphs/settlement-architecture.svg b/shared/graphs/settlement-architecture.svg index 88b84360..dcdb0655 100644 --- a/shared/graphs/settlement-architecture.svg +++ b/shared/graphs/settlement-architecture.svg @@ -1 +1 @@ - \ No newline at end of file + \ No newline at end of file From b57c4b83a34ee3e64f9b1761eddfd59c7e3ac86f Mon Sep 17 00:00:00 2001 From: Kincaid O'Neil Date: Mon, 12 Aug 2019 14:40:51 -0400 Subject: [PATCH 23/32] fix: minor diagram edits --- shared/graphs/settlement-architecture.svg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/shared/graphs/settlement-architecture.svg b/shared/graphs/settlement-architecture.svg index dcdb0655..be12fe89 100644 --- a/shared/graphs/settlement-architecture.svg +++ b/shared/graphs/settlement-architecture.svg @@ -1 +1 @@ - \ No newline at end of file + \ No newline at end of file From 3330782d68f83509ccc01a0d2a034839081738e1 Mon Sep 17 00:00:00 2001 From: Kincaid O'Neil Date: Thu, 15 Aug 2019 14:36:54 -0400 Subject: [PATCH 24/32] fix: apply suggestions Co-Authored-By: David Fuelling Co-Authored-By: Georgios Konstantopoulos --- 0000-settlement-engines/0000-settlement-engines.md | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/0000-settlement-engines/0000-settlement-engines.md b/0000-settlement-engines/0000-settlement-engines.md index 2046f16d..9e2ee2c6 100644 --- a/0000-settlement-engines/0000-settlement-engines.md +++ b/0000-settlement-engines/0000-settlement-engines.md @@ -60,9 +60,8 @@ Interledger connectors are RECOMMENDED to operate an **accounting system** which - Positive amount indicates its peer is indebted to the connector (an _asset_ to the connector). - Negative amount indicates its peer has sent a pre-payment to the connector. -(Note that "accounts payable" and "accounts receivable" refer to accounts as _records_, distinct from accounts referring to the _arrangement_ between counterparties.) -Thus, the connector's accounts payable with its peer should mirror its peer's accounts receivable with the connector, and respectively, the connector's accounts receivable should equal its peer's accounts payable. +Thus, a given connector's "accounts payable" balance should mirror its peer's "accounts receivable" balance. Likewise, a connector's "accounts receivable" balance should mirror its peer's "accounts payable" balance. ## Settlement @@ -93,7 +92,7 @@ Together, a settlement engine and an accounting system interface with one anothe #### Account for outgoing settlements -The accounting system is responsible for triggering outgoing settlements. For example, when the accounts payable reaches a particular threshold, the accounting system could trigger a settlement to reduce the amount owed to the peer to a predefined, lesser amount. +The accounting system is responsible for triggering outgoing settlements. For example, when the accounts payable reaches a particular threshold, the accounting system SHOULD trigger a settlement to reduce the amount owed to the peer to a predefined, lesser amount in order to be able to continue transacting with the peer. If the accounting system opts to trigger a settlement: @@ -108,15 +107,15 @@ If request retries fail per the [idempotence rules](#idempotence), the accountin If the settlement engine instructs the accounting system a settlement was received, the accounting system MUST credit the accounts receivable, subtracting the amount of the settlement. -The accounting system MUST respond with the amount it credited to the account. If it only credited a partial amount (due to lesser precision), the settlement engine tracks the leftover, uncredited amount. +The accounting system MUST respond with the amount it credited to the account. If it only credited a partial amount (due to lesser precision), the settlement engine tracks the leftover, uncredited amount and includes it in the next incoming settlement notification to the accounting system. ### Settlement symmetry invariant The fundamental expected behavior of a settlement engine implementation is the sum of amounts one instance is instructed to settle eventually equals the sum of amounts the recipient instance instructs its accounting system to credit as incoming settlements. -As long as the instructed settlements do not equal the acknowledged settlements, the double-entry bookkeeping is out-of-balance. Settlement engines MUST minimize the time that the bookkeeping is unbalanced. +As long as the instructed settlements do not equal the acknowledged settlements, the double-entry bookkeeping is out-of-balance. Settlement engines SHOULD minimize the time that the bookkeeping is unbalanced. -To trigger a settlement, the accounting system preemptively debits the accounts payable before the settlement has been initiated. For a period of time, the accounts payable of the peer sending the settlement is inconsistent with the accounts receivable of the peer who will later receive the settlement. +When the accounting system triggers a settlement, the accounting system should preemptively debit the accounts payable balance before any settlement has been initiated. During this time, the accounts payable balance of the settlement sender will be inconsistent with the accounts receivable balance of the settlement receiver. Many factors may result in these inconsistencies: @@ -136,7 +135,7 @@ Refunding failed settlements would enable the peer's accounting system balances After the settlement engine requests the accounting system to credit an incoming settlement, if the accounting system responds that it only credited a partial amount (due to lesser precision), the settlement engine MUST track the uncredited leftover amount. -If the request fails after retrying per the [idempotence rules](#idempotence), the settlement engine MUST track the uncredited amount to retry later. +If the request fails after retrying per the [idempotence rules](#idempotence), the settlement engine MUST track the uncredited amount in order to retry later. When a subsequent settlement is received, the settlement engine MUST request the accounting system to credit a new incoming settlement for the total amount yet to be credited, including the leftover, uncredited amount(s). From 76c717604ee8d51d8f61a9bc2cb92ba135738f09 Mon Sep 17 00:00:00 2001 From: Kincaid O'Neil Date: Sun, 18 Aug 2019 08:34:17 -0400 Subject: [PATCH 25/32] fix: apply suggestion Co-Authored-By: David Fuelling --- 0000-settlement-engines/0000-settlement-engines.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/0000-settlement-engines/0000-settlement-engines.md b/0000-settlement-engines/0000-settlement-engines.md index 9e2ee2c6..ee3adcce 100644 --- a/0000-settlement-engines/0000-settlement-engines.md +++ b/0000-settlement-engines/0000-settlement-engines.md @@ -389,6 +389,8 @@ To prevent overwhelming the server, the client SHOULD exponentially backoff each Endpoints to settle and endpoints to credit incoming settlements MUST support idempotency keys. -Before an endpoint responds to the request with a new idempotency key (one it hasn't seen before), it should persist the idempotency key and the state of its response. If a subsequent request is sent with the same idempotency key, the server should use the state from the initial request to return the same response. Servers MUST persist idempotency keys and response state for at least 24 hours after the initial request was performed. +Before an endpoint responds to the request with a new idempotency key (one it hasn't seen before), the endpoint should persist the idempotency key and the state of its response. If a subsequent request is encountered with the same idempotency key, the endpoint should use the state from the initial request to return the same response. + +Endpoints MUST persist idempotency keys and response state for at least 24 hours after the initial request was performed. Idempotency keys may be any length, but at a minimum, servers MUST support idempotency keys up to 32 characters long. From 9d5e5cff1e53da3cb49ac8f11138d5b70908da35 Mon Sep 17 00:00:00 2001 From: Kincaid O'Neil Date: Wed, 11 Sep 2019 16:49:05 -0400 Subject: [PATCH 26/32] fix: bring inline with existing implementations - response codes are now 201 to conform to REST - creating an account is a POST with an account ID in the body - other misc changes per comments --- .../0000-settlement-engines.md | 47 +++++++++++-------- 1 file changed, 27 insertions(+), 20 deletions(-) diff --git a/0000-settlement-engines/0000-settlement-engines.md b/0000-settlement-engines/0000-settlement-engines.md index 2046f16d..68c7104f 100644 --- a/0000-settlement-engines/0000-settlement-engines.md +++ b/0000-settlement-engines/0000-settlement-engines.md @@ -24,7 +24,7 @@ Counterparties may operate compatible settlement engines to settle their liabili ## Motivation -Settlement engines supercede the [Ledger Plugin Interface (LPIv2)](../deprecated/0024-ledger-plugin-interface-2/0024-ledger-plugin-interface-2.md), an earlier abstraction for settlement integrations with Interledger. This new model addresses these issues: +Settlement engines supercede the [Ledger Plugin Interface (LPIv2)](../deprecated/0024-ledger-plugin-interface-2/0024-ledger-plugin-interface-2.md) as an abstraction for settlement integrations with Interledger. This new model addresses these issues: 1. Multi-account plugins required logic for handling ILP packets, increasing implementation complexity. 2. Plugins bundled settlement and bilateral communication functionality together, limiting composability. @@ -47,9 +47,9 @@ Settlement engines provide a standardized mechanism for Interledger connectors t ## Accounting -Connectors MUST record their transactions with peers and/or the effects of their transactions with peers through the process of accounting. +Connectors are RECOMMENDED to record their transactions with peers through the process of accounting. -In this accounting context, an **account** represents amounts received (credits) and amounts owed (debits) for a set of transactions between counterparties. The **balance** of an account is the net difference between these credits and debits. All the balances and transactions of an account are denominated in a single, fungible asset. +In this financial accounting context, an **account** represents amounts received (credits) and amounts owed (debits) for a set of transactions between counterparties. The **balance** of an account is the net difference between these credits and debits. All the balances and transactions of an account are denominated in a single, fungible asset. Interledger connectors are RECOMMENDED to operate an **accounting system** which keeps a record of two accounts for each peer: @@ -60,8 +60,6 @@ Interledger connectors are RECOMMENDED to operate an **accounting system** which - Positive amount indicates its peer is indebted to the connector (an _asset_ to the connector). - Negative amount indicates its peer has sent a pre-payment to the connector. -(Note that "accounts payable" and "accounts receivable" refer to accounts as _records_, distinct from accounts referring to the _arrangement_ between counterparties.) - Thus, the connector's accounts payable with its peer should mirror its peer's accounts receivable with the connector, and respectively, the connector's accounts receivable should equal its peer's accounts payable. ## Settlement @@ -102,7 +100,7 @@ If the accounting system opts to trigger a settlement: If the settlement engine responds that it only queued a partial amount for settlement (due to lesser precision), the accounting system MUST credit back the accounts payable, adding the leftover amount. -If request retries fail per the [idempotence rules](#idempotence), the accounting system MUST credit back the accounts payable, adding the amount of the failed settlement. +If request retries fail per the [idempotence behavior](#idempotence), the accounting system MUST credit back the accounts payable, adding the amount of the failed settlement. #### Account for incoming settlements @@ -114,15 +112,15 @@ The accounting system MUST respond with the amount it credited to the account. I The fundamental expected behavior of a settlement engine implementation is the sum of amounts one instance is instructed to settle eventually equals the sum of amounts the recipient instance instructs its accounting system to credit as incoming settlements. -As long as the instructed settlements do not equal the acknowledged settlements, the double-entry bookkeeping is out-of-balance. Settlement engines MUST minimize the time that the bookkeeping is unbalanced. +As long as the instructed settlements do not equal the acknowledged settlements, the double-entry bookkeeping is out-of-balance. Settlement engines SHOULD minimize the time that the bookkeeping is unbalanced. -To trigger a settlement, the accounting system preemptively debits the accounts payable before the settlement has been initiated. For a period of time, the accounts payable of the peer sending the settlement is inconsistent with the accounts receivable of the peer who will later receive the settlement. +When the accounting system triggers a settlement, the accounting system preemptively debits the accounts payable balance before any settlement has been initiated. During this time, the accounts payable balance of the settlement sender will be inconsistent with the accounts receivable balance of the settlement recipient. Many factors may result in these inconsistencies: #### Settlement delay -All settlement engine implementations necessitate some settlement delay, or time until an instructed settlement is credited by the counterparty, due to network latency between peers or the time to finalize settlements on an underlying ledger or system. +Settlement engine implementations MAY have settlement delay, or time until an instructed settlement is credited by the counterparty, due to network latency between peers or the time to finalize settlements on an underlying ledger or system. #### Failed outgoing settlements @@ -136,7 +134,7 @@ Refunding failed settlements would enable the peer's accounting system balances After the settlement engine requests the accounting system to credit an incoming settlement, if the accounting system responds that it only credited a partial amount (due to lesser precision), the settlement engine MUST track the uncredited leftover amount. -If the request fails after retrying per the [idempotence rules](#idempotence), the settlement engine MUST track the uncredited amount to retry later. +If the request fails after retrying per the [idempotence behavior](#idempotence), the settlement engine MUST track the uncredited amount to retry later. When a subsequent settlement is received, the settlement engine MUST request the accounting system to credit a new incoming settlement for the total amount yet to be credited, including the leftover, uncredited amount(s). @@ -148,7 +146,7 @@ Operators and external factors outside the control of the settlement engine impl In order to settle or receive settlements with a peer, a settlement engine may first need to retrieve or communicate information with the peer's settlement engine. Two peered settlement engine instances may send and receive settlement-related messages among themselves, such as identifiers for their ledger accounts. -To support multiple interoperable settlement engine implementations for a particular settlement system, implementators may standardize the schema and type of messages their settlement engines use to communicate with one another. This work is out-of-scope of this RFC. +To support multiple interoperable settlement engine implementations for a particular settlement system, implementors may standardize the schema and type of messages their settlement engines use to communicate with one another. This work is out-of-scope of this RFC. Interledger connectors use a transport, such as HTTP or WebSockets, to send and receive data with peers. Settlement engine implementations SHOULD proxy all messages through its Interledger connector's existing transport like so: @@ -156,7 +154,7 @@ Interledger connectors use a transport, such as HTTP or WebSockets, to send and 2. Origin connector encodes the raw message within an ILP Prepare packet (described below), which is sent to the peer's connector using its existing transport. 3. Peer connector receives the message within the ILP Prepare, identifies which settlement engine instance the account is associated with, and sends a request to its settlement engine to handle the message. The peer connector MUST NOT forward the ILP Prepare to any other connectors. 4. Peer settlement engine processes the message and responds with its own message. -5. Peer connector sends the response message back across the transport to the origin connector within an ILP Fulfill or ILP Reject, depending upon the code of the response (described below). If the peer, connector was unable to process the request, it MUST respond with an ILP Reject. +5. Peer connector sends the response message back across the transport to the origin connector within an ILP Fulfill or ILP Reject, depending upon the code of the response (described below). If the peer connector was unable to process the request, it MUST respond with an ILP Reject. 6. Origin connector sends the response message back to the origin settlement engine. ### ILP Prepare @@ -199,6 +197,8 @@ For example, one cent represents an asset scale of 2 in the case of USD, whereas ### Selecting scales +TODO this is weird because we're not defining what settlements are -- what is "scale as its settlements" ? + Settlement engines are RECOMMENDED to perform or fulfill requests with amounts denominated in the same scale as its settlements. Account balances within the accounting systems are RECOMMENDED to be denominated in the same scale as their corresponding settlement engine, but MAY use different ones. For example, micropayments may require more precision than can actually be settled, or databases may limit precision to less than the settlement system is capable of. @@ -247,13 +247,20 @@ Informs the settlement engine that a new account was created within the accounti #### Request ```http -PUT /accounts/:id HTTP/1.1 +POST /accounts HTTP/1.1 +Content-Type: application/json +``` + +```json +{ + "id": +} ``` #### Response ```http -HTTP/1.1 200 OK +HTTP/1.1 201 Created ``` ### Delete an account @@ -269,12 +276,12 @@ DELETE /accounts/:id HTTP/1.1 #### Response ```http -HTTP/1.1 200 OK +HTTP/1.1 204 No-Content ``` ### Perform outgoing settlement -Asynchronously send an outgoing settlement. The accounting system sends this request and [accounts for outgoing settlements](#account-for-outgoing-settlements). +Asynchronously send an outgoing settlement. The accounting system sends this request and [accounts for outgoing settlements](#account-for-outgoing-settlements). (TODO link to other SE behavior here) #### Request @@ -290,7 +297,7 @@ Idempotency-Key: #### Response ```http -HTTP/1.1 202 ACCEPTED +HTTP/1.1 201 Created Content-Type: application/json ``` @@ -313,7 +320,7 @@ Content-Type: application/octet-stream #### Response ```http -HTTP/1.1 200 OK +HTTP/1.1 201 Created Content-Type: application/octet-stream ``` @@ -343,7 +350,7 @@ Idempotency-Key: #### Response ```http -HTTP/1.1 201 CREATED +HTTP/1.1 201 Created Content-Type: application/json ``` @@ -366,7 +373,7 @@ Content-Type: application/octet-stream #### Response ```http -HTTP/1.1 200 OK +HTTP/1.1 201 OK Content-Type: application/octet-stream ``` From 07a2be950396d466e20cc678b9bfcddcfad550b5 Mon Sep 17 00:00:00 2001 From: Kincaid O'Neil Date: Wed, 11 Sep 2019 17:15:45 -0400 Subject: [PATCH 27/32] fix: safer idempotency/retry behavior - max retry interval is 1 hour - servers may purge idempotency keys 24 hours after most recent request - clients must retry indefinitely in response to 5xx, 409, and no response errors - 4xx errors should have no retry - remove min max on idempotency key length - explain why retry behavior is specced --- 0000-settlement-engines/0000-settlement-engines.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/0000-settlement-engines/0000-settlement-engines.md b/0000-settlement-engines/0000-settlement-engines.md index bc4850ed..fa4bd801 100644 --- a/0000-settlement-engines/0000-settlement-engines.md +++ b/0000-settlement-engines/0000-settlement-engines.md @@ -387,11 +387,13 @@ Idempotent requests ensure each side effect only happens once, even though the s Requests to settle or requests to credit incoming settlements MUST include an idempotency key, or globally unique string, within an `Idempotency-Key: ` header. To avoid collisions, this key MUST be derived from a cryptographically secure source of randomness. -### Recommended retry behavior +### Retry behavior -If the client receives no response, it MUST retry the request with the same idempotency key. +This retry behavior ensures the client and server are eventually consistent and never perform any unsafe balance rollbacks that could result in lost funds or double payments. -To prevent overwhelming the server, the client SHOULD exponentially backoff each retry attempt and add random "jitter" to vary the retry interval. After several failed attempts, the client MUST rollback any side effects it performed, such as [refunding balances](#double-entry-bookkeeping). +If the client receives no response, an HTTP 5xx error, or an HTTP 409 Conflict error, the client MUST retry the request with the same idempotency key. If the response is a client error, such as another HTTP 4xx error, the client MUST rollback the balance update and NOT perform any subsequent retries. + +To prevent overwhelming the server, the client SHOULD exponentially backoff after each failed retry attempt and add random "jitter" to vary the retry interval. The maximum retry interval MUST be no greater than 1 hour. Clients also MUST retry indefinitely until they have received an acknowledgement from the server that the request was processed or failed. ### Handling idempotent requests @@ -399,6 +401,4 @@ Endpoints to settle and endpoints to credit incoming settlements MUST support id Before an endpoint responds to the request with a new idempotency key (one it hasn't seen before), the endpoint should persist the idempotency key and the state of its response. If a subsequent request is encountered with the same idempotency key, the endpoint should use the state from the initial request to return the same response. -Endpoints MUST persist idempotency keys and response state for at least 24 hours after the initial request was performed. - -Idempotency keys may be any length, but at a minimum, servers MUST support idempotency keys up to 32 characters long. +For safety, endpoints MUST persist idempotency keys and response state for at least 24 hours since the most recent request with the same idempotency key. From a8a0426ccd15a39b6badf4e49e80997d3aaca270 Mon Sep 17 00:00:00 2001 From: Kincaid O'Neil Date: Tue, 24 Sep 2019 01:00:20 -0400 Subject: [PATCH 28/32] fix: apply suggestions Co-Authored-By: pgarg-ripple <51800920+pgarg-ripple@users.noreply.github.com> --- 0000-settlement-engines/0000-settlement-engines.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/0000-settlement-engines/0000-settlement-engines.md b/0000-settlement-engines/0000-settlement-engines.md index fa4bd801..2318aa4d 100644 --- a/0000-settlement-engines/0000-settlement-engines.md +++ b/0000-settlement-engines/0000-settlement-engines.md @@ -24,7 +24,7 @@ Counterparties may operate compatible settlement engines to settle their liabili ## Motivation -Settlement engines supercede the [Ledger Plugin Interface (LPIv2)](../deprecated/0024-ledger-plugin-interface-2/0024-ledger-plugin-interface-2.md) as an abstraction for settlement integrations with Interledger. This new model addresses these issues: +Settlement engines supersede the [Ledger Plugin Interface (LPIv2)](../deprecated/0024-ledger-plugin-interface-2/0024-ledger-plugin-interface-2.md) as an abstraction for settlement integrations with Interledger. This new model addresses these issues: 1. Multi-account plugins required logic for handling ILP packets, increasing implementation complexity. 2. Plugins bundled settlement and bilateral communication functionality together, limiting composability. @@ -95,7 +95,7 @@ The accounting system is responsible for triggering outgoing settlements. For ex If the accounting system opts to trigger a settlement: -1. The accounting system MUST debit the accounts payable, subtracing the amount of the settlement. +1. The accounting system MUST debit the accounts payable, subtracting the amount of the settlement. 2. Then, the accounting system MUST send a request to the settlement engine to settle the amount. If the settlement engine responds that it only queued a partial amount for settlement (due to lesser precision), the accounting system MUST credit back the accounts payable, adding the leftover amount. @@ -305,7 +305,7 @@ Content-Type: application/json ### Handle incoming message -Process and respond to an incoming message from the peer's settlement engine. The connector sends this request when it receives it an incoming settlement message from the peer, and returns the response message back to the peer. +Process and respond to an incoming message from the peer's settlement engine. The connector sends this request when it receives an incoming settlement message from the peer, and returns the response message back to the peer. #### Request @@ -393,7 +393,7 @@ This retry behavior ensures the client and server are eventually consistent and If the client receives no response, an HTTP 5xx error, or an HTTP 409 Conflict error, the client MUST retry the request with the same idempotency key. If the response is a client error, such as another HTTP 4xx error, the client MUST rollback the balance update and NOT perform any subsequent retries. -To prevent overwhelming the server, the client SHOULD exponentially backoff after each failed retry attempt and add random "jitter" to vary the retry interval. The maximum retry interval MUST be no greater than 1 hour. Clients also MUST retry indefinitely until they have received an acknowledgement from the server that the request was processed or failed. +To prevent overwhelming the server, the client SHOULD exponentially backoff after each failed retry attempt and add random "jitter" to vary the retry interval. The maximum retry interval MUST be no greater than 1 hour. Clients also MUST retry indefinitely until they have received an acknowledgment from the server that the request was processed or failed. ### Handling idempotent requests From 79d6c1990d2f63fa32464991c11d8239131d00fb Mon Sep 17 00:00:00 2001 From: Kincaid O'Neil Date: Sat, 19 Oct 2019 19:53:37 -0400 Subject: [PATCH 29/32] docs(se): remove refunds, simplify, final edits --- .../0001-interledger-architecture.md | 2 +- .../0038-settlement-engines.md | 150 ++++++------------ README.md | 2 +- 3 files changed, 50 insertions(+), 104 deletions(-) rename 0000-settlement-engines/0000-settlement-engines.md => 0038-settlement-engines/0038-settlement-engines.md (64%) diff --git a/0001-interledger-architecture/0001-interledger-architecture.md b/0001-interledger-architecture/0001-interledger-architecture.md index 5489e804..cbf4a52b 100644 --- a/0001-interledger-architecture/0001-interledger-architecture.md +++ b/0001-interledger-architecture/0001-interledger-architecture.md @@ -57,7 +57,7 @@ If settlement of one account in the Interledger is contingent on the status of a Nodes can also choose never to settle their obligations. This configuration may be useful when several nodes representing different pieces of software or devices are all owned by the same person or business, and all their traffic with the outside world goes through a single "home router" connector. This is the model of [moneyd](https://github.com/interledgerjs/moneyd), one of the current implementations of Interledger. -Implementations of Interledger are recommended to use settlement engines as defined in [IL-RFC-00: Settlement Engines](../0000-settlement-engines/0000-settlement-engines.md) to settle obligations automatically while abstracting the differences between different settlement systems and ledgers. +Implementations of Interledger are recommended to use settlement engines as defined in [IL-RFC-00: Settlement Engines](../0038-settlement-engines/0038-settlement-engines.md) to settle obligations automatically while abstracting the differences between different settlement systems and ledgers. #### Link Protocols diff --git a/0000-settlement-engines/0000-settlement-engines.md b/0038-settlement-engines/0038-settlement-engines.md similarity index 64% rename from 0000-settlement-engines/0000-settlement-engines.md rename to 0038-settlement-engines/0038-settlement-engines.md index 2318aa4d..7bb645d0 100644 --- a/0000-settlement-engines/0000-settlement-engines.md +++ b/0038-settlement-engines/0038-settlement-engines.md @@ -11,45 +11,49 @@ The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "S ## Overview -This specification codifies a common interface for **settlement engines**. Settlement engines are services which perform two primary operations: +### Clearing -1. Send outgoing settlements -2. Receive incoming settlements +In the [Interledger protocol](../0001-interledger-architecture/0001-interledger-architecture.md), connectors maintain **peers**, or counterparty connectors whom they transact with. Connectors clear and fulfill ILP packets with their peers, representing conditional IOUs which affect financial accounting balances—**accounts**—between them. -Counterparties may operate compatible settlement engines to settle their liabilities between one another. Different implementations may utilize different settlement systems or types of settlements, such as: +A connector may extend a given peer a limited line of credit, or none at all, depending upon their trustworthiness. As the connector receives incoming ILP Prepare packets from a peer, forwards them, and returns corresponding ILP Fulfill packets, that peer's liabilities to the connector accrue. If the peer's liabilities exceed the credit limit assigned to it, the connector may reject and decline to forward additional packets. -- Transferring money or assets to the counterparty -- Sending payments on a shared ledger -- Signing and exchanging payment channel claims +### Settlement -## Motivation +In order to continue transacting, the peer must settle their liabilities. In most cases, this is accomplished through sending a payment on a settlement system that both peers have agreed to use. The connector should credit the incoming payment, irrevocably discharging a sum the peer owed to it, which enables it to clear subsequent Interledger packets from the peer. -Settlement engines supersede the [Ledger Plugin Interface (LPIv2)](../deprecated/0024-ledger-plugin-interface-2/0024-ledger-plugin-interface-2.md) as an abstraction for settlement integrations with Interledger. This new model addresses these issues: +A **settlement** is the irrevocable discharge of a liability via an unconditional transfer of an asset or financial instrument. Settlements may occur on a **settlement system**, or medium for exchanging value. Some settlements may transfer funds on a **ledger**, or registry of account balances and/or transactions, which is a type of settlement system. (Although not all settlement systems are ledgers, here, the terms are sometimes used interchangeably.) -1. Multi-account plugins required logic for handling ILP packets, increasing implementation complexity. -2. Plugins bundled settlement and bilateral communication functionality together, limiting composability. -3. JavaScript plugins limited interoperability with non-JavaScript connector implementations. -4. Plugins operated in the same process as the connector, which limited scaling the two services independently. +Examples of systems that settle Interledger credit relationships may include: + +- Traditional banking infrastructure +- Cryptocurrencies and distributed ledgers +- Payment channels and layer 2 networks +- Money transfer services +- Mobile money services +- Cash or physical delivery of assets -TODO Should I explicitly outline the "goals"? What is the aim of this specification? (seems to be some disagreement) +### Settlement Engines -## The Flow +_Settlement engines_ are services operated by two Interledger peers that facilitate sending and receiving settlements between them. Since an Interledger connector may have many peers that settle over the same system, a single settlement engine may manage multiple accounts, to settle with many different peers. -In the [Interledger protocol](../0001-interledger-architecture/0001-interledger-architecture.md), a **peer** is a connector who is a counterparty to another connector. An **account** is the connector's relationship with the counterparty, representing their arrangement to transact with one another. Connectors clear and fulfill ILP packets with their peers, which represent conditional IOUs that affect these accounts. +This specification defines a standardized HTTP API for Interledger connectors to interface with their settlement engines, and vice versa. Connectors trigger the settlement engine to perform settlements, and settlement engines trigger the connector to adjust accounting balances when incoming settlements are received, like so: -A connector may extend a given peer a limited line of credit, or none at all, depending upon their trustworthiness. As the connector receives incoming ILP Prepare packets from a peer, forwards them, and returns corresponding ILP Fulfill packets, that peer's liabilities to the connector accrue. If the peer's liabilities exceed the credit limit assigned to it, the connector may reject and decline to forward additional packets. +![Settlement architecture](../shared/graphs/settlement-architecture.svg) -In order to continue transacting, the peer must settle their liabilities. In most cases, this is accomplished through sending a payment on a settlement system that both peers have agreed to use. The connector should credit the incoming payment, irrevocably discharging a sum the peer owed to it, which enables clearing subsequent packets from the peer. +## Motivation -Settlement engines provide a standardized mechanism for Interledger connectors to coordinate and reconcile these settlements. +Settlement engines supercede the [Ledger Plugin Interface (LPIv2)](../deprecated/0024-ledger-plugin-interface-2/0024-ledger-plugin-interface-2.md) as an abstraction for settlement integrations with Interledger. This new model addresses these issues: -![Settlement architecture](../shared/graphs/settlement-architecture.svg) +1. Multi-account plugins necessitated non-trivial connector logic for handling ILP packets. +2. Plugin implementations were tightly coupled with a single bilateral communication mechanism. +3. JavaScript plugins limited interoperability with non-JavaScript connector implementations. +4. Plugins operated in the same process as the connector, which prevented scaling the two services independently. -## Accounting +## Concepts -Connectors are RECOMMENDED to record their transactions with peers through the process of accounting. +### Accounting -In this financial accounting context, an **account** represents amounts received (credits) and amounts owed (debits) for a set of transactions between counterparties. The **balance** of an account is the net difference between these credits and debits. All the balances and transactions of an account are denominated in a single, fungible asset. +Connectors are RECOMMENDED to record the effects of their transactions in an **accounting system**. In this financial accounting context, an **account** represents amounts received (credits) and amounts owed (debits) for a set of transactions between counterparties. The **balance** of an account is the net difference between these credits and debits. All the balances and transactions of an account are denominated in a single, fungible asset. Interledger connectors are RECOMMENDED to operate an **accounting system** which keeps a record of two accounts for each peer: @@ -62,87 +66,41 @@ Interledger connectors are RECOMMENDED to operate an **accounting system** which Thus, a given connector's "accounts payable" balance should mirror its peer's "accounts receivable" balance. Likewise, a connector's "accounts receivable" balance should mirror its peer's "accounts payable" balance. -## Settlement - -A **settlement** is the irrevocable discharge of a liability by providing something of value to the party owed. - -Settlements may occur on a **settlement system**, or medium for exchanging value. Some settlements may transfer funds on a **ledger**, or registry of account balances and/or transactions, which is a type of settlement system. (Although not all settlement systems are ledgers, here, the terms are sometimes used interchangeably.) - -Examples of settlement systems include: - -TODO Are these actually settlement systems? - -- Bank clearing houses -- Credit card processors -- Money transfer services -- Cryptocurrencies, blockchains, and distributed ledgers -- Payment channels and layer 2 networks -- Cash or physical delivery of assets - -## Double-entry bookkeeping - -TODO Evan: Is it correct to refer to it as "credit" if you were prefunding or settling things right away? (It might be, but then it would be worth clarifying the different uses of credit -- IOUs, "credits vs debits", and this usage) - -TODO Explain clearing vs settlement? - -Together, a settlement engine and an accounting system interface with one another to perform double-entry bookkeeping. To ensure accurate, balanced double-entry bookkeeping, settlement engine and accounting system implementations MUST enforce several invariants. - -### Accounting system invariants +Together, a settlement engine and an accounting system interface with one another to perform double-entry bookkeeping with eventual consistency. To ensure accurate, balanced double-entry bookkeeping, settlement engine and accounting system implementations MUST enforce several invariants. #### Account for outgoing settlements The accounting system is responsible for triggering outgoing settlements. For example, when the accounts payable reaches a particular threshold, the accounting system SHOULD trigger a settlement to reduce the amount owed to the peer to a predefined, lesser amount in order to be able to continue transacting with the peer. -If the accounting system opts to trigger a settlement: - -1. The accounting system MUST debit the accounts payable, subtracting the amount of the settlement. -2. Then, the accounting system MUST send a request to the settlement engine to settle the amount. - -If the settlement engine responds that it only queued a partial amount for settlement (due to lesser precision), the accounting system MUST credit back the accounts payable, adding the leftover amount. - -If request retries fail per the [idempotence behavior](#idempotence), the accounting system MUST credit back the accounts payable, adding the amount of the failed settlement. +When the accounting system triggers a settlement, it MUST debit the accounts payable, subtracting the amount of the settlement before sending the request to the settlement engine to settle the amount. #### Account for incoming settlements -If the settlement engine instructs the accounting system a settlement was received, the accounting system MUST credit the accounts receivable, subtracting the amount of the settlement. - -The accounting system MUST respond with the amount it credited to the account. If it only credited a partial amount (due to lesser precision), the settlement engine tracks the leftover, uncredited amount and includes it in the next incoming settlement notification to the accounting system. +If the settlement engine instructs the accounting system an incoming settlement was received, the accounting system MUST credit the accounts receivable, subtracting the amount of the settlement. -### Settlement symmetry invariant +### Settlement symmetry The fundamental expected behavior of a settlement engine implementation is the sum of amounts one instance is instructed to settle eventually equals the sum of amounts the recipient instance instructs its accounting system to credit as incoming settlements. As long as the instructed settlements do not equal the acknowledged settlements, the double-entry bookkeeping is out-of-balance. Settlement engines SHOULD minimize the time that the bookkeeping is unbalanced. -When the accounting system triggers a settlement, the accounting system preemptively debits the accounts payable balance before any settlement has been initiated. During this time, the accounts payable balance of the settlement sender will be inconsistent with the accounts receivable balance of the settlement recipient. - -Many factors may result in these inconsistencies: +The purpose of the settlement engine interface is to enable automated settlement and accounting. Settlement engines are not designed to enforce or guarantee that counterparties settle their liabilities or honor incoming payments. Accordingly, malicious counterparties or peers operating incompatible settlement engines break any settlement symmetry. #### Settlement delay -Settlement engine implementations MAY have settlement delay, or time until an instructed settlement is credited by the counterparty, due to network latency between peers or the time to finalize settlements on an underlying ledger or system. - -#### Failed outgoing settlements - -If an outgoing settlement fails, the settlement engine MUST track the amount of the settlement to retry it later. - -An intentional design decision was to hide failures from the accounting system rather than refunding failed settlements back to the accounts payable. Since the settlement engine tracks these failed amounts, if the conditions change so the settlement engine is later able to settle, a new settlement attempt may safely begin immediately. - -Refunding failed settlements would enable the peer's accounting system balances to appear synchronized. However, if settlement continues to fail, the credit limit would eventually be breached and prevent the peers from transacting, negating this utility. - -#### Uncredited incoming settlements +When the accounting system triggers a settlement, the accounting system preemptively debits the accounts payable balance before any settlement has been initiated. During this time, the accounts payable balance of the settlement sender will be inconsistent with the accounts receivable balance of the settlement recipient. -After the settlement engine requests the accounting system to credit an incoming settlement, if the accounting system responds that it only credited a partial amount (due to lesser precision), the settlement engine MUST track the uncredited leftover amount. +Settlement engine implementations incur settlement delay, or time until an instructed settlement is credited by the counterparty, due to network latency between peers or the time to finalize settlements on an underlying ledger or system. -If the request fails after retrying per the [idempotence behavior](#idempotence), the settlement engine MUST track the uncredited amount to retry later. +#### Retrying failed settlements -When a subsequent settlement is received, the settlement engine MUST request the accounting system to credit a new incoming settlement for the total amount yet to be credited, including the leftover, uncredited amount(s). +If an outgoing settlement fails, such as due to network connectivity issues, the settlement engine MUST track the amount of the settlement and retry it later. -#### Operating inconsistences +An intentional design decision was to hide failures from the accounting system rather than refunding failed settlements back to the accounts payable. Since the settlement engine tracks these failed amounts, if the settlement engine is able to settle later, a new settlement attempt may safely begin immediately. -Operators and external factors outside the control of the settlement engine implementation may also cause inconsistences, such as network connectivity, peers operating incompatible settlement engines, or malicious peers. +Refunding failed settlements would enable the peer's accounting system balances to appear synchronized. However, if settlement continues to fail, the credit limit would eventually be breached and prevent the peers from transacting, negating this utility. -## Exchanging messages +### Exchanging messages In order to settle or receive settlements with a peer, a settlement engine may first need to retrieve or communicate information with the peer's settlement engine. Two peered settlement engine instances may send and receive settlement-related messages among themselves, such as identifiers for their ledger accounts. @@ -177,13 +135,13 @@ Interledger connectors use a transport, such as HTTP or WebSockets, to send and - `message`: _Determined by connector_ - `data`: _Response message from recipient settlement engine, or empty if antecedent failure_ -## Accounts and identifiers +### Accounts and identifiers Each account MUST be identified by a unique, [URL-safe](https://tools.ietf.org/html/rfc3986#section-2.3) string, immutable for the lifetime of the account. The settlement engine MUST be responsible for correlating an account identifier to the peer's identity on the shared ledger or settlement system, if required. For separation of concerns between clearing and settlement, the accounting system is NOT RECOMMENDED to have knowledge of the peer's identity on the shared settlement system. -## Units and quantities +### Units and quantities Asset amounts may be represented using any arbitrary denomination. For example, one U.S. dollar may be represented as \$1 or 100 cents, each of which is equivalent in value. Likewise, one Bitcoin may be represented as 1 BTC or 100,000,000 satoshis. @@ -197,11 +155,7 @@ For example, one cent represents an asset scale of 2 in the case of USD, whereas ### Selecting scales -TODO this is weird because we're not defining what settlements are -- what is "scale as its settlements" ? - -Settlement engines are RECOMMENDED to perform or fulfill requests with amounts denominated in the same scale as its settlements. - -Account balances within the accounting systems are RECOMMENDED to be denominated in the same scale as their corresponding settlement engine, but MAY use different ones. For example, micropayments may require more precision than can actually be settled, or databases may limit precision to less than the settlement system is capable of. +Account balances within the accounting system are RECOMMENDED to be denominated in a scale corresponding to the unit settlements are denominated in, but MAY use different ones. For example, micropayments may require more precision than can actually be settled, or databases may limit precision to less than the settlement system is capable of. ### **`Quantity`** object @@ -228,11 +182,7 @@ To represent $2.54 in units of cents, where the amount is a multiple of $0.01: ### Scale conversions -If the accounting system or settlement engine receives a request with a **[`Quantity`](#quantity-json-type)** denominated in a unit more precise than its unit, it MUST convert the quantity into its native unit. If so, the resulting amount MUST be rounded down before fulfilling the request. - -The response to the request MUST include the converted, rounded **[`Quantity`](#quantity-json-type)** used to fulfill the request, which MUST be less than or equal to the amount sent in the original request. (If the amount rounds down to 0, then the amount in the response would be 0.) - -Then, the system with the additional precision initiating the request MUST track the leftover sum so it may accumulate and be retried in subsequent requests. +If the settlement engine receives a request with a **[`Quantity`](#quantity-object)** denominated in a unit more precise than it is capable of settling, it MUST persist the leftover amount. The leftover amounts MUST be processed later after they accumulate to a unit feasible for settlement. ## Settlement Engine HTTP API @@ -281,7 +231,7 @@ HTTP/1.1 204 No-Content ### Perform outgoing settlement -Asynchronously send an outgoing settlement. The accounting system sends this request and [accounts for outgoing settlements](#account-for-outgoing-settlements). (TODO link to other SE behavior here) +Asynchronously send an outgoing settlement. The accounting system sends this request and [accounts for outgoing settlements](#account-for-outgoing-settlements). #### Request @@ -292,7 +242,7 @@ Content-Type: application/json Idempotency-Key: ``` -> **[`Quantity`](#quantity-json-type)** to settle +> **[`Quantity`](#quantity-object)** to settle #### Response @@ -301,8 +251,6 @@ HTTP/1.1 201 Created Content-Type: application/json ``` -> **[`Quantity`](#quantity-json-type)** enqueued to settle, which MUST be less than or equal to the quantity in the original request - ### Handle incoming message Process and respond to an incoming message from the peer's settlement engine. The connector sends this request when it receives an incoming settlement message from the peer, and returns the response message back to the peer. @@ -334,7 +282,7 @@ HTTP/2 is RECOMMENDED for performance reasons, although HTTP/1.1 MAY also be use ### Credit incoming settlement -[Account for an incoming settlement](#account-for-incoming-settlements). The settlement engine sends this request as it receives incoming settlements in in order to ensure [settlement symmetry](#settlement-symmetry). If the accounting system responds that it only credited a partial amount, the settlement engine [tracks the uncredited amounts](#track-uncredited-incoming-settlements). +[Account for an incoming settlement](#account-for-incoming-settlements). The settlement engine sends this request as it receives incoming settlements in in order to ensure [settlement symmetry](#settlement-symmetry). #### Request @@ -345,7 +293,7 @@ Content-Type: application/json Idempotency-Key: ``` -> **[`Quantity`](#quantity-json-type)** to be credited to the account as an incoming settlement +> **[`Quantity`](#quantity-object)** to be credited to the account as an incoming settlement #### Response @@ -354,8 +302,6 @@ HTTP/1.1 201 Created Content-Type: application/json ``` -> **[`Quantity`](#quantity-json-type)** credited to the account as an incoming settlement, which MUST be less than or equal to the quantity in the original request - ### Send outgoing message Send the message to the given peer's settlement engine and return its response. The connector handles [proxying the message](#exchanging-messages) through the peer's connector. @@ -391,7 +337,7 @@ Requests to settle or requests to credit incoming settlements MUST include an id This retry behavior ensures the client and server are eventually consistent and never perform any unsafe balance rollbacks that could result in lost funds or double payments. -If the client receives no response, an HTTP 5xx error, or an HTTP 409 Conflict error, the client MUST retry the request with the same idempotency key. If the response is a client error, such as another HTTP 4xx error, the client MUST rollback the balance update and NOT perform any subsequent retries. +If the client receives no response or an HTTP error, the client MUST retry the request with the same idempotency key. To prevent overwhelming the server, the client SHOULD exponentially backoff after each failed retry attempt and add random "jitter" to vary the retry interval. The maximum retry interval MUST be no greater than 1 hour. Clients also MUST retry indefinitely until they have received an acknowledgment from the server that the request was processed or failed. diff --git a/README.md b/README.md index 33361192..99da100a 100644 --- a/README.md +++ b/README.md @@ -53,6 +53,6 @@ The default encoding rules for Interledger protocols are the Canonical Octet Enc ## Ledger Layer -- **[00: Settlement Engines](0000-settlement-engines/0000-settlement-engines.md)** +- **[38: Settlement Engines](0038-settlement-engines/0038-settlement-engines.md)** Specifies an interface to send and receive payments across different settlement systems and ledgers. From 07cf997c7f2d56cef1fb62131ccccc181ad8ee23 Mon Sep 17 00:00:00 2001 From: Kincaid O'Neil Date: Sat, 19 Oct 2019 19:59:02 -0400 Subject: [PATCH 30/32] fix: add settlement engine link to website sidebar --- tmpl/rfc.ejs.html | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tmpl/rfc.ejs.html b/tmpl/rfc.ejs.html index 7733eea5..6ea93777 100644 --- a/tmpl/rfc.ejs.html +++ b/tmpl/rfc.ejs.html @@ -84,10 +84,11 @@
  • Simple Payment Setup Protocol (SPSP)
  • Notes on OER encoding
  • Dynamic Configuration Protocol (ILDCP)
    • +
  • Settlement Engines
  • Peering, Clearing and Settlement
  • ILP over HTTP
  • SPSP Pull Payments
    • -
  •  
    • +
  •  
  • Payment Pointers
  • Web Monetization
  • W3C Web Payments
    • From c74b71a7a44a58a2652cdf640a2c73646a3a2b6d Mon Sep 17 00:00:00 2001 From: Kincaid O'Neil Date: Sat, 19 Oct 2019 21:10:31 -0400 Subject: [PATCH 31/32] docs(se): minor edits --- .../0038-settlement-engines.md | 40 +++++++++---------- 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/0038-settlement-engines/0038-settlement-engines.md b/0038-settlement-engines/0038-settlement-engines.md index 7bb645d0..1ed9c160 100644 --- a/0038-settlement-engines/0038-settlement-engines.md +++ b/0038-settlement-engines/0038-settlement-engines.md @@ -53,7 +53,7 @@ Settlement engines supercede the [Ledger Plugin Interface (LPIv2)](../deprecated ### Accounting -Connectors are RECOMMENDED to record the effects of their transactions in an **accounting system**. In this financial accounting context, an **account** represents amounts received (credits) and amounts owed (debits) for a set of transactions between counterparties. The **balance** of an account is the net difference between these credits and debits. All the balances and transactions of an account are denominated in a single, fungible asset. +In a financial accounting context, an **account** represents amounts received (credits) and amounts owed (debits) for a set of transactions between counterparties. The **balance** of an account is the net difference between these credits and debits. All the balances and transactions of an account are denominated in a single, fungible asset. Interledger connectors are RECOMMENDED to operate an **accounting system** which keeps a record of two accounts for each peer: @@ -64,15 +64,15 @@ Interledger connectors are RECOMMENDED to operate an **accounting system** which - Positive amount indicates its peer is indebted to the connector (an _asset_ to the connector). - Negative amount indicates its peer has sent a pre-payment to the connector. -Thus, a given connector's "accounts payable" balance should mirror its peer's "accounts receivable" balance. Likewise, a connector's "accounts receivable" balance should mirror its peer's "accounts payable" balance. +Thus, a given connector's accounts payable balance should mirror its peer's accounts receivable balance. Likewise, a connector's accounts receivable balance should mirror its peer's accounts payable balance. -Together, a settlement engine and an accounting system interface with one another to perform double-entry bookkeeping with eventual consistency. To ensure accurate, balanced double-entry bookkeeping, settlement engine and accounting system implementations MUST enforce several invariants. +Together, a settlement engine and an accounting system interface with one another to perform double-entry bookkeeping with eventual consistency. To ensure accurate, balanced double-entry bookkeeping, settlement engine and accounting system implementations MUST enforce several invariants: #### Account for outgoing settlements -The accounting system is responsible for triggering outgoing settlements. For example, when the accounts payable reaches a particular threshold, the accounting system SHOULD trigger a settlement to reduce the amount owed to the peer to a predefined, lesser amount in order to be able to continue transacting with the peer. +The accounting system is responsible for triggering outgoing settlements. For example, when the accounts payable reaches a particular threshold, the accounting system could trigger a settlement reducing the amount owed to the peer to a predefined, lesser amount. -When the accounting system triggers a settlement, it MUST debit the accounts payable, subtracting the amount of the settlement before sending the request to the settlement engine to settle the amount. +When the accounting system triggers a settlement, it MUST debit the accounts payable, subtracting the amount of the settlement, and then send a request to the settlement engine to settle the amount. #### Account for incoming settlements @@ -84,13 +84,13 @@ The fundamental expected behavior of a settlement engine implementation is the s As long as the instructed settlements do not equal the acknowledged settlements, the double-entry bookkeeping is out-of-balance. Settlement engines SHOULD minimize the time that the bookkeeping is unbalanced. -The purpose of the settlement engine interface is to enable automated settlement and accounting. Settlement engines are not designed to enforce or guarantee that counterparties settle their liabilities or honor incoming payments. Accordingly, malicious counterparties or peers operating incompatible settlement engines break any settlement symmetry. +The purpose of the settlement engine interface is to enable and account for automated settlements. Settlement engines are not designed to enforce or guarantee that counterparties settle their liabilities or honor incoming payments. Accordingly, malicious counterparties or peers operating incompatible settlement engine implementations break any settlement symmetry. #### Settlement delay When the accounting system triggers a settlement, the accounting system preemptively debits the accounts payable balance before any settlement has been initiated. During this time, the accounts payable balance of the settlement sender will be inconsistent with the accounts receivable balance of the settlement recipient. -Settlement engine implementations incur settlement delay, or time until an instructed settlement is credited by the counterparty, due to network latency between peers or the time to finalize settlements on an underlying ledger or system. +Settlement engine implementations necessarily incur settlement delay, or time until an instructed settlement is credited by the counterparty, due to network latency between peers or the time to finalize settlements on an underlying ledger or system. #### Retrying failed settlements @@ -106,7 +106,7 @@ In order to settle or receive settlements with a peer, a settlement engine may f To support multiple interoperable settlement engine implementations for a particular settlement system, implementors may standardize the schema and type of messages their settlement engines use to communicate with one another. This work is out-of-scope of this RFC. -Interledger connectors use a transport, such as HTTP or WebSockets, to send and receive data with peers. Settlement engine implementations SHOULD proxy all messages through its Interledger connector's existing transport like so: +Interledger connectors use network transports, such as HTTP or WebSockets, to send and receive ILP packets with peers. Settlement engine implementations SHOULD proxy all messages through its Interledger connector's existing transport like so: 1. Origin settlement engine sends a request to its connector with the settlement-related message to forward. 2. Origin connector encodes the raw message within an ILP Prepare packet (described below), which is sent to the peer's connector using its existing transport. @@ -115,7 +115,7 @@ Interledger connectors use a transport, such as HTTP or WebSockets, to send and 5. Peer connector sends the response message back across the transport to the origin connector within an ILP Fulfill or ILP Reject, depending upon the code of the response (described below). If the peer connector was unable to process the request, it MUST respond with an ILP Reject. 6. Origin connector sends the response message back to the origin settlement engine. -### ILP Prepare +#### ILP Prepare - `amount`: `0` - `expiresAt`: _Determined by connector_ @@ -123,12 +123,12 @@ Interledger connectors use a transport, such as HTTP or WebSockets, to send and - `destination`: `peer.settle` - `data`: _Request message from sender settlement engine_ -### ILP Fulfill +#### ILP Fulfill - `fulfillment`: `0000000000000000000000000000000000000000000000000000000000000000` - `data`: _Response message from recipient settlement engine_ -### ILP Reject +#### ILP Reject - `code`: _Determined by connector from HTTP status of forwarded request_ - `triggeredBy`: `peer.settle` @@ -139,7 +139,7 @@ Interledger connectors use a transport, such as HTTP or WebSockets, to send and Each account MUST be identified by a unique, [URL-safe](https://tools.ietf.org/html/rfc3986#section-2.3) string, immutable for the lifetime of the account. -The settlement engine MUST be responsible for correlating an account identifier to the peer's identity on the shared ledger or settlement system, if required. For separation of concerns between clearing and settlement, the accounting system is NOT RECOMMENDED to have knowledge of the peer's identity on the shared settlement system. +The settlement engine MUST be responsible for correlating an account identifier to the peer's identity on the shared ledger or settlement system, if required. To prevent tight coupling, the accounting system is NOT RECOMMENDED to have knowledge of the peer's identity on the shared settlement system. ### Units and quantities @@ -153,15 +153,15 @@ An **asset scale** is the difference in orders of magnitude between the standard For example, one cent represents an asset scale of 2 in the case of USD, whereas one satoshi represents an asset scale of 8 in the case of Bitcoin. -### Selecting scales +#### Selecting scales -Account balances within the accounting system are RECOMMENDED to be denominated in a scale corresponding to the unit settlements are denominated in, but MAY use different ones. For example, micropayments may require more precision than can actually be settled, or databases may limit precision to less than the settlement system is capable of. +Account balances in the accounting system SHOULD be denominated in a scale corresponding to the unit settlements are denominated in, but MAY be denominated in a different scale. For example, micropayments may require more precision than can actually be settled, or databases may limit precision to less than the settlement system is capable of. -### **`Quantity`** object +#### **`Quantity`** object An amount denominated in some unit of a single, fungible asset. (Since each account is denominated in a single asset, the type of asset is implied.) -#### Attributes +##### Attributes - **`amount`** — string - Amount of the unit, which is a non-negative integer. @@ -169,7 +169,7 @@ An amount denominated in some unit of a single, fungible asset. (Since each acco - **`scale`** — number - Asset scale of the unit, between `0` and the maximum 8-bit unsigned integer, `255` (inclusive). -#### Example +##### Example To represent $2.54 in units of cents, where the amount is a multiple of $0.01: @@ -180,9 +180,9 @@ To represent $2.54 in units of cents, where the amount is a multiple of $0.01: } ``` -### Scale conversions +#### Scale conversions -If the settlement engine receives a request with a **[`Quantity`](#quantity-object)** denominated in a unit more precise than it is capable of settling, it MUST persist the leftover amount. The leftover amounts MUST be processed later after they accumulate to a unit feasible for settlement. +If the settlement engine receives a request with a **[`Quantity`](#quantity-object)** denominated in a unit more precise than it is capable of settling, it MUST persist the leftover amount. The leftover amounts MUST be settled later after they accumulate to a unit feasible for settlement. ## Settlement Engine HTTP API @@ -203,7 +203,7 @@ Content-Type: application/json ```json { - "id": + "id": } ``` From 3a4b5285d70d87e950e1e0fb65270c151221489d Mon Sep 17 00:00:00 2001 From: Kincaid O'Neil Date: Sat, 19 Oct 2019 21:12:46 -0400 Subject: [PATCH 32/32] docs(se): fix create account json payload --- 0038-settlement-engines/0038-settlement-engines.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/0038-settlement-engines/0038-settlement-engines.md b/0038-settlement-engines/0038-settlement-engines.md index 1ed9c160..d8e47a9f 100644 --- a/0038-settlement-engines/0038-settlement-engines.md +++ b/0038-settlement-engines/0038-settlement-engines.md @@ -203,7 +203,7 @@ Content-Type: application/json ```json { - "id": + "id": "" } ```