[WIP] High-level service planning

[hrharder]

Overview

This issue will serve as a place to discuss and outline the high-level plans for the various services required to implement the Zaidan Dealer JSONRPC.

Description

1. Order book manager
   • Maintain in-memory order book data for CEX's and serve over GRPC
2. Hot wallet
   • Only service with hot-wallet mnemonic loaded
   • Signs and submits Ethereum transactions (doesn't necessarily prepare them)
   • Signs messages as needed
   • Provides gRPC API for token balances, allowances, and transfer functionality
3. Exchange manager
   • Only service that holds exchange API keys
   • Posts orders, cancels orders, serves gRPC API for other services
   • Provides query functionality for open orders,
4. Inventory manager
   • provides unified private API to manage "inventory" via Exchange manager and Hot wallet
5. Maker
6. Hedger
7. (Market watcher?)

[mtmuehl]

High-level plan for services responsible for market-making logic:

1. Maker

• Responsible for servicing quote requests, decides if a quote should be serviced and provides a price
• Pricing based on current order book state, market signals provided by 'Market Watcher' (tentative name), outstanding quotes and orders, taker profile, current exchange order states
• All the stateful information for these calculations should be stored in redis or in-memory in other relevant services that can be called on, pricing functions static within maker

2. Hedger

• Responsible for managing exchange strategies
• First iteration will be based on 'time buckets' for scalability, ie runs strategy based on orders in last x s/ms
• Relevant stateful information similar to Maker

3. Market Watcher

• Responsible for extracting market signals that can be used by other services
• Start with just short term volatility, but over time this service should grow

4. Taker Profiler (?)

• For now this functionality is just a blacklist so doesn't need an actual service running, but over time I think we want to move towards more dynamic taker profiling. Might make sense to make a whole service for this, but less sure on this one

[hrharder]

This comment attempts to outline the responsibility of each service in a typical user interaction.

1. Dealer receives quote request over public JSONRPC
   • performs basic quote request verification
   • creates DB reference for quote request and assigns a UUID (quoteId)
   • calls the maker via gRPC with quoteId (and/or request body)
2. Maker receives quote request over private gRPC
   • ensures requested maker and taker assets are supported
   • requests relevant order book data from order book manager
   • generates price quote by filling in either makerAssetSize or takerAssetSize
   • performs additional verification of price quote based on risk checks (balance, price, etc.)
   • uses signals from market watcher (or OBM) to influence quote (expiration, premium, etc.)
     • significant additional clarity needed on this step
   • updates quote DB reference with filled in size, updates status
   • indicates to the dealer the quote has been generated and is valid (or not)
3. Dealer delegates generation of the 0x order to the hot wallet
   • hot wallet generates and signs 0x order (w/ restricted senderAddress) based on quote parameters
   • one of the two will happen:
     • hot wallet stores signed order in DB and indicates success (or failure) to dealer
     • hot wallet sends signed order over the wire to dealer w/ gRPC
   • dealer OR hot wallet associates the order hash with the quoteId
4. Dealer sends quote (incl. signed order, 0x TX data, and/or hash) to client
5. [THIS STEP TAKES PLACE OUTSIDE DEALER IMPL.] Client decides to execute quote
   • Signs order or 0x transaction data according to the 0x specification and ZEIP-18
   • Submits signature, and quoteId back to dealer over public JSONRPC
6. Dealer receives fill request over JSONRPC
   • reflects status change (from "outstanding quote" to "fill requested") in DB
   • indicates to maker a request to fill has been received for the quoteId
7. Maker performs verification of request-to-fill for the given quoteId
   • checks quote expiration (distinct from order expiration)
   • checks any additional "soft-cancel" parameters (if applicable)
   • indicates to dealer one of the following:
     • the quote w/ quoteId is valid and SHOULD be executed
     • the quote w/ quoteId is NOT valid and SHOULD NOT be executed
   • maker or dealer reflects quote status change in DB; either:
     • quote validated (and passes) and approved for settlement
     • quote validated (and fails) and settlement is rejected
   • in the case of a rejection, this cycle exits early and the dealer informs the client of rejection
   • maker indicates completed verification (either outcome) to dealer over gRPC
8. Dealer submits fill request to hot wallet for final validation and submission
   • (all further steps assume maker approves fill for execution in prev. step)
   • dealer submits request-to-execute to hot wallet
     • there is a possibility this step will be executed in a dedicated (and distinct) service
   • hot wallet performs validation of fill; including:
     • validating taker has sufficient ERC-20 proxy allowances for the takerAsset
     • ensure taker has sufficient balance for the takerAsset
     • the fill will be executed in a timely manner; namely, that it will not expire during confirmation
   • hot wallet submits transaction calling executeTransaction on the 0x v3 exchange contract
   • following submission of fill transaction (and generation of a transaction hash), either:
     • hot wallet reflects status change ("fill approved" to "fill submitted") and stores TX ID
     • hot wallet reflects submission status to dealer over gRPC and dealer updates status in DB
9. Order watcher is alerted of submitted fill, and begins "watching" the transaction for status changes
   • upon inclusion of the (Ethereum) transaction in a block, order watcher indicates status change to maker over gRPC
   • maker process message from order watcher and:
     • in the case of a reverted transaction, marks the status as "fill failed"
     • in the case of successful transaction, marks the status as "filled; un-hedged"
   • in the case of a successful fill, the maker alerts the hedger of the fill over gRPC
     • an alternative would be the hedger polls the DB for quotes that need to be hedged
10. Hedger takes appropriate action to handle successful fill
    • @mtmuehl would be great if you can add some color here
    • in the case a hedge trade is necessary, hedger requests for an order to be placed on a supported CEX via the exchange manager
      • the specific service responsible to authenticated exchange interaction has not been finalized

---

Comments

• @mtmuehl @gchaincl @L-Kov @Freydal please review and comment with any questions
• there is plenty of specificity that is missing here
• additionally, please jump in to add specificity at steps that are lacking or incomplete
  • we also must begin thinking about things like the DB model, which to use, etc.

[L-Kov]

Thanks for writing this up @hrharder, I don't have any significant issues with this high level definition.

I have do have one question though. It specifically relates to the following bullet but it does seem applicable at other points:

hot wallet reflects status change ("fill approved" to "fill submitted") and stores TX ID

How would Dealer be made aware of a change to the status of the order so that they can respond correctly? Is the dealer essentially responsible for actively monitoring updates? It might make sense to make this more explicitly event driven (e.g. the second option that you referenced).

[qustavo]

@L-Kov:

hot wallet reflects submission status to dealer over gRPC and dealer updates status in DB

As I understand it, the Hot Wallet notifies the dealer right afer updating the database

[qustavo]

I made a small drawing to picture what's going on, some of the details are missing,
but overall it helped me to understand, here are some observations:

• The Dealer service is the central piece that orchestrates the process
• All the communication from the outside happens over JSON/RPC, all the local in-service communications happen over gRPC.
• Dealer stores quotes on the DB, quotes are updated by maker and HW
• HW stores signed Tx on the DB.
• We are using the pattern: 1) Update on DB, 2) Notify Dealer

Questions:

• What's the difference between

7.3:

indicates to dealer one of the following:

and 7.6:

maker indicates completed verification (either outcome) to dealer over gRPC

• Who alerts the Order Watcher? I presume is the Dealer, can you confirm?

9:

Order watcher is alerted of submitted fill, and begins "watching"

• What would be the implications of delegating all the DB access to the Dealer service?

[mtmuehl]

I want to change the end stage of our order processing in v2. Since the Hedger will be operating over epochs, I'd rather store the filled orders in redis, which at epoch end will be accessed by the Hedger and cleared. This eliminates grpc dependency and also allows epoch status to be used in pricing, e.g. if we have a lot of sells on a given pair during an epoch that hasn't been handled yet, we could offer better prices on the other side so we could match against ourselves.

[hrharder]

@L-Kov

How would Dealer be made aware of a change to the status of the order so that they can respond correctly?

@gchaincl is correct -- the idea is this would be a request/response pattern over gRPC where the hot-wallet (or whatever other service is taking action). Because most stateful data will be in a DB the messages can be very simple -- likely just including the relevant quote ID and a status code. We will certainly need some more clarity around this soon.

[hrharder]

@gchaincl
Not sure I understand what you're asking with regard to the difference between 7.3 and 7.6, can you clarify?

Who alerts the Order Watcher? I presume is the Dealer, can you confirm?

That is correct -- right now my thinking is the dealer will alert the order-watcher that a new transaction has been submitted and must be monitored. We could potentially make this the responsibility of whatever service actually submits the Tx.

What would be the implications of delegating all the DB access to the Dealer service?

I'm not sure -- this might be a good idea however. One thing I was not a huge fan of in the current system is how many services access the DB.

Personally I like the pattern of the dealer being this central service and orchestrator, in which case it seems to make sense for the dealer to do most of the (or all of) the DB interactions. What do you think?

[hrharder]

@mtmuehl

Since the Hedger will be operating over epochs, I'd rather store the filled orders in redis.

This sounds fine -- however, I'm personally in the boat that we should move away from redis as a catch-all DB, and only use it as a cache.

I would like to keep one main table in something like SQL or similar where each entry is a "quote", but it stores all relevant information, including the hedge trade.

In your mind, is redis the key word here, or rather just storing the filled orders in /some/ DB (not necessarily redis)?

[mtmuehl]

@hrharder

Yes although in this case the function is very 'cache-like' so I think redis might be the best choice. Since this is being accessed by the service we'd like to performance optimize the most whatever the fastest read option is should work

[hrharder]

@mtmuehl

Sounds good -- as long as any relevant data from hedge trades eventually gets stored in the persistent DB I think that is okay.

For example, the timestamp, exchange order ID, size/price/side of the hedge trade should be associated with the quote ID after we're done with it. Would you agree?

[qustavo]

Personally I like the pattern of the dealer being this central service and orchestrator, in which case it seems to make sense for the dealer to do most of the (or all of) the DB interactions. What do you think?

I agree. We can expose gRPC methods so that HotWallet and Maker can update the DB via Dealer

[hrharder]

Perfect.