Feat/XRPL connector v2 #128

mlguys · 2023-06-08T17:36:38Z

Before submitting this PR, please make sure:

Your code builds clean without any errors or warnings
You are using approved title ("feat/", "fix/", "docs/", "refactor/")

A description of the changes proposed in the pull request:

This PR reworks the Ripple DEX connector so that It implements the CLOBish class
Add a brand new order tracking system to help tracking and persisting trade orders submitted to Ripple DEX

Tests performed by the developer:

Tested posting and cancelling orders as integration tests
Tested tracking the state of orders as integration tests
Tested the CLOB Spot endpoints for Ripple DEX as routes tests
Tested using HB client to connect to the connector and trade on 2 markets USD/VND and XRP/VND on testnet

Tips for QA testing:

Bounty connected to this PR: Ripple DEX/Chain connector #103
Client PR connected to this PR: Feat/XRPL connector v2 hummingbot#6535
Proposal: https://snapshot.org/#/hbot-ip.eth/proposal/0x07b027fc420274d26add346ed65a2f7e3e5a662bd5317d2ddb2dd02562a7b2d2
For end-to-end tests, please contact me for getting test wallets on testnet
Link to trade USD/VND and XRP/VND on XRP DEX on testnet:

- USD/VND testnet book:
https://sologenic.org/trade?n[etwork=test](https://sologenic.org/trade?market=USD%2Brh8LssQyeBdEXk7Zv86HxHrx8k2R2DBUrx%2FVND%2Brh8LssQyeBdEXk7Zv86HxHrx8k2R2DBUrx)net&market=USD%2Brh8LssQyeBdEXk7Zv86HxHrx8k2R2DBUrx%2FVND%2Brh8LssQyeBdEXk7Zv86HxHrx8k2R2DBUrx

- XRP/VND testnet book:
https://sologenic.org/trade?n[etwork=](https://sologenic.org/trade?market=USD%2Brh8LssQyeBdEXk7Zv86HxHrx8k2R2DBUrx%2FVND%2Brh8LssQyeBdEXk7Zv86HxHrx8k2R2DBUrx)testnet&market=USD%2Brh8LssQyeBdEXk7Zv86HxHrx8k2R2DBUrx%2FVND%2Brh8LssQyeBdEXk7Zv86HxHrx8k2R2DBUrx

To trade on other markets, please contact me for how to do that as you will need to update connector config

Tests added:

Chain endpoint tests: test/chains/xrpl
Connector endpoint tests: test/connectors/xrpl

Test coverage:
Passed all tests on 22 of August using this command:

rm -rf ./db/xrpl.level && NODE_OPTIONS=--max_old_space_size=10240 yarn jest --runInBand --coverage --runTestsByPath ./test/chains/xrpl/*.test.ts ./test/connectors/xrpl/*.test.ts

Connector endpoints coverage:

Chain endpoints coverage:

nikspz · 2023-06-15T14:07:14Z

Steps:

Cloned and installed feature branch and Client development
Connected testnet XRPL wallet
Check balance command

Actual:

2023-06-15 13:48:44,687 - 847867 - hummingbot.core.utils.async_utils - ERROR - Unhandled error in background task: 'NoneType' object is not subscriptable
Traceback (most recent call last):
  File "/home/ubuntu/development/hummingbot/core/utils/async_utils.py", line 9, in safe_wrapper
    return await c
  File "/home/ubuntu/development/hummingbot/client/command/balance_command.py", line 84, in show_balances
    all_ex_bals = await asyncio.wait_for(
  File "/home/ubuntu/miniconda3/envs/hummingbot/lib/python3.8/asyncio/tasks.py", line 483, in wait_for
    return fut.result()
  File "/home/ubuntu/development/hummingbot/user/user_balances.py", line 140, in all_balances_all_exchanges
    await self.update_exchanges(client_config_map)
  File "/home/ubuntu/development/hummingbot/user/user_balances.py", line 136, in update_exchanges
    results = await safe_gather(*tasks)
  File "/home/ubuntu/development/hummingbot/core/utils/async_utils.py", line 22, in safe_gather
    return await asyncio.gather(*args, **kwargs)
  File "/home/ubuntu/development/hummingbot/user/user_balances.py", line 110, in update_exchange_balance
    return await self.add_exchange(exchange_name, client_config_map, **api_keys)
  File "/home/ubuntu/development/hummingbot/user/user_balances.py", line 86, in add_exchange
    market = UserBalances.connect_market(exchange, client_config_map, **api_details)
  File "/home/ubuntu/development/hummingbot/user/user_balances.py", line 25, in connect_market
    init_params = conn_setting.conn_init_parameters(
  File "/home/ubuntu/development/hummingbot/client/settings.py", line 263, in conn_init_parameters
    connector_name=connector_spec["connector"],
TypeError: 'NoneType' object is not subscriptable

nikspz · 2023-06-21T14:03:38Z

@mlguys Could you please add any updates on this PR?

nikspz · 2023-06-27T11:51:28Z

@mlguys Could you please resolve branch conflicts?

nikspz · 2023-06-28T14:54:34Z

hi @mlguys.
Please be informed about hummingbot ongoing migration to python3.10 this week.
The new PR referring to the hummingbot/hummingbot#6389 will be created and merged before all the connectors PRs to prevent connectors conflicts with python3.10 environment

mlguys · 2023-07-03T09:26:48Z

Hey @nikspz, sorry for being unresponsive. I've been busy fixing the order tracker logic since I found some bugs during making integration tests. Fortunately I have resolved the logic issue and pushed integration tests which should cover more than 75% cov of connector code.

I will now proceed to fix the bug you mentioned above and make sure this PR is conflict free.

mlguys · 2023-08-29T17:40:05Z

Hey guys, could you re-run the unit tests again? I have pushed fixes to address the test errors.

fengtality

See prelim comments

docs/swagger/definitions.yml

docs/swagger/xrpl-routes.yml

nikspz · 2023-10-06T15:45:41Z

Tests performed:
- cloned and installed XRPL connector: Client PR6535 + gateway PR128
- connected testnet succesfully
- created pureMM startegy using USD/VND pair
- started the strategy
- review order creation is inconsistent (logs fails to show orders not created on the exchange/ Client strategy created one order instead of pair of two, when balance is sufficient to place both)

gw128.zip
logs_puremm1.log
logs_gateway_app-xrpl.log

mlguys · 2023-10-11T07:13:27Z

Hey @nikspz, thanks for testing, currently I'm busy with moving to new home so I'm unable to work on this yet. Will resolve this next week.

nikspz · 2023-10-23T14:36:26Z

hi @mlguys Please give updates on this bounty when available

nikspz · 2023-10-30T12:02:03Z

@mlguys if you have any updates please post here

mlguys · 2023-10-31T02:53:03Z

Hey @nikspz, I'm still working on this but due to tight schedule with other connector and limited work hours during my moving to another living space, I'm unable to finish this yet. However, I get more time for this week so I will push this, hopefully will be able to push some fixes by today.

nikspz · 2023-11-08T14:14:55Z

hi @mlguys Could you please provide updates and fix branch conflicts?

mlguys · 2023-11-10T16:48:13Z

Hey @nikspz, I've resolved the conflicts and added fix to your issue, could you check it?

nikspz · 2023-11-13T21:03:52Z

Tests performed:
- cloned and installed XRPL connector: Client PR6535 + gateway PR128
- connected testnet succesfully
- created pureMM startegy using USD/VND pair
- started the strategy
- review order creation is inconsistent (Client failed to created orders for both sides, created only one)

xrpl.zip
logs_xrp4test.log

2023-11-13 20:48:43,407 - 3824428 - hummingbot.client.hummingbot_application - INFO - start command initiated.
2023-11-13 20:48:43,511 - 3824428 - hummingbot.connector.gateway.clob_spot.gateway_clob_spot.GatewayCLOBSPOT - INFO - Network status has changed to NetworkStatus.CONNECTED. Starting networking...
2023-11-13 20:48:43,931 - 3824428 - hummingbot.core.data_type.order_book_tracker - INFO - Initialized order book for USD-VND. 1/1 completed.
2023-11-13 20:48:45,001 - 3824428 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Creating 1 bid orders at (Size, Price): ['4 USD, 21532.5 VND']
2023-11-13 20:48:45,002 - 3824428 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Creating 1 ask orders at (Size, Price): ['4 USD, 21967.5 VND']
2023-11-13 20:48:45,801 - 3824428 - hummingbot.core.data_type.order_book_tracker - NETWORK - Unexpected error while fetching last trade price.
Traceback (most recent call last):
  File "/home/nikita/xrpl6535/hummingbot/core/data_type/order_book_tracker.py", line 160, in _update_last_trade_prices_loop
    last_prices = await self._data_source.get_last_traded_prices(**args)
  File "/home/nikita/xrpl6535/hummingbot/connector/gateway/clob_spot/gateway_clob_api_order_book_data_source.py", line 26, in get_last_traded_prices
    last_traded_prices = {
  File "/home/nikita/xrpl6535/hummingbot/connector/gateway/clob_spot/gateway_clob_api_order_book_data_source.py", line 27, in <dictcomp>
    trading_pair: float(await self._api_data_source.get_last_traded_price(trading_pair=trading_pair))
  File "/home/nikita/xrpl6535/hummingbot/connector/gateway/clob_spot/data_sources/xrpl/xrpl_api_data_source.py", line 196, in get_last_traded_price
    ticker_data = await self._get_ticker_data(trading_pair=trading_pair)
  File "/home/nikita/xrpl6535/hummingbot/connector/gateway/clob_spot/data_sources/xrpl/xrpl_api_data_source.py", line 346, in _get_ticker_data
    return ticker_data["markets"][trading_pair]
TypeError: list indices must be integers or slices, not str
2023-11-13 20:48:46,348 - 3824428 - hummingbot.core.rate_oracle.rate_oracle - INFO - Network status has changed to NetworkStatus.CONNECTED. Starting networking...
2023-11-13 20:48:50,054 - 3824428 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp": 1699908530.0, "order_id": "0x0000000000000000000998584e8f07792895a60a1fb1e1ee4fefdb10a1468da2", "exchange_order_id": "33596625", "event_name": "OrderCancelledEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-13 20:48:50,054 - 3824428 - hummingbot.connector.client_order_tracker - INFO - Successfully canceled order 0x0000000000000000000998584e8f07792895a60a1fb1e1ee4fefdb10a1468da2.
2023-11-13 20:48:50,132 - 3824428 - hummingbot.connector.client_order_tracker - INFO - Created LIMIT SELL order 0x0000000000000000000998584e8f07792895a60a1fb1e1ee4fefdb10a1469001 for 4.000000000000000 USD-VND.
2023-11-13 20:48:50,176 - 3824428 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp": 1699908530.0, "type": "OrderType.LIMIT", "trading_pair": "USD-VND", "amount": "4.000000000000000", "price": "21967.500000000000000", "order_id": "0x0000000000000000000998584e8f07792895a60a1fb1e1ee4fefdb10a1469001", "creation_timestamp": 1699908525.0, "exchange_order_id": "33596626", "leverage": 1, "position": "NIL", "event_name": "SellOrderCreatedEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-13 20:49:16,430 - 3824428 - hummingbot.core.data_type.order_book_tracker - NETWORK - Unexpected error while fetching last trade price.
Traceback (most recent call last):
  File "/home/nikita/xrpl6535/hummingbot/core/data_type/order_book_tracker.py", line 160, in _update_last_trade_prices_loop
    last_prices = await self._data_source.get_last_traded_prices(**args)
  File "/home/nikita/xrpl6535/hummingbot/connector/gateway/clob_spot/gateway_clob_api_order_book_data_source.py", line 26, in get_last_traded_prices
    last_traded_prices = {
  File "/home/nikita/xrpl6535/hummingbot/connector/gateway/clob_spot/gateway_clob_api_order_book_data_source.py", line 27, in <dictcomp>
    trading_pair: float(await self._api_data_source.get_last_traded_price(trading_pair=trading_pair))
  File "/home/nikita/xrpl6535/hummingbot/connector/gateway/clob_spot/data_sources/xrpl/xrpl_api_data_source.py", line 196, in get_last_traded_price
    ticker_data = await self._get_ticker_data(trading_pair=trading_pair)
  File "/home/nikita/xrpl6535/hummingbot/connector/gateway/clob_spot/data_sources/xrpl/xrpl_api_data_source.py", line 346, in _get_ticker_data
    return ticker_data["markets"][trading_pair]
TypeError: list indices must be integers or slices, not str
2023-11-13 20:49:46,777 - 3824428 - hummingbot.core.data_type.order_book_tracker - NETWORK - Unexpected error while fetching last trade price.
Traceback (most recent call last):
  File "/home/nikita/xrpl6535/hummingbot/core/data_type/order_book_tracker.py", line 160, in _update_last_trade_prices_loop
    last_prices = await self._data_source.get_last_traded_prices(**args)
  File "/home/nikita/xrpl6535/hummingbot/connector/gateway/clob_spot/gateway_clob_api_order_book_data_source.py", line 26, in get_last_traded_prices
    last_traded_prices = {
  File "/home/nikita/xrpl6535/hummingbot/connector/gateway/clob_spot/gateway_clob_api_order_book_data_source.py", line 27, in <dictcomp>
    trading_pair: float(await self._api_data_source.get_last_traded_price(trading_pair=trading_pair))
  File "/home/nikita/xrpl6535/hummingbot/connector/gateway/clob_spot/data_sources/xrpl/xrpl_api_data_source.py", line 196, in get_last_traded_price
    ticker_data = await self._get_ticker_data(trading_pair=trading_pair)
  File "/home/nikita/xrpl6535/hummingbot/connector/gateway/clob_spot/data_sources/xrpl/xrpl_api_data_source.py", line 346, in _get_ticker_data
    return ticker_data["markets"][trading_pair]
TypeError: list indices must be integers or slices, not str

mlguys · 2023-11-14T07:21:34Z

Hey @nikspz, Thanks for testing. I've added a fix on hummingbot/hummingbot#6535 to fix your issue, could you check?

nikspz · 2023-11-14T17:56:54Z

Steps to reproduce:

cloned and installed XRPL connector: Client PR6535 + gateway PR128
connected testnet succesfully
created pureMM startegy using USD/VND pair
started the strategy

Actual:

Order creation is inconsistent (failed to created orders for both sides, one order cancelled immediately and only one created)
Available balance in status not updated (should be changed depending on order amount)

6635.zip
logs_xrp2test.log

2023-11-14 17:43:53,388 - 4125298 - hummingbot.client.hummingbot_application - INFO - start command initiated.
2023-11-14 17:43:53,532 - 4125298 - hummingbot.connector.gateway.clob_spot.gateway_clob_spot.GatewayCLOBSPOT - INFO - Network status has changed to NetworkStatus.CONNECTED. Starting networking...
2023-11-14 17:43:53,908 - 4125298 - hummingbot.core.data_type.order_book_tracker - INFO - Initialized order book for USD-VND. 1/1 completed.
2023-11-14 17:43:55,002 - 4125298 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Creating 1 bid orders at (Size, Price): ['3 USD, 19710.5 VND']
2023-11-14 17:43:55,003 - 4125298 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Creating 1 ask orders at (Size, Price): ['3 USD, 19789.5 VND']
2023-11-14 17:43:56,833 - 4125298 - hummingbot.core.rate_oracle.rate_oracle - INFO - Network status has changed to NetworkStatus.CONNECTED. Starting networking...
2023-11-14 17:44:00,189 - 4125298 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp": 1699983840.0, "order_id": "0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd1fe59f870", "exchange_order_id": "33596726", "event_name": "OrderCancelledEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-14 17:44:00,189 - 4125298 - hummingbot.connector.client_order_tracker - INFO - Successfully canceled order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd1fe59f870.
2023-11-14 17:44:00,194 - 4125298 - hummingbot.connector.client_order_tracker - INFO - Created LIMIT SELL order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd1fe59fbd9 for 3.000000000000000 USD-VND.
2023-11-14 17:44:00,218 - 4125298 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp": 1699983840.0, "type": "OrderType.LIMIT", "trading_pair": "USD-VND", "amount": "3.000000000000000", "price": "19789.500000000000000", "order_id": "0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd1fe59fbd9", "creation_timestamp": 1699983835.0, "exchange_order_id": "33596727", "leverage": 1, "position": "NIL", "event_name": "SellOrderCreatedEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-14 17:44:55,002 - 4125298 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Canceling the limit order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd1fe59fbd9. [clock=2023-11-14 17:44:55+00:00]
2023-11-14 17:45:00,090 - 4125298 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp": 1699983900.0, "order_id": "0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd1fe59fbd9", "exchange_order_id": "33596727", "event_name": "OrderCancelledEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-14 17:45:00,090 - 4125298 - hummingbot.connector.client_order_tracker - INFO - Successfully canceled order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd1fe59fbd9.
2023-11-14 17:45:01,001 - 4125298 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Creating 1 bid orders at (Size, Price): ['3 USD, 19710.5 VND']
2023-11-14 17:45:01,002 - 4125298 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Creating 1 ask orders at (Size, Price): ['3 USD, 19789.5 VND']
2023-11-14 17:45:10,131 - 4125298 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp": 1699983910.0, "order_id": "0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20249086f", "exchange_order_id": "33596729", "event_name": "OrderCancelledEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-14 17:45:10,132 - 4125298 - hummingbot.connector.client_order_tracker - INFO - Successfully canceled order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20249086f.
2023-11-14 17:45:10,137 - 4125298 - hummingbot.connector.client_order_tracker - INFO - Created LIMIT SELL order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd202490a93 for 3.000000000000000 USD-VND.
2023-11-14 17:45:10,189 - 4125298 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp": 1699983910.0, "type": "OrderType.LIMIT", "trading_pair": "USD-VND", "amount": "3.000000000000000", "price": "19789.500000000000000", "order_id": "0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd202490a93", "creation_timestamp": 1699983901.0, "exchange_order_id": "33596730", "leverage": 1, "position": "NIL", "event_name": "SellOrderCreatedEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-14 17:46:01,002 - 4125298 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Canceling the limit order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd202490a93. [clock=2023-11-14 17:46:01+00:00]
2023-11-14 17:46:05,118 - 4125298 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp": 1699983965.0, "order_id": "0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd202490a93", "exchange_order_id": "33596730", "event_name": "OrderCancelledEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-14 17:46:05,118 - 4125298 - hummingbot.connector.client_order_tracker - INFO - Successfully canceled order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd202490a93.
2023-11-14 17:46:06,002 - 4125298 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Creating 1 bid orders at (Size, Price): ['3 USD, 19710.5 VND']
2023-11-14 17:46:06,003 - 4125298 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Creating 1 ask orders at (Size, Price): ['3 USD, 19789.5 VND']
2023-11-14 17:46:15,102 - 4125298 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp": 1699983975.0, "order_id": "0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20628de4b", "exchange_order_id": "33596732", "event_name": "OrderCancelledEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-14 17:46:15,102 - 4125298 - hummingbot.connector.client_order_tracker - INFO - Successfully canceled order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20628de4b.
2023-11-14 17:46:15,106 - 4125298 - hummingbot.connector.client_order_tracker - INFO - Created LIMIT SELL order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20628e01a for 3.000000000000000 USD-VND.
2023-11-14 17:46:15,116 - 4125298 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp": 1699983975.0, "type": "OrderType.LIMIT", "trading_pair": "USD-VND", "amount": "3.000000000000000", "price": "19789.500000000000000", "order_id": "0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20628e01a", "creation_timestamp": 1699983966.0, "exchange_order_id": "33596733", "leverage": 1, "position": "NIL", "event_name": "SellOrderCreatedEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-14 17:47:06,007 - 4125298 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Canceling the limit order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20628e01a. [clock=2023-11-14 17:47:06+00:00]
2023-11-14 17:47:15,088 - 4125298 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp": 1699984035.0, "order_id": "0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20628e01a", "exchange_order_id": "33596733", "event_name": "OrderCancelledEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-14 17:47:15,088 - 4125298 - hummingbot.connector.client_order_tracker - INFO - Successfully canceled order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20628e01a.
2023-11-14 17:47:16,001 - 4125298 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Creating 1 bid orders at (Size, Price): ['3 USD, 19710.5 VND']
2023-11-14 17:47:16,001 - 4125298 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Creating 1 ask orders at (Size, Price): ['3 USD, 19789.5 VND']
2023-11-14 17:47:20,544 - 4125298 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp": 1699984040.0, "order_id": "0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20a54f7e3", "exchange_order_id": "33596735", "event_name": "OrderCancelledEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-14 17:47:20,544 - 4125298 - hummingbot.connector.client_order_tracker - INFO - Successfully canceled order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20a54f7e3.
2023-11-14 17:47:21,021 - 4125298 - hummingbot.connector.client_order_tracker - INFO - Created LIMIT SELL order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20a54f9ad for 3.000000000000000 USD-VND.
2023-11-14 17:47:21,034 - 4125298 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp": 1699984041.0, "type": "OrderType.LIMIT", "trading_pair": "USD-VND", "amount": "3.000000000000000", "price": "19789.500000000000000", "order_id": "0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20a54f9ad", "creation_timestamp": 1699984036.0, "exchange_order_id": "33596736", "leverage": 1, "position": "NIL", "event_name": "SellOrderCreatedEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-14 17:48:16,003 - 4125298 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Canceling the limit order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20a54f9ad. [clock=2023-11-14 17:48:16+00:00]
2023-11-14 17:48:20,045 - 4125298 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp": 1699984100.0, "order_id": "0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20a54f9ad", "exchange_order_id": "33596736", "event_name": "OrderCancelledEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-14 17:48:20,046 - 4125298 - hummingbot.connector.client_order_tracker - INFO - Successfully canceled order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20a54f9ad.
2023-11-14 17:48:21,001 - 4125298 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Creating 1 bid orders at (Size, Price): ['3 USD, 19710.5 VND']
2023-11-14 17:48:21,002 - 4125298 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Creating 1 ask orders at (Size, Price): ['3 USD, 19789.5 VND']
2023-11-14 17:48:21,747 - 4125298 - hummingbot.core.gateway.gateway_http_client - NETWORK - Server disconnected
Traceback (most recent call last):
  File "/home/nikita/xrpl6535/hummingbot/core/gateway/gateway_http_client.py", line 184, in api_request
    response = await client.get(url, params=params)
  File "/home/nikita/miniconda3/envs/hummingbot/lib/python3.10/site-packages/aiohttp/client.py", line 586, in _request
    await resp.start(conn)
  File "/home/nikita/miniconda3/envs/hummingbot/lib/python3.10/site-packages/aiohttp/client_reqrep.py", line 905, in start
    message, payload = await protocol.read()  # type: ignore[union-attr]
  File "/home/nikita/miniconda3/envs/hummingbot/lib/python3.10/site-packages/aiohttp/streams.py", line 616, in read
    await self._waiter
aiohttp.client_exceptions.ServerDisconnectedError: Server disconnected
2023-11-14 17:48:21,815 - 4125298 - hummingbot.core.gateway.gateway_http_client - WARNING - Call to https://localhost:15888/clob/ticker failed. See logs for more details.
2023-11-14 17:48:21,815 - 4125298 - hummingbot.core.data_type.order_book_tracker - NETWORK - Unexpected error while fetching last trade price.
Traceback (most recent call last):
  File "/home/nikita/xrpl6535/hummingbot/core/data_type/order_book_tracker.py", line 160, in _update_last_trade_prices_loop
    last_prices = await self._data_source.get_last_traded_prices(**args)
  File "/home/nikita/xrpl6535/hummingbot/connector/gateway/clob_spot/gateway_clob_api_order_book_data_source.py", line 26, in get_last_traded_prices
    last_traded_prices = {
  File "/home/nikita/xrpl6535/hummingbot/connector/gateway/clob_spot/gateway_clob_api_order_book_data_source.py", line 27, in <dictcomp>
    trading_pair: float(await self._api_data_source.get_last_traded_price(trading_pair=trading_pair))
  File "/home/nikita/xrpl6535/hummingbot/connector/gateway/clob_spot/data_sources/xrpl/xrpl_api_data_source.py", line 196, in get_last_traded_price
    ticker_data = await self._get_ticker_data(trading_pair=trading_pair)
  File "/home/nikita/xrpl6535/hummingbot/connector/gateway/clob_spot/data_sources/xrpl/xrpl_api_data_source.py", line 340, in _get_ticker_data
    ticker_data = await self._get_gateway_instance().get_clob_ticker(
  File "/home/nikita/xrpl6535/hummingbot/core/gateway/gateway_http_client.py", line 921, in get_clob_ticker
    resp = await self.api_request(method="get", path_url="clob/ticker", params=request_payload)
  File "/home/nikita/xrpl6535/hummingbot/core/gateway/gateway_http_client.py", line 222, in api_request
    raise e
  File "/home/nikita/xrpl6535/hummingbot/core/gateway/gateway_http_client.py", line 184, in api_request
    response = await client.get(url, params=params)
  File "/home/nikita/miniconda3/envs/hummingbot/lib/python3.10/site-packages/aiohttp/client.py", line 586, in _request
    await resp.start(conn)
  File "/home/nikita/miniconda3/envs/hummingbot/lib/python3.10/site-packages/aiohttp/client_reqrep.py", line 905, in start
    message, payload = await protocol.read()  # type: ignore[union-attr]
  File "/home/nikita/miniconda3/envs/hummingbot/lib/python3.10/site-packages/aiohttp/streams.py", line 616, in read
    await self._waiter
aiohttp.client_exceptions.ServerDisconnectedError: Server disconnected
2023-11-14 17:48:25,390 - 4125298 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp": 1699984105.0, "order_id": "0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20e34ca93", "exchange_order_id": "33596738", "event_name": "OrderCancelledEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-14 17:48:25,390 - 4125298 - hummingbot.connector.client_order_tracker - INFO - Successfully canceled order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20e34ca93.
2023-11-14 17:48:25,843 - 4125298 - hummingbot.connector.client_order_tracker - INFO - Created LIMIT SELL order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20e34cc96 for 3.000000000000000 USD-VND.
2023-11-14 17:48:25,869 - 4125298 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp": 1699984105.0, "type": "OrderType.LIMIT", "trading_pair": "USD-VND", "amount": "3.000000000000000", "price": "19789.500000000000000", "order_id": "0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20e34cc96", "creation_timestamp": 1699984101.0, "exchange_order_id": "33596739", "leverage": 1, "position": "NIL", "event_name": "SellOrderCreatedEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-14 17:49:21,002 - 4125298 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Canceling the limit order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20e34cc96. [clock=2023-11-14 17:49:21+00:00]
2023-11-14 17:49:30,088 - 4125298 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp": 1699984170.0, "order_id": "0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20e34cc96", "exchange_order_id": "33596739", "event_name": "OrderCancelledEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-14 17:49:30,089 - 4125298 - hummingbot.connector.client_order_tracker - INFO - Successfully canceled order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20e34cc96.

nikspz · 2023-11-27T14:09:21Z

hi @mlguys, kindly Please add your updates here

rapcmia

LGTM

Ran tests with the Feat/XRPL connector v2 hummingbot#6535
Most of tests are done on testnet
- Added orders on orderbook using account provided by dev to simulate a real exchange scenario
Added xrpl wallet successfully and balance display ok
Setup simple PMM strategy and observe connector:
- Create, cancel and fetch orders ok (failed fetch update happens but thats due to nature of exchange)
- Setup multiple order levels, all ok
- Filled order events ok

Note: Current version does have issues with balance not updated orders are created on testnet, a separate issue will be created as part of improvement for the connector. Mainnet has not been tested.

fengtality · 2023-11-30T20:59:44Z

LGTM!

rapcmia · 2023-12-01T10:31:52Z

Merged to development and part of release version 1.23.0

mlguys changed the title ~~Feat/XRPL connector v2 [WIP]~~ Feat/XRPL connector v2 Jun 8, 2023

rapcmia requested review from rapcmia and nikspz June 9, 2023 03:32

Danilo-Araujo-Silva and others added 22 commits August 22, 2023 19:39

Updating .gitignore.

bb9e7df

Add XRPL & XRPLDEX connector (v1)

df59dad

add unified clob swagger schema

8ed27ad

Fix swagger missing field error for get orderbook

f01cf29

update yarn lock

14d0e87

wip

b0a0bc1

WIP

22b585c

WIP

a820ae3

Add order tracker logic wip

f2a2bbc

Finish connecter and add routes test

b1411d7

Remove old implementation and disable comments

ad5704a

Remove artifacts

8ba3776

Move order tracker to chain directory

3f49fba

add space

b2b589e

WIP

056e212

Add basic e2e test scripts + fix bugs

0af5945

Finish order posting and cancelling scenario test

6609840

Finish fixing order tracking

1111b27

merge development + fix test cases

4547c7e

Add tracking fills

d7844c8

add orderhash to filldata

edc2d9c

fix balances chain endpoint for xrpl

1737d24

Merge branch 'development' into feat/xrpl-connector-v2

0c74069

fengtality requested changes Sep 2, 2023

View reviewed changes

docs/swagger/definitions.yml Outdated Show resolved Hide resolved

docs/swagger/xrpl-routes.yml Outdated Show resolved Hide resolved

mlguys added 2 commits September 7, 2023 20:29

remove swagger doc changes

ceec75f

Merge branch 'development' into feat/xrpl-connector-v2

ba86d93

mlguys added 3 commits November 10, 2023 13:35

Merge branch 'development' into feat/xrpl-connector-v2

6280dff

fix UNKNOW state due to double offer cancel

a1a1ef5

Merge branch 'development' into feat/xrpl-connector-v2

8a2c1e2

mlguys added 2 commits November 11, 2023 11:01

update test

b84f8d7

update yarn

6f6bd8f

Merge branch 'development' into feat/xrpl-connector-v2

d35d850

rapcmia approved these changes Nov 29, 2023

View reviewed changes

fengtality approved these changes Nov 30, 2023

View reviewed changes

rapcmia merged commit 81576ca into hummingbot:development Dec 1, 2023
3 checks passed

This was referenced Dec 11, 2023

fix/ Fix account balance not reflect correctly + Improve websocket connection #249

Merged

fix/ Fix account balance not reflect correctly + Improve websocket connection hummingbot/hummingbot#6708

Merged

nikspz mentioned this pull request Dec 22, 2023

Sync Gateway / Gateway staging -> master for Hummingbot version 1.23.0 #258

Merged

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feat/XRPL connector v2 #128

Feat/XRPL connector v2 #128

mlguys commented Jun 8, 2023 •

edited

Loading

nikspz commented Jun 15, 2023

nikspz commented Jun 21, 2023 •

edited

Loading

nikspz commented Jun 27, 2023

nikspz commented Jun 28, 2023 •

edited

Loading

mlguys commented Jul 3, 2023

mlguys commented Aug 29, 2023

fengtality left a comment

nikspz commented Oct 6, 2023

mlguys commented Oct 11, 2023

nikspz commented Oct 23, 2023

nikspz commented Oct 30, 2023

mlguys commented Oct 31, 2023

nikspz commented Nov 8, 2023

mlguys commented Nov 10, 2023

nikspz commented Nov 13, 2023

mlguys commented Nov 14, 2023

nikspz commented Nov 14, 2023 •

edited

Loading

nikspz commented Nov 27, 2023

rapcmia left a comment

fengtality commented Nov 30, 2023

rapcmia commented Dec 1, 2023

Feat/XRPL connector v2 #128

Feat/XRPL connector v2 #128

Conversation

mlguys commented Jun 8, 2023 • edited Loading

nikspz commented Jun 15, 2023

nikspz commented Jun 21, 2023 • edited Loading

nikspz commented Jun 27, 2023

nikspz commented Jun 28, 2023 • edited Loading

mlguys commented Jul 3, 2023

mlguys commented Aug 29, 2023

fengtality left a comment

Choose a reason for hiding this comment

nikspz commented Oct 6, 2023

mlguys commented Oct 11, 2023

nikspz commented Oct 23, 2023

nikspz commented Oct 30, 2023

mlguys commented Oct 31, 2023

nikspz commented Nov 8, 2023

mlguys commented Nov 10, 2023

nikspz commented Nov 13, 2023

mlguys commented Nov 14, 2023

nikspz commented Nov 14, 2023 • edited Loading

nikspz commented Nov 27, 2023

rapcmia left a comment

Choose a reason for hiding this comment

fengtality commented Nov 30, 2023

rapcmia commented Dec 1, 2023

mlguys commented Jun 8, 2023 •

edited

Loading

nikspz commented Jun 21, 2023 •

edited

Loading

nikspz commented Jun 28, 2023 •

edited

Loading

nikspz commented Nov 14, 2023 •

edited

Loading