Skip to content

Overview

Jump to bottom
John Kearney edited this page Feb 11, 2021 · 7 revisions

PassivBot 101

2021 has yielded an accelerated interest in cryptocurrency as new price discovery coincides with gross inflation and the beginnings of a bull cycle. With that interest comes swaths of new traders, exposing themselves to complex derivatives like coin margined contracts, quarterly, and perpetual futures contracts, and options. PassivBot is an algorithmic open source grid trading utility designed to trade “chop” or indecision in leveraged perpetual futures positions. Leveraged futures contracts are a risk intense derivative and should be approached with extreme caution, especially as the bot is just entering live testing. This article serves as a “plain English” overview of the bot and how it’s configurations work.

Grid Trading

PassivBot uses a preferential buy / sell grid to scalp trade indecisive price action. A grid consists of a range of buy and sell orders wherein a position is entered, and doubled down on as the price moves against the trade. Dollar cost averaging into the position averages out the breakeven price closer to the mark price, and when the market rebounds or pulls back, limit “reduce” orders are used to slowly scale out of your position at the average mark price within your take profit range. Grid trading essentially trades sideways volatility in arbitrary ranges set by a certain risk tolerance, margin balance, and aggression coefficient. As such, they operate fairly quickly, executing order changes up to once per-second and sometimes filling orders even faster.

Traditional grid trading is slower, as properly weighting risk for the ranges requires a high step coefficient, and reduces the average purchased amount per-entry. Traditional grid trading often involves creating a buy and sell grid over a very large swath of price history with fixed values. Traders not willing to face potential liquidation are relegated to trading with a fraction of their futures balance, and must monitor the positions somewhat regularly to prevent fast price movement from destroying a trade or eating away at profits. As a result, traditional grid traders may have hundreds or thousands of entry orders and just as many exit orders. This prevents double down rules from letting the liquidation price go awry, but has a negative effect on short term P/L, especially when trading with small sums. Comparing these more conservative algorithms to other less risky investments, an interest account for example, reveals little added benefit to using a grid trading layout over just sticking your money in a bank, and often implies more risk.

Preferential Grid Trading

Preferential grid trading takes the fundamentals of grid trading and expunges upon them to attempt to form an algorithm that not only DCA’s it’s entries and exits based on relative price action, but tries to predict or go with the major trend using external data. Using indicators in tandem with exchange data (ex. volatility indicator + volume) allows a given program to add preference values to a given position (long or short) to influence the next re-entry. The indicators used can be anything, so long as there is a definite rule. In essence, simply adding an exponential moving average does nothing unless the bot has a ruleset by which to interpret the EMA.

PassivBot Implementation

PassivBot is a python3 implementation of a preferential grid trading framework designed around Binance USD-T Perpetual Futures derivatives and ByBit Inverse Perpetual Futures derivatives. The bot is designed simply as a framework for conducting grid trading, and doesn’t have any set preference, although it comes with example configurations to define expected formatting. PassivBot can be broken down into three distinct sections of operation:

  • Configuration & API Keys – Connecting your exchange with PassivBot using API keys, and understanding the configuration files.
  • Backtesting Configurations – Testing a given configuration over previous price history, and using iterating loops to find desirable / profitable settings.
  • Live Usage – Using the bot in a live account, expected behavior, typical quirks, risks, and troubleshooting.

For this document, all examples shown will be in Binance, as the process for ByBit configuration is essentially the same. This is new, largely unaudited code; use at your own risk.

Configuration & API Keys

As PassivBot is simply the framework for facilitating automated HFT, the user must specify some parameters for the bot to know how to place orders. These parameters are somewhat complex, so instead of running them appended to a command, they are specified within a .json file. The bot also needs to retrieve price data from your exchange as well as place orders using your account. For this, we use a websocket connection to your exchange’s API endpoints. Instead of specifying your API keys as params, they are loaded from a separate .json file. The directory should look something like this:

> /api_key_secrets/  
> /backtesting_settings/  
> /live_settings/  

/backtest.py
/backtesting_notes.ipynb
/binance.py
/bybit.py
/passivbot.py
/README.md
/rename_trade_data_csvs.py
/start_bot.py

The API keys will be placed in the 'api_key_secrets/{exchange}/example_user.json' file, and you’ll rename the file to whatever you would like the profile to be named. When the command is run to start the bot, you will specify a ‘profile’ name, which denotes the configuration file & API Key file to the bot. You may name the API key file anything you wish, as long as you retain the formatting. When adding your keys, leave the quotation marks, and only replace the relevant text:

["EXAMPLE_KEY", "EXAMPLE_SECRET"]

When creating your API keys, note that you are only allowed to see your secret key a single time, and you must manually enable the API’s ability to trade futures.

Backtesting Configurations

Next, you need to specify your settings. Generally, manually setting some variables can lead to the bot going awry over time, so exercise caution when using untested configurations. If you haven't installed the dependencies for the project yet, we will do that now. Open up a command line / terminal and run the following:

python3 -m pip install matplotlib pandas websockets ccxt

In some cases, it may be necessary to install using '--user'. This should not effect the program's functionality.

Utilizing the backtester is the best way to come up with new configurations, but requires that you have a basal understanding of the configuration file. The backtester's job is to look at a coin pair's price history (ETH/USDT for the last 30 days for example), look at some settings, and test those settings over the timeframe you selected. The bot will iterate through every trade as if it were doing it live, and return the results to the user. These setting (if they are profitable) are then used to generate new settings and test those. This process repeats itself as many times as the user chooses and upon completion; returns the most profitable configuration for the bot, over that timeframe. There is a default backtesting configuration with the bot located at:

/PassivBot/backtesting_settings/{exchange}/backtesting_settings.json

...which will look something like this:

"session_name": "unnamed_session_001",
"exchange": "binance",
"symbol": "XMRUSDT",
"n_days": 7,

"starting_candidate_preference": ["best", "given", "random"],
"starting_k": 0,
"n_jackrabbit_iterations": 100,

"starting_balance": 10.0,
"inverse": false,
"maker_fee": 0.00018,
"market_stop_loss": false,
"min_qty": 0.001,
"price_step": 0.01,
"qty_step": 0.001,
"taker_fee": 0.00036,
"min_close_qty_multiplier": 0.5,


"min_notional": 1.0,
"cross_mode": true,
"max_leverage": 75,
"do_long": true,
"do_shrt": true,

"break_on": [
    ["ON: break on first soft stop",
     "lambda trade, tick: trade['type'] == 'stop_loss'"],
    ["OFF: neg pnl sum",
     "lambda trade, tick: trade['pnl_sum'] < 0.0 and trade['progress'] > 0.3"],
    ["ON: liq diff too small",
     "lambda trade, tick: trade['liq_diff'] < 0.05"],
    ["OFF: time between consec trades",
     "lambda trade, tick: tick['timestamp'] - trade['timestamp'] > 1000 * 60 * 60 * 24"],
    ["OFF: pos price last price diff",
     "lambda trade, tick: calc_diff(trade['price'], tick['price']) > 1.05"],
    ["ON: adg too low",
     "lambda trade, tick: trade['average_daily_gain'] < 1.01 and trade['progress'] >= 0.1"]
],

"ddown_factor": 0.6,
"default_qty": -0.2624,
"grid_coefficient": 280.0,
"grid_spacing": 0.002,
"ema_span": 16500.0,
"leverage": 24.0,
"stop_loss_liq_diff": 0.04,
"stop_loss_pos_price_diff": 0.04,
"max_markup": 0.01694,
"min_markup": 0.00072,
"n_close_orders": 10.0,
"stop_loss_pos_reduction": 0.001

Note that there are two different kinds of configuration files. The formatting for a backtesting configuration file is not always the same as the formatting for a live usage configuration file. Always retain the default.json file as a reference for the formatting of your version. If an update makes your version obsolete, you will need a reference for the formatting if you don't plan to immediately update. When editing the backtesting configuration file, only edit the values between a colon and a comma. Some values can be positive or negative, some can only be positive, and some can only be boolean values. For more in-depth information on configuring each of the values, refer to the ‘README.md’ file or the 'Backtesting' Wiki page.

The values in 'backtesting_settings.json' specify how the backtest will run. These values will be tested, and the variables (grid size adjustments, leverage etc.) iterated through based on the values in the associated ‘ranges.json’ file. Upon finding a better configuration, the backtester will update the candidate to the best settings file ('best.json') and continue to backtest. This cycle continues until the max 'n_jackrabbit_iterations' value has been reached. This effectively equates to an ever-improving configuration tester. These settings can then be amputated into '/live_settings/{exchange}/‘ and run from the CLI. Note, the settings shown in the example above have not been backtested or used on a live account.

To run the backtester, navigate to the folder in your CLI and run the command as follows:

python3 backtest.py <exchange> <backtestConfigName>

You should receive some output as the backtester begins downloading the necessary price data from the exchange. The price data is cached on the machine under 'PassivBot/historical_data/'. This means if you interrupt or close the process, it will continue downloading price data where it left off. Once the necessary price data has been downloaded, the backtester will begin with the starting candidate, test against the price history, and continue iterating through the ranges for each variable. Since the backtester only utilizes the CPU (no GPU integration as of yet) and API requests are time-limited; downloading a 100 day price history can take around 36-48 hours to finish. Once the history is downloaded however, additional price history can simply be tacked on to the end of the cache, cutting down testing times. One methodology is to begin a random candidate backtest for the entire price history (in ‘n_days’) of a given trading pair. Once the backtester has finished downloading (but not iterating) customize your backtesting settings and re-run the backtest over your given period.

Once the backtester has downloaded the price history it begins testing your starting candidate against the price history. This process is also computed by the CPU, and can be time consuming depending on the testing period. Take this in to account when beginning your test. GPU support may become available over time. The backtester will store all relevant candidates in the ‘results.txt’ file, and the best current candidate under ‘best.json’. The backtest iterations are also cached, therefore allowing you to re-start the process assuming it is interrupted, although this is not suggested.

The key to finding new, profitable configurations is using the backtester often and familiarizing yourself with the settings and ranges. Adjusting the ranges narrows the proverbial area the program needs to search for good configurations, reducing the test time while potentially cutting more profitable settings out of the search range.

Once the backtest has concluded, the best results can be copied over to the ‘live_settings’ directory for usage, or analyzed in jupyter-lab. We recommend always analyzing your results with Jupyter before introducing them into a live account.

Analyzing Results: Jupyter-Lab

PassivBot includes some code to assist with visualizing the trade activity & bot performance. This allows users to visualize trade activity, sizes and duration along with some additional metrics to determine stability and intended usage for a given configuration. To use this part of the code, you’ll need to have Jupyter-lab installed:

pip3 install jupyter-lab

Next, open a command line / terminal and navigate to the PassivBot main directory using ‘cd’. You can then open the Jupyter Interface by running:

jupyter-lab

The Jupyter Interface should open, and display the PassivBot folder in the toolbar on the left. Navigate to the file ‘backtesting_notes.ipynb’. The ‘backtesting_notes.ipynb’ file only needs a single change before you can get the results of your backtest. Under cell 5 of the file, find and change the symbol to the currency pair your backtesting results are for. For example, if we are backtesting Ethereum:

symbol = 'ETHUSDT'

Save the file, and return focus to the first cell. You may now use ‘shift + enter’ to step through the code sequentially, or hit the ‘play’ icon. Ensure each step finishes before proceeding to run the next step. The cell ID will change from an asterisk ([*]) to the ID ([1]) after finishing.

This Information can then be amputated into custom scripts, manipulated to display new metrics, or saved as files.

Live Usage

Use at your own risk! The rule of thumb is "if you don't understand it, don't do it." We are not responsible for how your bot may behave.

If you have not installed the dependencies by now, do so with:

python3 -m pip install matplotlib pandas websockets ccxt

Let's assume you've run a backtest, and want to deploy the 'best.json' configuration to a live account trading Ethereum. If you have not yet run a backtest, but want to test the bot, a repository of community-contributed configurations is available at https://github.com/JohnKearney1/PassivBot-Configurations.

Navigate to your API KeyFile directory and either re-name or duplicate and re-name the KeyFile for the account you wish to run on. You should name your KeyFile and your desired Configuration file the same thing. For example, if I take the 'best.json' output from the backtest, copy it to my 'live_settings' directory, and re-name it 'J-ETHSomewhatAggressive.json'; I will need to name my API KeyFile 'J-ETHSomewhatAggressive.json'. This is your username. Save your files.

Now open up a terminal and navigate to the base folder of PassivBot.

cd Passivbot_futures

And run the following command:

python3 start_bot.py <exchange> <username>

Example:

python3 start_bot.py binance J-ETHSomewhatAggressive

Note that it is not necessary to append the file extension '.json' to the username. The bot will print your configuration file in the console, and proceed to connect to your exchange and enter a trade. The bot will only work for as long as you leave the console open, so do not close it unless you are ready to stop it from trading. Since the bot queues orders, make sure you cancel or monitor all open orders and positions through your exchange's trading interface. If you close the bot unexpectedly, it will not close its orders, leaving the account flying blind with an open position and potentially tens of open orders.

Clone this wiki locally