design/ml-concepts.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "   # Conceptual design for Liu ML approach"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Objectives\n",
    "\n",
    "* Automatically generate a predictive model, per strategy that predicts probability of openning a position for a given stock,\n",
    "* The framework should use off-hours calculations to train predictive models, based on various price `influencers`, \n",
    "* Extend the Strategy base class with functions for getting open / close position probability based on symbol and minute_data,\n",
    "* High-Performance, real time probability calculation,\n",
    "* Probability that could be used as a buy/sell signal, or risk measurement for position sizing."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Challenges / Open Questions\n",
    "\n",
    "* What are the set of `influence` variables on a stock price?\n",
    "* What should be the NN model and architecture?\n",
    "* Software Architecture changes for efficient off-hour calculations, \n",
    "* Software Archtecture for real-time model predition calculations,\n",
    "* Which libraries to use? PyTorch? Kares? \n",
    "* What hardware is required?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What are we tring to predict?\n",
    "\n",
    "Given a strategy, and a symbol's OHLC data - calculate the probability of the strategy to generate significant revenues today (> 1%), or the next few days (> 3%), or in the near future (> 10%), depending on the strategy objective (day-trading, or swing)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (Brain-Dump) Influences on a specific stock price\n",
    "\n",
    "### Excogenous\n",
    "\n",
    "* Market sentiment, where is the money flowing?\n",
    "* Insdusty / Segment influence - impact of similar/current industry trend on the stock price,\n",
    "\n",
    "### Endogenous\n",
    "\n",
    "* OHLC & Volume - impact of similar price & volume behaviours on stock up trend,\n",
    "* Proximity to stock calendar - past impact of proximity to an event (earning, dividend schedule),\n",
    "* Past performance of stock for the specific symbol,"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plan\n",
    "\n",
    "1. **[DONE]** Experiment with LSTM models,\n",
    "2. **[IN-PROCESS]** Experiment Attention NN models,\n",
    "3. Experiment CNN and conversion for time-series data to visual clasification,\n",
    "4. Tying Strategy results and coming up with a predictive model, per strategy,\n",
    "5. Software architecture for scale,\n",
    "6. Complete instration\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## LSTM\n",
    "\n",
    "### Configuration & Training\n",
    "* Created an LSTM model with various configurations, using PyTorch,\n",
    "* Training and testing on several symbols, using minute based data for 5 days (~5000 data points),\n",
    "* The NN configuration that worked the best (and appears in the notebook) is a 2 Stacked LSTMs, with 2 hidden layers each. Using 100 nodes, and 0.5 for drop-out configuration (to avoid over-fitting),\n",
    "* Training using 100 epochs, Adam optimizer w/ MSE lose-function, \n",
    "* Labeling every 60 minutes, and testing prediction on 120 minutes in total (last two hours of the day), \n",
    "* Tried a multivariant canculation, based on OHLC, but didn't see how that really helped the predictive capabilities.\n",
    "\n",
    "### Predictive capabilities\n",
    "\n",
    "* While running numerous configurations and dozens of tests, it seems like the network is able to accuratly predict several minutes, and then the network seems to get 'stuck' on specific values. \n",
    "* The results are somehow productive for a short period of time (10-20 minutes into the future) \n",
    "\n",
    "Notebook can be found here: https://github.com/amor71/LiuAlgoTrader/blob/master/analysis/notebooks/LSTM.ipynb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}