This is a supervised Recurrent Neural Network (RNN) learning project treating stock trading as a classification problem. Given input of a 60 day window of pricing data, choose the best action for maximum profit. This uses my earlier https://github.com/TimRivoli/Stock-Price-Trade-Analyzer project for a trading environment, and its SeriesPrediction module for data preparation and model training.
To begin with I use CalculateBestActions to generate the target action values. For each day, given the knowledge a stocks prior day's prices, and constraints of one trade per day, four consecutive trade days, and eight possible trade actions, it calculates the sequence of trades that will produce the highest value in 20 days time. The possible trades are: Aggressive Buy, Target Buy, Market Buy, Hold, Market Sell, Target Sell, Aggressive Sell, and CancelAllOrders. Target Buys and Sells are determined by the moving average price plus or minus 1/4 the 5 day average deviation of price. Aggressive Buys and Sells are determined by 1/2 the 5 day average deviation of price. CalculateBestActions is computationally expensive, so though you can calculate longer sequences they will take exponentially longer. A 4 trade sequence may take a day to calculate for a 30 year data set so you will hit a wall at about 6. The sequences are saved for re-use so you only have to generated them once for each ticker you test on and I use multi-threading to utilize all processor cores.
To train the model, I pass a sixty day window of market state values as input and use the best actions (above) for target values. The target values are sequences of actions because in many cases it will be a combination of actions that produces the best result, but in training I will choose the only first action for each day and then make a new determination of best action for the next day with updated price data.
The MakeState helper function takes the previous day's market prices and current portfolio positions as input and outputs one of seven types of state values as a list to pass as input into the model. Each type of state is a selection from 19 pricing variables that might be useful to the describe the current state of the market for a given stock. The state may also include the state of the model portfolio available funds, open orders, and long positions. This is an area where you can explore possibilities of new combinations of input that help the model better fit the target values. In my testing, I found that it learns best when you keep the input fairly simple but there is a lot of room for creativity here such as including moving averages, percentage change, and how much portfolio info to include. If you wanted to include other types of information then MakeState can easily be customized include it. The output of MakeState will be a list of items the network will take as input for learning. The seven states types I included are just seven combinations I used in my testing.
TraderTrain takes as input a stock ticker, date range, and state type (for MakeState). From this it will download the historical stock pricing data, calculate pricing data statistics, populate the state windows, and then train and test a RNN model on the data. The resulting trade history, daily values, and overall performance are saved to files in the data\trademodel folder. TraderTrain has options to create ticker specific models or use a generic model, to build on existing models or to delete the existing model and start over, and to use LSTM or CNN models. I had the best results with using a CNN model so that is the default. TraderTrain will do a train/test split for you, or you can use it just for training and use TraderRun for testing.
TraderRun restores a saved model and tests it on specified time periods. It can be useful to run it against the training data just to see how well it learned as well as testing against new data. The test results are saved in the same manner as TraderTrain. The overall performance of training is aggregated in the Performance.CSV file for comparison of all training scenarios. The computation of best action target data takes a long time so I included an option to compute the best actions while TraderRun is running so that it will periodically retrain from the results of past actions and also record those best actions to file for future training. In this way, it can run continually and learn with new data as it comes in.
TraderRandom and TraderBuyHold are two helper functions to evaluate the performance of trained models in a given epoch versus random actions and a BuyHold strategy. A good model should outperform both of these.
ExtensivePreparation runs CalculateBestActions for a given list of tickers. This is necessary to begin training.
ExtensiveTesting runs TraderTrain, TraderRun, TraderRandom, and TraderBuyHold on a given list of tickers for various periods of time between 1990 and 2018. The results are all saved to the data\trademodel folder.
This was developed using Python 3.6 and requires reasonably current versions of numpy, pandas, matplotlib, tensorflow and keras. Additionally, it uses the utilities of https://github.com/TimRivoli/Stock-Price-Trade-Analyzer for a trading environment. Once the requirements are installed, place the RobotTrader.py file in the Stock-Price-Trade-Analyzer-master and you should be good to go.
Have fun and keep programming! Oh, and don't use this for trading advice.
-Tim