Skip to content

Latest commit

 

History

History
116 lines (94 loc) · 7.55 KB

README.md

File metadata and controls

116 lines (94 loc) · 7.55 KB
Raw

Deep Learning for Time Series forecasting

Python 3.7 PyTorch 1.2

  This repo included a collection of models (transformers, attention models, GRUs) mainly focuses on the progress of time series forecasting using deep learning. It was originally collected for financial market forecasting, which has been organized into a unified framework for easier use.

  For beginners, we recommend you read this paper or the brief introduction we provided to learn about time series forecasting. And read the paper for a more comprehensive understanding.

 With limited ability and energy, we only test how well the repo works on a specific dataset. For problems in use, please leave an issue. The repo will be kept updated.

 See the dev branch for recent updates or the master branch not works.

Requirements

  • Python 3.7
  • matplotlib == 3.1.1
  • numpy == 1.21.5
  • pandas == 0.25.1
  • scikit_learn == 0.21.3
  • torch == 1.7.1

Dependencies can be installed using the following command:

pip install -r requirements.txt

Get Started

  1. Download data provided by the repo. You can obtain all the six benchmarks from Tsinghua Cloud or Google Drive. All the datasets are well pre-processed and can be used easily.
  2. Train the model and predict.
    • Step1. set model e.g. "autoformer".
    • Step2. set dataset, i.e. assign a Dataset class to feed data into the pre-determined model.
    • Step3. set some essential params included data_path, file_name,seq_len,label_len,pred_len,features,T,M,S,MS. Default params we has provided can be set from data_parser if data is provided.
    • Other. Sometimes it is necessary to revise _process_one_batch and _get_data in exp_main.

A simple command included three parameters correspond to the above three steps:

python -u main.py --model 'autoformer' --dataset 'ETTh1' --data "ETTh1"

Usage on customized data(comming soon)

 To run on your customized data, a DataSet class must be provided in data_loader.py, then add the Dataset to Exp_Basic.py. Need to be noted that elements ejected from the DataSet class must conform to the model's requirement.

See Colab Examples for detail: We provide google colabs to help reproducing and customing our repo, which includes experiments(train and test), forecasting, visualization and custom data.

Simple Results on ETT

Ses the repo for more details on ETT dataset.

Autoformer result for the task (only 2 epoches for train, test 300+ samples):



Figure 1. Autoformer results (randomly choose a sample).



Figure 2. Autoformer results (values distribution).

Simple Results on Oze Challenge

This challenge aims at introducing a new statistical model to predict and analyze energy consumptions and temperatures in a big building using observations stored in the Oze-Energies database. More details can be seen from the repo. A simple model(Seq2Seq) result for the task (only 2 epoches for train, test 2 samples):



Figure 3. GRU results (randomly choose a sample).



Figure 4. GRU results (values distribution).

Models currently supported

We will keep adding series forecasting models to expand this repo.

Year Model name Models Tasks
--- LSTM Vanilla Lstm many to one
2014 EDLstm Sequence to Sequence Learning with Neural Networks many to many
2014 EDGru Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation many to many
2016 EDGruAtt Neural Machine Translation by Jointly Learning to Align and Translate many to many
2017 Transformer Attention Is All You Need many to many
2017 DeepAR DeepAR:Probabilistic Forecasting with Autoregressive Recurrent Networks many to many
2017 LSTNet Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks many-to-one
2018 TPA TPA:Temporal Pattern Attention for Multivariate Time Series Forecasting many to one
2018 TCN TCN:An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling many to one
2019 LogTrans LogTrans:Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting many to many
2020 Informer Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting many to many
2021 Autoformer Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting many to many
Reformer Reformer many to many
Transformer XL Transformer XL many to many
-2019 N-BEATS N-BEATS: Neural basis expansion analysis for interpretable time series forecasting

DOING and TODO

  1. Add probability estimation function.
  2. Improve the network structure(especially attention network) according to our data scenario.
  3. Add Tensorboard to record exp.

About exp_single and exp_multi

Usually we will encounter three forms of data:

  1. multi files(usual caused by multi individual) which will cause oom if load all of them. Every separate file contaies train, vail, test.
  2. sigle file contaied train, vail, test.
  3. multi separate files (usual three) i.e. train, vail, test.

 For 1, we load a file (train, vail, test dataset) iteratively in a epoch until all files are loaded and fed to a model. i.e.exp_multi.py. For 2, 3, we load train, vail, test dataset before starting training. i.e. exp_single.py.

Contact

If you have any questions, feel free to contact hyliu through Email (hyliu_sh@outlook.com) or Github issues.

Acknowlegements

To complete the project, we referenced the following repos. Informer2020, AdjustAutocorrelation, flow-forecast, pytorch-seq2seq, Autoformer.