Tensor Tracking, Streaming/Online/Adaptive/Incremental Tensor Decomposition and Dynamic Tensor Analysis

A list of up-to-date research papers on streaming tensor decomposition, tensor tracking, and dynamic tensor analysis.

Contributions to improve the completeness of this list are greatly appreciated. Please feel free to open an issue, make a pull request, contribute, and revise (or contact me) if you find any missed references or errors.

Table of Content

• Survey and Overview Paper

• Dynamic Tensor Analysis Under CP/PARAFAC Format

  • Subspace-based Methods
  • Block-Coordinate Descent (BCD)
    • BCD + Adaptive Least-Squares Filters
    • BCD + Stochastic Gradient Solvers
    • BCD + ADMM
  • Bayesian Inference
  • Multi-aspect Streaming CP Decomposition
  • Streaming PARAFAC2 Decomposition

• Dynamic Tensor Analysis Under Tucker/HOSVD Format

  • Online Tensor Dictionary Learning
    • Incremental Subspace Learning on Tensor Unfolding Matrices
    • Online Multimodal Dictionary Learning
  • Tensor Subspace Tracking
  • Bayesian Inference
  • Multi-aspect Streaming Tucker Decomposition

• Dynamic Tensor Analysis Under Tensor-Train Format

  • Dynamic Decomposition of Time-series Tensors with Fix-Size
  • Incremental Decomposition of Tensors in Stationary Environments
  • Streaming Decomposition of Tensors in Non-Stationary Environments

• Dynamic Tensor Analysis Under Block-Term Decomposition Format

• Dynamic Tensor Analysis Under T-SVD Format

• Dynamic Tensor Analysis Under Tensor-Ring/Tensor Network Format

• Dynamic/Adaptive Coupled Tensor Decomposition

• Research Challenges, Open Problems, and Future Directions

• Related Sources

  • Good Surveys of (Batch) Tensor Decomposition and Analysis
  • Tensor Toolbox and Software
    • MATLAB
    • Python
    • R
    • Julia
  • Some Applications and Datasets

• Citation

Survey and Overview Paper

A Contemporary and Comprehensive Survey on Streaming Tensor Decomposition, in IEEE TKDE, 2023, PDF

Authors: Thanh Trung Le, Karim Abed-Meraim, Nguyen Linh Trung and Adel Hafiane

Dynamic Tensor Analysis Under CP/PARAFAC Format

Subspace-based Methods

• PARAFAC-SDT/-RLS: “Adaptive algorithms to track the PARAFAC decomposition of a third-order tensor,” in IEEE Trans. Signal Process., 2009, Paper, PDF, Code

• 3D-OPAST: ``Fast adaptive PARAFAC decomposition algorithm with linear complexity", in IEEE ICASSP, 2016, Paper, PDF

• CP-PETRELS: ``Adaptive PARAFAC decomposition for third-order tensor completion", in IEEE ICCE, 2016, Paper, PDF

• SOAP: "Second-order optimization based adaptive PARAFAC decomposition of three-way tensors", in Digital Signal Process., 2017, Paper, Code

Block-Coordinate Descent (BCD) Methods

BCD + Adaptive Least-Squares Filters

• OLSTEC: "Fast online low-rank tensor subspace tracking by CP decomposition using recursive least squares from incomplete observations", in NeuroComput., 2017, Paper, PDF, Code

  • Conference version: "Online low-rank tensor subspace tracking from incomplete data by CP decomposition using recursive least squares", in IEEE ICASSP, 2016 Paper, PDF, Code

• CP-NLS: "Nonlinear least squares updating of the canonical polyadic decomposition", in EUSIPCO, 2017, Paper, PDF, Code

• CP-stream: "Streaming tensor factorization for infinite data sources", in SDM, 2018, Paper, PDF, Code in Splatt Toolbox

• InParTen: "Incremental PARAFAC decomposition for three-dimensional tensors using Apache Spark", in ICWE, 2019, Paper

• TenNOODL: "Provable online CP/PARAFAC decomposition of a structured tensor via dictionary learning", in NeurISP, 2021, Paper, PDF, Code

• SliceNStitch: "Slicenstitch: Continuous CP decomposition of sparse tensor streams", in IEEE ICDE, 2021, Paper, PDF, Code

• STF: "Accurate online tensor factorization for temporal tensor streams with missing value", in ACM CIKM, 2021, Paper, PDF, Code

• ROLCP: "A Fast Randomized Adaptive CP Decomposition for Streaming Tensors", in IEEE ICASSP, 2021, Paper, PDF, Code

• OnlineCPDL: "Online nonnegative CP-dictionary learning for Markovian data" in J. Mach. Learn. Res., 2022, PDF, Code

• ACP: "Tracking online low-rank approximations of higher-order incomplete streaming tensors", in Cell Patterns, 2023, Paper, PDF, Code

• ALTO: "Dynamic Tensor Linearization and Time Slicing for Efficient Factorization of Infinite Data Streams", in IEEE IPDPS, 2023, Paper, Code

• OnlineGCP: "Streaming Generalized Canonical Polyadic Tensor Decompositions", in PASC, 2023, Paper, PDF, Code

BCD + Stochastic Gradient Solvers

• TeCPSGD: "Subspace Learning and Imputation for Streaming Big Data Matrices and Tensors", in IEEE Trans. Signal Process., 2015, Paper, PDF, Code

• OLCP: "Accelerating Online CP Decompositions for Higher Order Tensors", in ACM SIGKDD, 2016, Paper, PDF, Code

• OnlineSCP: "Online CP Decomposition for Sparse Tensors", in IEEE ICDM, 2018, Paper, PDF, Code

• SOFIA: "Robust Factorization of Real-world Tensor Streams with Patterns, Missing Values, and Outliers", in IEEE ICDE, 2020, Paper, PDF, Code

• iCP-AM: "Incremental CP tensor decomposition by alternating minimization method", in SIAM J. Matrix Anal. Appl, 2020, Paper

• DAO-CP: "DAO-CP: Data Adaptive Online CP Decomposition", in Plus One, 2021, Paper, PDF, Code

BCD + ADMM

• spCP-stream: "High-Performance Streaming Tensor Decomposition", in IEEE IPDPS, 2021, Paper, PDF, Code

• RACP: "Robust Tensor Tracking with Missing Data and Outliers: Novel Adaptive CP Decomposition and Convergence Analysis" in IEEE Trans. Signal Process., 2022, Paper, PDF

• T-MUST: "Robust online tensor completion for IoT streaming data recovery", in IEEE Trans. Neural Netw. Learn. Syst., 2022, Paper

Bayesian Inference Methods

• POST: "Probabilistic streaming tensor decomposition", in IEEE ICDM, 2018, Paper, PDF, Code

• BRST: "Variational Bayesian inference for robust streaming tensor factorization and completion", in IEEE ICDM, 2018, Paper, PDF, Code

• SBDT: "Streaming Bayesian deep tensor factorization", in ICML, 2021, Paper, PDF, Code

• SFTL: "Streaming Factor Trajectory Learning for Temporal Tensor Decomposition", in NeurIPS, 2023, Paper, PDF, PDF, Code

Multi-aspect Streaming CP Decomposition Methods

• MASTA: "Multi-aspect-streaming tensor analysis", in Knowl.-Based Syst., 2015, Paper, PDF, Code

• MAST: "Multi-aspect streaming tensor completion", in ACM SIGKDD, 2017, Paper, PDF, Code

• OR-MSTC: "Outlier-Robust Multi-Aspect Streaming Tensor Completion and Factorization", in IJCAI, 2019, Paper, PDF

• InParTen2: "Multi-aspect incremental tensor decomposition based on distributed in-memory big data systems", in J. Data Inf. Sci., 2020, Paper

• DisMASTD: "Dismastd: An efficient distributed multi-aspect streaming tensor decomposition", in IEEE ICDE, 2021, Paper, PDF

• GOCPT: "GOCPT: Generalized Online Canonical Polyadic Tensor Factorization and Completion", in IJCAI, 2022, Paper, PDF, Code

Streaming PARAFAC2 Decomposition Methods

• SPADE: "SPADE: Streaming PARAFAC2 decomposition for large datasets", in SDM, 2020, Paper, PDF, Code

• Dpar2: "Dpar2: Fast and scalable parafac2 decomposition for irregular dense tensors", in IEEE ICDE, 2022, Paper, PDF, Code

• ATOM: "Accurate PARAFAC2 Decomposition for Temporal Irregular Tensors with Missing Values", in IEEE BigData, 2022, Paper, PDF, Code

• DASH: "Fast and Accurate Dual-Way Streaming PARAFAC2 for Irregular Tensors--Algorithm and Application", in ACM SIGKDD, 2023, Paper, PDF, Code

• tPARAFAC2: "A Time-aware tensor decomposition for tracking evolving patterns", in ArXiv, 2023, Paper, Code

Others

• DEMOTE: "Dynamic Tensor Decomposition via Neural Diffusion-Reaction Processes", in NeurIPS, 2023, Paper, PDF, Code

Dynamic Tensor Analysis Under Tucker/HOSVD Format

Online Tensor Dictionary Learning Methods

Incremental Subspace Learning on Tensor Unfolding Matrices

• DTA and STA: "Beyond streams and graphs: dynamic tensor analysis", in ACM SIGKDD, 2007, Paper, PDF, TKDD, Code

• IRTSA: "Robust visual tracking based on incremental tensor subspace learning", in IEEE ICCV, 2007, Paper, PDF, IJCV, M-Code, P-Code

• RTSL: "Robust tensor subspace learning for anomaly detection", in Int. J. Mach. Learn. Cybern, 2011, Paper, PDF

• ITF: "An incremental tensor factorization approach for web service recommendation", in IEEE ICDM Works, 2014, Paper

• Online-LRTL: "Accelerated online low rank tensor learning for multivariate spatiotemporal streams", in ICML, 2015, Paper, PDF, Matlab, Python

• IHOSVD: "A tensor-based approach for big data representation and dimensionality reduction", in IEEE Trans. Emerg. Topics Comput., 2014, Paper, PDF

• Ho-RLSL: "Recursive tensor subspace tracking for dynamic brain network analysis", in IEEE Trans. Signal Inf. Process. Netw., 2017, Paper, PDF

• DHOSVD: "A distributed HOSVD method with its incremental computation for big data in cyber-physical-social systems", in IEEE Trans. Comput. Social Syst., 2018, Paper, PDF

• MDHOSVD: "A multi-order distributed HOSVD with its incremental computing for big services in cyber-physical-social systems", in IEEE Trans. Big Data, 2018, Paper

• IMDHOSVD: "Improved multi-order distributed HOSVD with its incremental computing for smart city services", in IEEE Trans. Sustain. Comput., 2018, Paper

Online Multimodal Dictionary Learning

• Singleshot: "Singleshot: A scalable Tucker tensor decomposition", in NeurIPS, 2019, Paper, PDF

• OTDL: "Online multimodal dictionary learning", in NeuroComput., 2019, Paper, PDF

• ODL: "Learning separable dictionaries for sparse tensor representation: An online approach", in IEEE Trans. Circuits Syst. II, 2019, Paper

• ORLTM: "Online robust low-rank tensor modeling for streaming data analysis", in IEEE Trans. Neural Netw. Learn. Syst., 2019, Paper

• TTMTS: "Low-rank Tucker approximation of a tensor from streaming data", in SIAM J. Math. Data Sci., 2020, Paper, PDF, Code

• Zoom-Tucker: "Fast and Memory-Efficient Tucker Decomposition for Answering Diverse Time Range Queries" in ACM SIGKDD, 2021, Paper, PDF, Code

• OLRTR: "Streaming data preprocessing via online tensor recovery for large environmental sensor networks", in ACM Trans. Knowl. Disc. Data, 2022, Paper, PDF, Code

• D-L1-Tucker: "Dynamic L1-norm Tucker tensor decomposition", in IEEE J. Sel. Topics Signal Process., 2021, Paper, PDF, Code

• ROLTD: "Robust Online Tucker Dictionary Learning from Multidimensional Data Streams", in APSIPA-ASC, 2022, Paper, PDF, Code

Tensor Subspace Tracking Methods

• LRUT: "Accelerated low-rank updates to tensor decompositions", in IEEE HPEC, 2016, Paper, PDF

• Riemannian-based method: "Low-rank tensor completion: a Riemannian manifold preconditioning approach", in ICML, 2016, Paper, PDF, Code

• RT-NTD and BK-NTD: " Incremental nonnegative Tucker decomposition with block-coordinate descent and recursive approaches", in Symmetry, 2022, Paper, Code

• D-TuckerO: "Static and Streaming Tucker Decomposition for Dense Tensors", in ACM Trans. Knowl. Disc. Data, 2022, Paper, PDF, Code

• ATD: "Tracking online low-rank approximations of higher-order incomplete streaming tensors", in Cell Patterns, 2023, Paper, PDF, Code

Bayesian Inference Methods

• SNBTD: "Streaming nonlinear Bayesian tensor decomposition", in UAI, 2020, Paper, PDF, Code

• BASS-Tucker: "Bayesian streaming sparse Tucker decomposition", in UAI, 2021, Paper, PDF, Code

• BCTT: "Bayesian Continuous-Time Tucker Decomposition", in ICML, 2022, Paper, PDF, Code

Multi-aspect Streaming Tucker Decomposition Methods

• SIITA: "Inductive Framework for Multi-Aspect Streaming Tensor Completion with Side Information", in ACM CIKM, 2018, Paper, PDF, Code

• eOTD: "eOTD: An efficient online tucker decomposition for higher order tensors", in IEEE ICDM, 2018, Paper, PDF

Dynamic Tensor Analysis Under Tensor-Train Format

Dynamic Decomposition of Time-series Tensors with Fix-Size (Non-Streaming)

• DATT: "Dynamical approximation by hierarchical Tucker and tensor-train tensors", in SIAM J. Matrix Anal. Appl., 2013, Paper, PDF

• DATT: "Time integration of tensor trains", in SIAM J. Numer. Anal., 2015, Paper, PDF

Incremental Decomposition of Tensors in Stationary Environments (i.e., TT-Cores are Fixed Over Time)

• ITTD: "An incremental tensor-train decomposition for cyber-physical-social big data", in IEEE Trans. Big Data, 2018, Paper, PDF

• DTT: "DTT: A highly efficient distributed tensor train decomposition method for IIoT big data", in IEEE Trans. Ind. Inf, 2021, Paper

Streaming Decomposition of Tensors in Non-Stationary Environments (i.e., TT-Cores Can Change Over Time)

• TT-FOA: "Adaptive Algorithms for Tracking Tensor-Train Decomposition of Streaming Tensors", in EUSIPCO, 2020, Paper, PDF, Code

• ROBOT: "Robust Tensor Tracking With Missing Data Under Tensor-Train Format", in EUSIPCO, 2022, Paper, PDF, Code

• ATT: "A Novel Recursive Least-Squares Adaptive Method For Streaming Tensor-Train Decomposition With Incomplete Observations", in Signal Process., 2023, Paper, PDF, Code

• STTA: "Streaming tensor train approximation", in SIAM J. Sci. Comput., 2023, Paper, PDF, Code

• SPTT: "Streaming probabilistic tensor train decomposition", in ArXiv, 2023, Paper

• TT-ICE: "An Incremental Tensor Train Decomposition Algorithm", in SIAM J. Scient. Comput., 2024. Paper, Code

Dynamic Tensor Analysis Under Block-Term Decomposition Format

• OnlineBTD: "OnlineBTD: Streaming algorithms to track the block term decomposition of large tensors", in DSAA, 2020, Paper, PDF, Code

• O-BTD-RLS: "Online rank-revealing block-term tensor decomposition", in Signal Process., 2023, Paper, PDF

• SBTD: "A Novel Tensor Tracking Algorithm For Block-Term Decomposition of Streaming Tensors", in IEEE SSP, 2023, Paper, PDF

Dynamic Tensor Analysis Under T-SVD Format

• TO-RPCA: "An online tensor robust PCA algorithm for sequential 2D data", in IEEE ICASSP, 2016, Paper, PDF

• TOUCAN: "Grassmannian optimization for online tensor completion and tracking with the T-SVD", in IEEE Trans. Signal Process., 2022, Paper, PDF, Code

  • Conference version: "Online Tensor Completion and Free Submodule Tracking With The T-SVD", in IEEE ICASSP, 2020, Paper, PDF, Code

• "Effective streaming low-tubal-rank tensor approximation via frequent directions", in IEEE Trans. Neural Netw. Learn. Syst., 2022, Paper, PDF

Dynamic Tensor Analysis Under Tensor-Ring/Tensor Network Format

• "Multi-Aspect Streaming Tensor Ring Completion for Dynamic Incremental Data", in IEEE Signal Process. Lett., 2022, Paper
• OLSTR-SGD/RLS: "Online subspace learning and imputation by Tensor-Ring decomposition", in Neural Netw., 2022, Paper
• STRC & TRSSD: "Patch tracking-based streaming tensor ring completion for visual data recovery", in IEEE Trans. Circuits Syst. Video Techn., 2022, Paper, PDF, Code
• STR: "Tracking Tensor Ring Decompositions of Streaming Tensors", in ArXiv, 2023, Paper

Dynamic/Adaptive Coupled Tensor Decomposition

• C-CPD-T: "An Adaptive Algorithm for Tracking Third-Order Coupled Canonical Polyadic Decomposition", in IEEE ICASSP, 2024, Paper

Research Challenges, Open Problems, and Future Directions

• Data Imperfection and Corruption
  • Non-Gaussian and Colored Noises
  • Outliers and Missing Data
• Rank Revealing and Tracking
• Efficient and Scalable Tensor Tracking
  • Randomized Sketching
  • Parallel and Distributed Computing
  • Neural Networks-based Methods
• Provable Tensor Tracking Methods
• Symbolic Tensor Tracking
• Tracking under BTD, t-SVD, TN, and other variants

Related Sources

Good Surveys of (Batch) Tensor Decomposition, Analysis, and Applications

• Evrim Acar et al. "Unsupervised multiway data analysis: A literature survey", in IEEE TKDE, 2008, PDF

• Tamara G. Kolda et al. "Tensor decompositions and applications", in SIAM Rev., 2009, PDF

• Lieven De Lathauwer et al. "Breaking the curse of dimensionality using decompositions of incomplete tensors: Tensor-based scientific computing in big data analysis", in IEEE Signal Process. Mag., 2014, PDF

• Andrzej Cichocki & Lieven De Lathauwer et al. "Tensor decompositions for signal processing applications: From two-way to multiway component analysis", in IEEE Signal Process. Mag., 2015, PDF

• Andrzej Cichocki & Ivan Oseledets et al. "Tensor networks for dimensionality reduction and large-scale optimization: Part 1 low-rank tensor decompositions", in Found. Trends Mach. Learn., 2016, PDF

• Andrzej Cichocki & Ivan Oseledets et al. "Tensor networks for dimensionality reduction and large-scale optimization: Part 2 Applications and future perspectives", in Found. Trends Mach. Learn., 2017, PDF

• Nicholas D. Sidiropoulos & Lieven De Lathauwer et al. "Tensor Decomposition for Signal Processing and Machine Learning", in IEEE Trans. Signal Process., 2017, PDF

• Nicholas D. Sidiropoulos et al. "Tensors for data mining and data fusion: Models, applications, and scalable algorithms", in ACM Trans. Intell. Syst. Technol, 2017, PDF

• Yannis Panagakis et al. "Tensor methods in computer vision and deep learning", in Proc. IEEE, 2021, PDF

Tensor Toolbox and Software

MATLAB

• Nway-Toolbox, Link
• TensorToolbox, Link
• TensorLab, Link
• TensorBox, Link
• Tensor-Tensor Product Toolbox, Link
• Splatt, Link
• TT-Toolbox, Link
• Hierarchical Tucker Toolbox, Link

Python

• TT-Toolbox, Link
• Pyttb Toolbox, Link
• TensorLy, Link
• Hottbox, Link
• Tensorfac, Link
• TensorD, Link

R

• Tensor, Link
• rTensor, Link
• nnTensor, Link
• TensorBF, Link

Julia

• TensorDecomp, Link
• Tensortoolbox, Link
• iTensor, Link

Some Applications and Time-Series Datasets

Computer Vision

• Video/visual tracking: ICCV 2007, ICCV 2015, TPAMI 2016, TSP 2022, TNNLS 2022

• Online video denoising: ICCV 2017, TIP 2018

• Segmentation and Classification: TPAMI 2008, SIAM JMDS 2020

• Video datasets: Link-1, Link-2, Link-3

NeuroScience

• Dynamic functional connectivity networks analysis: TBE 2016, TSIPN 2017, Front. Neurosci. 2022

• Online EEG Completion and Analysis: ISMICT 2018, TSP 2022, Patterns 2023

• Datasets: Link-1, Link-2, Link-3, Link-4

Anomaly Detection

• Sensor Networks: KIS 2014, TNSM 2016, DMKD 2018

• Visual anomaly detection: TVCG 2017

• Others: See Survey

Citation

If you find this repository helpful for your work, please cite

[1] L.T. Thanh, K. Abed-Meraim, N. L. Trung and A. Hafiane. “A Contemporary and Comprehensive Survey on Streaming Tensor Decomposition”. IEEE Trans. Knowl. Data Eng., 2023 PDF.