Recent AI Advances: Tutorials and Surveys in Top AI Conferences and Journals

A professionally curated list of tutorials (keynote, invited talk, etc.) and surveys of recent AI advances, including Deep Learning, Machine Learning, Data Mining, Computer Vision (CV), Natural Language Processing (NLP), Speech, etc., at the Top AI Conferences and Journals, which is updated ASAP (the earliest time) once the accepted tutorials and surveys are announced in the corresponding top AI conferences/journals. Hope this list would be helpful for researchers and engineers who are interested in various AI areas.

The top conferences including:

• Machine Learning: NeurIPS, ICML, ICLR
• Computer Vision: CVPR, ICCV, ACMMM
• NLP and Speech: ACL, EMNLP
• Speech: ICASSP, INTERSPEECH
• Artificial Intelligence: AAAI, IJCAI
• Data Mining: KDD, WWW
• Data Management: SIGMOD, VLDB, ICDE
• Misc (selected): AISTAT, CIKM, ICDM, WSDM, SIGIR, etc.

The top journals including: CACM, PIEEE, TPAMI, TKDE, TNNLS, TITS, TIST, TSP, TASLP, TIP, TACL, SPM, IJCV, JMLR, JAIR, CSUR, DMKD, KAIS, arXiv(selected), etc.

If you found any missed resources (paper/code) or errors, please feel free to open an issue or make a pull request.

Table of Contents

• AI Tutorials

  • Tutorials 2023
  • Tutorials 2022
  • Tutorials 2021
  • Tutorials 2020
  • Tutorials 201X

• AI Surveys

  • General
  • Transformer and Attention
  • Self-Supervised Learning
  • Graph Neural Networks
  • Federated Learning
  • XAI
  • AutoML
  • Deep Generative Models
  • N-Shot Learning
  • Anomaly Detection and OOD
  • Label-noise Learning
  • Imbalanced-data Learning
  • Deep Reinforcement Learning
  • Domain Adaptation
  • Others

AI Tutorials

Tutorials 2023

• Everything You Need to Know about Transformers: Architectures, Optimization, Applications, and Interpretation, AAAI 2023. [Link]
• On Explainable AI: From Theory to Motivation, Industrial Applications, XAI Coding & Engineering Practices, AAAI 2023. [Link]

Tutorials 2022

• Causality and deep learning: synergies, challenges& opportunities for research, ICML 2022. [Link TBD]
• Bridging Learning and Decision Making, ICML 2022. [Link TBD]
• Facilitating a smoother transition to Renewable Energy with AI (AI4Renewables), ICLR 2022 Social. [Link] [slides]
• Optimization in ML and DL - A discussion on theory and practice, ICLR 2022 Social. [slides]
• Beyond Convolutional Neural Networks, CVPR 2022. [Link]
• Evaluating Models Beyond the Textbook: Out-of-distribution and Without Labels, CVPR 2022. [Link]
• Sparsity Learning in Neural Networks and Robust Statistical Analysis, CVPR 2022. [Link]
• Denoising Diffusion-based Generative Modeling: Foundations and Applications, CVPR 2022. [Link]
• On Explainable AI: From Theory to Motivation, Industrial Applications, XAI Coding & Engineering Practices, AAAI 2022. [Link]
• Deep Learning on Graphs for Natural Language Processing, AAAI 2022. [Link]
• Bayesian Optimization: From Foundations to Advanced Topics, AAAI 2022. [Link]

Tutorials 2021

• The Art of Gaussian Processes: Classic and Contemporary, NeurIPS 2021. [Link] [slides]
• Self-Supervised Learning: Self-Prediction and Contrastive Learning, , NeurIPS 2021. [slides] [vedio]
• Self-Attention for Computer Vision, ICML 2021. [Link]
• Continual Learning with Deep Architectures, ICML 2021. [Link]
• Responsible AI in Industry: Practical Challenges and Lessons Learned, ICML 2021. [Link]
• Self-Supervision for Learning from the Bottom Up, ICLR 2021 Talk. [Link]
• Geometric Deep Learning: the Erlangen Programme of ML, ICLR 2021 Talk. [Link]
• Moving beyond the fairness rhetoric in machine learning, ICLR 2021 Talk. [Link]
• Is My Dataset Biased, ICLR 2021 Talk. [Link]
• Interpretability with skeptical and user-centric mind, ICLR 2021 Talk. [Link]
• AutoML: A Perspective where Industry Meets Academy, KDD 2021. [Link]
• Automated Machine Learning on Graph, KDD 2021. [Link]
• Toward Explainable Deep Anomaly Detection, KDD 2021. [Link]
• Fairness and Explanation in Clustering and Outlier Detection, KDD 2021. [Link]
• Real-time Event Detection for Emergency Response, KDD 2021. [Link]
• Machine Learning Explainability and Robustness: Connected at the Hip, KDD 2021. [Link]
• Machine Learning Robustness, Fairness, and their Convergence, KDD 2021. [Link]
• Counterfactual Explanations in Explainable AI: A Tutorial, KDD 2021. [Link]
• Causal Inference and Machine Learning in Practice with EconML and CausalML: Industrial Use Cases at Microsoft, TripAdvisor, Uber, KDD 2021. [Link]
• Normalization Techniques in Deep Learning: Methods, Analyses, and Applications, CVPR 2021. [Link]
• Normalizing Flows and Invertible Neural Networks in Computer Vision, CVPR 2021. [Link]
• Theory and Application of Energy-Based Generative Models, CVPR 2021. [Link]
• Adversarial Machine Learning in Computer Vision, CVPR 2021. [Link]
• Practical Adversarial Robustness in Deep Learning: Problems and Solutions, CVPR 2021. [Link]
• Leave those nets alone: advances in self-supervised learning, CVPR 2021. [Link]
• Interpretable Machine Learning for Computer Vision, CVPR 2021. [Link]
• Learning Representations via Graph-structured Networks, CVPR 2021. [Link]
• Reviewing the Review Process, ICCV 2021. [Link]
• Meta Learning and Its Applications to Natural Language Processing, ACL 2021. [Link]
• Deep generative modeling of sequential data with dynamical variational autoencoders, ICASSP 2021. [Link]

Tutorials 2020

• Deep Implicit Layers - Neural ODEs, Deep Equilibirum Models, and Beyond, NeurIPS 2020. [Link]
• Practical Uncertainty Estimation and Out-of-Distribution Robustness in Deep Learning, NeurIPS 2020. [Link]
• Explaining Machine Learning Predictions: State-of-the-art, Challenges, and Opportunities, NeurIPS 2020. [Link]
• Advances in Approximate Inference, NeurIPS 2020. [Link]
• There and Back Again: A Tale of Slopes and Expectations, NeurIPS 2020. [Link]
• Federated Learning and Analytics: Industry Meets Academia, NeurIPS 2020. [Link]
• Machine Learning with Signal Processing, ICML 2020. [Link]
• Bayesian Deep Learning and a Probabilistic Perspective of Model Construction, ICML 2020. [slides] [video]
• Representation Learning Without Labels, ICML 2020. [slides] [video]
• Recent Advances in High-Dimensional Robust Statistics, ICML 2020. [Link]
• Submodular Optimization: From Discrete to Continuous and Back, ICML 2020. [Link]
• Deep Learning for Anomaly Detection, in KDD 2020. [Link] [video]
• Learning with Small Data, in KDD 2020. [Link]

Tutorials 201X

• Adversarial Machine Learning, ICLR 2019 Keynote. [slides]
• Introduction to GANs, CVPR 2018. [slides]
• Which Anomaly Detector should I use, ICDM 2018. [slides]

AI Surveys

General

• Deep learning, in Nature 2015. [paper]
• Deep learning in neural networks: An overview, in Neural networks 2015. [paper]

Transformer and Attention

• A survey on visual transformer, in IEEE TPAMI 2022. [paper]
• Transformers in vision: A survey, in ACM Computing Surveys 2021. [paper]
• Efficient transformers: A survey, in arXiv 2022. [paper]
• A General Survey on Attention Mechanisms in Deep Learning, in IEEE TKDE 2022. [paper]
• Attention, please! A survey of neural attention models in deep learning, in Artificial Intelligence Review 2022. [paper]
• An attentive survey of attention models, in ACM TIST 2021. [paper]
• Attention in natural language processing, in IEEE TNNLS 2020. [paper]

Self-Supervised Learning

• Self-supervised visual feature learning with deep neural networks: A survey, in IEEE TPAMI 2020. [paper]
• Self-supervised Learning: Generative or Contrastive, TKDE'21. [paper]
• Self-Supervised Representation Learning: Introduction, advances, and challenges, SPM'22. [paper]

Graph Neural Networks

• A comprehensive survey on graph neural networks, TNNLS'20. [paper]
• Deep learning on graphs: A survey, TKDE'20. [paper]
• Graph neural networks: A review of methods and applications, AI Open'20. [paper]
• Self-Supervised Learning of Graph Neural Networks: A Unified Review, TPAMI'22. [paper]
• Graph Self-Supervised Learning: A Survey, TKDE'22. [paper]
• Self-supervised learning on graphs: Contrastive, generative, or predictive, TKDE'21. [paper]

Federated Learning

• Federated machine learning: Concept and applications, TIST'19. [paper]
• Advances and open problems in federated learning, now'21. [paper]
• A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection, TKDE'21. [paper]
• A comprehensive survey of privacy-preserving federated learning: A taxonomy, review, and future directions, CSUR'21. [paper]
• A survey on federated learning, Knowledge-Based Systems'21. [paper]
• A Survey on Federated Learning: The Journey From Centralized to Distributed On-Site Learning and Beyond, JIOT'20. [paper]
• Federated learning: Challenges, methods, and future directions, SPM'20. [paper]

XAI

• Explaining deep neural networks and beyond: A review of methods and applications, PIEEE'21. [paper]
• Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI, Information Fusion'20. [paper]
• A survey on the explainability of supervised machine learning, JAIR'21. [paper]
• Techniques for Interpretable Machine Learning, CACM'19. [paper]

AutoML

• AutoML: A survey of the state-of-the-art, Knowledge-Based Systems'21. [paper]
• Benchmark and survey of automated machine learning frameworks, JAIR'21. [paper]
• AutoML to Date and Beyond: Challenges and Opportunities, CSUR'22. [paper]
• Automated Machine Learning on Graphs: A Survey, IJCAI'21. [paper]
• Others: awesome-automl-papers. [repo]

Deep Generative Models

• NIPS 2016 Tutorial: Generative Adversarial Networks, arXiv'17. [paper]
• Generative adversarial networks: An overview, SPM'18. [paper]
• A review on generative adversarial networks: Algorithms, theory, and applications, TKDE'21. [paper]
• A survey on generative adversarial networks: Variants, applications, and training, CSUR'22. [paper]
• An Introduction to Variational Autoencoders, now'19. [paper]
• Dynamical Variational Autoencoders: A Comprehensive Review, now'21. [paper]
• Advances in variational inference, TPAMI'19. [paper]
• Normalizing flows: An introduction and review of current methods, TPAMI'20. [paper]
• Normalizing Flows for Probabilistic Modeling and Inference, JMLR'21. [paper]

N-Shot Learning

• A survey of zero-shot learning: Settings, methods, and applications, in TIST 2019. [paper]
• Generalizing from a few examples: A survey on few-shot learning, in CSUR 2020. [paper] [Link]
• What Can Knowledge Bring to Machine Learning?—A Survey of Low-shot Learning for Structured Data, in TIST 2022. [paper]
• A Survey of Few-Shot Learning: An Effective Method for Intrusion Detection, in SCN 2022. [paper]
• Few-Shot Learning on Graphs: A Survey, in arXiv 2022. [paper]
• A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities, in arXiv 2022. [paper]

Anomaly Detection and OOD

• A unifying review of deep and shallow anomaly detection, PIEEE'21. [paper]
• Deep learning for anomaly detection: A review, CSUR'20. [paper]
• A Comprehensive Survey on Graph Anomaly Detection with Deep Learning, TKDE'21. [paper]
• Graph based anomaly detection and description: a survey, DMKD'15. [paper]
• Anomaly detection in dynamic networks: a survey, WICS'15. [paper]
• Anomaly detection: A survey, CSUR'09. [paper]
• A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challenges, arXiv'21. [paper]
• Self-Supervised Anomaly Detection: A Survey and Outlook, arXiv'21. [paper]

Label-noise Learning

• A Survey of Label-noise Representation Learning: Past, Present and Future, arXiv'21. [paper] [link]
• Learning from Noisy Labels with Deep Neural Networks: A Survey, TNNLS'22. [paper] [link]
• Classification in the presence of label noise: a survey, TNNLS'13. [paper]

Imbalanced-data Learning

• Learning from imbalanced data, TKDE'09. [paper]
• A Systematic Review on Imbalanced Data Challenges in Machine Learning: Applications and Solutions, CSUR'20. [paper]
• Imbalance problems in object detection: A review, TPAMI'20. [paper]

Deep Reinforcement Learning

Domain Adaptation

• Generalizing to unseen domains: A survey on domain generalization, TKDE'22. [paper]
• A survey of unsupervised deep domain adaptation, TIST'21. [paper]
• A review of domain adaptation without target labels, TPAMI'19. [paper]

Others

• A continual learning survey: Defying forgetting in classification tasks, in IEEE TPAMI 2021. [paper]
• Learning under concept drift: A review, in IEEE TKDE 2018. [paper]
• Learning in nonstationary environments: A survey, MCI'15. [paper]
• Online learning: A comprehensive survey, Neucom'21. [paper]
• A survey on transfer learning, TKDE'09. [paper]
• A Comprehensive Survey on Transfer Learning, PIEEE'21. [paper]
• A survey on multi-task learning, TKDE'21. [paper]
• Bayesian statistics and modelling, Nature Reviews Methods Primers'21. [paper]
• Meta-learning in neural networks: A survey, arXiv'21. [paper]
• Deep Long-Tailed Learning: A Survey, arXiv'21. [paper] [link]
• Learning to optimize: A primer and a benchmark, arXiv'21. [paper]