迁移学习 Transfer Learning

Everything about Transfer Learning (Probably the most complete repository?). Your contribution is highly valued! If you find this repo helpful, please cite it as follows:

关于迁移学习的所有资料，包括：介绍、综述文章、最新文章、代表工作及其代码、常用数据集、硕博士论文、比赛等等。(可能是目前最全的迁移学习资料库？) 欢迎一起贡献！ 如果认为本仓库有用，请在你的论文和其他出版物中进行引用！

@Misc{transferlearning.xyz,
howpublished = {\url{http://transferlearning.xyz}},   
title = {Everything about Transfer Learning and Domain Adapation},  
author = {Wang, Jindong and others}  
}  

Contents
0.Papers (论文)	1.Introduction and Tutorials (简介与教程)
2.Transfer Learning Areas and Papers (研究领域与相关论文)	3.Theory and Survey (理论与综述)
4.Code (代码)	5.Transfer Learning Scholars (著名学者)
6.Transfer Learning Thesis (硕博士论文)	7.Datasets and Benchmarks (数据集与评测结果)
8.Transfer Learning Challenges (迁移学习比赛)	Applications (迁移学习应用)
Other Resources (其他资源)	Contributing (欢迎参与贡献)

关于机器学习和行为识别的资料，请参考：行为识别｜机器学习

---

NOTE: You can directly open the code in Gihub Codespaces on the web to run them without downloading!

0.Papers (论文)

• Arxitics: a good website to see the latest arXiv papers: Transfer learning, Domain adaptation

• Paperweekly: A good website to recommend and read paper notes (一个推荐、分享论文的网站比较好，上面会持续整理相关的文章并分享阅读笔记)

• Awesome transfer learning papers (迁移学习文章汇总)

• Latest papers: (These papers are also put in docs/awesome_papers.md)

Latest papers (2021-08-09)

• A Data Augmented Approach to Transfer Learning for Covid-19 Detection

  • Data augmentation to transfer learning for COVID
  • 迁移学习使用数据增强，用于COVID-19

• MM-21 Few-shot Unsupervised Domain Adaptation with Image-to-class Sparse Similarity Encoding

  • Few-shot DA with image-to-class sparse similarity encoding
  • 小样本的领域自适应

• Dual-Tuning: Joint Prototype Transfer and Structure Regularization for Compatible Feature Learning

  • Prototype transfer and structure regularization
  • 原型的迁移学习

Latest papers (2021-08-06)

• Finetuning Pretrained Transformers into Variational Autoencoders
  • Finetune transformer to VAE
  • 把transformer迁移到VAE

Latest papers (2021-08-05)

• Pre-trained Models for Sonar Images

  • Pre-trained models for sonar images
  • 针对声纳图像的预训练模型

• Domain Adaptor Networks for Hyperspectral Image Recognition

  • Finetune for hyperspectral image recognition
  • 针对高光谱图像识别的迁移学习

Latest papers (2021-07-30)

• CVPR-21 Efficient Conditional GAN Transfer With Knowledge Propagation Across Classes

  • Transfer conditional GANs to unseen classes
  • 通过知识传递，迁移预训练的conditional GAN到新类别

• CVPR-21 Ego-Exo: Transferring Visual Representations From Third-Person to First-Person Videos

  • Transfer learning from third-person to first-person video
  • 从第三人称视频迁移到第一人称

Latest papers (2021-07-28)

• Toward Co-creative Dungeon Generation via Transfer Learning

  • Game scene generation with transfer learning
  • 用迁移学习生成游戏场景

• Transfer Learning in Electronic Health Records through Clinical Concept Embedding

  • Transfer learning in electronic health record
  • 迁移学习用于医疗记录管理

Latest papers (2021-07-27)

• CVPR-21 Conditional Bures Metric for Domain Adaptation

  • A new metric for domain adaptation
  • 提出一个新的metric用于domain adaptation

• CVPR-21 Wasserstein Barycenter for Multi-Source Domain Adaptation

  • Use Wasserstein Barycenter for multi-source domain adaptation
  • 利用Wasserstein Barycenter进行DA

• CVPR-21 Generalized Domain Adaptation

  • A general definition for domain adaptation
  • 一个更抽象更一般的domain adaptation定义

• CVPR-21 Reducing Domain Gap by Reducing Style Bias

  • Syle-invariant training for adaptation and generalization
  • 通过训练图像对style无法辨别来进行DA和DG

• CVPR-21 Uncertainty-Guided Model Generalization to Unseen Domains

  • Uncertainty-guided generalization
  • 基于不确定性的domain generalization

• CVPR-21 Adaptive Methods for Real-World Domain Generalization

  • Adaptive methods for domain generalization
  • 动态算法，用于domain generalization

Latest papers (2021-07-16)

• 20210716 ICML-21 Continual Learning in the Teacher-Student Setup: Impact of Task Similarity

  • Investigating task similarity in teacher-student learning
  • 调研在continual learning下teacher-student learning问题的任务相似度

• 20210716 BMCV-extend Exploring Dropout Discriminator for Domain Adaptation

  • Using multiple discriminators for domain adaptation
  • 用分布估计代替点估计来做domain adaptation

• 20210716 TPAMI-21 Lifelong Teacher-Student Network Learning

  • Lifelong distillation
  • 持续的知识蒸馏

• 20210716 MICCAI-21 Few-Shot Domain Adaptation with Polymorphic Transformers

  • Few-shot domain adaptation with polymorphic transformer
  • 用多模态transformer做少样本的domain adaptation

• 20210716 InterSpeech-21 Speech2Video: Cross-Modal Distillation for Speech to Video Generation

  • Cross-model distillation for video generation
  • 跨模态蒸馏用于语音到video的生成

• 20210716 ICML-21 workshop Leveraging Domain Adaptation for Low-Resource Geospatial Machine Learning

  • Using domain adaptation for geospatial ML
  • 用domain adaptation进行地理空间的机器学习

---

1.Introduction and Tutorials (简介与教程)

Want to quickly learn transfer learning？想尽快入门迁移学习？看下面的教程。

• Books 书籍

  • 《迁移学习导论》Introduction to Transfer Learning Homepage Buy

• Blogs 博客

  • Zhihu blogs - 知乎专栏《小王爱迁移》系列文章

• Video tutorials 视频教程

  • Recent advance of transfer learning - 2021年最新迁移学习发展现状探讨

  • Domain generalization - 迁移学习新兴研究方向领域泛化

  • Domain adaptation - 迁移学习中的领域自适应方法(中文)

  • Transfer learning by Hung-yi Lee @ NTU - 台湾大学李宏毅的视频讲解(中文视频)

• Brief introduction and slides 简介与ppt资料

  • Recent advance of transfer learning

  • Domain generalization survey

  • Brief introduction in Chinese

    • PPT (English) | PPT (中文)

  • 迁移学习中的领域自适应方法 Domain adaptation: PDF ｜ Video on Bilibili | Video on Youtube

  • Tutorial on transfer learning by Qiang Yang: IJCAI'13 | 2016 version

• Talk is cheap, show me the code 动手教程、代码、数据

  • Pytorch官方迁移学习示意代码
    • Pytorch的finetune Fine-tune based on Alexnet and Resnet
    • 用Pytorch进行深度特征提取
    • 更多 More...

• Transfer Learning Scholars and Labs - 迁移学习领域的著名学者、代表工作及实验室介绍

• Negative transfer - 负迁移

---

2.Transfer Learning Areas and Papers (研究领域与相关论文)

• Survey
• Theory
• Per-training/Finetuning
• Knowledge distillation
• Traditional domain adaptation
• Deep domain adaptation
• Domain generalization
• Source-free domain adaptation
• Multi-source domain adaptation
• Heterogeneous transfer learning
• Online transfer learning
• Zero-shot / few-shot learning
• Multi-task learning
• Transfer reinforcement learning
• Transfer metric learning
• Federated transfer learning
• Lifelong transfer learning
• Transfer learning applications

---

3.Theory and Survey (理论与综述)

Here are some articles on transfer learning theory and survey.

Survey (综述文章)：

• 2021 Domain generalization: IJCAI-21 Generalizing to Unseen Domains: A Survey on Domain Generalization | 知乎文章 | 微信公众号
  • First survey on domain generalization
  • 第一篇对Domain generalization (领域泛化)的综述
• 2020 迁移学习最新survey，来自中科院计算所庄福振团队，发表在Proceedings of the IEEE: A Comprehensive Survey on Transfer Learning
• 2020 负迁移的综述：Overcoming Negative Transfer: A Survey
• 2020 知识蒸馏的综述: Knowledge Distillation: A Survey
• 用transfer learning进行sentiment classification的综述：A Survey of Sentiment Analysis Based on Transfer Learning
• 2019 一篇新survey：Transfer Adaptation Learning: A Decade Survey
• 2018 一篇迁移度量学习的综述: Transfer Metric Learning: Algorithms, Applications and Outlooks
• 2018 一篇最近的非对称情况下的异构迁移学习综述：Asymmetric Heterogeneous Transfer Learning: A Survey
• 2018 Neural style transfer的一个survey：Neural Style Transfer: A Review
• 2018 深度domain adaptation的一个综述：Deep Visual Domain Adaptation: A Survey
• 2017 多任务学习的综述，来自香港科技大学杨强团队：A survey on multi-task learning
• 2017 异构迁移学习的综述：A survey on heterogeneous transfer learning
• 2017 跨领域数据识别的综述：Cross-dataset recognition: a survey
• 2016 A survey of transfer learning。其中交代了一些比较经典的如同构、异构等学习方法代表性文章。
• 2015 中文综述：迁移学习研究进展
• 2010 A survey on transfer learning
• Survey on applications - 应用导向的综述：
  • 视觉domain adaptation综述：Visual Domain Adaptation: A Survey of Recent Advances
  • 迁移学习应用于行为识别综述：Transfer Learning for Activity Recognition: A Survey
  • 迁移学习与增强学习：Transfer Learning for Reinforcement Learning Domains: A Survey
  • 多个源域进行迁移的综述：A Survey of Multi-source Domain Adaptation。

Theory （理论文章）:

• ICML-20 Few-shot domain adaptation by causal mechanism transfer
  • The first work on causal transfer learning
  • 日本理论组大佬Sugiyama的工作，causal transfer learning
• CVPR-19 Characterizing and Avoiding Negative Transfer
  • Characterizing and avoid negative transfer
  • 形式化并提出如何避免负迁移
• ICML-20 On Learning Language-Invariant Representations for Universal Machine Translation
  • Theory for universal machine translation
  • 对统一机器翻译模型进行了理论论证
• NIPS-06 Analysis of Representations for Domain Adaptation
• ML-10 A Theory of Learning from Different Domains
• NIPS-08 Learning Bounds for Domain Adaptation
• COLT-09 Domain adaptation: Learning bounds and algorithms
• MMD paper：A Hilbert Space Embedding for Distributions and A Kernel Two-Sample Test
• Multi-kernel MMD paper: Optimal kernel choice for large-scale two-sample tests

---

4.Code (代码)

Unified codebases for:

• Deep domain adaptation
• Deep domain generalization

More: see HERE and HERE for an instant run using Google's Colab.

---

5.Transfer Learning Scholars (著名学者)

Here are some transfer learning scholars and labs.

全部列表以及代表工作性见这里

Please note that this list is far not complete. A full list can be seen in here. Transfer learning is an active field. If you are aware of some scholars, please add them here.

---

6.Transfer Learning Thesis (硕博士论文)

Here are some popular thesis on transfer learning.

这里, 提取码：txyz。

---

7.Datasets and Benchmarks (数据集与评测结果)

Please see HERE for the popular transfer learning datasets and benchmark results.

这里整理了常用的公开数据集和一些已发表的文章在这些数据集上的实验结果。

---

8.Transfer Learning Challenges (迁移学习比赛)

• Visual Domain Adaptation Challenge (VisDA)

---

Applications (迁移学习应用)

See HERE for transfer learning applications.

迁移学习应用请见这里。

---

Other Resources (其他资源)

• Call for papers:

  • Advances in Transfer Learning: Theory, Algorithms, and Applications, DDL: October 2021

• Related projects:

  • Salad: A semi-supervised domain adaptation library

---

Contributing (欢迎参与贡献)

If you are interested in contributing, please refer to HERE for instructions in contribution.

---

Copyright notice

[Notes]This Github repo can be used by following the corresponding licenses. I want to emphasis that it may contain some PDFs or thesis, which were downloaded by me and can only be used for academic purposes. The copyrights of these materials are owned by corresponding publishers or organizations. All this are for better adademic research. If any of the authors or publishers have concerns, please contact me to delete or replace them.