This pandect (πανδέκτης is Ancient Greek for encyclopedia) was created to help you find almost anything related to Natural Language Processing that is available online.
- Awesome NLP by keon [GitHub, 11370 stars]
- Speech and Natural Language Processing Awesome List by elaboshira [GitHub, 1999 stars]
- Awesome Deep Learning for Natural Language Processing (NLP) [GitHub, 878 stars]
- Text Mining and Natural Language Processing Resources by stepthom [GitHub, 339 stars]
- Made with ML List by madewithml.com
- Brainsources for #NLP enthusiasts by Philip Vollet
- Awesome AI/ML/DL - NLP Section [GitHub, 794 stars]
- 100 Must-Read NLP Papers 100 Must-Read NLP Papers [GitHub, 2978 stars]
- NLP Paper Summaries by dair-ai [GitHub, 1259 stars]
- Curated collection of papers for the NLP practitioner [GitHub, 1004 stars]
- Papers on Textual Adversarial Attack and Defense [GitHub, 733 stars]
- The Most Influential NLP Research of 2019
- Recent Deep Learning papers in NLU and RL by Valentin Malykh [GitHub, 287 stars]
- Some Notable Recent ML Papers and Future Trends by Aran Komatsuzaki [Blog, Oct. 2020]
- NLP top 10 conferences Compendium by soulbliss [GitHub, 331 stars]
- NLP Conferences Calendar
- ICLR 2020 Trends
- SpacyIRL 2019 Conference in Overview
- Paper Digest - Conferences and Papers in Overview
- NLP Progress by sebastianruder [GitHub, 17519 stars]
- NLP Tasks by Kyubyong [GitHub, 2872 stars]
- Reading list for Awesome Sentiment Analysis papers by declare-lab [GitHub, 273 stars]
- Awesome Sentiment Analysis by xiamx [GitHub, 802 stars]
- NLP Datasets by niderhoff [GitHub, 4414 stars]
- Datasets by Huggingface [GitHub, 5942 stars]
- Big Bad NLP Database
- 25 Best Parallel Text Datasets for Machine Translation Training
- UWA Unambiguous Word Annotations - Word Sense Disambiguation Dataset
- 20 Best German Language Datasets for Machine Learning
- Awesome Embedding Models by Hironsan [GitHub, 1419 stars]
- Awesome list of Sentence Embeddings by Separius [GitHub, 1716 stars]
- Awesome BERT by Jiakui [GitHub, 1589 stars]
- The Super Duper NLP Repo [Website, 2020]
- NLP Resources for Bahasa Indonesian [GitHub, 138 stars]
- Indic NLP Catalog [GitHub, 207 stars]
- Pre-trained language models for Vietnamese [GitHub, 285 stars]
- Natural Language Toolkit for Indic Languages (iNLTK) [GitHub, 686 stars]
- Indic NLP Library [GitHub, 332 stars]
- AI4Bharat-IndicNLP Portal
- ARBML - Implementation of many Arabic NLP and ML projects [GitHub, 157 stars]
- zemberek-nlp - NLP tools for Turkish [GitHub, 880 stars]
- List of pre-trained NLP models [GitHub, 114 stars]
- Natural Language Processing in 2020: The Year In Review [Blog, December 2020]
- ML and NLP Research Highlights of 2020 [Blog, January 2021]
- NLP Highlights [Years: 2017 - now, Status: active]
- TWIML AI [Years: 2016 - now, Status: active]
- Practical AI [Years: 2018 - now, Status: active]
- The Data Exchange [Years: 2019 - now, Status: active]
- Gradient Dissent [Years: 2020 - now, Status: active]
- Machine Learning Street Talk [Years: 2020 - now, Status: active]
- The Super Data Science Podcast [Years: 2016 - now, Status: active]
- Data Hack Radio [Years: 2018 - now, Status: active]
- AI Game Changers [Years: 2020 - now, Status: active]
- NLP News by Sebastian Ruder
- dair.ai Newsletter by dair.ai
- This Week in NLP by Robert Dale
- Papers with Code
- The Batch by deeplearning.ai
- Paper Digest by PaperDigest
- NLP Cypher by QuantumStat
- NLP Zurich [YouTube Recordings]
- NY-NLP (New York)
- Online NLP Meetup
- Hacking-Machine-Learning [YouTube Recordings]
- Yannic Kilcher
- HuggingFace
- Kaggle Reading Group
- Rasa Paper Reading
- Stanford CS224N: NLP with Deep Learning
- NLPxing
- ML Explained - A.I. Socratic Circles - AISC
- Deeplearning.ai
- Machine Learning Street Talk
- GLUE - General Language Understanding Evaluation (GLUE) benchmark
- SuperGLUE - benchmark styled after GLUE with a new set of more difficult language understanding tasks
- decaNLP - The Natural Language Decathlon (decaNLP) for studying general NLP models
- RACE - ReAding Comprehension dataset collected from English Examinations
- dialoglue - DialoGLUE: A Natural Language Understanding Benchmark for Task-Oriented Dialogue
- DynaBench - Dynabench is a research platform for dynamic data collection and benchmarking
- WikiAsp - WikiAsp: Multi-document aspect-based summarization Dataset
- SQuAD - Stanford Question Answering Dataset (SQuAD)
- XQuad - XQuAD (Cross-lingual Question Answering Dataset) for cross-lingual question answering
- GrailQA - Strongly Generalizable Question Answering (GrailQA)
- CSQA - Complex Sequential Question Answering
- XTREME - Massively Multilingual Multi-task Benchmark
- GLUECoS - A benchmark for code-switched NLP
- IndoNLU Benchmark - collection of resources for training, evaluating, and analyzing NLP for Bahasa Indonesia
- IndicGLUE - Natural Language Understanding Benchmark for Indic Languages
- LinCE - Linguistic Code-Switching Evaluation Benchmark
- BLURB - Biomedical Language Understanding and Reasoning Benchmark
- BLUE - Biomedical Language Understanding Evaluation benchmark
- Long-Range Arena - Long Range Arena for Benchmarking Efficient Transformers (Pre-print) [GitHub, 164 stars]
- CodeXGLUE - A benchmark dataset for code intelligence
- CrossNER - CrossNER: Evaluating Cross-Domain Named Entity Recognition
- MultiNLI - Multi-Genre Natural Language Inference corpus
- A Recipe for Training Neural Networks by Andrej Karpathy [Keywords: research, training, 2019]
- Pre-trained ELMo Representations for Many Languages [GitHub, 1295 stars]
- sense2vec - Contextually-keyed word vectors [GitHub, 1135 stars]
- wikipedia2vec [GitHub, 617 stars]
- StarSpace [GitHub, 3514 stars]
- fastText [GitHub, 22074 stars]
- Language Models and Contextualised Word Embeddings by David S. Batista [Blog, 2018]
- An Essential Guide to Pretrained Word Embeddings for NLP Practitioners by AnalyticsVidhya [Blog, 2020]
- Polyglot Word Embeddings Discover Language Clusters [Blog, 2020]
- The Illustrated Word2vec by Jay Alammar [Blog, 2019]
- vecmap - VecMap (cross-lingual word embedding mappings) [GitHub, 515 stars]
- bpemb - Pre-trained subword embeddings in 275 languages, based on Byte-Pair Encoding (BPE) [GitHub, 862 stars]
- subword-nmt - Unsupervised Word Segmentation for Neural Machine Translation and Text Generation [GitHub, 1592 stars]
- python-bpe - Byte Pair Encoding for Python [GitHub, 136 stars]
- The Transformer Family by Lilian Weng [Blog, 2020]
- Keeping up with the BERTs: a review of the main NLP benchmarks by Manuel Tonneau [Blog, 2020]
- Playing the lottery with rewards and multiple languages - about the effect of random initialization [ICLR 2020 Paper]
- Attention? Attention! by Lilian Weng [Blog, 2018]
- the transformer … “explained”? [Blog, 2019]
- Attention is all you need; Attentional Neural Network Models by Łukasz Kaiser [Talk, 2017]
- Understanding and Applying Self-Attention for NLP [Talk, 2018]
- The Annotated Transformer by Harvard NLP [Blog, 2018]
- The Illustrated Transformer by Jay Alammar [Blog, 2018]
- Illustrated Guide to Transformers by Hong Jing [Blog, 2020]
- Sequential Transformer with Adaptive Attention Span by Facebook. Blog [Blog, 2019]
- Evolution of Representations in the Transformer by Lena Voita [Blog, 2019]
- Reformer: The Efficient Transformer [Blog, 2020]
- Longformer — The Long-Document Transformer by Viktor Karlsson [Blog, 2020]
- TRANSFORMERS FROM SCRATCH [Blog, 2019]
- Universal Transformers by Mostafa Dehghani [Blog, 2019]
- Transformers in Natural Language Processing — A Brief Survey by George Ho [Blog, May 2020]
- Lite Transformer - Lite Transformer with Long-Short Range Attention [GitHub, 371 stars]
- A Visual Guide to Using BERT for the First Time by Jay Alammar [Blog, 2019]
- The Dark Secrets of BERT by Anna Rogers [Blog, 2020]
- Understanding searches better than ever before [Blog, 2019]
- Demystifying BERT: A Comprehensive Guide to the Groundbreaking NLP Framework [Blog, 2019]
- SemBERT - Semantics-aware BERT for Language Understanding [GitHub, 179 stars]
- BERTweet - BERTweet: A pre-trained language model for English Tweets [GitHub, 257 stars]
- Optimal Subarchitecture Extraction for BERT [GitHub, 396 stars]
- CharacterBERT: Reconciling ELMo and BERT [GitHub, 66 stars]
- T5 Understanding Transformer-Based Self-Supervised Architectures [Blog, August 2020]
- T5: the Text-To-Text Transfer Transformer [Blog, 2020]
- multilingual-t5 - Multilingual T5 (mT5) is a massively multilingual pretrained text-to-text transformer model [GitHub, 526 stars]
- Big Bird: Transformers for Longer Sequences original paper by Google Research [Paper, July 2020]
- Reformer: The Efficient Transformer - [Paper, February 2020] [Video, October 2020]
- Longformer: The Long-Document Transformer - [Paper, April 2020] [Video, April 2020]
- Linformer: Self-Attention with Linear Complexity - [Paper, June 2020] [Video, June 2020]
- Rethinking Attention with Performers - [Paper, September 2020] [Video, September 2020]
- performer-pytorch - An implementation of Performer, a linear attention-based transformer, in Pytorch [GitHub, 425 stars]
- Switch Transformers: Scaling to Trillion Parameter Models original paper by Google Research [Paper, January 2021]
- The Illustrated GPT-2 by Jay Alammar [Blog, 2019]
- The Annotated GPT-2 by Aman Arora
- OpenAI’s GPT-2: the model, the hype, and the controversy by Ryan Lowe [Blog, 2019]
- How to generate text by Patrick von Platen [Blog, 2020]
- Zero Shot Learning for Text Classification by Amit Chaudhary [Blog, 2020]
- GPT-3 A Brief Summary by Leo Gao [Blog, 2020]
- GPT-3, a Giant Step for Deep Learning And NLP by Yoel Zeldes [Blog, June 2020]
- GPT-3 Language Model: A Technical Overview by Chuan Li [Blog, June 2020]
- Is it possible for language models to achieve language understanding? by Christopher Potts
- Aweseome GPT-3 - list of all resources related to GPT-3 [GitHub, 2723 stars]
- GPT-3 Projects - a map of all GPT-3 start-ups and commercial projects
- OpenAI API - API Demo to use GPT-3 for commercial applications
- GPT-Neo - in-progress GPT-3 open source replication
- What is Two-Stream Self-Attention in XLNet by Xu LIANG [Blog, 2019]
- Visual Paper Summary: ALBERT (A Lite BERT) by Amit Chaudhary [Blog, 2020]
- Turing NLG by Microsoft
- Multi-Label Text Classification with XLNet by Josh Xin Jie Lee [Blog, 2019]
- ELECTRA [GitHub, 1594 stars]
- Performer implementation of Performer, a linear attention-based transformer, in Pytorch [GitHub, 425 stars]
- Distilling knowledge from Neural Networks to build smaller and faster models by FloydHub [Blog, 2019]
- David over Goliath: towards smaller models for cheaper, faster, and greener NLP by Manuel Tonneau [Blog, 2020]
- PEGASUS: A State-of-the-Art Model for Abstractive Text Summarization by Google AI [Blog, June 2020]
- CTRLsum - CTRLsum: Towards Generic Controllable Text Summarization [GitHub, 15 stars]
- LemmInflect - A python module for English lemmatization and inflection
- In Search of Best Practices for NLP Projects [Slides, Dec. 2020]
- EMNLP 2020: High Performance Natural Language Processing by Google Research [Slides, Recording, Nov. 2020]
- Practical Natural Language Processing - A Comprehensive Guide to Building Real-World NLP Systems [Book, June 2020]
- Why BERT Fails in Commercial Environments by Intel AI [Blog, 2020]
- Fine Tuning BERT for Text Classification with FARM by Sebastian Guggisberg [Blog, 2020]
- Pretrain Transformers Models in PyTorch using Hugging Face Transformers [GitHub, 47 stars]
- Practical NLP for the Real World [Presentation, 2019]
- From Paper to Product – How we implemented BERT by Christoph Henkelmann [Talk, 2020]
- embedding-as-service [GitHub, 147 stars]
- Bert-as-service [GitHub, 8708 stars]
- NLP Recipes by microsoft [GitHub, 5321 stars]
- NLP with Python by susanli2016 [GitHub, 1822 stars]
- Basic Utilities for PyTorch NLP by PetrochukM [GitHub, 1846 stars]
- Blackstone - A spaCy pipeline and model for NLP on unstructured legal text [GitHub, 432 stars]
- Sci spaCy - spaCy pipeline and models for scientific/biomedical documents [GitHub, 767 stars]
- FinBERT: Pre-Trained on SEC Filings for Financial NLP Tasks [GitHub, 129 stars]
- LexNLP - Information retrieval and extraction for real, unstructured legal text [GitHub, 419 stars]
- NerDL and NerCRF - Tutorial on Named Entity Recognition for Healthcare with SparkNLP
- Legal Text Analytics - A list of selected resources dedicated to Legal Text Analytics [GitHub, 128 stars]
- WildNLP - Corrupt an input text to test NLP models' robustness [GitHub, 63 stars]
- Great Expectations - Write tests for your data [GitHub, 3297 stars]
- CheckList - Beyond Accuracy: Behavioral Testing of NLP models [GitHub, 1146 stars]
- TextAttack - framework for adversarial attacks, data augmentation, and model training in NLP [GitHub, 1154 stars]
- wav2letter - Automatic Speech Recognition Toolkit [GitHub, 5577 stars]
- DeepSpeech - Baidu's DeepSpeech architecture [GitHub, 16157 stars]
- Acoustic Word Embeddings by Maria Obedkova [Blog, 2020]
- kaldi - Kaldi is a toolkit for speech recognition [GitHub, 9913 stars]
- awesome-kaldi - resources for using Kaldi [GitHub, 351 stars]
- FastSpeech - The Implementation of FastSpeech based on pytorch [GitHub, 561 stars]
- Topic Modelling with PySpark and Spark NLP by Maria Obedkova [Spark, Blog, 2020]
- Top2Vec [GitHub, 810 stars]
- Anchored Correlation Explanation Topic Modeling [GitHub, 263 stars]
- Topic Modeling in Embedding Spaces [GitHub, 274 stars] Paper
- TopicNet - A high-level interface for BigARTM library [GitHub, 96 stars]
- BERTopic - Leveraging BERT and a class-based TF-IDF to create easily interpretable topics [GitHub, 504 stars]
- PyTextRank - PyTextRank is a Python implementation of TextRank as a spaCy pipeline extension [GitHub, 1376 stars]
- textrank - TextRank implementation for Python 3 [GitHub, 964 stars]
- rake-nltk - Rapid Automatic Keyword Extraction algorithm using NLTK [GitHub, 783 stars]
- yake - Single-document unsupervised keyword extraction [GitHub, 529 stars]
- RAKE-tutorial - A python implementation of the Rapid Automatic Keyword Extraction [GitHub, 340 stars]
- rake-nltk - Rapid Automatic Keyword Extraction algorithm using NLTK [GitHub, 783 stars]
- flashtext - Extract Keywords from sentence or Replace keywords in sentences [GitHub, 4592 stars]
- BERT-Keyword-Extractor - Deep Keyphrase Extraction using BERT [GitHub, 150 stars]
- NLP Profiler - A simple NLP library allows profiling datasets with text columns [GitHub, 170 stars]
- keyBERT - Minimal keyword extraction with BERT [GitHub, 268 stars]
- Language Interpretability Tool (LIT) [GitHub, 2296 stars]
- WhatLies - Toolkit to help visualise - what lies in word embeddings [GitHub, 208 stars]
- Interpret-Text - Interpretability techniques and visualization dashboards for NLP models [GitHub, 191 stars]
- InterpretML - Fit interpretable models. Explain blackbox machine learning [GitHub, 3348 stars]
- ecco - Tools to visuals and explore NLP language models [GitHub, 77 stars]
- Machine Learning as a Software Engineering Enterprise - NeurIPS 2020 Keynote [Presentation, Dec 2020]
- Computational Ethics for NLP - course resources from the Carnegie Mellon University [Lecture Notes, Spring 2020]
- Ethics in NLP - resources from ACLs Ethics in NLP track
- The Institute for Ethical AI & Machine Learning
- Privacy Considerations in Large Language Models [Blog, Dec 2020]
- DeepWordBug - Generation of Adversarial Text Sequences to Evade Deep Learning Classifiers [GitHub, 45 stars]
- spaCy by Explosion AI [GitHub, 17910 stars]
- flair by Zalando [GitHub, 9733 stars]
- AllenNLP by AI2 [GitHub, 9524 stars]
- stanza (former Stanford NLP) [GitHub, 4999 stars]
- spaCy stanza [GitHub, 484 stars]
- nltk [GitHub, 9512 stars]
- gensim - framework for topic modeling [GitHub, 11534 stars]
- NLP Architect - A Deep Learning NLP/NLU library by Intel® AI Lab [GitHub, 2563 stars]
- polyglot - Multi-lingual NLP Framework [GitHub, 1737 stars]
- FARM [GitHub, 1057 stars]
- gobbli by RTI International [GitHub, 249 stars]
- headliner - training and deployment of seq2seq models [GitHub, 218 stars]
- SyferText - A privacy preserving NLP framework [GitHub, 143 stars]
- DeText - Text Understanding Framework for Ranking and Classification Tasks [GitHub, 976 stars]
- TextHero - Text preprocessing, representation and visualization [GitHub, 2017 stars]
- textblob - TextBlob: Simplified Text Processing [GitHub, 7466 stars]
- AdaptNLP - A high level framework and library for NLP [GitHub, 242 stars]
- textacy - NLP, before and after spaCy [GitHub, 1578 stars]
- WildNLP Text manipulation library to test NLP models [GitHub, 63 stars]
- snorkel Framework to generate training data [GitHub, 4368 stars]
- NLPAug Data augmentation for NLP [GitHub, 1495 stars]
- SentAugment Data augmentation by retrieving similar sentences from larger datasets [GitHub, 250 stars]
- faker - Python package that generates fake data for you [GitHub, 11837 stars]
- TextAttack - framework for adversarial attacks, data augmentation, and model training in NLP [GitHub, 1154 stars]
- CleverHans - adversarial example library for constructing NLP attacks and building defenses [GitHub, 4868 stars]
- textblob-de - TextBlob: Simplified Text Processing for German [GitHub, 79 stars]
- Kashgari Transfer Learning with focus on Chinese [GitHub, 1979 stars]
- Underthesea - Vietnamese NLP Toolkit [GitHub, 793 stars]
- transformers by HuggingFace [GitHub, 38665 stars]
- Adapter Hub and its documentation - Adapter modules for Transformers [GitHub, 287 stars]
- haystack - Transformers at scale for question answering & neural search. [GitHub, 1206 stars]
- DeepPavlov by MIPT [GitHub, 4911 stars]
- ParlAI by FAIR [GitHub, 6866 stars]
- rasa - Framework for Conversational Agents [GitHub, 10419 stars]
- wav2letter - Automatic Speech Recognition Toolkit [GitHub, 5577 stars]
- Spark NLP [GitHub, 1776 stars]
- COMET -A Neural Framework for MT Evaluation [GitHub, 49 stars]
- PolyFuzz - Fuzzy string matching, grouping, and evaluation [GitHub, 264 stars]
- ConvoKit - Cornell Conversational Analysis Toolkit [GitHub, 229 stars]
- Learn NLP the practical way [Blog, Nov. 2019]
- Dive into Deep Learning - An interactive deep learning book with code, math, and discussions
- Natural Language Processing and Computational Linguistics - Speech, Morphology and Syntax (Cognitive Science)
- Top NLP Books to Read 2020 - Blog post by Raymong Cheng [Blog, Sep 2020]
- NLP Course | For You - Great and interactive course on NLP
- Choosing the right course for a Practical NLP Engineer
- 12 Best Natural Language Processing Courses & Tutorials to Learn Online
- nlp-tutorial - A list of NLP(Natural Language Processing) tutorials built on PyTorch [GitHub, 1153 stars]
- nlp-tutorial - Natural Language Processing Tutorial for Deep Learning Researchers [GitHub, 8003 stars]
- Hands-On NLTK Tutorial [GitHub, 405 stars]
- Modern Practical Natural Language Processing [GitHub, 250 stars]
- r/LanguageTechnology - NLP Reddit forum
- NeuralCoref 4.0: Coreference Resolution in spaCy with Neural Networks by HuggingFace [GitHub, 2154 stars]
- tokenizers - Fast State-of-the-Art Tokenizers optimized for Research and Production [GitHub, 4113 stars]
- SentencePiece - Unsupervised text tokenizer for Neural Network-based text generation [GitHub, 4673 stars]
- SoMaJo - A tokenizer and sentence splitter for German and English web and social media texts [GitHub, 82 stars]
- WildNLP Text manipulation library to test NLP models [GitHub, 63 stars]
- snorkel Framework to generate training data [GitHub, 4368 stars]
- NLPAug Data augmentation for NLP [GitHub, 1495 stars]
- SentAugment Data augmentation by retrieving similar sentences from larger datasets [GitHub, 250 stars]
- TextAttack - framework for adversarial attacks, data augmentation, and model training in NLP [GitHub, 1154 stars]
- A Visual Survey of Data Augmentation in NLP [Blog, 2020]
- Weak Supervision: A New Programming Paradigm for Machine Learning [Blog, March 2019]
- Datasets for Entity Recognition [GitHub, 831 stars]
- Datasets to train supervised classifiers for Named-Entity Recognition [GitHub, 216 stars]
- Bootleg - Self-Supervision for Named Entity Disambiguation at the Tail [GitHub, 61 stars]
- tacred-relation TACRED: position-aware attention model for relation extraction [GitHub, 262 stars]
- tacrev TACRED Revisited: A Thorough Evaluation of the TACRED Relation Extraction Task [GitHub, 33 stars]
- tac-self-attention Relation extraction with position-aware self-attention [GitHub, 55 stars]
- Neural Adaptation in Natural Language Processing - curated list [GitHub, 109 stars]
- CMU LTI Low Resource NLP Bootcamp 2020 - CMU Language Technologies Institute low resource NLP bootcamp 2020 [GitHub, 449 stars]
- NeuSpell - A Neural Spelling Correction Toolkit [GitHub, 62 stars]
- SymSpellPy - Python port of SymSpell [GitHub, 395 stars]
- nlp-gym - NLPGym - A toolkit to develop RL agents to solve NLP tasks [GitHub, 68 stars]
- TPOT - Python Automated Machine Learning tool [GitHub, 7716 stars]
- Auto-PyTorch - Automatic architecture search and hyperparameter optimization for PyTorch [GitHub, 1101 stars]
- HungaBunga - Brute-Force all sklearn models with all parameters using .fit .predict [GitHub, 606 stars]
- AutoML Natural Language - Google's paid AutoML NLP service
License CC0
- All linked resources belong to original authors
- Akropolis by parkjisun from the Noun Project
- Book of Ester by Gilad Sotil from the Noun Project
- quill by Juan Pablo Bravo from the Noun Project
- acting by Flatart from the Noun Project
- olympic by supalerk laipawat from the Noun Project
- aristocracy by Eucalyp from the Noun Project
- Horn by Eucalyp from the Noun Project
- temple by Eucalyp from the Noun Project
- constellation by Eucalyp from the Noun Project
- ancient greek round pattern by Olena Panasovska from the Noun Project
- Harp by Vectors Point from the Noun Project
- Atlas by parkjisun from the Noun Project
- Parthenon by Eucalyp from the Noun Project
- papyrus by IconMark from the Noun Project
- papyrus by Smalllike from the Noun Project
- pegasus by Saeful Muslim from the Noun Project