The changelog will record what content was changed (e.g. changed an existing paragraph to a better-worded version, re-ran the notebook using an updated version of the package, introduced new content to existing notebook), added (e.g. a completely new jupyter notebook).
- Direct Preference Optimization (DPO) [nbviewer][html]
- LLM Pairwise Judge (PyTorch Lightning) [nbviewer][html]
- Introduction to CLIP (Contrastive Language-Image Pre-training), LiT, ViT [nbviewer][html]
- Massive overhaul to the content using latest version of PyTorch 2.
- Switched to using huggingface transformer ViT image encoder, instead of timm's ResNet.
- Added quantitative evaluation with retrieval recall@k.
- Added additional introduction to LiT, ViT.
- Finetuning Pre-trained BERT Model on Text Classification Task And Inferencing with ONNX Runtime. [nbviewer][html]
- Overhaul the content using latest version of PyTorch 2.
- Use huggingface trainer for model training/evaluation instead of custom implementation.
- Benchmarked ONNX versus PyTorch on both GPU and CPU.
- Deep Learning for Tabular Data - PyTorch. [nbviewer][html]
- Overhaul the content using latest version of PyTorch 2.
- Use huggingface trainer for model training/evaluation instead of custom implementation.
- Removed outdated content around ONNX which are not relevant for this particular topic.
- Training Bi-Encoder Models with Contrastive Learning Notes. [nbviewer][html]
- Update to include a section on data augmentation as well as various clarification on wordings.
- Response Knowledge Distillation for Training Student Model. [nbviewer][html]
- Re-ran with huggingface dataset logging disabled, this is to prevent messages from flooding main content.
- Sentence Transformer: Training Bi-Encoder via Contrastive Loss. [nbviewer][html]
- Re-ran with huggingface dataset logging disabled, this is to prevent messages from flooding main content.
- Training Bi-Encoder Models with Contrastive Learning Notes. [nbviewer][html]
- Introduction to CLIP (Contrastive Language-Image Pre-training) [nbviewer][html]
- Machine Translation with Huggingface Transformers mT5. [nbviewer][html]
- This is a complete overhaul of the original Machine Translation with Huggingface Transformers article, which was a bit obsolete.
- Response Knowledge Distillation for Training Student Model. [nbviewer][html]
- Added a final notes section on distilbert, well read students read well.
- Quick Introduction to Graph Neural Network Node Classification Task (DGL, GraphSAGE). [nbviewer][html]
- Probability Calibration for deep learning classification models with Temperature Scaling. [nbviewer][html]
- Finetuning Pre-trained BERT Model on Text Classification Task And Inferencing with ONNX Runtime. [nbviewer][html]
- Removed mlutils: Machine learning utility function package. A lot of its contents are not well-maintained, as a result, are already out-dated.
- LightGBM API walkthrough and a discussion about categorical features in tree-based models. [nbviewer][html]
- Upgrade LightGBM to 3.0.0, and deprecate out-dated content.
- Xgboost API walkthrough (includes hyperparameter tuning via scikit-learn like API). [nbviewer][html]
- Upgrade XGBoost to 1.2.1, and deprecate out-dated content.
- Probability Calibration for classification models. [nbviewer][html]
- Massive overhaul to the content. e.g. introducing two additional calibration methods, histogram binning and Plat Scaling Binning. Bundling all helper utility function in a package structure for ease of re-use.
- Multi-Label Text Classification with Fasttext and Huggingface Tokenizers. [nbviewer][html]
- Update Huggingface Tokenizers to 0.8.1 API.
- Product Quantization for Model Compression. [nbviewer][html]
- Maximum Inner Product for Speeding Up Generating Recommendations. [nbviewer][html]
- Extremely Quick Guide to Unicode. [markdown]
- MultiLabel Text Classification with Fasttext and Huggingface Tokenizers. [nbviewer][html]
- FastAPI & Azure Kubernetes Cluster. End to end example of training a model and hosting it as a service. [folder]
- Added application load testing with Apache Jmeter.
- FastAPI & Azure Kubernetes Cluster. End to end example of training a model and hosting it as a service. [folder]
- Added more best practices when specifying a deployment.
- FastAPI & Azure Kubernetes Cluster. End to end example of training a model and hosting it as a service. [folder]
- Parallel programming with Python (threading, multiprocessing, concurrent.futures, joblib). [nbviewer][html]
- Added a short section to asynchronous programming.
- Monotonic Constraint with Boosted Tree. [nbviewer][html]
- The original notebook uses xgboost to demonstrate the feature. Added lightgbm example.
- Logging module. [nbviewer][html]
- Added a section that emphasizes the importance of logging the full stack trace of an exception.
- Kaggle: Quora Insincere Questions Classification Predicting insincere questions. [folder]
- Seq2Seq for German to English Machine Translation - PyTorch. Includes quick intro to torchtext [nbviewer][html]
- Added more introduction to torchtext.
- Byte Pair Encoding (BPE) from scratch and quick walkthrough of sentencepiece. [nbviewer][html]
- Sentencepiece Subword tokenization for Text Classification. [nbviewer][html]
- Gaussian Mixture Model from scratch; AIC and BIC for choosing the number of Gaussians. [nbviewer][html]
- Fix erroneous log likelihood calculation.
- Update deprecated function for plotting contour plots.
- Leveraging Pre-trained Word Embedding for Text Classification. [nbviewer][html]
- Monotonic Constraint with Boosted Tree. [nbviewer][html]
- Probability Calibration for classification models. [nbviewer][html]
- Kaggle: Rossman Store Sales Predicting daily store sales. Also introduces deep learning for tabular data. [folder]
- Optimizing Pandas (e.g. reduce memory usage using category type). [nbviewer][html]
- Added helper function to automatically determine optimal data type.
- Framing time series problem as supervised-learning. [nbviewer][html]
- Added window-based features.
- Word2vec (skipgram + negative sampling) using Gensim. [nbviewer][html]
- Update to the more efficient file-based training.
- Quick Example of Factory Design Pattern. [nbviewer][html]
- Introduction to Multi-armed Bandits. [nbviewer][html]
- Quantile Regression and its application in A/B testing.
- Kullback-Leibler (KL) Divergence. [nbviewer][html]
- Calibrated Recommendation for reducing bias/increasing diversity in recommendation. [nbviewer][html]
- Influence Maximization from scratch. Includes discussion on Independent Cascade (IC), Submodular Optimization algorithms including Greedy and Lazy Greedy, a.k.a Cost Efficient Lazy Forward (CELF) [nbviewer][html]
Introduction to Residual Networks (ResNets) and Class Activation Maps (CAM). [nbviewer][html]
Hosted html-version of all jupyter notebook on github pages.
- (Text) Content-Based Recommenders. Introducing Approximate Nearest Neighborhood (ANN) - Locality Sensitive Hashing (LSH) for cosine distance from scratch. [nbviewer]
- Benchmarking ANN implementations (nmslib). [nbviewer]
- Getting started with time series analysis with Exponential Smoothing (Holt-Winters). [nbviewer]
- Framing time series problem as supervised-learning. [nbviewer]
- Tuning Spark Partitions. [nbviewer]
- Evaluation metrics for imbalanced dataset. [nbviewer]
- H2O API walkthrough (using GBM as an example). [nbviewer]
- Moved H2O notebook to its own sub-folder.
- Added model interpretation using partial dependence plot.
- RNN, LSTM - PyTorch hello world. [nbviewer]
- Recurrent Neural Network (RNN) - language modeling basics. [nbviewer]
- Long Short Term Memory (LSTM) - Tensorflow. [nbviewer]
- Vanilla RNN - Tensorflow. [nbviewer]
- WARP (Weighted Approximate-Rank Pairwise) Loss using lightfm. [nbviewer]
- Local Hadoop cluster installation on Mac. [markdown]
- Spark MLlib Binary Classification (using GBM as an example). [raw zeppelin notebook][Zepl]
- H2O API walkthrough (using GBM as an example). [nbviewer]
- Factorization Machine from scratch. [nbviewer]
- The
sparkfolder has been renamed tobig_datato incorporate other big data tools.
- Partial Dependence Plot (PDP), model-agnostic approach for directional feature influence. [nbviewer]
- Parallel programming with Python (threading, multiprocessing, concurrent.futures, joblib). [nbviewer]
- LightGBM API walkthrough and a discussion about categorical features in tree-based models. [nbviewer]
- Curated tips and tricks for technical and soft skills. [nbviewer]
- Detecting collinearity amongst features (Variance Inflation Factor for numeric features and Cramer's V statistics for categorical features), also introduces Linear Regression from a Maximum Likelihood perspective and the R-squared evaluation metric. [nbviewer]
- Random Forest from scratch and Extra Trees. [nbviewer]
- Refactored code for visualizating tree's feature importance.
- Building intuition on Ridge and Lasso regularization using scikit-learn. [nbviewer]
- Include section when there are collinear features in the dataset.
- mlutils: Machine learning utility function package [folder]
- Refer to its changelog for details.
- data_science_is_software. [nbviewer]
- Mention notebook extension, a project that contains various functionalities that makes jupyter notebook even more pleasant to work with.
- Introduction to Singular Value Decomposition (SVD), also known as Latent Semantic Analysis/Indexing (LSA/LSI). [nbviewer]
- mlutils: Machine learning utility function package [folder]
- Bernoulli and Multinomial Naive Bayes from scratch. [nbviewer]
- Fixed various typos and added a more efficient implementation of Multinomial Naive Bayes.
- TF-IDF (text frequency - inverse document frequency) from scratch. [nbviewer]
- Moved to its own tfidf folder.
- Included the full tfidf implementation from scratch.
- Using built-in data structure and algorithm. [nbviewer]
- Merged the content from the two notebooks
namedtuple and defaultdictandsorting with itemgetter and attrgetterinto this one and improved the section on priority queue.
- Merged the content from the two notebooks
- Understanding iterables, iterator and generators. [nbviewer]
- Word2vec (skipgram + negative sampling) using Gensim (includes text preprocessing with spaCy). [nbviewer]
- Frequentist A/B testing (includes a quick review of concepts such as p-value, confidence interval). [nbviewer]
- AUC (Area under the ROC, precision/recall curve) from scratch (includes building a custom scikit-learn transformer). [nbviewer]
- Optimizing Pandas (e.g. reduce memory usage using category type). [nbviewer]
- This is a revamp of the old content Pandas's category type.
- cohort : Cohort analysis. Visualize user retention by cohort with seaborn's heatmap and illustrating pandas's unstack. [nbviewer]
- Bayesian Personalized Ranking (BPR) from scratch & AUC evaluation. [nbviewer]
- A more efficient matrix operation using Hadamard product.
- Cython and Numba quickstart for high performance python. [nbviewer]
- Added Numba parallel prange.
- ALS-WR for implicit feedback data from scratch & mean average precision at k (mapk) and normalized cumulative discounted gain (ndcg) evaluation. [nbviewer]
- Included normalized cumulative discounted gain (ndcg) evaluation.
- Gradient Boosting Machine (GBM) from scratch. [nbviewer]
- Added a made up number example on how GBM works.
- data_science_is_software. [nbviewer]
- Mention
nbdime, a tool that makes checking changes in jupyter notebook on github a lot easier. - Mention semantic versioning (what each number in the package version usually represents).
- Mention
configparser, a handy library for storing and loading configuration files.
- Mention
- K-fold cross validation, grid/random search from scratch. [nbviewer]
- Minor change in Kfolds educational implementation (original was passing redundant arguments to a method).
- Minor change in random search educational implementation (did not realize scipy's .rvs method for generating random numbers returns a single element array instead of a number when you pass in size = 1).
This is the first time that the changelog file is added, thus every existing notebook will fall under the added category. Will try to group the log by month (one or two) in the future. Note that this repo will be geared towards Python3. Hence, even though the repo contains some R-related contents, they are not that well maintained and will most likely be translated to Python3. As always, any feedbacks are welcomed.
- Others (Genetic Algorithm)
- Regression (Linear, Ridge/Lasso)
- Market Basket Analysis (Apriori)
- Clustering (K-means++, Gaussian Mixture Model)
- Deep Learning (Feedforward, Convolutional Neural Nets)
- Model Selection (Cross Validation, Grid/Random Search)
- Dimensionality Reduction (Principal Component Analysis)
- Classification (Logistic, Bernoulli and Multinomial Naive Bayes)
- Text Analysis (TF-IDF, Chi-square feature selection, Latent Dirichlet Allocation)
- Tree Models (Decision Tree, Random Forest, Extra Trees, Gradient Boosting Machine)
- Recommendation System (Alternating Least Squares with Weighted Regularization, Bayesian Personalized Ranking)
- Python Programming (e.g. logging, unittest, decorators, pandas category type)