Visit the docs link for Literature Review.
Check all the colab notebooks here.
Check the document on LLM Quantization and Benchmarking here. It includes all my implementation details, challenges and adaptations.
Check the document on Deploying LLM to Mobile here. It includes all my implementation details, challenges and adaptations.
Published studio at Lightning AI
- Post-Training Quantization to GGUF format and Evaluation
- Post-Training Quantization to GPTQ format and Evaluation
- Benchmark GGUF format quantized model using lm-evaluation-harness and llama-cpp-python
- HumanEval benchmark (non-quantized, quantized (GPTQ, GGUF))
- Research SmoothQuant
- Pruning
- Distillation