v0.8.4

What's Changed

• Add codellama to tokenizer list for set_pad_token by @Infernaught in #3598
• Set default eval batch size to 2 for LLM fine-tuning by @arnavgarg1 in #3599
• [CI] Explicitly set eval batch size in determinism tests, introduce a new integration test group, and exclude slow tests. by @justinxzhao in #3590
• [CI] Run sudo apt-get update in GHAs. by @justinxzhao in #3608
• Store steps_per_epoch in Trainer by @hungcs in #3601
• Updated characters, underscore and comma preprocessors to be TorchScriptable. by @martindavis in #3602
• [CI] Deflake: Explicitly set eval batch size for mlflow test. by @justinxzhao in #3612
• Fix registration for char error rate. by @justinxzhao in #3604
• fix: Load 8-bit quantized models for eval after fine-tuning by @jeffkinnison in #3606
• Add Code Alpaca and Consumer Complaints Datasets by @connor-mccorm in #3611
• Add support for gradient checkpointing for LLM fine-tuning by @arnavgarg1 in #3613
• Bump min support transformers to 4.33.0 by @tgaddair in #3616
• [CI] Fix failing tests on master by @arnavgarg1 in #3617
• Eliminate short-circuiting for loading from local by @Infernaught in #3600
• Refactor integration tests into matrix by @tgaddair in #3618
• fix: Check underlying model device type when moving 8-bit quantized models to GPU at eval by @jeffkinnison in #3622
• Fixed range validation for text generation penalty parameters by @tgaddair in #3623
• Update comment for predict to update Ludwig docs by @Infernaught in #3535
• Avoid deprecation warnings on pandas Series.fillna by @carlogrisetti in #3631
• QoL: Default to using fast tokenizer for Llama models by @arnavgarg1 in #3625
• fixed typo in EfficientNet's model variant from v2_ to v2_s by @saad-palapa in #3628
• Add pytorch profiler and additional tensorboard logs for GPU memory usage. by @justinxzhao in #3607
• Pin minimum transformers version to 4.33.2 by @arnavgarg1 in #3637

New Contributors

• @saad-palapa made their first contribution in #3628

Full Changelog: v0.8.3...v0.8.4