[GPT2] Fix gradient checkpointing bug #21772
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The documentation is not available anymore as the PR was closed or merged. |
|
cc @gante |
|
Hey @yhl48 👋 to make our CI pass, you'll have to run In a nutshell, we have a system in place that ensures that the code in a few models stays the synchronized. |
5 tasks
|
@yhl48 uhmmm something went wrong. What does your terminal print after running |
|
|
I think the file Also because I was working on the main branch locally and pushed to a different branch, so that has caused some confusion with #21773. I should probably keep just one of #21772 and #21773, fixing the two models in one PR. |
gante
approved these changes
Feb 24, 2023
Closed
42 tasks
gongbaitao
pushed a commit
to pioliverse/transformers
that referenced
this pull request
Feb 27, 2023
* fix gradient checkpointing bug * fix gradient checkpointing bug * ran make fix-copies * fixed bug * fixed bug
ArthurZucker
pushed a commit
to ArthurZucker/transformers
that referenced
this pull request
Mar 2, 2023
* fix gradient checkpointing bug * fix gradient checkpointing bug * ran make fix-copies * fixed bug * fixed bug
oushu1zhangxiangxuan1
added a commit
to oushu1zhangxiangxuan1/transformers
that referenced
this pull request
Mar 20, 2023
* Fix 2 quicktour file doctest (#21742)
* Update expect output values - as Hub repo. files are updated
* Update expect output values - as librosa is from 0.9.2 to 0.10.0 on CI docker
* fix
* update one more
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* [`GPTNeo`] Fix gradient checkpointing bug (#21733)
* fix bug
* forward contrib credits from discussions
* change logic
---------
Co-authored-by: edbeeching <edbeeching@users.noreply.github.com>
* Generate: Fix GIT batched captioning (#21738)
* Skip test_log_level for now
* Added Type Hints for modeling_tf_encoder_decoder.py (#21673)
* Ran Black formatting
* Added imports and reformatted
* Update src/transformers/models/encoder_decoder/modeling_tf_encoder_decoder.py
---------
Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>
* Auto api Value Error addition to Troubleshoot (#21708)
* troubleshooting guide: added an error description for missing auto-mapping
* minor polishing
* changed the example
* Apply suggestions from code review
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/troubleshooting.mdx
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* [deepspeed tests] fix issues introduced by #21700 (#21769)
* [deepspeed tests] fix issues introduced by #21700
* fix
* fix
* Graphormer fix (#21699)
* Removed useless check for backend
* fix style check for graphormer
* Reverted change and corrected requires_backend for cython
* code qual
* fix: Change is_last chunk calc and add conditional break in chunk_iter (#21612)
* fix: Change is_last chunk calc and add conditional break
* format fix
* account for 0 and full stride_rights, add comment
* add new test
* make style
* update slow whisper asr test timestamps
* use nested_simplify on output and round timestamp to hundreths place
* [Flax] adding support for batch norm layers (#21581)
* [flax] adding support for batch norm layers
* fixing bugs related to pt+flax integration
* cleanup, batchnorm support in sharded pt to flax
* support for batchnorm tests in pt+flax integration
* simplifying checking batch norm layer
* [Examples] Generalise run audio classification for log-mel models (#21756)
* [Examples] Generalise run audio classification for log-mel models
* batch feature extractor
* make style
* Different behavior in DistilBERT when using "inputs_embeds" (#21752)
* Different behavior in DistilBERT when using "inputs_embeds"
Fixes #21089
* fix failing test
* [Flax] Fix erroneous kwargs being passed to generate config (#21765)
* [Whisper] Add SpecAugment (#21298)
* Return and rescale attention_mask
* Add SpecAugment to Whisper modeling
* Fix test
* Update docstring
* Add SpecAug related parameters to model config
* Add the _mask_input_features function to doc
* Fix quality
* Apply suggestions from code review
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* Remove dev comments
* Add test
* Resolve conflict
* feat: mask {feature, time} prob fast tests
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
---------
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
Co-authored-by: sanchit-gandhi <sanchit@huggingface.co>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Fix-ci-whisper (#21767)
* fix history
* input_features instead of input ids for TFWhisport doctest
* use translate intead of transcribe
* Generate - update cookie cutters to not initialize cache with training and gradient checkpointing (#21759)
* [time series] updated expected values for integration test. (#21762)
* updated expected
* prediction_length fix
* prediction_length default value
* default prediction_length 24
* revert back prediction_length default
* move prediction_length test
* [GPT2, ProphetNet] Fix gradient checkpointing bug (#21772)
* fix gradient checkpointing bug
* fix gradient checkpointing bug
* ran make fix-copies
* fixed bug
* fixed bug
* [SpeechT5] Fix HiFiGAN tests (#21788)
* Fix resume_from_checkpoint for deepspeed (#21735)
* Fix resume_from_checkpoint for deepspeed
Fix resume_from_checkpoint for deepspeed, by ensuring that the deepspeed engine is the one to load the checkpoint.
* Empty commit to trigger CI
* Removed deepspeed skipping
Removed deepspeed skipping inside the _load_from_checkpoint function, as it is obsolete
* another adjustment
* Trigger CI
* trigger circleci
* style
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: Stas Bekman <stas@stason.org>
* [examples/summarization] deal with `max_length` and `num_beams` (#21740)
* Override the decoding parameters of Seq2SeqTrainer
* Fix quality
* Fix max_length parameter
* Fix quality
* Remove redundant parameter max_length
* Separate the preprocess of train and validation to use different max_target_length
* Fix type in gpt2 config docstring (#21782)
Fix docstring gpt2 config
* Fix en documentation typos (#21799)
* fix wrong url
* typos in english documentation
* [FX tracer] Make `concrete_args` from outside available (#21775)
make concrete_args from outside available
* [Pipeline] Add zero shot audio classificatoin pipeline (#21600)
* add pipeline
* update init
* add zero shot to init
* update inits and correct checkpoints
* update base to support input features
* add tests
* Update src/transformers/pipelines/zero_shot_audio_classification.py
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
* Update src/transformers/pipelines/zero_shot_audio_classification.py
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
* update pieline code
* use tiny checkpoint
* nits and expected value with tiny model
* style
* last nit on tests values
* fix styling
* fix collate fn that was casting t float
* update
---------
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
* [torch] remove deprecated uint8 in favor of bool (#21384)
* uint8 -> bool
* fix copies
* style
* update test modeling commen when checking attention buffers
* style
* use logical not on random mask instead of subtraction with 1
* remove torch uint8
* quality
* remove modified modeling utils
* Update based on review
Co-authored-by: sgugger <sylvain.gugger@gmail.com>
---------
Co-authored-by: sgugger <sylvain.gugger@gmail.com>
* [`tests`] add `accelerate` marker (#21743)
* add `accelerate` marker
* add to docs
* Update docs/source/en/testing.mdx
* Fix PyTorch Perceiver `PerceiverFourierPositionEncoding` with fp16 (#21787)
* fix perceiver fp16
* hopefully fix tests
* Fix nn.init.trunc_normal_ call on torch.float16 data (#21789)
fix nn.init.trunc_normal_ call on half data
* Fix gradient checkpointing bug in gptneox (#21815)
* Fix gradient checkpointing bug in gptneox
* Remove use_cache block
* Inheritance-based framework detection (#21784)
* Fix quality with `ruff==0.0.253` (#21828)
fix quality with ruff 0.0.253
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* introduce `logger.warning_once` and use it for grad checkpointing code (#21804)
* logger.warning_once
* style
* Rename `MobileViTModelTest` to `TFMobileViTModelTest` (#21825)
Let's give TF a bit more love ❤️ 🙏
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* Fix gradient checkpointing bug BioGpt (#21844)
Co-authored-by: saswatmeher <saswatmeher@cse.iitb.ac.in>
* check for None forced tokens (#21793)
* Fix gradient checkpointing bug in git (#21818)
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Fix gradient checkpointing imagegpt (#21816)
* Fix gradient checkpointing bug in gptneox
* Fix gradient checkpointing bug in modeling_imagegpt.py
* Revert gpt neox changes
---------
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Fix tf random token masking probability in data collator (#21834)
* fix tf random mask tokens probability
* fix tf random mask tokens probability in collator for langauge modelling
* [`T5`] Fix torchquant issue (#21843)
* fix torchquant issue
* add tests
* [`Blip2`] Add `Blip2Model` (#21817)
* add v1
* add `Blip2Model`
- add relevant functions
- add tests
- add on automapping
* fix docs
* fix doctest
* Fix the issue of blip model returning loss even when the label is not provided. (#21811)
* Fix the issue of blip model returning loss even when the label is not provoided
* Fix ruff failure
* Incorporate PR feedbacks
* Incorporate PR feedbacks
* Incorporate PR feedbacks
* Incorporate PR feedbacks
* [GPTJ] Fix gradient checkpointing bug (#21794)
* If applied, this commit fixes generate bug in gptj
* Remove extra same code block
* formatting and test fix
* Conflict fix and declaration error fix
---------
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Add: task guide for zero shot object detection (#21829)
* zero shot object detection part 1
* added batch prediction section
* added image guided object detection section
* make style
* added the task guide to the TOC
* minor polishing
* Apply suggestions from code review
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Co-authored-by: Alara Dirik <8944735+alaradirik@users.noreply.github.com>
* added embedded owlvit demo
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* minor fix
* make style
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Co-authored-by: Alara Dirik <8944735+alaradirik@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Make Slack CI reporting stronger (#21823)
* Use token
* Avoid failure
* better error
* Fix
* fix style
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* [`Blip2`] Fix Blip-2 multi gpu (#21707)
* fix blip multi gpu
* fix
* final changes
* adapt suggestions
* fix failing slow test
* forward contrib credits from testing and suggestions
* reformat
---------
Co-authored-by: akkikiki <akkikiki@users.noreply.github.com>
* Add loss for BridgeTowerForMaskedLM and BridgeTowerForImageAndTextRetrieval (#21684)
* Add loss for BridgeTowerForMaskedLM and BridgeTowerForImageAndTextRetrieval
* minor fix return_dict
* implement test for loss computation
---------
Co-authored-by: Tiep Le <97980157+tileintel@users.noreply.github.com>
Co-authored-by: Tiep Le <tiep.le@intel.com>
* 🔥Rework pipeline testing by removing `PipelineTestCaseMeta` 🚀 (#21516)
* Add PipelineTesterMixin
* remove class PipelineTestCaseMeta
* move validate_test_components
* Add for ViT
* Add to SPECIAL_MODULE_TO_TEST_MAP
* style and quality
* Add feature-extraction
* update
* raise instead of skip
* add tiny_model_summary.json
* more explicit
* skip tasks not in mapping
* add availability check
* Add Copyright
* A way to diable irrelevant tests
* update with main
* remove disable_irrelevant_tests
* skip tests
* better skip message
* better skip message
* Add all pipeline task tests
* revert
* Import PipelineTesterMixin
* subclass test classes with PipelineTesterMixin
* Add pipieline_model_mapping
* Fix import after adding pipieline_model_mapping
* Fix style and quality after adding pipieline_model_mapping
* Fix one more import after adding pipieline_model_mapping
* Fix style and quality after adding pipieline_model_mapping
* Fix test issues
* Fix import requirements
* Fix mapping for MobileViTModelTest
* Update
* Better skip message
* pipieline_model_mapping could not be None
* Remove some PipelineTesterMixin
* Fix typo
* revert tests_fetcher.py
* update
* rename
* revert
* Remove PipelineTestCaseMeta from ZeroShotAudioClassificationPipelineTests
* style and quality
* test fetcher for all pipeline/model tests
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* Improve TF weight loading, especially PT crossloading (#21792)
* First commit for the improved PT-TF weight loading
* Remove workarounds from TFEncoderDecoder tests
* Allow a custom weight renaming function in from_pretrained and use that to clean up EncoderDecoder
* make fixup
* First attempt at visionencoderdecoder
* Disable tensorfloat32 in tests to get consistent outputs
* Quick fix to tf_vision_encoder_decoder tests
* make fixup
* Update Blenderbot tests
* Remove unused arg in modeling_tf_opt
* load_tf_sharded_weights had strict=True! This meant transfer learning was impossible, so I'm setting it to False.
* Support prefixes when loading sharded TF checkpoints
* make fixup
* Add test to load sharded models with a weight prefix
* Fix sharded weight loading test
* Add a test for transfer from a sharded checkpoint
* make fixup
* Add test to check that crossloading from PT with a prefix works
* Refactor from_pretrained in the encoderdecoder classes
* Refactor from_pretrained in the encoderdecoder classes
* missmatched -> mismatched
* Explicitly check for None
* No comments showing my very impressive and attractive knowledge of Py3.9+
* Disable TF32 across all TF tests
* Fix flaky test for log level (#21776)
* Fix flaky test for log level
* Fix other flaky test
* prepare for "__floordiv__ is deprecated and its behavior will change in a future version of pytorch" (#20211)
* rounding_mode = "floor" instead of // to prevent behavioral change
* add other TODO
* use `torch_int_div` from pytrch_utils
* same for tests
* fix copies
* style
* use relative imports when needed
* Co-authored-by: sgugger <sylvain.gugger@gmail.com>
* [ConvBert] Fix #21523 (#21849)
* fix reshaping
Fixes #21523
* add test
* styling
* last fixes
* Update src/transformers/models/convbert/modeling_convbert.py
* code quallity
* Flax beam search fix (#21857)
* Fix gradient checkpointing bug Bart (#21866)
Co-authored-by: saswatmeher <saswatmeher@cse.iitb.ac.in>
* [deepspeed] check whether model is NLP one instead of counting on input type (#21800)
* trying to figure out whether model is NLP
* drop my changes and apply easier fix
* trying to handle all int input types
* fix logic
---------
Co-authored-by: Stas Bekman <stas@stason.org>
* Change the way tensor is reshaped in BartAttention (from .view to .reshape) (#21860)
* Change the .view call to .reshape
* Change the .view call to .reshape to all the copies from bart attention
* Fix copies and style
* Fix copies and style
* Fix copies and style
* Fix copies and style
* Fix copies and style
* Revert unneccessary changes
* Revert unneccessary changes
* Revert unneccessary changes
* Revert unneccessary changes
* Italian translation of community.mdx (#21871)
Italian translation of community.mdx gh-17459
* [`Blip`] Fix blip doctest (#21868)
fix blip doctest
* Removed BLIP mention from the troubleshooting guide (#21872)
removed BLIP mention from the troubleshooting guide
* update FSDP and add XLA-FSDP documentation (#21812)
* update FSDP and add XLA-FSDP documentation
* resolving comments
* minor update
* fix xla-fsdp docs
* [doc] deepspeed tests (#21859)
* Add an utility file to get information from test files (#21856)
* Add an utility file to get information from test files
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* Add check for different embedding types in examples (#21881)
* Add check for different embedding types in examples
* Correctly update summarization example
* Make loading of pretrained gpt2 faster by avoiding initialization of Conv1D's weights (#21879)
apply normal_ after assigning weight as nn.Parameter to avoid unnecessary initialization computation
* Add TFVisionTextDualEncoder (#21873)
* Temporary commit to stash everything so far
* Temporary commit to stash everything so far
* stash commit
* Refactor from_pretrained
* Fix final test, make fixup
* Update dummies
* Add model to TEST_FILES_WITH_NO_COMMON_TESTS
* Update src/transformers/models/vision_text_dual_encoder/modeling_tf_vision_text_dual_encoder.py
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
* Update src/transformers/models/vision_text_dual_encoder/modeling_tf_vision_text_dual_encoder.py
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
* Update src/transformers/models/vision_text_dual_encoder/modeling_tf_vision_text_dual_encoder.py
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
* Update src/transformers/models/vision_text_dual_encoder/modeling_tf_vision_text_dual_encoder.py
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
* Add TFVisionTextDualEncoder to utils/documentation_tests.txt
* make fixup
---------
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
* Add ALIGN to transformers (#21741)
Adds the ALIGN model to transformers. ALIGN is introduced in "Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision" by Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc V. Le, Yunhsuan Sung, Zhen Li, Tom Duerig.
* Fix Gradient checkpointing bug BigBird (#21882)
Co-authored-by: saswatmeher <saswatmeher@cse.iitb.ac.in>
* Fix `WhisperModelTest` (#21883)
* force on the same device
* fix tests
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* Fix `test_load_default_pipelines_pt` for `ClapModel` (#21886)
* fix tests
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* fix checkpoint (#21874)
* [Refactor] Relative imports wherever we can (#21880)
* initial commit
* update
* second batch
* style
* fix imports
* fix relative import on pipeline
* [ZAC] fix ci daily (#21893)
add correct revision after model was overwritten
* Use PyAV instead of Decord in examples (#21572)
* Use PyAV instead of Decord
* Get frame indices
* Fix number of frames
* Update src/transformers/models/videomae/image_processing_videomae.py
* Fix up
* Fix copies
* Update timesformer doctests
* Update docstrings
* Add `inputs_embeds` functionality when generating with BioGPT (#21889)
* initial commit to add inputs_embeds to generation
* formatting
* [T5 doc] Fix confusing documentation about `d_kv` (#21896)
* Confusing documentation in T5
* Fix onfusing documentation in T5 configuration file
* [Whisper] Add rescaling function with `do_normalize` (#21263)
* add `zero_mean_unit_var_norm` function
* normalize before MEL computation
* fixup
* add simple test
* quality
* Update tests/models/whisper/test_feature_extraction_whisper.py
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
* fixup
* use attention masks if padding was applied
* Update based on review
Co-authored-by: bofeng huang <bofenghuang7@gmail.com>
---------
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
Co-authored-by: bofeng huang <bofenghuang7@gmail.com>
* fix typo in Bart's attention (#21898)
* [GPT-J] add deprecation warning (#21869)
* add deprecation warning
* remove pos ids from args docstirng
* fix failing test
* fsdp bf16 enable autocast (#21847)
* Fix gradient checkpointing bug LED (#21840)
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Fix gradient checkpointing bug M2M 100 (#21841)
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Fix gradient checkpointing bug marian (#21842)
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Mark pipeline tests to skip them easily (#21887)
* Mark pipeline tests to skip them easily
* Mark the mixin as pipeline test
* Update src/transformers/testing_utils.py
Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>
---------
Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>
* Clean up auto mapping names (#21903)
* add new test
* fix after new test
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* Prophetnet batch dimension inversion fix (#21870)
* decoder forward pass is working
* no model has forward pass returning attentions
* decoder ngram changed to not mix batch size
* current basic forward pass returns identical result
* passed test_model attentions
* passed test_encoder_decoder_model_generate
* passed test_headmasking
* removed old block
* removed comments bug/fixme
* removed bug comments
* applied styling
* applied fix-copies
* applied ngram forward comments
* corrected dimension notation
* applied styling and comment fixes
* changed asserts for raise ValueError
* changed question gen test
* updated hidden_states integration test
* applied styling
* Make schedulers picklable by making lr_lambda fns global (#21768)
* Make schedulers picklable by making lr_lambda fns global
* add unused _get_constant_schedule_lr_lambda arg
* remove unneeded _get_constant_schedule_lr_lamda
* add test
* make style
* rebase, remove torch dep, put lambda back
* repo-consistency and style
* Refactor whisper asr pipeline to include language too. (#21427)
* [WIP] whisper refacto to support language output.
* Handling merges.
* A bit more cleanup and comments.
* Many improvements.
Lots of details everywhere.
* Cleanup old code and tests.
* Handle lone timestamp tokens (just recover when something bad happens).
* Adding return_language example.
* No ffmpeg.
* Hmm.
* Some corrections.
* Both fast and slow.
* New black.
* Update src/transformers/models/whisper/tokenization_whisper.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* Update src/transformers/models/whisper/tokenization_whisper.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* Remove print.
* Undoing tests modifications.
* Smaller test modifications.
* Rename.
* Remove maxDiff.
---------
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* Add Blip and Blip2 for pipeline tests (#21904)
* fix
* add to tests
* style and quality
* add missing
---------
Co-authored-by: NielsRogge <NielsRogge@users.noreply.github.com>
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* Temporarily skip 3 tests in `BridgeTowerModelTest` (#21908)
skip for now
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* Faster zero shot image (#21897)
* Make ZeroShotImageClassificationPipeline faster
The pipeline makes separate calls to model for each candidate label.
This commit combines all labels into one call.
Original code takes more that 60 seconds to process one image and 1000
candidate labels. Updated code takes less than 2 seconds.
* implement batching
* code formatting
* Creating an even faster zero-shot-image-classifiction.
Unfortunately super tailored towards CLIP.
Co-Authored-By: Yessen Kanapin <yessen@deepinfra.com>
* Quality.
* Cleanup.
* Order different on the CI it seems.
* Cleanup.
* Quality.
---------
Co-authored-by: Yessen Kanapin <yessen@deepinfra.com>
* [time series] Add Time series inputs tests (#21846)
* intial test of inputs
* added test for generation
* remove asserts
* fixed test
* Update tests/models/time_series_transformer/test_modeling_time_series_transformer.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
---------
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Avoid modeling tests run in pipeline CI jobs (#21911)
* rework is_pipeline_test
* bring back 3 tests
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* Fix doctests for TFVisionTextDualEncoder (#21910)
* faster forward following what is done for images (#21906)
* faster forward following what is done for images
* add missing licence
* Fix gradient checkpointing bug in MBart (#21918)
* Fix gradient checkpointing bug in mvp (#21920)
* Fix gradient checkpointing megatron bert (#21921)
* Update `model_split_percents` for `WhisperModelTest` (#21922)
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* Use large VM for `repo_utils_job` (#21928)
upgrade to large VM
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* Cleanup more auto mapping names (#21909)
* fix auto 2
* fix auto 2
* fix task guide issue
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* feat: filter try/except when looking at custom code (#21914)
* feat: filter try/except
* Update src/transformers/dynamic_module_utils.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
---------
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Fix `AlignModelTest` tests (#21923)
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* Avoid failure in `check_repo.py` due to missing backends (#21930)
* Update utils/check_repo.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update utils/check_repo.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Fix wrong documentation about DataCollator padding defaults (#21919)
* Fix wrong documentation about DataCollator padding defaults
* Fix styling
* [Flan-UL2] Add-flan-ul2 (#21929)
* add doc and readme
* add model docs
* update toctree and fix copies
* update
* update doc file
* fix
* add FLAN-UL2 to configuration mapping
* fixup
* Apply suggestions from code review
* more clarification
---------
Co-authored-by: younesbelakda <younesbelkada@gmail.com>
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
* Update README logo (#21933)
* [CLAP] Support batched inputs for CLAP. Fixes pipeline issues (#21931)
* fix pipeline
* fix feature_extraction clap
* you can now batch the `is_longer` attribute
* add tests
* fixup
* add expected scores
* comment on is_longert
* [Whisper] Fix feature normalization in `WhisperFeatureExtractor` (#21938)
Fix feature normalization in WhisperFeatureExtractor
* Fix gradient checkpointing bug in OPT (#21943)
* Fix gradient checkpointing bug in Pegasus (#21944)
* Fix gradient checkpointing bug in Rembert (#21945)
* Fix gradient checkpointing bug in Roformer (#21946)
* Fixed gradient_checkpointing/use_cache bug in blenderbot (#21833)
* Fixed gradient_checkpointing/use_cache bug in blenderbot
* Update modeling_blenderbot.py
* Added back if statement
* Formatted using black
* Update expected values in `XLMProphetNetModelIntegrationTest` (#21957)
update values
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* [CI] Fix ci (#21940)
* fix `get_proposal_pos_embed`
* fix order
* style
* zero shot simplify test
* add approximate values for zero shot audio classification
* Disable DDP for neuron (#21953)
Disable DDp for neuron
Co-authored-by: EC2 Default User <ec2-user@ip-172-31-42-72.us-west-2.compute.internal>
* Fix bert issue (#21963)
Co-authored-by: saswatmeher <saswatmeher@cse.iitb.ac.in>
* [Generate] Fix gradient_checkpointing and use_cache bug for BLOOM (#21956)
Step 1 - Change use_cache fix
* Add missing parameter definition in layoutlm config (#21960)
Four parameters in `LayoutLM` config were missing definitions, Added their definition (copied from BertConfig).
* Use larger atol in `torch.allclose` for some tests (#21966)
Use larger atol
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* Add TF contrastive image text finetuning example (#21939)
* Initial commit
* stash commit
* Add model checkpointing and pushing
* Fix model name inference
* Update README
* Update README
* Remove a couple of Torch references
* Update copyright date
* make fixup
* Update PushToHubCallback args!
* Remove the torch summary
* Add strategy.scope
* Update expected values for `test_xglm_sample` (#21975)
update expected values for xglm
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* Fix gradient checkpointing bug in BigBird Pegasus (#21976)
* Fix gradient checkpointing bug in Blenderbot Small (#21977)
* Fix gradient checkpointing bug in BlipText (#21978)
Make Format
* Fix gradient checkpointing bug in Codegen (#21979)
* Fix gradient checkpointing bug in ESM (#21980)
* docs: improve clarity for language modeling (#21952)
* docs: improve clarity for clm/mlm
* docs: remove incorrect explanation
* docs: remove incorrect explanation
---------
Co-authored-by: pdhall99 <pdhall99>
* Update `Jukebox` tests (#21984)
* update expected values for jukebox
* update expected values for jukebox
* update expected values for jukebox
* update expected values for jukebox
* update expected values for jukebox
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* Add check before int casting for PIL conversion (#21969)
* Add check before int casting for PIL conversion
* Line length
* Tidier logic
* Fix MinNewTokensLengthLogitsProcessor when used with a list of eos tokens (#21959)
* Fix MinNewTokensLengthLogitsProcessor when used with a list of eos tokens
* fix docs
* Empty commit
* formatting
* [DETR, YOLOS] Fix device bug (#21974)
* Fix integration test
* Add test
* Add test
* Remove unneeded casts to bool (#21983)
Remove cast to Bool
* Update `notification_service.py` (#21992)
* better check
* better check
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* Skip `test_multi_gpu_data_parallel_forward` for some model tests (#21991)
skip test_multi_gpu_data_parallel_forward for some model tests
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* [Whisper] Add model for audio classification (#21754)
* [Whisper] Add model for audio classification
* make fix-copies
* add to docs
* add docstring
* empty returns
* add code example
* switch to fleurs
* stick everything on one line
* Stop requiring Torch for our TF examples! (#21997)
* Stop requiring Torch for our TF examples!
* Slight tweak to logging in the example itself
* [TF] Fix creating a PR while pushing in TF framework (#21968)
* add create pr arg
* style
* add test
* ficup
* update test
* last nit fix typo
* add `is_pt_tf_cross_test` marker for the tsts
* [DETR and friends] Remove is_timm_available (#21814)
* First draft
* Fix to_dict
* Improve conversion script
* Update config
* Remove timm dependency
* Fix dummies
* Fix typo, add integration test
* Upload 101 model as well
* Remove timm dummies
* Fix style
---------
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
* [Time-Series] informer model (#21099)
* added informer to gitignore
* added informer to gitignore
* WIP informer2020
* added checking that instantiate works
* added config using gluonTS by kashif
* WIP config
* adding informeConfig. need to remove FeatureEmbedder
* done InformerConfig, but need to change the names
* Done informer model init. working on enc-dec
* added things to address, after reading again enc-dec in the paper
* done modeling - checking initialization work
* added informer to gitignore
* WIP informer2020
* added checking that instantiate works
* added config using gluonTS by kashif
* WIP config
* adding informeConfig. need to remove FeatureEmbedder
* done InformerConfig, but need to change the names
* Done informer model init. working on enc-dec
* added things to address, after reading again enc-dec in the paper
* done modeling - checking initialization work
* moved enc-dec init to InformerEncoder/Decoder init
* added 'init_std' to config, now model init works!
* WIP conversion script, and added code sources
* WIP conversion script: loading original informer pth works
* WIP conversion script: change defaults in the config
* WIP conversion script: supporting Informer input embedding
* WIP conversion script: added parameters for the informer embed
* WIP conversion script: change dim_feedforward=2048
* WIP conversion script: remove unused args for loading checkpoint
* just cleaning up
* DataEmbedding removed, after thinking with Kashif
* working on forward pass
* WIP forward pass: trying to establish working batch for forward pass
* cleaning and finalizing
* adding HF names and docs
* init after cleaning works
* WIP in tests
* added docs for the informer specific args
* fix style
* undo change
* cleaning informer, now need to work only enc-dec
* initial enc-dec classes
* added encoder and decoder
* added todo
* add todos for conv_layers
* added decoder docs from vanilla
* added encoder docs from vanilla
* remove encoder decoder from the original informer
* removed AttentionLayer from the original paper
* removed TriangularCausalMask, same as decoder_attention_mask
* initial sparse attention
* use conv_layers
* fixed test_config test
* fix parenthesis when itearting zip(layers, conv_layers)
* error found in prob attention, added sizes as comments
* fix sizes
* added proposal for q_reduce indexing, and remove unused
* WIP ProbMask, and changed factor=2 for testing
* remove unused libs for this PR for creating the env
* fix checking the attn_weights.size() after bmm
* Q_reduce: changed from torch.gather to simple slicing
* WIP calculate final attn_output
* finish adding v_aggregated, attn_output ready
* changed tgt_len to u in attention_mask, need to fix the size error
* comment attention_mask for encoder, and fix if cond for v_agg
* added ProbMask support (wip), removed old original code
* finished ProbMask 😃
* Revert "remove unused libs for this PR for creating the env"
This reverts commit 11a081e09e92771e51a5d2758d53a9afb59547f0.
* fixes
* make style
* fix initial tests
* fix more tests
* dry
* make style
* remove unused files
* style
* added integration tests
* fix num_static_real_features
* fix header
* remove unused function
* fix example
* fix docs
* Update src/transformers/models/informer/configuration_informer.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/informer/modeling_informer.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/informer/configuration_informer.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/informer/configuration_informer.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/informer/configuration_informer.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update src/transformers/models/informer/configuration_informer.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* fixes for reviewer
* use prediction_length from model
* fix style
* fixed informer.mdx
* added to index
* updated readme
* undo
* make fix-copies
* typo
* fix copy
* added Informer to toctree
* in order
* fixed comments
* remove unneeded new lines in docs
* make static real and cat optional
* fix use of distil conv layers
* fixed integration test
* added checkpoint for convlayer
* make fix-copies
* updated from time series model
* make fix-copies
* copy decoder
* fix unit tests
* updated scaling config
* fix integration tests
* IGNORE_NON_TESTED
* IGNORE_NON_AUTO_CONFIGURED
* IGNORE_NON_AUTO_CONFIGURED
* updated check configs
* fix formatting
* undo change from time series
* prediction_length should not be None
* aliign with the blog: prettify ProbSparse and change attention_factor to sampling_factor
* make style
* make fix-copies
* niels CR: update contributed by
* niels CR: update configuration_informer.py
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* niels CR: update kashif -> huggingface
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* niels CR: `sampling_factor` only relevant when `attention_type`=prob
* make style
* fixed U_part: added multiplication by `L_Q`
* fixed bug: remove `is not None` from `if config.distil`
* fixed test: `decoder_seq_length` to `encoder_seq_length` in cross_attentions check
* fix integration tests
* updated model hub
* do not shift as in training
* undo
* fix make-copies
* make fix-copies
* added `if prediction_length is None`
* changed `ProbSparseAttention` to `InformerProbSparseAttention`
* changed `V_sum` -> `v_mean_dim_time`
* changed `ConvLayer` to `InformerConvLayer` and fixed `super()`
* TimeSeriesTansformer->Informer in decoder's Copied from
* more descriptive in ProbSparse
* make style
* fix coped from
* Revert "added `if prediction_length is None`"
This reverts commit b4cbddfa05e3bd739b79569cd3c3b89e316f2451.
* fixed indent
* use InformerSinusoidalPositionalEmbedding
* make fix-style
* fix from #21860
* fix name
* make fix-copies
* use time series utils
* fix dec num_heads
* docstring
* added time series util doc
* _import_structure
* formatting
* changes from review
* make style
* fix docs
* fix doc
* removed NegativeLogLikelihood
---------
Co-authored-by: Kashif Rasul <kashif.rasul@gmail.com>
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
* Update tiny model creation script and some others files (#22006)
* Update 1
* Update 2
* Update 3
* Update 4
* Update 5
* Update 6
* Update 7
* Update 8
* Update 9
* Update 10
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* Generate - add 1 to cur_len to make up the new beam length (#21993)
* add 1 to cur_len to make up the new beam length
cur_len is 1 token shorter comparing to the length of the sequence whose best_sum_logprobs is the numerator.
* cur_len+=1 before check if beam hyp is done
* format code
* reformat with black
---------
Co-authored-by: Chiming <chiming@biomap.com>
* VideoMAE doctest - use valid dummy pixel values (#22022)
Use valid dummy pixel values
* update: bertology paper (#22012)
* Update `AudioClassificationPipelineTests::test_small_model_pt` for PT 2.0.0 (#22023)
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* [`bnb`] Fix bnb error message (#22026)
* fix error message
* make style
* [WIP] Add BridgeTowerForContrastiveLearning (#21964)
* Add BridgeTower for ITC
* Fix review feedback
* Rename BridgeTowerForITC, cleanup
* Fix style and quality
* implement tests
---------
Co-authored-by: Tiep Le <97980157+tileintel@users.noreply.github.com>
Co-authored-by: Tiep Le <tiep.le@intel.com>
* Fix test for torchneuroncore in Trainer (#22028)
* Add tokenize_kwargs parameter definition in the FeatureExtractionPipeline (#22031)
add tokenize_kwargs doc in the FeatureExtractionPipeline
* [examples/speech-recognition] Add SpecAugment to run_speech_recognition_seq2seq.py (#21942)
* Add specaugment to run_speech_recognition_seq2seq.py
* Remove useless argument: text_column
* Fix quality
* Update return_attention_mask condition
* Update specaugment arguments only for whisper models
* Remove SpecAugment arguments from ModelArguments, only leave default values for simplicity
* Apply suggestions from code review
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
* Update apply_spec_augment only for whisper models
* Apply suggestions from code review
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
* Rename return_attention_mask to forward_attention_mask to avoid confusion with wav2vec2 models
---------
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
* fixes the gradient checkpointing of whisper (#22019)
* fixing
* Update modeling_whisper.py
* Update modeling_whisper.py
* Update src/transformers/models/whisper/modeling_whisper.py
---------
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
* Avoid `text_config_dict` and `vision_config_dict` being saved for CLIP-like models (#22035)
* Avoid text_config_dict and vision_config_dict being saved
* for other CLIP-like models
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* Mark all `BridgeTower` tests slow for now (#22039)
* slow me
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* Bug fix: token classification pipeline while passing offset_mapping (#22034)
fix slow tokenizers with passing offset_mapping
* Update ALIGN docs (#22025)
* Fix typos and add code examples, resources
* [21737][T5]: Fix gradient checkpoint bug (#22036)
* [21737][T5]: Fix gradient checkpoint bug
* [21737][T5]: Fix gradient checkpoint bug
* [21737][T5]: Fix gradient checkpoint bug
* Update src/transformers/models/mt5/modeling_mt5.py
* Update src/transformers/models/t5/modeling_t5.py
---------
Co-authored-by: njindal <njindal@adobe.com>
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
* Docs Improvement - In ZSH, not using ' ' around pip install fails, fix it (#22045)
In ZSH, not using ' ' around pip install fails
Running
```
pip install transformers[torch]
```
in the default ZSH terminal will fail with the error `zsh: no matches found: transformers[torch]`
The solution is to wrap the installation path in ' ' like
```
pip install 'transformers[torch]'
```
Relevant StackOverflow: https://stackoverflow.com/questions/30539798/zsh-no-matches-found-requestssecurity
* Can't install tf2 on M1 Chip by default (#22046)
* Remove set_access_token usage + fail tests if FutureWarning (#22051)
* Remove set_access_token usage + fail tests if FutureWarning
* do not fail on FutureWarning in CI
---------
Co-authored-by: testbot <lucainp@hf.co>
* Show the number of `huggingface_hub` warnings in CI report (#22054)
* show hfh warnings
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* Return analysis for hyperparameter_search with Ray backend (#22040)
* return analysis for hyperparameter_search with ray backend
* Revert "return analysis for hyperparameter_search with ray backend"
This reverts commit cd5179070930e03020d96d98eb51dec3eb21ef75.
* add run_summary attribute to BestRun and return analysis for ray backend
* fix typo
* add doc for run_summary for ray backend
* pt-to-tf model architecture override (#22055)
* Add an argument to pt-to-tf to allow overriding the model class
* make fixup
* Minor fix to error message
* Remove unused extra conversion from the script
* rm $ symbol from code block from contributing.md (#22057)
rm $ symbol from code block
Removed the $ symbol from the code block to make copy-pasting easier.
* [deepspeed] offload + non-cpuadam optimizer exception (#22043)
* [deepspeed] offload + non-cpuadam optimizer exception
* flip
* revert min version
* Edit the docstring of `image_processing_donut` to match code (#22033)
* Edit the docstring of `image_processing_donut` to match code
* improve style
* more style improvement after installing quality
* Skip 3 tests for `WhisperEncoderModelTest` (#22060)
* skip 3 tests
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* Add setters by type of args to TrainingArguments (#21570)
* Add setters by type of args to TrainingArguments
* Define more setters
* Update tiny model creation script (#22058)
Update the script
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* Fix case when using --gradient_accumulation_steps with DDP disabled. (#22007)
Co-authored-by: EC2 Default User <ec2-user@ip-172-31-42-72.us-west-2.compute.internal>
* Add a progress bar for the total download of shards (#22062)
* Add a progress bar for the total download of shards
* Check for no cache at all
* Fix check
* Fix gradient checkpointing bug in Speech2Text (#22079)
* Fix gradient checkpointing bug in Speech2Text
* Update modeling_speech_to_text.py
* Update modeling_speech_to_text_2.py
* Fix gradient checkpointing bug in switch transformer (#22081)
* [GPT2] Propose fix for #21080 (#21853)
* Make sure position ids are masked
* test that padded input produce the same results
* fix failing tests
* fixup
* fix batch test
* Fix small typo in flan-ul2.mdx (#22068)
* Update flan-ul2.mdx
* Update flan-ul2.mdx
* Generate - Fix broken documentation links (#22078)
fix broken links
* Fix gradient checkpointing bug in Speecht5 (#22080)
* Fix gradient checkpointing bug in Speecht5
* Update modeling_speech_to_text.py
* Update src/transformers/models/speech_to_text/modeling_speech_to_text.py
* Fix change errors
---------
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
* Fix hint in src/transformers/modeling_utils.py (#22074)
fix hint
* handle numpy inputs in whole word mask data collator (#22032)
* GPT-J specific half precision on CPU note (#22086)
* re: #21989
* update re: #21989
* removed cpu option
* make style
* Fix imports of TF MobileViT (#22065)
* Fix imports of TF MobileViT
* Fix copies
* Revert "[GPT2] Propose fix for #21080" (#22093)
Revert "[GPT2] Propose fix for #21080 (#21853)" to avoid CI failure
This reverts commit a3fef89b2694fac4dd642a3f77d3e96d0c3df82a.
* [Whisper] Remove embed_tokens from encoder docstring (#21996)
* [Whisper] Remove embed_tokens from encoder docstring
* new line to retrigger CI
* remove new line
* Add AutoModelForZeroShotImageClassification (#22087)
Adds AutoModelForZeroShotImageClassification to transformers
* add new model of MGP-STR (#21418)
* add new model of MGP-STR
* fix the check failings
* remove torch and numpy from mgp_tokenization
* remove unused import from modeling_mgp_str
* add test_processing_mgp_str
* rm test_processing_mgp_str.py
* add test_processing_mgp_str
* add test_processing_mgp_str
* add test_processing_mgp_str
* rm test_processing_mgp_str and add softmax outs to model
* rm test_processing_mgp_str and add softmax outs to model
* rewrite the code of mgp-str according to PR suggestions
* rewrite the code of mgp-str according to PR suggestions
* add new model of MGP-STR
* fix the check failings
* remove torch and numpy from mgp_tokenization
* remove unused import from modeling_mgp_str
* add test_processing_mgp_str
* rm test_processing_mgp_str.py
* add test_processing_mgp_str
* add test_processing_mgp_str
* add test_processing_mgp_str
* rm test_processing_mgp_str and add softmax outs to model
* rewrite the code of mgp-str according to PR suggestions
* rewrite the code of mgp-str according to PR suggestions
* remove representation_size from MGPSTRConfig
* reformat configuration_mgp_str.py
* format test_processor_mgp_str.py
* add test for tokenizer and complete model/processer test and model file
* rm Unnecessary tupple in modeling_mgp_str
* reduce hidden_size/layers/label_size in test_model
* add integration tests and change MGPSTR to Mgpstr
* add test for logit values
* reformat test model file
---------
Co-authored-by: yue kun <yuekun.wp@alibaba-inc.com>
* Add pr_checks.mdx Italian translation (#17459) (#22116)
* Add pr_checks.mdx Italian translation (#17459)
* Updated pr_checks.mdx Italian translation (#17459)
* Fix gradient checkpointing bug in xglm (#22127)
* Fix gradient checkpointing bug in Trajectory Transformer (#22125)
* Fix gradient checkpointing bug in xlm_roberta_xl (#22128)
* Added big_models.mdx italian translation #17600 (#22115)
* updated toctree
* italian translation big_model.mdx
* italian translation big_models
* [`Blip2`] skip accelerate test (#22124)
skip accelerate test
* Fix gradient checkpointing bug in xmod (#22129)
* Fix gradient checkpointing bug in LongT5 (#22130)
* Fix gradient checkpointing bug in trocr (#22126)
* Fix gradient checkpointing bug in trocr
* Fix format
* Update src/transformers/models/trocr/modeling_trocr.py
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
---------
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
* Zero-shot image classification task guide (#22132)
* WIP
* WIP
* manual inference example
* make style
* Apply suggestions from code review
Co-authored-by: Alara Dirik <8944735+alaradirik@users.noreply.github.com>
---------
Co-authored-by: Alara Dirik <8944735+alaradirik@users.noreply.github.com>
* Fix doc link for MGP-STR (#22138)
* Adding Type Hints to TF_Pegasus model (#21941)
* Adding Type Hints to TF_Pegasus model
* Updated some parameters per maintainer comments
* Add a new script to check model testers' config (#22063)
* Add script
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* Update configuration_align.py (projected_dim=640) (#22139)
Update configuration_align.py
updated projected_dim=640 from 512 in arguments of AlignConfig
* [`Whiper`] add `get_input_embeddings` to `WhisperForAudioClassification` (#22133)
* add `get_input_embeddings` to `WhisperForAudioClassification`
* add common tests
* fix another common test
* Update tests/models/whisper/test_modeling_whisper.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* fix style
---------
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* Trainer: let generate pick its inputs (#22108)
* Let generate pick its inputs
* fix squad seq2seq example
* Enforce same behavior as PyTorch 2.0 for older versions (#22136)
* [trainer] fix bug in grad accum with multiple epochs (#22098)
* [trainer] fix bug in grad accum
* comment out debug
* fix one-off
* rename counter
* [deepspeed docs] Activation Checkpointing (#22099)
* [deepspeed docs] Activation Checkpointing
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update deepspeed.mdx
---------
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Remove backend check for torch.compile (#22140)
* Remove backend enforcment for torch.compile
* Update error
* Update src/transformers/training_args.py
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
* Style
---------
Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
* [Safetensors] Add explicit flag to from pretrained (#22083)
* [Safetensors] Add explicit flag to from pretrained
* add test
* remove @
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
---------
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Prepare daily CI for torch 2.0.0 (#22135)
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* docs: New terms and updates to glossary (#21982)
* Updated glossary with new terms, added abbreviations for certain terms and merged autoencoding models, autoregressive models and causal language modeling into encoder and decoder models
* Update docs/source/en/glossary.mdx
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/glossary.mdx
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/glossary.mdx
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/glossary.mdx
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/glossary.mdx
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/glossary.mdx
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/glossary.mdx
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/glossary.mdx
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/glossary.mdx
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/glossary.mdx
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/glossary.mdx
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/glossary.mdx
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Added link to 'Pipeline for inference' tutorial
* Trigger CI
* Update docs/source/en/glossary.mdx
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update docs/source/en/glossary.mdx
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Added entry for self supervised learning, added deleted entries + fixed broken links
* Update docs/source/en/glossary.mdx
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* [🛠️] Fix-whisper-breaking-changes (#21965)
* temp fix
* temporary fix
* update
* fix tests
* fixup
* update based on reveiew
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
* update to fix tests
* update docstring
---------
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
* Move `is_pipeline_test_to_skip` to specific model test classes (#21999)
* Move `is_pipeline_test_to_skip` to specific model test classes
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* Add ConvNeXT V2 (#21679)
* Add ConvNeXt V2 to transformers
* TF model is separated from the PR to fix issues
* Update 2 doctest expected values for torch 2.0.0 (#22148)
update values
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* Translation Italian: perf_train_cpu and perf_train_cpu_many (#22151)
* added translated files
added perf_train_cpu and perf_train_cpu_many
* updated toctree
* Fix big model inference for T5 models in float16 (#22095)
* Fix big model inference for T5 models in float16
* Apply suggestions from code review
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
* Style
* Trigger CI with latest release
---------
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
* Create MaskedImageCompletionOutput and fix ViT docs (#22152)
* create MaskedImageCompletionOutput
* fix bugs
* fix bugs
* to_pil - don't rescale if int and in range 0-255 (#22158)
* Don't rescale if in and in range 0-255
* Raise value error if int values too large
* Update tests/test_image_transforms.py
* Update tests/test_image_transforms.py
* [trainer] add `--optim adamw_torch_fused` for pt-2.0+ (#22144)
* [trainer] add --optim adamw_torch_fused
* change optim default
* deal with non-torch
* revert default change; prep; add fp16/amp assert
* typo
* typo
* Revert "Enforce same behavior as PyTorch 2.0 for older versions" (#22163)
Revert "Enforce same behavior as PyTorch 2.0 for older versions (#22136)"
This reverts commit 1c801d65eb42a71ea52db797af760bd96c8b113f.
* v4.28.0.dev0
* Load optimizer state on CPU to avoid CUDA OOM (#22159)
* Run all tests by default (#22162)
* Fix: unfinished_sequences with correct device (#22184)
Fix: unfinished_sequences with correct device
The original code was causing errors when running torch.jit.trace due to the tensor options being incorrect. I fixed this by using torch.ones to create a tensor with the correct device and dtype. This should resolve the issue with running torch.jit.trace.
* Revert 22152 MaskedImageCompletionOutput changes (#22187)
Revert changes
* Regression pipeline device (#22190)
* Fix regression in pipeline when device=-1 is passed
* Add regression test
* Update BridgeTowerForContrastiveLearning (#22145)
* Use return_loss for BridgeTowerForContrastiveLearning, add example
* fix tests
* Update example in BridgeTowerForContrastiveLearning
* Update test_modeling_bridgetower.py
* update model output format
* minor update
* Update src/transformers/models/bridgetower/modeling_bridgetower.py
* make style
---------
Co-authored-by: Tiep Le <97980157+tileintel@users.noreply.github.com>
Co-authored-by: Tiep Le <tiep.le@intel.com>
Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* t5 remove data dependency (#22097)
* t5 remove data dependency
* make style
* make fix-copies
---------
Co-authored-by: Prathik Rao <prathikrao@microsoft.com>
* Fix DeepSpeed CI (#22194)
* Deal with torch-tensorrt
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* Fix typo in Align docs (#22199)
Fix align docs typo
* Update expected values in `MgpstrModelIntegrationTest` (#22195)
Update values
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* Italian Translation of migration.mdx (#22183)
* Tranlstion Italian: migration
* Update migration.mdx
minor fixes
* Update _toctree.yml
* Delete migration.mdx
* Add italian translation of migration.mdx
* Update of migration.mdx translation and toctree
* LLaMA Implementation (#21955)
* LLaMA
* sharding and docs
* tweak
* black
* inits
* ruff
* LLAMA_PRETRAINED_CONFIG_ARCHIVE_MAP
* init
* no checkpoint
* docs
* ruff
* type_vocab_size
* tokenizer fixes
* tokenizer fixes
* Update tokenization_llama.py
* Update tokenization_llama.py
* Update configuration_llama.py
* Update modeling_llama.py
* tokenizer add_bos by default
* licenses
* remove decoder
* norms and mlp
* rope overhaul
* tweaks
* black
* mention OPT implementation
* off-by-one naming
* typo
* fix
* tokenization fix and slicing bug
* padding config
* cleanup
* black
* update tests
* undo typo
* fix vocab caching logic
* ruff
* docbuilder
* attn fix from BlackSamorez
* initial feedback
* typo
* docs
* llama case
* llama case
* load checkpoint docs
* comment about tokenizer
* tokenizer defaults
* clear past_key_values if use_cache=False
* last tweaks
* last tweaks
* last tweaks
* last tweaks
---------
Co-authored-by: Stella Biderman <stellabiderman@gmail.com>
* LLaMA Implementation (#21955)
* LLaMA
* sharding and docs
* tweak
* black
* inits
* ruff
* LLAMA_PRETRAINED_CONFIG_ARCHIVE_MAP
* init
* no checkpoint
* docs
* ruff
* type_vocab_size
* tokenizer fixes
* tokenizer fixes
* Update tokenization_llama.py
* Update tokenization_llama.py
* Update configuration_llama.py
* Update modeling_llama.py
* tokenizer add_bos by default
* licenses
* remove decoder
* norms and mlp
* rope overhaul
* tweaks
* black
* mention OPT implementation
* off-by-one naming
* typo
* fix
* tokenization fix and slicing bug
* padding config
* cleanup
* black
* update tests
* undo typo
* fix vocab caching logic
* ruff
* docbuilder
* attn fix from BlackSamorez
* initial feedback
* typo
* docs
* llama case
* llama case
* load checkpoint docs
* comment about tokenizer
* tokenizer defaults
* clear past_key_values if use_cache=False
* last tweaks
* last tweaks
* last tweaks
* last tweaks
---------
Co-authored-by: Stella Biderman <stellabiderman@gmail.com>
* Update tiny model creation script (#22202)
* Update UNCONVERTIBLE_MODEL_ARCHITECTURES
* Deal with 2 model tester classes in single test file
* Deal with 2 model tester classes in single test file
* Deal with 2 model tester classes in single test file
* make style and quality
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* Temporarily fix ONNX model exporting error (#21830)
* Temporarily fix https://github.com/microsoft/onnx-converters-private/issues/143
* Reduced column width
* Fix formatting.
* Revert "Temporarily fix https://github.com/microsoft/onnx-converters-private/issues/143"
This reverts commit 6e95a108042118d204da447729f3834affa354fc.
* Fix export error.
* Revert "Fix formatting."
This reverts commit 8310f60da10358edbdf77a2a2f3c83ee55066cb8.
* Propagated changes made in SwinV2 to Swin2SR
* [`XGLM`] Add `accelerate` support for XGLM (#22207)
* add `accelerate` support for XGLM
* fix order
* fixes a typo in WhisperFeatureExtractor docs. (#22208)
* fixes a typo
* .
* 🔥py38 + torch 2 🔥🔥🔥🚀 (#22204)
* py38 + torch 2
* increment cache versions
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* Hotfix for natten issue with torch 2.0.0 on CircleCI (#22218)
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* fix typos in llama.mdx (#22223)
* fix code example in mgp-str doc (#22219)
Co-authored-by: yue kun <yuekun.wp@alibaba-inc.com>
* Use `dash==2.8.1` for now for daily CI (#22227)
Use dash 2.8.1 for now
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* Depth estimation task guide (#22205)
* added doc to toc, auto tip with supported models, mention of task guide in model docs
* make style
* removed "see also"
* minor fix
* LLaMA house-keeping (#22216)
* LLaMA house-keeping
* Doc links
* fix AutoTP in deepspeed could not work for bloom (#22196)
* fix AutoTP in deepspeed could not work for bloom
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
* add a method in BloomModel to build ailib
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
---------
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
* Add LlamaForSequenceClassification (#22209)
* Add LlamaForSequenceClassification
* Update src/transformers/models/llama/modeling_llama.py
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
* Update src/transformers/models/llama/modeling_llama.py
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
* Add docstring
* Add test
* Add input embedding getter and setter
* Remove dead code
---------
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
* Removed .mdx extension in two links (#22230)
removed …
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes #21737
Before submitting
Pull Request section?
to it if that's the case. -- [
Generate] Fixgradient_checkpointinganduse_cachebug for generate-compatible models #21737documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.