Fix generate with inputs_embeds as input

[molbap]

What does this PR do?

Seems recent refactor broke generation based on inputs_embeds if I understand correctly.
This aims at catching correctly the sequence_length by using the passed model_inputs, creating the correctly-sized causal_mask in generation step.

Core change is

        if inputs_embeds is not None and cache_position[0] == 0:
            model_inputs = {"inputs_embeds": inputs_embeds, "input_ids": None}
        else:
            # The clone here is for the same reason as for `position_ids`.
            model_inputs = {"input_ids": input_ids.clone(memory_format=torch.contiguous_format), "inputs_embeds": None}
        # or any Cache class
        if isinstance(past_key_values, HybridCache) and attention_mask.ndim == 2:
            if model_inputs["inputs_embeds"] is not None:
                batch_size, sequence_length, _ = model_inputs["inputs_embeds"].shape
                device = model_inputs["inputs_embeds"].device
            else:
                batch_size, sequence_length = model_inputs["input_ids"].shape
                device = model_inputs["input_ids"].device

Fixes #32479

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[ArthurZucker]

Thanks, will be included in a patch.
Can you add a test in the generate tests this should clearly have been caught

[molbap]

Yes investigating that to add a test, a bit surprised that this slipped through as we're supposed to be testing this already 🤔

[SunMarc]

Hey @molbap, I think that we forgot to skip a few tests related to this PR : test_1 and test_2 . Can you confirm me that we indeed need to skip them just like bert model ? I'll open a PR if that's the case ! Not sure why these tests are only triggered on my PR: #32627

[molbap]

Hi @SunMarc , there was an issue with the fetching of tests and it is causing a lot of failures currently, opening a PR to fix it - in the meantime you can indeed skip them!

[molbap]

In the end #32651 was also needed to fix breaking tests introduced here, FYI!