Fix for use_safetensors parameters, allow use of parameter on loading submodels (#9576)

[elismasilva]

What does this PR do?

This PR fixes the issue described in (#9576) where use_safetensors parameter from pipeline is not passed to submodules when them is loaded from "from_pretrained" method.

Reproduction tests:

1. If I do not pass use_safetensors in the pipeline, it will set use_safetensors to True and allow_pickle to True and then an alert will be issued if it does not find ".safetensors" in the directory, however it will continue loading the .bin files normally.

from diffusers import StableDiffusionXLPipeline
import torch
pipe = StableDiffusionXLPipeline.from_pretrained('stablediffusionapi/yamermix-v8-vae', torch_dtype=torch.float16)

Log:

███████████████████████████████████████████████████████████████████████▏                                                                                              | 3/7 [00:00<00:00,  4.13it/s]An error occurred while trying to fetch F:/projetos/DiffusersWebUI/models/stablediffusionapi/yamermix-v8-vae\unet: Error no file named diffusion_pytorch_model.safetensors found in directory F:/projetos/DiffusersWebUI/models/stablediffusionapi/yamermix-v8-vae\unet.
Defaulting to unsafe serialization. Pass `allow_pickle=False` to raise an error instead.
Loading pipeline components...:  71%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▌                                               | 5/7 [00:03<00:01,  1.27it/s]An error occurred while trying to fetch F:/projetos/DiffusersWebUI/models/stablediffusionapi/yamermix-v8-vae\vae: Error no file named diffusion_pytorch_model.safetensors found in directory F:/projetos/DiffusersWebUI/models/stablediffusionapi/yamermix-v8-vae\vae.
Defaulting to unsafe serialization. Pass `allow_pickle=False` to raise an error instead.
Loading pipeline components...: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:03<00:00,  2.06it/s]

f:\Projetos\diffusers\venv\lib\site-packages\transformers\models\clip\modeling_clip.py:491: UserWarning: 1Torch was not compiled with flash attention. (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:555.)

2. If I pass use_safetensors True in the pipeline, it will not issue the warning, it will simply give the error message that it did not find ".safetensors" files in the directory.

from diffusers import StableDiffusionXLPipeline
import torch
pipe = StableDiffusionXLPipeline.from_pretrained('stablediffusionapi/yamermix-v8-vae', torch_dtype=torch.float16, use_safetensors=True)

Log:

"...Error no file named diffusion_pytorch_model.safetensors found in directory F:/projetos/DiffusersWebUI/models/stablediffusionapi/yamermix-v8-vae\vae"

3. If I pass use_safetensors False in the pipeline (the scenario where this PR is fixing), then it means that it will directly search for the .bin files in the directory without throwing errors, except if the model is .safetensors, then it will throw an error saying that it didn't find any .bin files in the directory.

IF model is .bin:

from diffusers import StableDiffusionXLPipeline
import torch
pipe = StableDiffusionXLPipeline.from_pretrained('stablediffusionapi/yamermix-v8-vae', torch_dtype=torch.float16, use_safetensors=False)

IF model is .safetensors:

from diffusers import StableDiffusionXLPipeline
import torch
pipe = StableDiffusionXLPipeline.from_pretrained('wangqixun/YamerMIX_v8', torch_dtype=torch.float16, use_safetensors=False)

Log:

"...raise EnvironmentError(
OSError: Error no file named pytorch_model.bin, model.safetensors, tf_model.h5, model.ckpt.index or flax_model.msgpack found in directory F:/projetos/DiffusersWebUI/models/YamerMIX_v8\text_encoder."

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline?
• Did you read our philosophy doc (important for complex PRs)?
• Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

@yiyixuxu and @asomoza