Error with flux models.

[benew666]

Since a few days ago, I can not use any Flux model with Krita. It used to work fine, and it still works fine with comfyui.
But when I try to use it with Krita. I got this errors, even with the latest version.
Other models beside Flux works well.

model weight dtype torch.float8_e4m3fn, manual cast: torch.bfloat16
model_type FLUX
Requested to load FluxClipModel_
Loading 1 new model
loaded completely 0.0 3303.39990234375 True
!!! Exception during processing !!! 'NoneType' object has no attribute 'device'
Traceback (most recent call last):
File "d:\AI\ComfyUI_windows_portable_nightly_pytorch\ComfyUI\execution.py", line 323, in execute
output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "d:\AI\ComfyUI_windows_portable_nightly_pytorch\ComfyUI\execution.py", line 198, in get_output_data
return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "d:\AI\ComfyUI_windows_portable_nightly_pytorch\ComfyUI\execution.py", line 169, in _map_node_over_list
process_inputs(input_dict, i)
File "d:\AI\ComfyUI_windows_portable_nightly_pytorch\ComfyUI\execution.py", line 158, in process_inputs
results.append(getattr(obj, func)(**inputs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "d:\AI\ComfyUI_windows_portable_nightly_pytorch\ComfyUI\nodes.py", line 65, in encode
output = clip.encode_from_tokens(tokens, return_pooled=True, return_dict=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "d:\AI\ComfyUI_windows_portable_nightly_pytorch\ComfyUI\comfy\sd.py", line 127, in encode_from_tokens
o = self.cond_stage_model.encode_token_weights(tokens)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "d:\AI\ComfyUI_windows_portable_nightly_pytorch\ComfyUI\comfy\text_encoders\flux.py", line 56, in encode_token_weights
l_out, l_pooled = self.clip_l.encode_token_weights(token_weight_pairs_l)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "d:\AI\ComfyUI_windows_portable_nightly_pytorch\ComfyUI\comfy\sd1_clip.py", line 41, in encode_token_weights
o = self.encode(to_encode)
^^^^^^^^^^^^^^^^^^^^^^
File "d:\AI\ComfyUI_windows_portable_nightly_pytorch\ComfyUI\comfy\sd1_clip.py", line 238, in encode
return self(tokens)
^^^^^^^^^^^^
File "d:\AI\ComfyUI_windows_portable_nightly_pytorch\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "d:\AI\ComfyUI_windows_portable_nightly_pytorch\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "d:\AI\ComfyUI_windows_portable_nightly_pytorch\ComfyUI\comfy\sd1_clip.py", line 210, in forward
outputs = self.transformer(tokens, attention_mask_model, intermediate_output=self.layer_idx, final_layer_norm_intermediate=self.layer_norm_hidden_state, dtype=torch.float32)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "d:\AI\ComfyUI_windows_portable_nightly_pytorch\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "d:\AI\ComfyUI_windows_portable_nightly_pytorch\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "d:\AI\ComfyUI_windows_portable_nightly_pytorch\ComfyUI\comfy\clip_model.py", line 136, in forward
x = self.text_model(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "d:\AI\ComfyUI_windows_portable_nightly_pytorch\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "d:\AI\ComfyUI_windows_portable_nightly_pytorch\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "d:\AI\ComfyUI_windows_portable_nightly_pytorch\ComfyUI\comfy\clip_model.py", line 112, in forward
x, i = self.encoder(x, mask=mask, intermediate_output=intermediate_output)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "d:\AI\ComfyUI_windows_portable_nightly_pytorch\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "d:\AI\ComfyUI_windows_portable_nightly_pytorch\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "d:\AI\ComfyUI_windows_portable_nightly_pytorch\ComfyUI\comfy\clip_model.py", line 69, in forward
x = l(x, mask, optimized_attention)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "d:\AI\ComfyUI_windows_portable_nightly_pytorch\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "d:\AI\ComfyUI_windows_portable_nightly_pytorch\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "d:\AI\ComfyUI_windows_portable_nightly_pytorch\ComfyUI\comfy\clip_model.py", line 50, in forward
x += self.self_attn(self.layer_norm1(x), mask, optimized_attention)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "d:\AI\ComfyUI_windows_portable_nightly_pytorch\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "d:\AI\ComfyUI_windows_portable_nightly_pytorch\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "d:\AI\ComfyUI_windows_portable_nightly_pytorch\ComfyUI\comfy\clip_model.py", line 17, in forward
q = self.q_proj(x)
^^^^^^^^^^^^^^
File "d:\AI\ComfyUI_windows_portable_nightly_pytorch\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "d:\AI\ComfyUI_windows_portable_nightly_pytorch\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "d:\AI\ComfyUI_windows_portable_nightly_pytorch\ComfyUI\comfy\ops.py", line 68, in forward
return self.forward_comfy_cast_weights(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\AI\ComfyUI_windows_portable_nightly_pytorch\ComfyUI\custom_nodes\ComfyUI-GGUF\ops.py", line 153, in forward_comfy_cast_weights
return super().forward_comfy_cast_weights(input, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "d:\AI\ComfyUI_windows_portable_nightly_pytorch\ComfyUI\comfy\ops.py", line 63, in forward_comfy_cast_weights
weight, bias = cast_bias_weight(self, input)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "d:\AI\ComfyUI_windows_portable_nightly_pytorch\ComfyUI\comfy\ops.py", line 47, in cast_bias_weight
weight = comfy.model_management.cast_to(s.weight, dtype, device, non_blocking=non_blocking, copy=has_function)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "d:\AI\ComfyUI_windows_portable_nightly_pytorch\ComfyUI\comfy\model_management.py", line 847, in cast_to
if device is None or weight.device == device:
^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'device'

[noyart]

Is this the normal flux model or gguf or something?

[benew666]

Both have the same problems with Krita. It works fine with comfyui.

[Acly]

Not sure, but you can try to export the workflow: https://github.com/Acly/krita-ai-diffusion/wiki/Common-Issues#comfyui-workflow-export
And load it in ComfyUI to compare. It might be a wrong model is picked up as text encoder / vae.

[benew666]

It seems like it picks up the wrong clip. It choose EVA02_CLIB_336_psz14_s6B.pt. If I change it to clip_l.safetensors, it works in Comfyui.
Now, how do I force Krita to use the correct clip?