Can I get some help with matching tensor shape?

[luxiant]

I'm doing a project, referring the example codes in candle: candle-examples/examples/quantized-t5

And here are my project
https://github.com/luxiant/quantized_t5_batch

What I am intended to do are the followings.

1. Convert a t5 model https://huggingface.co/csebuetnlp/mT5_multilingual_XLSum into quantized model.
   I downloaded the model to convert_to_quantized_model/src/summarizer_model/ of my project, and then run

cargo run quantize --out-file ./src/summarizer_model/model.gguf --quantization q6k ./src/summarizer_model/model.safetensors

to make a gguf file to quantize.

2. and then run the model to summarize the corpus abstractive_summary/src/firstProcessed.json of my project.
   I moved the quantized model to /src/summarizer_model.

However, when I run

cargo run ./src/firstProcessed.json

The error appears in forwarding the embedded text batch, which is line 134, telling

tensor_batch: Tensor[dims 1, 4, 304; u32]
decoder_token_ids: Tensor[dims 1, 4, 304; u32]
thread 'main' panicked at src/main.rs:134:53:
called `Result::unwrap()` on an `Err` value: ShapeMismatchBinaryOp { lhs: [1, 4, 304, 768], rhs: [1, 4, 12, 64], op: "reshape" }

It seems like the shapes of tensors are not matching, but I have no idea where the mismatch appeared. What am I missing?

[LaurentMazare]

One thing to try is running the command with RUST_BACKTRACE=1 and debug symbols activated (e.g. in the main candle repo candle run --profile=release-with-debug). This should provide you with a proper backtrace for where the error is actually occurring.