Question on multi-image input

[auhowielau]

Some models (e.g. LLaVA 1.5) cannot input multiple (>3) images limited by input length (e.g., 2048). However, Evaluation Dimension 17-24 of SeedBench 2 may require inputs of up to 8 images. How do you handle such situations? Thanks!

[Bohao-Lee]

In our code, we concat images to handle such situations just like other models. In our experiment, llava model can output reasonable loss.

[auhowielau]

For the LLaVA 1.5 model, does the concat operation transform N input images into Nx576 visual tokens? If so, for an input of 8 frames, would there be a truncation issue, as 576x8=4608 far exceeds the input length limit of 2048? Thanks!