Does the current code supports generation of text + image + image ? #25
-
|
Hi, does the current code supports generation of text + image + image ? as see in the paper? where they are predicting text image tokens and then again text tokens |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 3 replies
-
|
@Wonder1905 it can do anything, any number of modalities, any order the sky is the limit |
Beta Was this translation helpful? Give feedback.
-
|
the only thing missing at the moment is allowing for learned encoder / decoder flanking the transformer (unet in paper, but can also be anything) i'm getting that done this morning, hopefully |
Beta Was this translation helpful? Give feedback.
-
|
@Wonder1905 once the dust has settled, i'll put together a small demo doing simple arithmetic (1 + 3 = 4), but with numbers substituted with mnist images |
Beta Was this translation helpful? Give feedback.
-
|
Niceeeeeeeeeeeeeeee! |
Beta Was this translation helpful? Give feedback.
@Wonder1905 it can do anything, any number of modalities, any order
the sky is the limit