how to implement the phrase 1 pretraining by contrasive learning?

[indeedzcr]

There are 3 stages of training in your paper, in which the first one is contrasive learning.
which token do you use to get the embedding of text in the image-text pair?
And how do you get the embedding of the image in the image-text pair?

thanks