Add option to store TI embeddings in png chunks, and load from same.

[dfaker]

Making the preview pngs themselves sharable embeddings.

So images like this:

Can go straight into your embeddings folder...maybe wait until you've done a little more training before sharing them in general though.

Initially concerned about torch.load doing unsafe pickling but it seems as if it's restricted on the type of classes it'll deserialise to str, I'm not sure if we're happy that's protection enough.

[dfaker]

Actually thinking this is too risky, upon reviewing the ongoing pytorch attempts to mitigate the risks of pickle.

[dfaker]

will rework to directly pass the embedding without torch.load

[dfaker]

Okay, Zero-🥒, now serialises to and from a safe json representation for embedding in the images without a torch.load of untrusted files, added a little styling to the embedded-images, they all go in a separate folder, standard images are left untouched.

I've left the original above-below captioning method in under images.captionImge as it's a nice generalization of adding captions and backgrounds.

[dfaker]

Updated the way the data is embedded in the image pixels:

Side blocks representing the stored tensor expand as more bytes are needed to embed the data.

[R-N]

if filename.upper().endswith('.PNG'):
    embed_image = Image.open(path)
    if 'sd-ti-embedding' in embed_image.text:
        data = embedding_from_b64(embed_image.text['sd-ti-embedding'])
        name = data.get('name', name)
    else:
        data = extract_image_data_embed(embed_image)
        name = data.get('name', name)
else:
    data = torch.load(path, map_location="cpu")

Why does image embedding try to use loaded name, while .pt embedding doesn't?

[dfaker]

The .pt doesn't use the name field because not all embeddings are guaranteed to have them, only ones trained in this repository, and to allow users to rename the file to whatever tokens they want to trigger the embedding.

The image embeddings should always contain a name field, already advertise their trained name in the image itself, and are more prone to having their original filenames destroyed by image hosts.

[R-N]

But then that won't allow multiple image embeddings that happened to be trained with same name. And it's not like the original filename matters, or does it? The user can set the filename like the .pt ones.

[Jonseed]

@dfaker this is an awesome feature. I have a question. My best embedding during training didn't turn out a good image. How do I make a new image embedding with a better image?

It would be cool if there was a checkbox option, perhaps on the txt2img page or settings, that could generate new embeddings along with the generated images? Then you could pick the one that best represents the embedding?

[dfaker]

@Jonseed there's actually a script for that: https://github.com/dfaker/embedding-to-png-script

I do like the honesty of using the training images though.

[Jonseed]

Cool! I'll check that out.

Yeah, the prompt I used to generate preview images during training just wasn't that great to begin with.