Cloud GPU via Interstice.cloud

[Acly]

Interstice.cloud

In short: this is an online service which provides all the functionality needed by the Krita plugin, with the same results you'd get locally.

No installation and no GPU needed!

Why?

One of the goals of this project is to make generative AI accessible. Local Stable Diffusion requires a powerful GPU, and some time and technical skill to set it up. No matter what I try to make it easier, the ecosystem is so volatile and changes so fast, it will be a struggle for some time to come.

Of course there are lots of services out there already, but they are usually restrictive, closed, and lack the tools Krita brings. They also tend to be either very simple, or very technical.

Sustainability is also a reason. This project requires a lot of time, but makes no money. It will always be open source and local will always be an option. But if it can make some money by offering optional convenience, that would be great!

What about the exsting Runpod/Vast template?

It has several issues:

• Start up time is terrible (~15 minutes). The new solution has no or minimal wait time.
• More models have to be added which only makes it worse and will run into hard limit soon.
• It cannot take advantage of many people using the same solution. Cloud becomes efficient when it can scale.

Work in progress

Currently this is an experiment, up for testing:

• No money involved, it's free until you run out.
• Most features are supported, but there are a few missing. Notably no SDXL yet, but will set that up soon.
• Performance and availability will have to be improved. Live mode doesn't work great yet.

Feedback

If you're interested please give it a try. Some data is needed to figure out what it has to cost to cover expenses.

You will need an account on www.interstice.cloud (only requires a valid email), and version 1.16.0 of the plugin.

[croosn]

Thanks, this is some hard and nice work 👍 any way to refresh the remaining tokens? What do you expect the price to be in the future?

[akpenou]

Hello !

Thank you for this wonderful work.
We face the same issue with our clients in training (in France).
Perhaps we can help you get this project off the ground, whether in coding or in business.

We're also looking to have trainers on Krita AI to help creatives push beyond the limits of Midjourney.

Looking forward to discussing this further.

[Acly]

Thanks for testing!

any way to refresh the remaining tokens? What do you expect the price to be in the future?

Currently tokens are limited to keep cost under some control, might extend it later. Still need to evaluate statistics to figure out pricing. I'll post an update once that's done and if it looks reasonable go forward (this may then take some time due to business/legal stuff though...)

In the meantime, SDXL is now deployed too, version 1.16.1 is required to use it.

[Kobayashimaru82]

I would MUCH rather pay for your cloud than jump through all the hoops of Vast.ai. Plus you deserve the financial support. Question: how would the LoRas work - would we just upload them like Vast.ai or could we use what's on our local directory?

[Acly]

Time for some updates.

About Tokens

Generating images requires GPU compute time, and that costs money. The idea of Tokens is to give a rough estimate on how much compute is required, so people have an idea what a generation will cost.

Ideally it shouldn't be too complicated. There are three factors which have a large impact:

• Image resolution
• Number of sampling steps (indirectly influenced by strength)
• Batch size (how many images are computed at once)

Also SDXL is a bigger up-front and maintenance cost than SD1.5.

The Data

I did some initial tests and whipped up a formula that was used to compute Tokens. Now there's actual usage data, which allows to compare Tokens predicted versus actual compute time required. Initially the variance was quite high (+/-80% on average). After the latest updates it's down to +/-30%, but it hasn't been live very long.

Resolution

Image size is tricky. Other services usually offer some fixed choices (1MP, 2MP) with various aspect ratios and things are easy. But when working with a canvas, resolutions need to be very flexible. The gist is, large resolutions are really expensive.

Non-linear progression

Where a 2048x1024 image takes 18 seconds, 2048x2048 already takes 60 - twice the pixels, more than three times the cost. Double resolution again, 220 seconds. It quickly becomes very inefficient. The latest version now takes this into account.

Maximum resolution

The plugin's performance settings have a "Maximum resolution" setting, which limits the image size used for diffusion. If the canvas is bigger, some relatively cheap but decent upscaling is used to make it fit. This is especially helpful for people who aren't aware of how costly high resolutions are, and load fotos or illustrations at camera/print resolutions.

In v1.17.2 the default maximum resolution is now 6MP (6 millionen pixels). This number can be tweaked in performance settings, but the maximum the service accepts is 8MP (and that's already very expensive). If you want diffusion at higher resolutions use the Upscaling workspace, it does not have a limit and uses tiling to scale better.

Fill

Actually what is the resolution of the image? It's not always clear. Inpainting (selections at 100% strength) for example will run an initial pass on a downscaled image with a lot of context, then upscale, but also crop to a smaller region around the selection. So there are multiple resolutions. Also because it's two passes, the number of steps that run is actually higher than what is configured.

The latest changes take some of this into account, but it's perhaps still not close enough. The biggest outliers currently fall into this category.

Conclusions

It's quite difficult to find a good trade off between keeping it simple, but also making it fair!
I don't really like subscriptions, but I can now see the appeal...

Examples

Mode	SD	Resolution (Batch)	Cost
Live	1.5	640x512	1
Normal	1.5	1024x1024 (x2)	9
Normal	XL	1024x1024 (x2)	10
Refine (50%)	XL	1024x1024 (x2)	6
Normal	XL	1920x1080	14
Normal	XL	3840x2160	58
Upscale (40%)	1.5	3840x2160	38

Formula

I'm not going to post it here as it may still change, but hey, it's open source.

UI

Also in v1.17.2 the UI now shows you how much generation will cost when you hover over the button.

Next

• There are some outstanding updates to models and support for a few more Control layers. But they have to be synced up with Plugin development.
• Payment options roll out in the next days
• LoRA support will be a bit further into the future. It's tricky to run them efficiently on on-demand servers with limited caching. From user perspective upload and caching can be automated, but it still takes some time, which should probably be transparent.

[dkqjhoifh]

If he can allow users to choose their own local checkpoints and loras, then I think more people will choose him, otherwise you will need to prepare a large number of checkpoints and loras to meet the needs of different users.

[dkqjhoifh]

If can solve this problem, I am willing to become a paid user of this project for a long time

[Acly]

Custom LoRA upload has been added in v1.23.0. Forgot to update this discussion.

Custom Checkpoint is not possible yet, it's a bit more questionable. There are many checkpoints, but even though they are many GB large, they are almost all the same with very superficial differences. So it's quite inefficient. You can do almost everything with LoRA instead.

[dkqjhoifh]

I'm going to try this today, anyway, buy a GPU first, and hopefully this meager support will help you

[Muskelmagier]

If you would Add Autsimmix as a base model for the cloudservice
https://civitai.com/models/288584?modelVersionId=324619
its is one of the most used models