Skip to content

Latest commit

 

History

History
570 lines (361 loc) · 20.2 KB

README.old.md

File metadata and controls

570 lines (361 loc) · 20.2 KB
Raw

1. 🛋️ CushyStudio

1.1. The AI and Generative Art platform for everyone

Maturity Build Status Activity

🔶 NOTICE: CushyStudio is still in development. It updates and breaks often. We are working hard to make a stable version. Current Focus is on improving the install experience and switching to some real production-ready database 🔶


Join our Discord Follow _rvion on Twitter

🖥️ Cushy Studio: A modern and cross-platform software to Play and Work with Generative AI Art. Explore and build generative Apps. 🌠 Image, 🎥 Video, and 🧊 3d. Welcome to the future.

💎 Cushy Apps: Self-contained, use-case-centric applications with dedicated UIs that make generative art simple for everyone. Find the app for you need in the Cushy Library, from general-purpose Apps to very specific ones for dedicated use cases.

🚀 Cushy SDK: Automate your work or tackle specific use cases by building your own App. CushyStudio is packed with power-user tools to support you in building the "app" of your dreams. Once ready, share it with the world through the cushy app library.

🎓 Cushy Foundation: Cushy is made to last. Free and Open source. It builds on proven technologies like ComfyUI, Stable Diffusion, FFmpeg, or ImageMagick. It features an extensible core and a professional UI with dockable panes. It is LLM-friendly. The project is early, but I'm (rvion) committed to growing this. If you like this, please, sponsor my work:

media 1: CushyStudio

media 2: CushyStudio gallery

  • a list of apps ready to be used
    • Apps with the Cushy Action label are native CushyApps using the CushyStudio SDK.
    • Apps with the ComfyUI label are auto-imported ComfyUI workflow with an autogenerated interface added on top.
    • Apps with the A1111 label are imported images from Automatic1111. they use a custom Static Badge

media 3: ComfyUI on steroids

Art blooms in bytes, vast,
In Cushy realms, ideas soar,
For all, art awakes.

media 4: Sample Video (placeholder until real video)

2023-11-15_01-17-02.mp4

2. Installation

  1. download this repository,

    git clone https://github.com/rvion/CushyStudio --depth 1
cd CushyStudio
git remote set-branches origin '*' # so you can switch to dev branch despite the shallow clone

  2. click

    1. _windows_INSTALL.bat (on windows)
    2. _macos_INSTALL.sh (on macos)
    3. _linux_INSTALL.sh (on linux)

3. Quickstart Guide

  1. click
    1. _windows_START.bat (on windows)
    2. _macos_START.sh (on macos)
    3. _linux_START.sh (on linux)

3.2. Updating

You can update CushyStudio in two ways:

    1. Safest: Close Cushy, and run the 1_update.sh (mac, Linux) or 1_update.bat (window)
    1. Fastest: Click the update button in the top navbar (may not work when dependencies changes)

3.3. Moving around

  • CushyStudio uses a powerful dock system to organize your workspace.
  • Every panel can be moved around, and even detached from the main window.
  • Use ctrl+w or cmd+w to close the current tab

3.3.1. The top bar

3.4. Use built-in apps

  • Use the Library panel to select the CushyStudio/prompt card.
  • Pick a model and click run to see it in action

3.5. Install custom apps

  • Open the Library panel to select the CushyStudio/prompt card.

3.6. Create custom Apps

3.7. Misc Tips

  • Lots of Shorcuts, and way more to come.
  • Right-clicking any tooltip / menu / dropdown handle make the tooltip persistently visible
  • left and right borders fold back when tab handle is clicked
  • click any version number to reveal a update system

4. Features

4.1. First-class ComfyUI support

4.1.1. Work with Local, Managed, Remote, and Cloud ComfyUI instances

  • ✅ Quickly switch between various ComfyUI setups
  • ✅ Parallelize work across multiple instances

4.1.2. Parallelize load across multiple ComfyUI instances

  • ✅ Automatically download images in your local outputs folder
  • ✅ Automatically upload missing files
  • ✅ Powerful image hashing system to avoid duplicate uploads

4.1.3. Support ComfyUI workflow natively

🛋️ CushyStudio is directly compatible with ComfyUI workflow json format and images. Add them to the action folder to quickly use them inside CushyStudio.

You can convert them to actions in cone click t

4.2. Full LLM support

  • baked in prompt-master system prompt to quickly try llms
  • ability to define custom agents / customize every system prompts
  • open-router based
    • => you have access to all the models chatgpt4, claude 2, various unfiltered llama flavors, etc.

4.3. Real-time enabled

2023-10-19_00-39-13.mp4

4.4. CivitAI integration

  • ✅ Embedded Civitai browser so you can find and copy models refs for download
  • ✅ Civitai custom ComfyUI Nodes support
  • ✅ Dedicated Civitai widgets to augment your allow your apps to select CivitAI models directly

4.5. Built-in full-featured Image Editor

Layers, effects, masks, blending modes, ...and more. Always one click away

4.6. Easy to extend

👉 In case you have a problem, check the logs and report what you see

Click here to see how to reveal the debug console

4.7. Create your own Apps to streamline any image or video production

Light         Dark

  1. Define your own UI
  2. Build one or many prompts with custom logic in TypeScript
  3. Type-safe experience pushed to the MAXIMUM
    1. Every single value/enum is typed
    2. A built-in standard library made to quickly build your dream workflow
    3. Use lambda to get completion only for the node that produces the value needed

5. Quickstart Guide For Action Creators

Creating Apps is easy. In this section, we will

  • Understand how CushyStudio works,
  • See how to create a custom deck
  • See how to edit your apps
  • See how to publish your deck on the library

5.1. How does CushySudio work ?

  1. On startup, CushyStudio connect to your ComfyUI server(s)

  2. CushyStudio then scans your ComfyUI install and generates the whole TypeScript SDK supporting all your custom nodes, assets, models installed, etc.

  3. All your custom nodes, models, and images will be converted to enums, classes, helpers, etc, allowing you to create Apps with maximum type safety and completion. you can check the generated SDK at schema/global.d.ts.

    • 👉 this file is updated after each ComfyUI connection.
    • 👉 It means it will only be created after the first successful connection to your ComfyUI instance

5.2. What is an App?

An Action is self-contained.

  • A widget definition ( "ui" function)
  • An execution logic ( "run" function)
    • The "run" function will receive the widget value when you click the run button

Here is a very simple card that prints the value you enter

card({
    ui: (form) => ({ name: form.str({}) }),
    run: (runtime) => runtime.print('Hello World')
})

5.3. What is an App?

A Package is a folder containing

  • A list of Apps
  • A list of assets
  • A list of prefabs
  • Some documentation about how to use those apps
  • Some metadata for proper library indexing.

Decks are made to be shared.

  • You can download Decks from the Library panel
  • You can publish Decks from the Library panel

5.4. Create your first deck

  1. Click the [+ Create Package] button at the top of the Library

  2. Enter your github username and deck name

    1. 👉 Your github username will be your dedicated namespace in the library
    2. 👉 All your decks will live under the library/<username>/ folder

  3. Click [OK]

    1. It will create those files
      1. library/<username>/<deckname>/readme.md
        • A readme file to explain what your deck is about
      2. library/<username>/<deckname>/cushy-deck.json
        • Manifest that lists all your apps for better library integration
      3. library/<username>/<deckname>/_prefab.ts
        • A file where you can place your prefabs.
        • It contains some example prefabs for you to build upon
        • The file starts with an _ so it's not listed in CushyStudio
      4. library/<username>/<deckname>/sample-card-1.ts
        • First sample action
      5. library/<username>/<deckname>/sample-action-2.ts
        • Second sample action

  4. Open the sample-action-1.ts card

  5. Click the [edit] button to open the card in vscode

    • 👉 Everything is preconfigured to run correctly in vscode and make you productive right away
    • 👉 For this, you need to open the whole CushyStudio folder within vscode
    • 👉 (NOT just the action folder, NOR the action file itself)
card('demo1-basic', {
    author: 'rvion',
    ui: (form) => ({ positive: form.str({ label: 'Positive', default: 'flower' }), }),
    run: async (action, form) => {
        // Build a ComfyUI graph
        const graph = action.nodes
        const ckpt = graph.CheckpointLoaderSimple({ ckpt_name: 'albedobaseXL_v02.safetensors' })
        const seed = action.randomSeed()
        const sampler = graph.KSampler({
            seed: seed,
            steps: 20,
            cfg: 14,
            sampler_name: 'euler',
            scheduler: 'normal',
            denoise: 0.8,
            model: ckpt,
            positive: graph.CLIPTextEncode({ text: form.positive, clip: ckpt }),
            negative: graph.CLIPTextEncode({ text: '', clip: ckpt }),
            latent_image: graph.EmptyLatentImage({ width: 512, height: 512, batch_size: 1 }),
        })

        graph.SaveImage({
            images: graph.VAEDecode({ samples: sampler, vae: ckpt }),
            filename_prefix: 'ComfyUI',
        })

        // Run the graph you built
        await action.PROMPT()
    },
})

A Card is a file containing

  • An UI definition (widgets, form, styles, default values, tabs, etc...)
  • A piece of code that runs your action
  • ...And more

5.5. Prefabs so you don't repeat yourself

The simplest way to modularize your Apps is to build prefabs.

  • Prefabs allow you to re-use parts of your apps in other apps.
  • Prefabs allow you to re-use parts of your apps multiple times in your card
  • Prefabs are functions that can add widgets (ui part) or subgraphs (execution part).
  • Prefabs allow you to build complex apps very efficiently.

Best practices:

  • Usually, your Package will contain one or many prefab files,
  • You can The main prefab used by the built-in deck here: library/CushyStudio/default/_prefab.ts:

5.5.1. Prefabs are just functions.

  • 👉 A ui prefab is a function that takes a formBuilder and returns a Widget.
  • 👉 A run prefab is a function that takes a runtime and modifies its graph or performs other actions.
  • 👉 A util prefab is a helper function that does whatever.

Example:

// FILE: `_ui.ts`
import type { Builder } from 'src/controls/Builder'

// 📝 this is a ui prefab you can use in any card you want.
export const ui_startImage = (form: Builder) =>
    form.group({
        items: {
            startImage: form.imageOpt({ group: 'latent' }),
            width: form.int({ default: 512, group: 'latent', step: 128, min: 128, max: 4096 }),
            height: form.int({ default: 512, group: 'latent', step: 128, min: 128, max: 4096 }),
            batchSize: form.int({ default: 1, group: 'latent', min: 1, max: 20 }),
        },
    })

5.5.2. Naming Conventions

5.5.3. Use a UI Prefab

To use a prefab in your card, you can simply import it and use it.

  • ui prefabs are made to be used in the ui function of your card.
  • run prefabs are made to be used in the run function of your card.
  • run prefabs are made to be used in the run function of your card.
// FILE: `card1.ts`
import { ui_startImage } from './_ui'

card({
    name: 'card1',
    ui: (formBuilder) => {
        return {
            a: ui_startImage(formBuilder), // 👈 HERE
            b: ui_startImage(formBuilder), // 👈 HERE
            c: formBuilder.int({ default: 1 }),
        }
    },
    run: async (flow, p) => {
        flow.print(`startImage: ${p.a.startImage}`)
        flow.print(`startImage: ${p.b.startImage}`)
    },
})

5.5.4. Execution Prefab

The resulting card looks like this:

5.6. Various Function notations

CushyStudio aims to be accessible to non-programmers, while still being powerful enough for advanced users. You don't need to know typescript to get started, you can just learn what you need along the way.

This section is not a typescript tutorial, but rather a list of points of attention you may need to pay attention to when building apps if you're not a typescript developer.

Be careful to understand those various notations for functions

5.6.1. The special () => ({ }) notation to return objects in a single line

const fn = () => ({ a: 1 })
//               👆      👆

This is equivalent to:

const fn = () => {
    return {a: 1}
}

Or to

function fn() {
    return {a: 1}
}

5.7. Publish your Package

Publishing your deck/apps is easy!

  1. When creating a deck, CushyStudio automatically

    1. initializes the git repository for you.
    2. configures remote to point to the GitHub repository with the same name as your deck

  2. Create a GitHub repository. (https://github.com/new)

  3. Commit your deck files

    1. You can one-click the publish button in your deck entry in the library
    2. You can follow the instructions given by Git Hub on the new repository page

  4. Open an issue asking to add your Package to the Library.

SHOW EXAMPLE 
cd library/rvion

git init
Initialized empty Git repository in /Users/loco/dev/CushyStudio/library/rvion/.git/

git add .

git commit -m "first commit"
[master (root-commit) 602fab1] first commit
 4 files changed, 146 insertions(+)
 create mode 100644 mask-face.ts
 create mode 100644 rembg.ts
 create mode 100644 replace-part.ts
 create mode 100644 test.ts

git remote add origin git@github.com:rvion/cushy-example-deck.git

Then open an issue asking there

6. Frequently Asked Questions:

In case you want to know more about the CLA, here are some answers to questions you might have: If your questions are not covered, you can take a look at the full FAQ: https://github.com/rvion/CushyStudio/blob/master/docs/FAQ.md

6.1. Q: Will CushyStudio remain open source and free?

A: Absolutely! I'm committed to keeping CushyStudio 100% open source and free to use as a tool. Create assets, art, games – you name it. It's your creation, and I'm just here to help you make it.

6.2. Q: Why the AGPL and not the GPLV3?

A: For now, CushyStudio is licensed under AGPL as I aim to guide the project towards sustainability. I plan to transition to GPLV3 in the future, once the project achieves a stable foundation. This choice is designed to safeguard the project and its dedicated community. Rest assured, utilizing CushyStudio for asset creation remains unrestricted. My primary intention is to prevent the unauthorized resale or commercialization of CushyStudio as a service.

6.3. Q: Any plans for a paid version of CushyStudio?

A: No, and that's a promise. CushyStudio will always be free and open source. While I do have plans to offer complementary services (like cloud GPU services), the core of CushyStudio will remain free for all. Again, I simply wish to prevent the unauthorized resale or commercialization of CushyStudio as a service.

Thank you for your understanding and patience in going through this agreement. Together, let's make CushyStudio better for everyone! 💪🎉

7. Theming

Full theming capabilities using DaisyUI themes.

TODO: add screenshots here.