Skip to content

A system for Prompt generation to improve Text-to-Image performance.

License

Notifications You must be signed in to change notification settings

ananosleep/KGen

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

29 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

KGen - A System for Prompt Generation to Improve Text-to-Image Performance

*WIP*

KGen is a project that utilizes Large Language Models (LLMs) to generate prompts for Text-to-Image (T2I) models.

The goal is to enable T2I models to use more complicated and detailed captions during training while maintaining good usability.

Usage

Installation:

pip install tipo-kgen

Use in code: Read the Example code for more informations.

DanTagGen

DanTagGen is an early project under KGen, trained on the Danbooru tag system. Danbooru tags often have "overlaps" or "duplicates", such as:

  • "long hair" and "very long hair"
  • "thighhighs", "black thighhighs", and "black legwears"

Although users naturally avoid these duplications, the model may benefit from having the complete set of tags for better results, as that is how they were trained.

In addition to overlapping tags, "character tags" also need to be mentioned. For simplicity, a "DreamBooth style" prompt can be used to illustrate this:

  • Original: a dog
  • DreamBooth: a [V] dog
  • What User wants: [V]

As shown above, users tend to ignore all the "descriptions" that directly point to the character. In this situation, utilizing LLMs to "connect" the "trigger word" and "related description" is a promising approach. DanTagGen is an experimental project to prove this concept.

Architecture

DanTagGen uses the LLaMA architecture with 400M parameters.

Training

DanTagGen is trained on posts with the top 75% favorite count in Danbooru, which amounts to 5 million entries.

More details about the architecture and training can be found on the Hugging Face page: KBlueLeaf/DanTagGen-beta · Hugging Face

Usage

About

A system for Prompt generation to improve Text-to-Image performance.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%