Skip to content

🌟 AniScale 2 & AniScale 2 Refiner

Jump to bottom
Sirosky edited this page Nov 4, 2023 · 12 revisions

Introduction

AniScale 2 is a versatile and faithful anime model trained for use on a variety of post ~2000 sources. As the name suggests, this is the successor to the original AniScale, and a substantial upgrade in nearly every respect. Superb blur and depth of field handling, thorough WEB and DVD compression repair, and pleasing line art refinement are the hallmarks of AniScale 2.

AniScale 2 Variants

AniScale 2 is trained first and foremost as an OmniSR model, but it is also intended to be a platform to explore multiple SISR archs.

Currently, the following variants of AniScale 2 exist (roughly in order of my favorite to least favorite):

  • OmniSR [👑 best overall]: I consider this the best overall AniScale 2 variant. While nowhere near as fast as Compact, it is a good balance of speed and quality. It is the king of all tested archs in terms of blur/DOF handling, and one of the best in terms of detail retention. If you don't know which version to start with, pick OmniSR!
  • Compact [speeeeed, tensorrt]: Unsurprisingly, the Compact variant is incredibly fast, and even faster when using tensorrt for inference. It seems to do a better job of fixing certain issues with line art compared to the OmniSR variant. While it suffers in the detail retention department compared to its bigger brother, the detail retention is still more than adequate.
  • ESRGAN Lite [balance of speed and quality]: For the uninitiated, ESRGAN Lite is just the ESRGAN architecture with certain parameters halved. There might be a slight hit to quality, but the results are still very close to a full-sized ESRGAN model while being significantly faster (nearly as fast as Compact) without tensorrt. However, I haven't seen anything that supports tensorrt inference of ESRGAN Lite, which seems like a massively missed opportunity.
  • ESRGAN [tensorrt?]: I haven't been able to get this model working with tensorrt-- it seems like tensorrt support for ESRGAN is a bit uneven. Without tensorrt, ESRGAN is actually slower than OmniSR. Also note that the ESRGAN architecture as a whole struggles a bit with blur handling. AniScale2's ESRGAN model shouldn't have any serious issues there, but it's worth noting given that ESRGAN is after all an older architecture.
  • SwinIR Small [slow as shit, possibly more faithful]: SwinIR has been the golden standard in single image super resolution for sometime-- but for the purposes of video, it really is tests the limits of my patience in terms of inference speed. Based on my testing, OmniSR also produces better results overall. So why bother releasing this? Well, in some cases, the SwinIR version seems to produce more faithful results, so in the interest of freedom of choice, have at it!
  • DITN [not recommended]: DITN is one of the fastest transformer archs out there, roughly on par with ESRGAN in terms of inference speed. The problem is, it doesn't seem to be on par with ESRGAN in terms of output quality. For one thing, it is absolutely horrid at handling DOF and blur effects. It also has noticeably worse detail retention. On top of all that, none of the inference GUIs (Chainner for example) support DITN at this time, so you have to use CLI in neosr for inference. All this means that I cannot recommend DITN.
  • SRFormer Small [slow as shit, and bad]: Pretty much everything about it is mediocre or simply bad-- detail retention, de-compression, blur and DOF handling, etc. And you though SwinIR was slow? Well this is even slower! Given all this, I won't be releasing the model.

AniScale Refiner

AniScale Refiner is an optional add-on model which can be used if AniScale 2's output isn't sharp enough, and also serves the following two purposes:

  • When run before AniScale 2, it'll fix help fix line art that AniScale 2 doesn't do enough work on. Example here.
  • When run after AniScale 2, it'll apply apply line thinning, though this effect will likely be lessened on newer sources. Example here.

Thus, Refiner is as the name suggests-- a lightweight model that helps with touch-up of AniScale 2's output. Note that the sharpening effect will unavoidably impact shallower blur / DOF effects. Still, it remains a helpful tool to increase the versatility of the base model.

Specific Credits and Acknowledgements

  • muslll for creating neosr, on which these models are trained, and his extensive testing of the optimizers, losses, archs, etc.
  • the_database for letting me bounce my dumb ideas off him, and assistance with blur/lineart handling.
  • .kuronoe. for listening to my daily griping about OmniSR and helping me review the output of the countless release candidates.
Clone this wiki locally